LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 5

Reply
 
Thread Tools Display Modes
  #1  
Old 05-05-2013, 10:17 PM
mvellon mvellon is offline
Junior Member
 
Join Date: Apr 2013
Posts: 9
Default Q8-10: Problem parsing instructions

I understand how to generate f(x) and the N data points in X. I also know how to create permutations of these data points and that in each "epoch" of the algorithm I will want to visit each of the N points in a (newly) permuted data set. From here things get a little fuzzier. Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector w initialized to zeros. What do I do with the weight vectors resulting from SGD executions within an epoch? Do I average them? How do I terminate the SGD algorithm in the execution? Do I let it run for a prescribed number of iterations or do I terminate when the gradient gets small?

Most of my confusion stems from interpreting:
Run Logistic Regression with Stochastic Gradient Descent to find g and estimate Eout (the cross entropy error) by generating a sufficiently large separate set of points to evaluating the error. Repeat the experiment for 100 runs with different targets and take the average.
The first part of it makes sense: run the experiment (multiple epochs) to get a final set of weights (g) then generate a large set of points on which to evaluate the error. The second part, "Repeat the experiment for 100 runs with different targets and take the average", I find confusing. What "experiment"? The whole multiple-epoch experiment? "With different targets" - does this mean different samples? Different permutations?

I fear I made a wrong turn in Albuquerque. Some clarification would be useful.

thanks
Reply With Quote
  #2  
Old 05-05-2013, 10:56 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Q8-10: Problem parsing instructions

Quote:
Originally Posted by mvellon View Post
Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector w initialized to zeros.
In each of these executions, you start with the weight vector {\bf w} that came out of the previous execution. Only the initial weights at the very beginning of the algorithm are set to zero.

Quote:
I fear I made a wrong turn in Albuquerque.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 05-05-2013, 11:01 PM
mvellon mvellon is offline
Junior Member
 
Join Date: Apr 2013
Posts: 9
Default Re: Q8-10: Problem parsing instructions

Ah - that helps a lot. Thanks.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:42 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.