LFD Book Forum Q8-10: Problem parsing instructions

#1
05-05-2013, 10:17 PM
 mvellon Junior Member Join Date: Apr 2013 Posts: 9
Q8-10: Problem parsing instructions

I understand how to generate and the data points in . I also know how to create permutations of these data points and that in each "epoch" of the algorithm I will want to visit each of the N points in a (newly) permuted data set. From here things get a little fuzzier. Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector initialized to zeros. What do I do with the weight vectors resulting from SGD executions within an epoch? Do I average them? How do I terminate the SGD algorithm in the execution? Do I let it run for a prescribed number of iterations or do I terminate when the gradient gets small?

Most of my confusion stems from interpreting:
Run Logistic Regression with Stochastic Gradient Descent to find g and estimate Eout (the cross entropy error) by generating a sufficiently large separate set of points to evaluating the error. Repeat the experiment for 100 runs with different targets and take the average.
The first part of it makes sense: run the experiment (multiple epochs) to get a final set of weights () then generate a large set of points on which to evaluate the error. The second part, "Repeat the experiment for 100 runs with different targets and take the average", I find confusing. What "experiment"? The whole multiple-epoch experiment? "With different targets" - does this mean different samples? Different permutations?

I fear I made a wrong turn in Albuquerque. Some clarification would be useful.

thanks
#2
05-05-2013, 10:56 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477
Re: Q8-10: Problem parsing instructions

Quote:
 Originally Posted by mvellon Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector initialized to zeros.
In each of these executions, you start with the weight vector that came out of the previous execution. Only the initial weights at the very beginning of the algorithm are set to zero.

Quote:
 I fear I made a wrong turn in Albuquerque.
__________________
Where everyone thinks alike, no one thinks very much
#3
05-05-2013, 11:01 PM
 mvellon Junior Member Join Date: Apr 2013 Posts: 9
Re: Q8-10: Problem parsing instructions

Ah - that helps a lot. Thanks.

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 08:53 AM.