LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 5 (http://book.caltech.edu/bookforum/forumdisplay.php?f=134)
-   -   Q8-10: Problem parsing instructions (http://book.caltech.edu/bookforum/showthread.php?t=4266)

mvellon 05-05-2013 11:17 PM

Q8-10: Problem parsing instructions
 
I understand how to generate f(x) and the N data points in X. I also know how to create permutations of these data points and that in each "epoch" of the algorithm I will want to visit each of the N points in a (newly) permuted data set. From here things get a little fuzzier. Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector w initialized to zeros. What do I do with the weight vectors resulting from SGD executions within an epoch? Do I average them? How do I terminate the SGD algorithm in the execution? Do I let it run for a prescribed number of iterations or do I terminate when the gradient gets small?

Most of my confusion stems from interpreting:
Run Logistic Regression with Stochastic Gradient Descent to find g and estimate Eout (the cross entropy error) by generating a sufficiently large separate set of points to evaluating the error. Repeat the experiment for 100 runs with different targets and take the average.
The first part of it makes sense: run the experiment (multiple epochs) to get a final set of weights (g) then generate a large set of points on which to evaluate the error. The second part, "Repeat the experiment for 100 runs with different targets and take the average", I find confusing. What "experiment"? The whole multiple-epoch experiment? "With different targets" - does this mean different samples? Different permutations?

I fear I made a wrong turn in Albuquerque. Some clarification would be useful.

thanks

yaser 05-05-2013 11:56 PM

Re: Q8-10: Problem parsing instructions
 
Quote:

Originally Posted by mvellon (Post 10710)
Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector w initialized to zeros.

In each of these executions, you start with the weight vector {\bf w} that came out of the previous execution. Only the initial weights at the very beginning of the algorithm are set to zero.

Quote:

I fear I made a wrong turn in Albuquerque.
:D

mvellon 05-06-2013 12:01 AM

Re: Q8-10: Problem parsing instructions
 
Ah - that helps a lot. Thanks.


All times are GMT -7. The time now is 05:38 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.