LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 5

Thread Tools Display Modes
Prev Previous Post   Next Post Next
Old 05-05-2013, 11:17 PM
mvellon mvellon is offline
Junior Member
Join Date: Apr 2013
Posts: 9
Default Q8-10: Problem parsing instructions

I understand how to generate f(x) and the N data points in X. I also know how to create permutations of these data points and that in each "epoch" of the algorithm I will want to visit each of the N points in a (newly) permuted data set. From here things get a little fuzzier. Each epoch will require a set of N "executions" of the SGD algorithm, running through all the points of its permuted data set. In each of these executions I start with the weight vector w initialized to zeros. What do I do with the weight vectors resulting from SGD executions within an epoch? Do I average them? How do I terminate the SGD algorithm in the execution? Do I let it run for a prescribed number of iterations or do I terminate when the gradient gets small?

Most of my confusion stems from interpreting:
Run Logistic Regression with Stochastic Gradient Descent to find g and estimate Eout (the cross entropy error) by generating a sufficiently large separate set of points to evaluating the error. Repeat the experiment for 100 runs with different targets and take the average.
The first part of it makes sense: run the experiment (multiple epochs) to get a final set of weights (g) then generate a large set of points on which to evaluate the error. The second part, "Repeat the experiment for 100 runs with different targets and take the average", I find confusing. What "experiment"? The whole multiple-epoch experiment? "With different targets" - does this mean different samples? Different permutations?

I fear I made a wrong turn in Albuquerque. Some clarification would be useful.

Reply With Quote

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -7. The time now is 01:54 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.