LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 5

Reply
 
Thread Tools Display Modes
  #1  
Old 05-06-2013, 12:10 AM
bargava bargava is offline
Junior Member
 
Join Date: Apr 2013
Posts: 8
Default Q7 - understanding co-ordinate descent

I didn't entirely understand what co-ordinate descent meant. This is what I believe it to be: Instead of descending "simultaneously" along all the co-ordinates as in gradient descent(in this eg: both u and v), we first descend along u, find the new u and then find v. So, when computing v, the new value of u is to be used. Am I right?
Reply With Quote
  #2  
Old 05-06-2013, 01:06 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Q7 - understanding co-ordinate descent

Quote:
Originally Posted by bargava View Post
I didn't entirely understand what co-ordinate descent meant. This is what I believe it to be: Instead of descending "simultaneously" along all the co-ordinates as in gradient descent(in this eg: both u and v), we first descend along u, find the new u and then find v. So, when computing v, the new value of u is to be used. Am I right?
Correct. After each update along one coordinate, you compute the derivative at the new point, then descend along the other coordinate. This is not an efficient method, and is meant for comparison with gradient descent.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 05-06-2013, 11:12 PM
Darcy Daugela Darcy Daugela is offline
Junior Member
 
Join Date: Apr 2013
Location: Edmonton
Posts: 3
Default Re: Q7 - understanding co-ordinate descent

I am struggling to understand what I did wrong on this question.

The instructions are clear, I followed the method above (I think?), my answers to related questions (5 and 6) were correct, but my answer to question 7 is far far less than the correct answer. I got the answer level of accuracy in only 5 iterations (instead of 15), so I must have a serious problem with my algorithm.

I am wondering if I understand the term "only to reduce error". I took this to mean that after each step I recalculate the error, and if the error increased I do not apply the update. This helped rapid convergence significantly.

Upon researching why I got this answer wrong, I ran across some conflicting references that suggest "coordinate descent" can be much more efficient algorithm than GD because of some tricks to re-use parts of the calculation. I'm not sure what to think.
Reply With Quote
  #4  
Old 05-06-2013, 11:55 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Q7 - understanding co-ordinate descent

Quote:
Originally Posted by Darcy Daugela View Post
I am wondering if I understand the term "only to reduce error". I took this to mean that after each step I recalculate the error, and if the error increased I do not apply the update. This helped rapid convergence significantly.
I see where the misunderstanding is. The word 'only' is meant to qualify the previous part: move along the u coordinate only to reduce the error.' Having said that, evaluating the error then undoing the step is not indicated given the part that follows: '(assume first-order approximation holds like in gradient descent).'
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 02:54 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.