- **Homework 5**
(*http://book.caltech.edu/bookforum/forumdisplay.php?f=134*)

- - **Evaluating a gradient function with vectors**
(*http://book.caltech.edu/bookforum/showthread.php?t=450*)

Evaluating a gradient function with vectorsLet's say we are using SGD with a gradient function
-(yn* xn)/(1 + e^(yn*w*xn)) where xn and w are 3-element vectors.When I evaluate this function, can I evaluate it 3 times, once for each corresponding x[i] and w[i], and thus get a 3-element gradient vector, then update w from that vector?Something like: Code:
`double gradient(double yn, double xn, double wn) {` |

Re: Evaluating a gradient function with vectorsI think your approach is wrong for the denominator. My understanding is that w*x is a inner product that makes a scalar, so the only vector part of the gradient is the yn*xn on the top of the fraction.
(In particular, all 3 w and x terms would be used in the exponential in the denominator for each of the 3 values in the gradient. But your numerator would use the 3 different x's for the 3 different terms.) |

Re: Evaluating a gradient function with vectorsBasically you have to calculate the partial derivative of the error function with respect to each dimension of w vector.
Then update each dimension of w using its partial derivative and the original w vector, and also the -eta. This will help: http://www.youtube.com/watch?v=U7HQ_...hannel&list=UL |

Re: Evaluating a gradient function with vectorsI realized that the numerator term (y_n*x_n) is a size 3 x n matrix that doesn't change for a particular data-set. So I evaluate it before I start the logistic regression loop. Inside the loop, the denominator has to be evaluated because
w is changing each time. |

All times are GMT -7. The time now is 03:04 AM. |

Powered by vBulletin® Version 3.8.3

Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.