View Single Post
  #1  
Old 05-05-2012, 12:46 PM
kurts kurts is offline
Invited Guest
 
Join Date: Apr 2012
Location: Portland, OR
Posts: 70
Default Evaluating a gradient function with vectors

Let's say we are using SGD with a gradient function
-(yn*xn)/(1 + e^(yn*w*xn)) where xn and w are 3-element vectors.

When I evaluate this function, can I evaluate it 3 times, once for each corresponding x[i] and w[i], and thus get a 3-element gradient vector, then update w from that vector?
Something like:

Code:
double gradient(double yn, double xn, double wn) {
  double num = -1.0*yn*xn;
  double denom = 1 + exp(yn*wn*xn);
  return num / denom;
}

double g0 = gradient(yn, 1.0, w0);
double g1 = gradient(yn, x1n, w1);
double g2 = gradient(yn, x2n, w2);
w0 = w0 - eta*g0;
w1 = w1 - eta*g1;    
w2 = w2 - eta*g2;
I'm not entirely confident that this is correct.
Reply With Quote