Let's say we are using SGD with a gradient function
-(yn*
xn)/(1 + e^(yn*
w*
xn)) where
xn and
w are 3-element vectors.
When I evaluate this function, can I evaluate it 3 times, once for each corresponding x[i] and w[i], and thus get a 3-element gradient vector, then update
w from that vector?
Something like:
Code:
double gradient(double yn, double xn, double wn) {
double num = -1.0*yn*xn;
double denom = 1 + exp(yn*wn*xn);
return num / denom;
}
double g0 = gradient(yn, 1.0, w0);
double g1 = gradient(yn, x1n, w1);
double g2 = gradient(yn, x2n, w2);
w0 = w0 - eta*g0;
w1 = w1 - eta*g1;
w2 = w2 - eta*g2;
I'm not entirely confident that this is correct.