Q8 implementation question

pyguy · #1 05-06-2013, 12:37 PM

I'm trying to compute the stochastic gradient descent for linear regression, and the formula I'm using is:

$\nabla e_n(\mathbf w) = \frac{-y_n\mathbf x_n}{1 + e^{y_n\mathbf{w^T}\mathbf{x_n}}}$

I'm running the experiment with what I believe to be the correct inputs, but I'm not getting the what I expect to be the output, so I'm trying to trace my steps and see where I went wrong. I was looking at the formula, and one part that I was uncertain about was the $\mathbf{w^T}\mathbf{x_n}$ part. Aren't $\mathbf{w}$ and $\mathbf{x_n}$ both 1x3 row vectors? If I transpose $\mathbf{w}$ , and multiply, I'd get a 3x3 matrix which didn't make sense to me in the calculation, so I'm essentially multiplying them right now as if they were just 1x3 row vectors.

yaser · #2 05-06-2013, 01:11 PM

Quote:

Originally Posted by pyguy

I'm trying to compute the stochastic gradient descent for linear regression, and the formula I'm using is:

$\nabla e_n(\mathbf w) = \frac{-y_n\mathbf x_n}{1 + e^{y_n\mathbf{w^T}\mathbf{x_n}}}$

I'm running the experiment with what I believe to be the correct inputs, but I'm not getting the what I expect to be the output, so I'm trying to trace my steps and see where I went wrong. I was looking at the formula, and one part that I was uncertain about was the $\mathbf{w^T}\mathbf{x_n}$ part. Aren't $\mathbf{w}$ and $\mathbf{x_n}$ both 1x3 row vectors? If I transpose $\mathbf{w}$ , and multiply, I'd get a 3x3 matrix which didn't make sense to me in the calculation, so I'm essentially multiplying them right now as if they were just 1x3 row vectors.

The convention is that vectors are column vectors, so $\mathbf{w}^{\rm T}\mathbf{x_n}$ is an inner product, not an outer product.

pyguy · #3 05-06-2013, 07:15 PM

Thank you! I knew I was missing something with the math and my best guess was I was doing something wrong there.

jlevy · #4 05-07-2013, 05:18 AM

Looks to me like the derivative is incorrect

Thread Tools
Show Printable Version Email this Page
Display Modes
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode