LFD Book Forum - View Single Post

magdon · #4 09-20-2012, 04:24 AM

Just a small correction on notation. The normal equations for linear regression are

The solution to the normal equations (for

) is given by the formula that you mention, and indeed the two solutions are equivalent. A proof of this fact is using the singular value decomposition:

$X=U\Sigma V^T$ and $X^\dagger=V\Sigma^\dagger U^T$

where

and

. So,

$(X^TX)^\dagger X^T=(V\Sigma U^TU\Sigma V^T)^\dagger V\Sigma U^T=V(\Sigma^2)^\dagger V^T V\Sigma U^T=V\Sigma^\dagger U^T=X^\dagger$

Quote:

Originally Posted by lorddoskias

From coursera's ML course I've known that the normal equation is calculated as follows:

pinv((X'*X))*X'*Y; (octave code) but apparently this is equivalent to just pinv(X)*Y;

Can anyone explain why this is the case?