
#1




Normal equation in linear regression
From coursera's ML course I've known that the normal equation is calculated as follows:
pinv((X'*X))*X'*Y; (octave code) but apparently this is equivalent to just pinv(X)*Y; Can anyone explain why this is the case? 
#2




Re: Normal equation in linear regression
pinv is already the pseudo inverse function. When applied to an invertible matrix (in this case X'*X), it returns the regular inverse.
__________________
Where everyone thinks alike, no one thinks very much 
#3




Re: Normal equation in linear regression
Dr. Ng derived the Normal Equation in class, see Lecture 46 and he also cautioned about the case where X'*X is noninvertible which meant that there were redundant features (linearly dependent) or too many features (m <= n).
Daniel 
#4




Re: Normal equation in linear regression
Just a small correction on notation. The normal equations for linear regression are
The solution to the normal equations (for ) is given by the formula that you mention, and indeed the two solutions are equivalent. A proof of this fact is using the singular value decomposition: and where and . So,
__________________
Have faith in probability 
#5




Re: Normal equation in linear regression
Thanks you all for useful input. We have some reading to do... :)
__________________
williamjhone 
Thread Tools  
Display Modes  

