View Single Post
Old 09-25-2012, 02:57 AM
htlin's Avatar
htlin htlin is offline
Join Date: Aug 2009
Location: Taipei, Taiwan
Posts: 610
Default Re: On the 'linear model I'',one step learning in not what linear...

I'd like to share two pieces of thoughts on your view of linear regression.

(0) While linear regression can be described as one-step learning, calculating the pseudo inverse of \mathrm{X} (which is N by d+1) generally contains multiple steps in the order of O(poly(N, d)). So whether to call it one-step learning depends on how you view the process.

(1) While using the pseudo inverse is arguably the most common way of doing linear regression since the early years of Statistics, indeed it is not the only way. Applying gradient decent on linear regression, as you describe, can also work and would allow you to enjoy something similar to PLA (or logistic regression with gradient decent) --- making the line iteratively approach the optimal one.
When one teaches, two learn.
Reply With Quote