View Single Post
Old 04-25-2013, 02:03 PM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Question 4 linear regression hypothesis

Originally Posted by View Post
On question 4, I tried to fit each of two sample points through (i) h = ax, and (ii) h = ax+b. I found that hypothesis (i) gave me an average of "a" quite different to any of the answers, but hypothesis (ii) gave me an average of "a" very close to one of the answer options and the average of b is virtually close to 0. If average of b is 0 in (ii), why the average of a are different in (i) and (ii)? Can anyone help me explaining these?
Let me address why they can be different. The model h(x)=ax+b can fit both points in the training set {\cal D} perfectly, while the model h(x)=ax finds a compromise that minimizes the mean-squared error on those points. Because of this, we have different fits that can have different averages. Symmetry dictates that b will average to zero for the first model, but that does not mean that the average a should be the same as the second model.

Having said that, you should get an answer that matches one of the 5 choices when you fit the h(x)=ax model properly.
Where everyone thinks alike, no one thinks very much
Reply With Quote