Thread: HW 2 Problem 6
View Single Post
Old 07-23-2012, 04:59 PM
rakhlin rakhlin is offline
Join Date: Jun 2012
Posts: 24
Default Re: HW 2 Problem 6

Originally Posted by yaser View Post
Does variation in the average error mean that you repeat the entire experiment you described (including the target, training set, and resulting linear fit) and look at the different averages you get?
Not quite so. When I repeat entire experiment (including the target, training set, and resulting linear fit) I always get approximately same averages. I get different averages (0.01 ... 0.13) when I use one target and average over 1000 out-of-sample sets (for a single target).

Now as I plotted both lines (target and hypothesis) per your advice I begin to think that this is maybe what we should expect. Linear regression not always fits well. Usually it looks good giving small in-sample error. But sometimes disagreement is visually large (>0.1 in-sample error). This is the root of variation in average error when I use same "bad" regression for all 1000 iterations. I hope this type of experiment isn't implied by the problem. Otherwise it has no certain answer - at least 2 answers match.

So there is another question. Is >0.1 in-sample error and visually non-optimal fit still valid outcome of linear regression for linearly separable data?
Reply With Quote