Thread: Question 1
View Single Post
Old 07-15-2013, 06:30 AM
hsolo hsolo is offline
Join Date: Jul 2013
Posts: 12
Default Re: Question 1

Originally Posted by yaser View Post
First, just to make sure, the inside 'E' is not an expectation, but the value of the in-sample error that corresponds to the weight vector {\bf w}_{\rm lin}. The (outside) expected value is with respect to the training data set, and it means the average value (of the in-sample error) as you train with different data sets.

Training data has d dimensions in the x's. If one ignored some of the dimensions and did linear regression with reduced number d' of dimensions one would have larger in-sample errors presumably, compared to considering all d dimensions?

Why then is the expected in-sample error averaged over all data sets increasing with the number of dimensions?
Reply With Quote