View Single Post
#2
08-17-2012, 02:26 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Overfitting with Polynomials : deterministic noise

Quote:
 Originally Posted by Kevin Chapter 4 state that: - A 2nd order polynomial could be better than a 10th order polynomial to fit a 50th order polynomial target and it's due to the deterministic noise. --> So I conclude that there is more deterministic noise with 10th order than with the 2nd order.
The confusion is justified, since there are two opposing factors here (see Exercise 4.3). There is more deterministic noise with the 2nd order model than with the 10th order model (which would suggest more overfitting with the 2nd order), but the model itself is simpler so that would suggest less overfitting. It turns out that the latter factor wins here.

If you want to isolate the impact of deterministic noise on overfitting without interference from the model complexity, you can fix the model and change the complexity of the target function.
__________________
Where everyone thinks alike, no one thinks very much