
#1




Figure 4.3(b)
The title of Figure 4.3(b) is "Deterministic noise". However, the "overfitting level first decrease, hit sweet spot, then increase" trend with increase of Qf is the result of not only the deterministic noise, but of the stochastic noise as well. The reason that the fixed hypothesis set overfits the data when the target function is extremely simple is the existence of stochastic noise. If we eliminate that, a 10th order polynomial hypothesis set will fit a 2nd order polynomial target function exactly as it is.
In this sense, the title "Deterministic noise" of figure 4.3(b) is a bit misleading. Actually, I became aware of this while doing exercise 4.3. At first I thought the answer should be "deterministic noise will go up all the way with increase of target complexity", then I looked at Fig. 4.3(b) and thought "hey, it says deterministic noise will first go down and then go up!". But come to think of it, the figure is really Eout(H10)Eout(H2), which include effect of both stochastic and deterministic noise, hence the post. 
#2




Re: Figure 4.3(b)
An explanation by @yaser was posted on the 2014 edX forum:
Quote:

#3




Re: Figure 4.3(b)
Another question about this figure: is there any intuition as to why regions of a given color (fixed overfit measure) are roughly linear in the stochastic Nsigma^2 graph, but nonlinear (exponential?) in the deterministic NQf graph (for Qf > 10)?
Thank you! 
#4




Re: Figure 4.3(b)
Quote:
BTW, the LaTeX stuff is done by delimiters [ math ] and [ /math ] (without the spaces) instead of $ and $. A bit cumbersome when you use a lot of math.
__________________
Where everyone thinks alike, no one thinks very much 
Tags 
figure, noise, suggestion 
Thread Tools  
Display Modes  

