LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 4 - Overfitting

Thread Tools Display Modes
Prev Previous Post   Next Post Next
Old 10-21-2014, 04:10 PM
physicsme physicsme is offline
Junior Member
Join Date: Oct 2014
Posts: 2
Default Figure 4.3(b)

The title of Figure 4.3(b) is "Deterministic noise". However, the "overfitting level first decrease, hit sweet spot, then increase" trend with increase of Qf is the result of not only the deterministic noise, but of the stochastic noise as well. The reason that the fixed hypothesis set overfits the data when the target function is extremely simple is the existence of stochastic noise. If we eliminate that, a 10th order polynomial hypothesis set will fit a 2nd order polynomial target function exactly as it is.
In this sense, the title "Deterministic noise" of figure 4.3(b) is a bit misleading.

Actually, I became aware of this while doing exercise 4.3. At first I thought the answer should be "deterministic noise will go up all the way with increase of target complexity", then I looked at Fig. 4.3(b) and thought "hey, it says deterministic noise will first go down and then go up!". But come to think of it, the figure is really Eout(H10)-Eout(H2), which include effect of both stochastic and deterministic noise, hence the post.
Reply With Quote

figure, noise, suggestion

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -7. The time now is 02:32 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.