LFD Book Forum - E_in E_out trade-off, Couldn't E

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)

- Chapter 1 - The Learning Problem (http://book.caltech.edu/bookforum/forumdisplay.php?f=108)

- - E_in E_out trade-off, Couldn't E_in always be 0? (http://book.caltech.edu/bookforum/showthread.php?t=4493)

E_in E_out trade-off, Couldn't E_in always be 0?

QUESTION: Page 25-26. $E_{in}$ and $E_{out}$ tradeoff. Shouldn't $E_{in}$ always be able to be 0? Since we can always be lucky and just came up with a polynomial with

(number of sample points in $\mathcal{D}$ ) degree that fits the sample points perfectly. Here since we try to keep

(number of hypothesis) to be 1, we assume we didn't look at the data (thus

) and just straight-forwardly set $\mathcal{H}=\{h\}$ (thus avoiding complex $\mathcal{H}$ ), where

is the

degree perfect-fit polynomial. Wouldn't this case make $E_{in}$ always possible to be 0?

Thank you so much!

Re: E_in E_out trade-off, Couldn't E_in always be 0?

Quote:

Originally Posted by chemaoxfz (Post 11711)

QUESTION: Page 25-26. $E_{in}$ and $E_{out}$ tradeoff. Shouldn't $E_{in}$ always be able to be 0? Since we can always be lucky and just came up with a polynomial with

(number of sample points in $\mathcal{D}$ ) degree that fits the sample points perfectly. Here since we try to keep

(number of hypothesis) to be 1, we assume we didn't look at the data (thus

) and just straight-forwardly set $\mathcal{H}=\{h\}$ (thus avoiding complex $\mathcal{H}$ ), where

is the

degree perfect-fit polynomial. Wouldn't this case make $E_{in}$ always possible to be 0?

Thank you so much!

The statement "can always be zero" needs to be clarified a bit. If you mean "it is possible that the error will be zero using a singleton hypothesis set," then that's true. If you mean "we can always choose a singleton hypothesis set that makes the error zero" then that's not true, because we don't know which hypothesis to pick if we didn't look at the data.

Being lucky is similar to being lucky with the lottery; you can always pick the winning lottery ticket if you are lucky enough, but that does not mean that there is a way to make this happen.