Quote:
Originally Posted by noahdavis
I'm a little bit confused by the section "Generalization error" on Page 40 of the LFD textbook. What is the purpose of establishing a "Generalization bound"? Why is the generalization bound a useful rephrasing of Hoeffding Inequality?
|
The generalization bound form does not add to the Hoeffding form in terms of substance. It only articulates the tradeoff based on error and complexity terms.
Quote:
|
I'm also confused of the use of the word "tolerance" - since epsilon and delta are both used to refer to a kind of "tolerance"
|
You are right. One can start with either epsilon or delta as a tolerance parameter and derive what the other quantity needs to be. The cause and effect may be a bit more natural starting with epsilon and seeing what delta needs to be, but the other direction is also legitimate.