Quote:
Originally Posted by yaser
It is unlikely (as a practical observation) to be far, but it is likely to be different.

Sorry to be harping on this question, but I just wanted to ask: is there any intuitive way to see that the average hypothesis will be close to the best hypothesis from the hypothesis set, beyond "practical observation"? E.g. for hypothesis sets satisfying certain wellbehavedness criteria, such as being parameterized by a finite number of parameters, containing only continuous functions, etc. The lectures rely in crucial ways on this assumption and it would help to get some more intuition for why it is true for the typically used hypothesis sets, if possible.