Quote:
Originally Posted by Andrew87
Hello,
I'm getting confused about  . Why is it the best approximation of the target function  we could obtain in the unreal case of infinite training sets ?
Thank you in advance,
Andrea |
It is not necessarily the best approximation of the target function, but it is often close. If we have one, infinite-size training set, and we have infinite computational power that goes with it, we can arrive at the best approximation. In the bias-variance analysis, we are given an infinite number of
finite training sets, and we are restricted to using one of these finite training sets at a time, then averaging the resulting hypotheses. This restriction can take us away from the absolute optimal, but usually not by much.