View Single Post
Old 03-29-2015, 10:48 AM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Bias-Variance Analysis

Originally Posted by Andrew87 View Post

I'm getting confused about \bar{g}(x). Why is it the best approximation of the target function f we could obtain in the unreal case of infinite training sets ?

Thank you in advance,
It is not necessarily the best approximation of the target function, but it is often close. If we have one, infinite-size training set, and we have infinite computational power that goes with it, we can arrive at the best approximation. In the bias-variance analysis, we are given an infinite number of finite training sets, and we are restricted to using one of these finite training sets at a time, then averaging the resulting hypotheses. This restriction can take us away from the absolute optimal, but usually not by much.
Where everyone thinks alike, no one thinks very much
Reply With Quote