View Single Post
Old 08-01-2012, 07:32 PM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Why would variance be non-zero?

Originally Posted by samirbajaj View Post
If we are sampling uniformly from the interval [-1, 1] for the calculation of g_bar, as well as for each data set (g_d), why would the variance be anything but a very small quantity? In the general case, when the data sets are not drawn from a uniform distribution, a non-zero variance makes sense, but if there is sufficient overlap in the data sets, it makes intuitive sense that the variance should be close to zero.
\bar g is the average of g^{({\cal D})} over different data sets {\cal D}. You will get different g^{({\cal D})}'s when you pick different {\cal D}'s, since the final hypothesis depends on the data set used for training. Therefore, there will be a variance that measures how different these g^{({\cal D})}'s are around their expected value \bar g (which does not depend on {\cal D} as {\cal D} gets integrated out in the calculation of \bar g).

This argument holds for any probability distribution, uniform or not, that is used to generate the different {\cal D}'s.
Where everyone thinks alike, no one thinks very much
Reply With Quote