View Single Post
Old 06-10-2013, 01:55 PM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Exchange of expectations in derivation of bias-variance decomposition

Originally Posted by kynnjo View Post
[I originally posted this question to the wrong sub-forum. My apologies.]

In the derivation of the bias-variance decomposition (on p. 63), there is a step in which the taking of expectation wrt \mathcal{D} and wrt \mathbf{x} are exchanged.

It's not clear to me that these two expectations commute: the choice of \mathbf{x} depends on the choice of \mathcal{D}, and viceversa.

I would appreciate a clarification on this point.

Thanks in advance,


P.S. Pardon the "raw LaTeX" above. Is there a better way to include mathematical notation in these posts.
It is not independence that allows us to change the order of integration. It is the fact that the integrand is always nonnegative. Think of it as a double summation. You are adding up the same set of numbers whether you start with one sum or the other. The problem arises when some of these numbers are positive and some are negative, since in that case you can play tricks with different orders of the summation to converge to different values. Look up "absolute convergence" versus "conditional convergence."
Where everyone thinks alike, no one thinks very much
Reply With Quote