Bias-Variance Analysis

[prithagupta.nsit]

Thank you professor, I think I can work from here.

[Jackwsimpson]

I am confused in trying to get from the first line to the second line for the first set of equations on page 63: ... ED[Ex[(g... on the first line to ...Ex[ED[( on the second line.

I sort of see the first line: expected value with respect to data set x (a subset of D I assume) is averaged over all possible data set x's in D. On the second line we have what might be the average of the argument over all of D inside the outer brackets. I don't know how to interpret Ex outside the outer brackets.

In short, I certainly don't understand what exactly is meant by the 2nd line, and I may well not understand the first line. Any further explanation possible?

[Vu Van Tu]

Dear Prof. Yaser,

Can you explain more about variance, the simple idea and example of this?
I read this paragraph but I still cannot fully understand.

'One can also view the variance as a measure of 'instability' in the learning model. Instability manifests in wild reactions to small variations or idiosyn*- crasies in the data, resulting in vastly different hypotheses.'
page 64

how does "small variations or idiosyn*-crasies in the data" effect our final hypothesis (variance of linear model vs constant model for example)?

Thank you very much!