LFD Book Forum - Page 63 and excercise 2.8

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)

- Chapter 2 - Training versus Testing (http://book.caltech.edu/bookforum/forumdisplay.php?f=109)

- - Page 63 and excercise 2.8 (http://book.caltech.edu/bookforum/showthread.php?t=4500)

Page 63 and excercise 2.8

On page 63 its given that g_(x) is approximately the mean of all gk(x) for any x. Why is it an estimate and not exactly the mean?

Since g_(x) is the average for any x, then its possible for it to have non binary values for a binary classification problem. But this seems to be a bit counter intuitive to me. Can you please clarify if my understanding of g_(x) is correct

Re: Page 63 and excercise 2.8

I think $\overline{g}(\textbf{x}) \approx \frac{1}{K} \sum\limits_{k = 1}^{K} g_k(\textbf{x})$ because it's computed over a finite set of points, whereas the actual value for $\overline{g}(\textbf{x})$ is an exact solution

Re: Page 63 and excercise 2.8

$\overline{g}(\textbf{x}) \approx \frac{1}{K} \sum\limits_{k = 1}^{K} g_k(\textbf{x})$ because $\overline{g}(\textbf{x})$ is defined as an expectation with respect to data sets of g(x). The average over data sets approximates this expectation.

Yes, $\overline{g}(\textbf{x})$ is not a valid hypothesis: it may not be in your hypothesis set; it may not even be binary. It is never used as a classifier. It is just used to represent "what would happen on average after learning", and this abstract function plays a role in defining the bias in the bias variance decomposition.

Quote:

Originally Posted by Newbrict (Post 11733)