Thread: role of P(X) ?
View Single Post
Old 04-11-2013, 09:55 AM
Elroch Elroch is offline
Invited Guest
Join Date: Mar 2013
Posts: 143
Default Re: role of P(X) ?

Originally Posted by kokrah View Post
The Hoeffding bound for the model H in chapter one, only requires that
we make the assumption that the input examples are a random sample
from the bin; so we can generalize the sample error.

What role does the distribution on X play? It appears to me that we don't need
it. (at least the way the issue of feasibility is setup in chapter 1)
ie. true mismatch ~ sample mismatch.

As I see it, the theory of generalisation relies on the fact that the distribution P(X, y) which gives the examples used to generate a model (both training and test data) is the same as the distribution of examples which we are trying to learn. There are two things that can go wrong. Either P(X) may be different, or P(y | X) may be different. In the first case, the examples may be concentrated in some subset of the input space, and this may be a region where the models work better. Obviously the second case can also lead to misleading conclusions.

[This may appear to be a trivial assumption when sampling from some populations, but it is likely to be non-trivial in many cases where we are attempting to infer future behavior from past behavior in a system whose characteristics may change]
Reply With Quote