View Single Post
Old 04-08-2012, 04:01 PM
htlin's Avatar
htlin htlin is offline
Join Date: Aug 2009
Location: Taipei, Taiwan
Posts: 610
Default Re: question about probability

Originally Posted by canon1230 View Post
Does the Hoeffding Inequality allow us to say something about this probability?

P[|Ein - Eout| > epsilon] <= 2e^(-2 * epslion^2 * N)

Since Ein = 0, N = 10, setting epsilon to 0.5, the inequality gives us:

P[Eout > 0.5] <= 2e^(-5) = 0.013+

This seems to be saying something nontrivial about Eout.
The P in Hoeffding is subject to the process of generating the sample (i.e. E_{in}), not the probability on E_{out}. Indeed it tells us something nontrivial (and that's how we use it in the learning context), but it does not answer your original question.

The question that got answered by Hoeffding is roughly

"What is the probability of a big-Eout urn (many red) for generating such an Ein (all green)?"


"What is the probability of Eout being small in the first place?"

The answer to the latter question remains unknown, but even so, we know that having a big Eout is unlikely because of Hoeffding.

Hope this helps.
When one teaches, two learn.
Reply With Quote