LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 1 - The Learning Problem

Reply
 
Thread Tools Display Modes
  #1  
Old 04-08-2012, 12:32 AM
canon1230 canon1230 is offline
Junior Member
 
Join Date: Apr 2012
Posts: 2
Default question about probability

If an urn contains 100 green or red marbles, and you sample 10 and they are all green, what is the probability that they are all green?
Reply With Quote
  #2  
Old 04-08-2012, 02:30 AM
htlin's Avatar
htlin htlin is offline
NTU
 
Join Date: Aug 2009
Location: Taipei, Taiwan
Posts: 601
Default Re: question about probability

Quote:
Originally Posted by canon1230 View Post
If an urn contains 100 green or red marbles, and you sample 10 and they are all green, what is the probability that they are all green?
The short reply is that probability does not tell us this answer, unless there are more assumptions.

Put it to the simplest case, if you flip a coin and get a head, what is the head probability of the coin? The answer can be "any non-zero number" but there is no further information to pin it down.

Hope this helps.
__________________
When one teaches, two learn.
Reply With Quote
  #3  
Old 04-08-2012, 03:26 AM
GraceLAX GraceLAX is offline
Junior Member
 
Join Date: Apr 2012
Location: LAX
Posts: 4
Default Re: question about probability

Quote:
Originally Posted by canon1230 View Post
If an urn contains 100 green or red marbles, and you sample 10 and they are all green, what is the probability that they are all green?
A physics professor friend wrote this up to help clarify a common statistical misconception.
Do these help? Read through the second one for a similar example.

http://badmomgoodmom.blogspot.com/20...rt-one_16.html
http://badmomgoodmom.blogspot.com/20...-part-two.html
Reply With Quote
  #4  
Old 04-08-2012, 08:44 AM
canon1230 canon1230 is offline
Junior Member
 
Join Date: Apr 2012
Posts: 2
Default Re: question about probability

Does the Hoeffding Inequality allow us to say something about this probability?

P[|Ein - Eout| > epsilon] <= 2e^(-2 * epslion^2 * N)

Since Ein = 0, N = 10, setting epsilon to 0.5, the inequality gives us:

P[Eout > 0.5] <= 2e^(-5) = 0.013+

This seems to be saying something nontrivial about Eout.
Reply With Quote
  #5  
Old 04-08-2012, 03:01 PM
htlin's Avatar
htlin htlin is offline
NTU
 
Join Date: Aug 2009
Location: Taipei, Taiwan
Posts: 601
Default Re: question about probability

Quote:
Originally Posted by canon1230 View Post
Does the Hoeffding Inequality allow us to say something about this probability?

P[|Ein - Eout| > epsilon] <= 2e^(-2 * epslion^2 * N)

Since Ein = 0, N = 10, setting epsilon to 0.5, the inequality gives us:

P[Eout > 0.5] <= 2e^(-5) = 0.013+

This seems to be saying something nontrivial about Eout.
The P in Hoeffding is subject to the process of generating the sample (i.e. E_{in}), not the probability on E_{out}. Indeed it tells us something nontrivial (and that's how we use it in the learning context), but it does not answer your original question.

The question that got answered by Hoeffding is roughly

"What is the probability of a big-Eout urn (many red) for generating such an Ein (all green)?"

not

"What is the probability of Eout being small in the first place?"

The answer to the latter question remains unknown, but even so, we know that having a big Eout is unlikely because of Hoeffding.

Hope this helps.
__________________
When one teaches, two learn.
Reply With Quote
  #6  
Old 04-10-2012, 07:19 AM
rukacity rukacity is offline
Member
 
Join Date: Apr 2012
Posts: 21
Default Re: question about probability

I would think this way:

if p is probability of the outcome then for 10 trials there is p^10 probability of getting all favorable outcome.
Reply With Quote
  #7  
Old 04-13-2012, 11:00 PM
jsarrett jsarrett is offline
Member
 
Join Date: Apr 2012
Location: Sunland, CA
Posts: 13
Default Re: question about probability

Hoeffding's inequality has a free parameter in your question, namely \epsilon. It lest you say that since you have 10 samples, if you want to be 80% sure of the distribution of marbles in the jar (\epsilon = 0.2), then the jar is at most E_{out} percent different from the sample.

where
P(|E_{in}-E_{out}|
 < \epsilon) \le 2e^{-2\epsilon^2N}

substituting from your example:
P(|0-E_{out}| < 0.2) \le 2e^{-2(0.2^2)10}

simplifying:
P(|0-E_{out}| < 0.2) \le 0.898657928234443
so we think that with 80% confidence, the jar is at least 10.2% green.

We probably need a bigger N!
Reply With Quote
  #8  
Old 04-14-2012, 06:24 PM
pventurelli@gotoibr.com pventurelli@gotoibr.com is offline
Member
 
Join Date: Apr 2012
Posts: 12
Default Re: question about probability

In the second lecture, the Professor asked a question about flipping a coin:
What is the probability of getting all ten heads if you flip a coin 10 times count the number of heads, and then you repeat the experiment 1000 times.

The answer he gave was 63% -- I would like to know how this was computed.

Any help would be greatly appreciated.

Thanks!
Reply With Quote
  #9  
Old 04-14-2012, 09:11 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: question about probability

Quote:
Originally Posted by pventurelli@gotoibr.com View Post
In the second lecture, the Professor asked a question about flipping a coin:
What is the probability of getting all ten heads if you flip a coin 10 times count the number of heads, and then you repeat the experiment 1000 times.

The answer he gave was 63% -- I would like to know how this was computed.

Any help would be greatly appreciated.

Thanks!
The probability of getting 10 heads for one coin is {1 \over 2} \times {1 \over 2} \times \cdots \times {1 \over 2} (10 times) which is aprroximately {1 \over 1000}.

Therefore, the probability of not getting 10 heads for one coin is approximately (1-{1 \over 1000}).

This means that the probability of not getting 10 heads for any of 1000 coins is this number multiplied by itself 1000 times, once for every coin. This probability is therefore \approx (1-{1 \over 1000})^{1000}.

This is approximately {1 \over e} since \lim_{n\to\infty} (1-{1\over n})^n = {1\over e}. Numerically, {1 \over e}\approx{1\over 2.718}\approx 0.37.

Therefore, the probability of this not happening, namely that at least one coin of the 1000 coins will give 10 heads, is 1 minus that. This gives us the answer of approximately 0.63 or 63% that I mentioned in the lecture.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #10  
Old 04-15-2012, 04:51 PM
pventurelli@gotoibr.com pventurelli@gotoibr.com is offline
Member
 
Join Date: Apr 2012
Posts: 12
Default Re: question about probability

Thank you Professor, that was extremely helpful.
Reply With Quote
Reply

Tags
marble, probability, urn

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 06:08 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.