LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 1 - The Learning Problem (http://book.caltech.edu/bookforum/forumdisplay.php?f=108)
-   -   question about probability (http://book.caltech.edu/bookforum/showthread.php?t=287)

canon1230 04-08-2012 01:32 AM

question about probability
 
If an urn contains 100 green or red marbles, and you sample 10 and they are all green, what is the probability that they are all green?

htlin 04-08-2012 03:30 AM

Re: question about probability
 
Quote:

Originally Posted by canon1230 (Post 1027)
If an urn contains 100 green or red marbles, and you sample 10 and they are all green, what is the probability that they are all green?

The short reply is that probability does not tell us this answer, unless there are more assumptions. :clueless:

Put it to the simplest case, if you flip a coin and get a head, what is the head probability of the coin? The answer can be "any non-zero number" but there is no further information to pin it down.

Hope this helps.

GraceLAX 04-08-2012 04:26 AM

Re: question about probability
 
Quote:

Originally Posted by canon1230 (Post 1027)
If an urn contains 100 green or red marbles, and you sample 10 and they are all green, what is the probability that they are all green?

A physics professor friend wrote this up to help clarify a common statistical misconception.
Do these help? Read through the second one for a similar example.

http://badmomgoodmom.blogspot.com/20...rt-one_16.html
http://badmomgoodmom.blogspot.com/20...-part-two.html

canon1230 04-08-2012 09:44 AM

Re: question about probability
 
Does the Hoeffding Inequality allow us to say something about this probability?

P[|Ein - Eout| > epsilon] <= 2e^(-2 * epslion^2 * N)

Since Ein = 0, N = 10, setting epsilon to 0.5, the inequality gives us:

P[Eout > 0.5] <= 2e^(-5) = 0.013+

This seems to be saying something nontrivial about Eout.

htlin 04-08-2012 04:01 PM

Re: question about probability
 
Quote:

Originally Posted by canon1230 (Post 1042)
Does the Hoeffding Inequality allow us to say something about this probability?

P[|Ein - Eout| > epsilon] <= 2e^(-2 * epslion^2 * N)

Since Ein = 0, N = 10, setting epsilon to 0.5, the inequality gives us:

P[Eout > 0.5] <= 2e^(-5) = 0.013+

This seems to be saying something nontrivial about Eout.

The P in Hoeffding is subject to the process of generating the sample (i.e. E_{in}), not the probability on E_{out}. Indeed it tells us something nontrivial (and that's how we use it in the learning context), but it does not answer your original question.

The question that got answered by Hoeffding is roughly

"What is the probability of a big-Eout urn (many red) for generating such an Ein (all green)?"

not

"What is the probability of Eout being small in the first place?"

The answer to the latter question remains unknown, but even so, we know that having a big Eout is unlikely because of Hoeffding.

Hope this helps.

rukacity 04-10-2012 08:19 AM

Re: question about probability
 
I would think this way:

if p is probability of the outcome then for 10 trials there is p^10 probability of getting all favorable outcome.

jsarrett 04-14-2012 12:00 AM

Re: question about probability
 
Hoeffding's inequality has a free parameter in your question, namely \epsilon. It lest you say that since you have 10 samples, if you want to be 80% sure of the distribution of marbles in the jar (\epsilon = 0.2), then the jar is at most E_{out} percent different from the sample.

where
P(|E_{in}-E_{out}|
 < \epsilon) \le 2e^{-2\epsilon^2N}

substituting from your example:
P(|0-E_{out}| < 0.2) \le 2e^{-2(0.2^2)10}

simplifying:
P(|0-E_{out}| < 0.2) \le 0.898657928234443
so we think that with 80% confidence, the jar is at least 10.2% green.

We probably need a bigger N!

pventurelli@gotoibr.com 04-14-2012 07:24 PM

Re: question about probability
 
In the second lecture, the Professor asked a question about flipping a coin:
What is the probability of getting all ten heads if you flip a coin 10 times count the number of heads, and then you repeat the experiment 1000 times.

The answer he gave was 63% -- I would like to know how this was computed.

Any help would be greatly appreciated.

Thanks!

yaser 04-14-2012 10:11 PM

Re: question about probability
 
Quote:

Originally Posted by pventurelli@gotoibr.com (Post 1287)
In the second lecture, the Professor asked a question about flipping a coin:
What is the probability of getting all ten heads if you flip a coin 10 times count the number of heads, and then you repeat the experiment 1000 times.

The answer he gave was 63% -- I would like to know how this was computed.

Any help would be greatly appreciated.

Thanks!

The probability of getting 10 heads for one coin is {1 \over 2} \times {1 \over 2} \times \cdots \times {1 \over 2} (10 times) which is aprroximately {1 \over 1000}.

Therefore, the probability of not getting 10 heads for one coin is approximately (1-{1 \over 1000}).

This means that the probability of not getting 10 heads for any of 1000 coins is this number multiplied by itself 1000 times, once for every coin. This probability is therefore \approx (1-{1 \over 1000})^{1000}.

This is approximately {1 \over e} since \lim_{n\to\infty} (1-{1\over n})^n = {1\over e}. Numerically, {1 \over e}\approx{1\over 2.718}\approx 0.37.

Therefore, the probability of this not happening, namely that at least one coin of the 1000 coins will give 10 heads, is 1 minus that. This gives us the answer of approximately 0.63 or 63% that I mentioned in the lecture.

pventurelli@gotoibr.com 04-15-2012 05:51 PM

Re: question about probability
 
Thank you Professor, that was extremely helpful.

ivanku 04-18-2012 12:59 AM

Re: question about probability
 
Quote:

Originally Posted by yaser (Post 1288)
This is approximately {1 \over e} since \lim_{n\to\infty} (1-{1\over n})^n = {1\over e}. Numerically, {1 \over e}\approx{1\over 2.718}\approx 0.37.

Is it correct to assume that \nu_{min} from the coins tossing simulation in homework 2 assignment one should be close to that limit (where we're tossing 1000 coins 10 times and \nu_{min} is the fraction of heads obtained for the coin which had the minimum frequency of heads)?

elkka 04-18-2012 03:49 AM

Re: question about probability
 
The law of big numbers states that the average $\nu_min$ is close to the $E{\nu_min}$.

$E\nu_min$ can be calculated directly for this experiment.
$P(\nu_min=0)$=0.623576
$P(\nu_min = 0.1)$ = 0.3764034
$P(\nu_min = 0.2)$ = 0.00002;
and $P(\nu_min>=0.3)=0$ for the purposes of calculating the mean.

Therefore, $E(\nu_min)$=0.037644, and the average proportion of heads for c_min should be close to this number.

SamK52 04-18-2012 06:43 AM

Re: question about probability
 
Allow me to format your post:

Quote:

The law of big numbers states that the average \nu_{min} is close to the E{\nu_{min}}.

E\nu_{min} can be calculated directly for this experiment.
P(\nu_{min}=0)=0.623576
P(\nu_{min} = 0.1)= 0.3764034
P(\nu_{min} = 0.2) = 0.00002
and P(\nu_{min}>=0.3)=0 for the purposes of calculating the mean.

Therefore, E(\nu_{min})=0.037644, and the average proportion of heads for c_{min} should be close to this number.

elkka 04-18-2012 11:26 AM

Re: question about probability
 
Quote:

Originally Posted by SamK52 (Post 1399)
Allow me to format your post:

Please, allow me to ask how you did it?

yaser 04-18-2012 01:57 PM

Re: question about probability
 
Quote:

Originally Posted by elkka (Post 1413)
Please, allow me to ask how you did it?

http://book.caltech.edu/bookforum/sh...77&postcount=1

rohanag 04-18-2012 01:58 PM

Re: question about probability
 
how did you calculate those probability values? ( nu_min = 0, 0.1, 0.2 )

elkka 04-19-2012 04:10 AM

Re: question about probability
 
Thank you, Professor.

This how I calculate the probabilities. Let h_{min} = 10*\nu_{min} - the number of heads for c_{min}, and let h_i be the number of heads in i-th experiment (out of 1000). Then, as Professor has shown previously,

P(\nu_{min}=0) =P(h_{min}=0) = P(\exists i\in[1,1000]\,\, h_i = 0)=1-P(\forall i\in[1,1000]\, h_i > 0)
=1-P(h_1 > 0)^{1000};

Now, P(h_1>0) = 1-P(h_1 = 0) = 1-0.5^{10} . Therefore,
P(\nu_{min}=0)=1-(1-0.5^{10})^{1000} = 0.623576.

Next,
P(\nu_min=0.1)=P(h_{min}=1) =P(\forall i\in[1,1000] \,\,h_i > 0)-P(\forall i\in[1,1000] \,\,h_i > 1)
= P^{1000}(h_1>0)-P^{1000}(h_1>1) = (1-P(h_1=0))^{1000}-(1-P(h_1=0)-P(h_1=1))^{1000}
=(1-0.5^{10})^{1000}-(1-0.5^{10}-C_{10}^1*0.5^{10})^{1000} =0.3764034.

Next,
P(\nu_min=0.2)=P(h_{min}=2) =P(\forall i\in[1,1000]\,\, h_i > 1)-P(\forall i\in[1,1000] \,\,h_i > 2)
= P^{1000}(h_1>1)-P^{1000}(h_1>2)
= (1-P(h_1=0)-P(h_1=1))^{1000}-(1-P(h_1=0)-P(h_1=1)-P(h_1=2))^{1000}
=(1-0.5^{10}-C_{10}^1*0.5^{10})^{1000}-(1-0.5^{10}-C_{10}^1*0.5^{10}-C_{10}^2*0.5^{10})^{1000} =0.000204.

The rest can be calculated directly too, but they are essenctially 0 for the purpose of calculating the mean.

rohanag 04-19-2012 08:44 AM

Re: question about probability
 
thank you for the detailed explanation.

nyxee 07-13-2012 05:55 PM

Re: question about probability
 
the answer is very clear but how do we know when to use this not.

to clarify my question, if say P(ten heads)=p and P(not ten heads)=q (=1-p). why does using (p^1000) give the wrong answer?

htlin 07-15-2012 04:56 AM

Re: question about probability
 
Quote:

Originally Posted by nyxee (Post 3399)
the answer is very clear but how do we know when to use this not.

to clarify my question, if say P(ten heads)=p and P(not ten heads)=q (=1-p). why does using (p^1000) give the wrong answer?

p^{1000} means the probability of getting ten heads in each of the independent random trials. Is that the event you are interested in? ;)

coolguy 09-02-2012 08:05 AM

Re: question about probability
 
Quote:

Originally Posted by yaser (Post 1288)
The probability of getting 10 heads for one coin is {1 \over 2} \times {1 \over 2} \times \cdots \times {1 \over 2} (10 times) which is aprroximately {1 \over 1000}.

Therefore, the probability of not getting 10 heads for one coin is approximately (1-{1 \over 1000}).

This means that the probability of not getting 10 heads for any of 1000 coins is this number multiplied by itself 1000 times, once for every coin. This probability is therefore \approx (1-{1 \over 1000})^{1000}.

This is approximately {1 \over e} since \lim_{n\to\infty} (1-{1\over n})^n = {1\over e}. Numerically, {1 \over e}\approx{1\over 2.718}\approx 0.37.

Therefore, the probability of this not happening, namely that at least one coin of the 1000 coins will give 10 heads, is 1 minus that. This gives us the answer of approximately 0.63 or 63% that I mentioned in the lecture.

WOW!Thanks for the detailed explanation!It was over my head for a long time.

weehong 08-28-2013 03:32 AM

Re: question about probability
 
Quote:

Originally Posted by yaser (Post 1288)
The probability of getting 10 heads for one coin is {1 \over 2} \times {1 \over 2} \times \cdots \times {1 \over 2} (10 times) which is aprroximately {1 \over 1000}.

Therefore, the probability of not getting 10 heads for one coin is approximately (1-{1 \over 1000}).

This means that the probability of not getting 10 heads for any of 1000 coins is this number multiplied by itself 1000 times, once for every coin. This probability is therefore \approx (1-{1 \over 1000})^{1000}.

This is approximately {1 \over e} since \lim_{n\to\infty} (1-{1\over n})^n = {1\over e}. Numerically, {1 \over e}\approx{1\over 2.718}\approx 0.37.

Therefore, the probability of this not happening, namely that at least one coin of the 1000 coins will give 10 heads, is 1 minus that. This gives us the answer of approximately 0.63 or 63% that I mentioned in the lecture.

I captured the above idea, however I am confused why the following approach gives wrong answer:

Probability of all heads in 10 flips by one coin: p = 0.5^10
Probability of all heads in 10 flips by any of 1000 coin: 1000*p = 97.7%

In above I assumed there were 1000 10-flips.

yaser 08-30-2013 02:02 AM

Re: question about probability
 
Quote:

Originally Posted by weehong (Post 11459)
I captured the above idea, however I am confused why the following approach gives wrong answer:

Probability of all heads in 10 flips by one coin: p = 0.5^10
Probability of all heads in 10 flips by any of 1000 coin: 1000*p = 97.7%

In above I assumed there were 1000 10-flips.

In order to multiply the probabilities in the "all" case, you need the events to be independent, and we have that for the coin flips. In order to add the probabilities in the "any of" case, you need the events to be disjoint, i.e., they cannot simultaneously occur. The first coin giving 10 heads is not disjoint from the second coin giving 10 heads, so when you add their probabilities you are double counting the overlap which is "both coins giving 10 heads each."

weehong 09-01-2013 12:41 AM

Re: question about probability
 
Thank you very much, Professor. The explanation is spot on, very helpful.


All times are GMT -7. The time now is 01:21 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.