LFD Book Forum Q19
 Register FAQ Calendar Mark Forums Read

#1
09-14-2012, 03:29 PM
 DeanS Member Join Date: Jul 2012 Posts: 16
Q19

I was wondering if the problem assumes that some learning has been done to determine P(D|h=f) for some population or if the person with the heart attack is the only person in D. Obviously, I may not understand Bayesian analysis.
#2
09-14-2012, 04:37 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Q20

Quote:
 Originally Posted by DeanS I was wondering if the problem assumes that some learning has been done to determine P(D|h=f) for some population or if the person with the heart attack is the only person in D. Obviously, I may not understand Bayesian analysis.
The set is the set of available data points, so in this case it is that one person with a heart attack. This problem will help you understand the Bayesian reasoning better.
__________________
Where everyone thinks alike, no one thinks very much
#3
09-15-2012, 08:17 AM
 DeanS Member Join Date: Jul 2012 Posts: 16
Re: Q20

Thank you very much for the quick reply. This has been an amazing course!!
#4
09-16-2012, 09:56 PM
 fgpancorbo Senior Member Join Date: Jul 2012 Posts: 104
Re: Q20

I am still a bit confused about the setup of the problem. Is it correct to assume that what we are trying to determine is the underlying probability of somebody picked at random from the population to have a heart attack out of a single sample? If so, shouldn't be relevant? If a single point is all we have, call it the binary variable - equal to 1 if the patient has a heart attach; 0 if he doesn't-, that would be the probability of generating a single point with a patient having a heart attack, given the underlying probability that a person has a heart attack, right? In that case, the posterior is going to have two cases and . The question refers only to case right?
#5
09-16-2012, 10:12 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Q20

Quote:
 Originally Posted by fgpancorbo Is it correct to assume that what we are trying to determine is the underlying probability of somebody picked at random from the population to have a heart attack out of a single sample?

It should be based on rater than out of. A source of confusion here is that is a probability, but then we have a probability distribution over . Let us just call the fraction of heart attacks in the population. Then the problem is addressing the probability distribution of that fraction - Is the fraction more likely to be 0.1 or 0.5 or 0.9 etc. The prior is that that fraction is equally likely to be anything (uniform probability). The problem then asks how this probability is modified if we get a sample of a single patient and they happen to have a heart attack.

__________________
Where everyone thinks alike, no one thinks very much
#6
09-16-2012, 10:44 PM
 fgpancorbo Senior Member Join Date: Jul 2012 Posts: 104
Re: Q20

Quote:
 Originally Posted by yaser (emphasis added) It should be based on rater than out of. A source of confusion here is that is a probability, but then we have a probability distribution over . Let us just call the fraction of heart attacks in the population. Then the problem is addressing the probability distribution of that fraction - Is the fraction more likely to be 0.1 or 0.5 or 0.9 etc. The prior is that that fraction is equally likely to be anything (uniform probability). The problem then asks how this probability is modified if we get a sample of a single patient and they happen to have a heart attack.
I see. If my understanding is correct, I think that I can safely assume that , in which is made of a single random variable say , has a Bernoulli distribution with parameter . Is that right?
#7
09-16-2012, 10:59 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Q20

Quote:
 Originally Posted by fgpancorbo I see. If my understanding is correct, I think that I can safely assume that , in which is made of a single random variable say , has a Bernoulli distribution with parameter . Is that right?
Right. In terms of , that would be parameter .
__________________
Where everyone thinks alike, no one thinks very much
#8
03-19-2013, 08:04 AM
 ilya19 Junior Member Join Date: Feb 2013 Location: Montreal, Canada Posts: 7
Re: Q20

If I understand the problem correctly, P(X=1) is independent on P(h=f). Correct?
#9
03-19-2013, 09:36 AM
 Haowen Member Join Date: Jan 2013 Posts: 24
Re: Q20

Quote:
 Originally Posted by ilya19 If I understand the problem correctly, P(X=1) is independent on P(h=f). Correct?
P(X=1) is defined over the full joint distribution, i.e. . The h is marginalized out by the summation. However it doesn't mean that X and h are independent.

The reason why you can ignore P(X=1) is because in Bayesian analysis you usually don't care about the absolute probability of the dataset since it is just a constant that all of your hypotheses are divided by, equally, so it doesn't affect which hypothesis is a-posteriori most probable.
#10
03-19-2013, 04:09 PM
 boulis Member Join Date: Feb 2013 Location: Sydney, Australia Posts: 29
Re: Q20

These are some fine points here. We have to use with exact meaning of terms, using them loosely can create misunderstandings.
At a first reading I thought that Haowen's answer was not correct, and also ilya's remark was not correct too. On a second reading, Haowen's answer is correct, but I am not sure that it answers the initial question, since the initial question/remark by ilya was ambiguous. Let me explain.

When we talk about independency we can talk about it in probability terms, where we have specific rules on random variables being independent, or we can talk about it in more loose/everyday terms when we want to express that something affects something else.

Ilya's question is expressed loosely. It talks about independence of Probabilities not random events. It can be taken with several different meanings.
1) If you are really asking whether X=1 and h=f are independent events, then we can clearly say they are not. The choice of h clearly affects the probability of X=1. More specifically, the choice of h is the probability of X=1.
2) If you are asking whether the distribution of h=f affects P(X=1) for all possible h, (which can be taken as the more literal interpretation of what you are asking) then again: yes there is a connection and Haowen gives you the formula.
3) If you are asking in general "should we care about calculating the value for P(X=1)", then Haowen gives you the answer again.
4) If you are asking whether the event X=1 affects the probability of h=f, it depends whether you are really referring to the a-priori or the a-posteriori. It does not affect the apriori and it does affect the a-posteriori (and Q20 asks how).

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 07:28 AM.