LFD Book Forum Doubt from lecture 2(Is learning feasible?)
 User Name Remember Me? Password
 Register FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
02-08-2013, 08:13 PM
 shirin Junior Member Join Date: Feb 2013 Posts: 2
Doubt from lecture 2(Is learning feasible?)

At 31 minutes mark professor has assumed that the input samples come from a probability distribution. My question is why do we make this assumption? Because throughout the lecture we haven't make use of this assumption anywhere.
#2
02-08-2013, 09:17 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Doubt from lecture 2(Is learning feasible?)

The assumption made it possible to invoke Hoeffding inequality. Without a probability distribution, one cannot talk about the probability of an event (the left-hand-side of the inequality). The specifics of the probability distribution don't matter here, any distribution will do.
__________________
Where everyone thinks alike, no one thinks very much
#3
02-09-2013, 09:49 PM
 shirin Junior Member Join Date: Feb 2013 Posts: 2
Re: Doubt from lecture 2(Is learning feasible?)

Can't I start talking about hypothesis analogy without making this assumption?

I mean if i say that a hypothesis is analogous to a bin and then I say that for any hypothesis there is a probability that that it will make a wrong classification in the bin and in the sample with probability \mu & \vu.

And then go ahead with hooeffding's inequality.

In doing so do I really need that assumption?
#4
02-09-2013, 10:01 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Doubt from lecture 2(Is learning feasible?)

Quote:
 Originally Posted by shirin Can't I start talking about hypothesis analogy without making this assumption? I mean if i say that a hypothesis is analogous to a bin and then I say that for any hypothesis there is a probability that that it will make a wrong classification in the bin and in the sample with probability \mu & \vu. And then go ahead with hooeffding's inequality. In doing so do I really need that assumption?
The introduction of a probability is not needed to make the analogy between a hypthesis and a bin, but it is needed to invoke Hoeffding inequality on the bin (and the hypothesis). Think of it this way. If I choose 3000 voters according to a deterministic criterion (say the richest 3000 people in the country) and poll them about who they are going to vote for, this sample will not indicate how the population as a whole will vote. If I introduce a probability distribution (say each voter in the population is as likely to be chosen for the poll as every other voter), then I can apply statistical results like Hoeffding to infer from a random sample of 3000 people how the population as a whole will vote.
__________________
Where everyone thinks alike, no one thinks very much

 Tags is learning feasible?

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 09:56 AM.

 Contact Us - LFD Book - Top