LFD Book Forum *ANSWER* Homework 1 Question 3, 4
 User Name Remember Me? Password
 Register FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
06-09-2014, 05:53 PM
 jacox Junior Member Join Date: Jun 2014 Posts: 1
*ANSWER* Homework 1 Question 3, 4

I was a little confused at first by what Q3 was asking, so I thought I would describe my reasoning here, in case anybody else had the same question.

So what the question is saying in layman's terms is:

Quote:
 You have a hypothesis function, h(x) that you train on a target function, y=f(x), but it makes an error with probability mu (oh well, at least you tried ). Now, after finding h(x), you apply that to some noisy data set (real-world data). Now, the probability that h(x) makes an error in noiseless data is mu and the probability that it doesn't is 1-mu. In addition, the probability that you make an error, simply due to noise, is 1-lambda, with lambda probability that the noise produces no error. Therefore, since these are binary functions, the probability that you actually make an error when you apply h(x) to a noisy version of f(x) is: "the probability that there is an error due to noise (1-lambda), AND no error due to the "deterministic" error (1-u) OR the probability that there is no error due to noise (lambda) AND there is a "deterministic" error (mu). Note, the probability distributions for "mu" and "lambda" are statistically independent (this is the assumption). Therefore: P_{error} = P_{noise error}*P_{no mu error} + P_{no noise error}*P_{mu error} = (1-lambda)*(1-mu) + lambda*mu.
For question 4, you can see if you set lambda = 0.5, that P_{error} reduces to 1/2, and mu drops out. Intuitively, what you are saying if lambda = 0.5 is that your noise is so bad, that half the time, you are making errors. Well, in this situation, you don't expect mu to influence the outcome because your data is already uniformly random.

Please feel free to chime in with corrections and comments as necessary.
#2
06-10-2014, 05:53 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: *ANSWER* Homework 1 Question 3, 4

Thank you for the nice explanation.
__________________
Where everyone thinks alike, no one thinks very much
#3
10-18-2017, 02:20 PM
 hhprogram Junior Member Join Date: Oct 2017 Posts: 7
Re: *ANSWER* Homework 1 Question 3, 4

Thanks for the post. Help me understand as I was unclear on number 3. I think the key is 'h' approximating 'y'. I saw that and just read it as still 'h' approximating 'f' but reading it like that discounts the noise introduced to the reading into 'y'

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 09:48 AM.

 Contact Us - LFD Book - Top