LFD Book Forum In-sample error and Max Likelihood
 User Name Remember Me? Password
 Register FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
09-26-2013, 03:08 AM
 hsolo Member Join Date: Jul 2013 Posts: 12
In-sample error and Max Likelihood

The learning bounds we learnt in the course relate the generalization error of h, where h is the best hypothesis in terms of in-sample error, to the VC dimension.

In practice very often the best hypothesis h' is computed/estimated using Max Likelihood.

Is there a connection between the in-sample error of h' (the max likelihood hypothesis) and the minimum in-sample error possible?
#2
09-27-2013, 11:57 AM
 magdon RPI Join Date: Aug 2009 Location: Troy, NY, USA. Posts: 596
Re: In-sample error and Max Likelihood

For classificatiion, the maximum likelihood error is not very easy to define for a hypothesis that returns . However you can define a maximum likelihood error for logistic regression. You can compute the weights which maximize the likelihood in logistic regression and indeed these weights will be quite good in-sample for the classification problem where the in-sample error is the number of mistakes. Unfortunately, however, there is no known relationship between the number of mistakes made by the logistic regression solution and the minimum number of mistakes possible.

In general, for example using the linear perceptron, there is no know algorithm to efficiently compute the hypothesis which makes the minimum number of mistakes in-sample.

Quote:
 Originally Posted by hsolo The learning bounds we learnt in the course relate the generalization error of h, where h is the best hypothesis in terms of in-sample error, to the VC dimension. In practice very often the best hypothesis h' is computed/estimated using Max Likelihood. Is there a connection between the in-sample error of h' (the max likelihood hypothesis) and the minimum in-sample error possible?
__________________
Have faith in probability
#3
10-05-2013, 12:13 AM
 hsolo Member Join Date: Jul 2013 Posts: 12
Re: In-sample error and Max Likelihood

Thank you, Prof Magdon.
#4
03-07-2018, 01:35 PM
 mostafa3030 Junior Member Join Date: Mar 2018 Posts: 6
Re: In-sample error and Max Likelihood

Thank you, Prof Magdon.
دستگاه فایبر

 Tags in-sample, likelihood, mle

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 12:18 PM.

 Contact Us - LFD Book - Top

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.