LFD Book Forum Question on VC dimension

#1
01-26-2013, 08:51 PM
 Suhas Patil Senior Member Join Date: Dec 2012 Posts: 57
Question on VC dimension

I am trying to understand this concept: "If VC dimension is finite, the final hypothesis will generalize". But somehow not able to relate VC dimension and learning. Can someone help.
#2
01-26-2013, 09:11 PM
 magdon RPI Join Date: Aug 2009 Location: Troy, NY, USA. Posts: 595
Re: Question on VC dimension

If , then with enough data, for every hypothesis in your learning model. This is why the final hypothesis will generalize, because it is one of the hypotheses in . The connection to learning comes when you pick the hypothesis with minimum Ein.

Because , by looking at Ein and picking the function with minimum Ein, that hypothesis you pick will also have minimum Eout, and so you will be able to learn (figure out) the best hypothesis in your model using the data.

Well, that is the first step in learning - can you pick/figure out/learn the best hypothesis available to you? That does not mean that that hypothesis is a good hypothesis, but it is the first step.

The second step is to ask whether this hypothesis that was "learned" is good enough. You will know whether the hypothesis is good or not by looking at its Ein (since it is close to Eout). If you had chosen a good then the answer to this second step will be yes. If your is bad then the answer will be no, and you will declare that you failed, but you know you failed.

Quote:
 Originally Posted by Suhas Patil I am trying to understand this concept: "If VC dimension is finite, the final hypothesis will generalize". But somehow not able to relate VC dimension and learning. Can someone help. Thank you for your attention.
__________________
Have faith in probability

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 08:35 PM.