LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 5 - Three Learning Principles

Reply
 
Thread Tools Display Modes
  #1  
Old 12-15-2017, 08:04 AM
PCdimension PCdimension is offline
Junior Member
 
Join Date: Dec 2017
Posts: 1
Smile Paradox in VC dimension

Letís say two reserachers are locked up in 2 separate rooms provided with the same training set. One smart researcher (A) learnt about neural network and SVM, the other B only know about neural network. Letís say the ultimate truth is, neural network is the best model for this particular learning problem and both research A and B submitted the same neural network model.

B happen to have a smaller VC dimension than A as B has a smaller hypothesis test, but both end up choosing the same neural network model as a result.

It looks paradoxical that the less educated researcher B submitted a better model (less VC dimension and requires less number of sample).
______________
Another scenario is that a researcher C had developed a great algorithm to solve a particular learning problem. Later years, countless number of researchers had tried different models but all failed to improve the learning performance. Now the learning problem has grown its VC dimension over time as the total hypothesis space increase. Practically as time pass, the hypothesis will grow to infinity. These all sound paradoxical.

How can we charge for the VC dimension accordingly?
Reply With Quote
  #2  
Old 12-20-2017, 11:00 AM
Burine Burine is offline
Junior Member
 
Join Date: Dec 2017
Posts: 1
Default Re: Paradox in VC dimension

Quote:
Originally Posted by PCdimension View Post
...both research A and B submitted the same neural network model.

It looks paradoxical that the less educated researcher B submitted a better model (less VC dimension and requires less number of sample).
B's model cannot be better, because A and B used the same model.

If you mean B happened to choose a simpler network than A did (e.g. less layers), then since A is more educated clearly he would know how to put weights regularization, dropout,... to avoid overfitting.
______________
Quote:
Originally Posted by PCdimension View Post
Later years, countless number of researchers had tried different models but all failed to improve the learning performance.
On which ground could we assume that countless other models failed while C's model is sub-optimal?
Reply With Quote
Reply

Tags
paradox, vc dimension

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 12:11 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.