LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 4 - Overfitting

Reply
 
Thread Tools Display Modes
  #1  
Old 11-10-2016, 04:45 AM
CountVonCount CountVonCount is offline
Member
 
Join Date: Oct 2016
Posts: 17
Default Validaten and VC-Bound in practice

Hello,

after reading the validation chapter of the book, I got some questions about the VC-bound.

I understand, that especially for cross-validation, the generalization theory is not completely applicable anymore. I think that's why your wrote the "model complexity penalty term" in all equations with the O(.) notation instead of using the exact bound from the second chapter (2.12).

However in a practical situation you must give an estimate about the E_out to the customer. And a constant factor like \sqrt8 would make a direct difference in the numbers given to the customer. With O(.) notation you lose those factors, that leads to a more optimistic estimation.

So in practice, when using the validation error to estimate Eout for the customer, do you simply apply the full formula for the VC-Bound or Hoeffding-Bound (in case of discrete models to select)?

Or does the validation error just gives an relative estimate and when you have to provide absolute numbers to customer you have to use a separated test-set and apply the simple Hoeffding-Bound to Etest?

Best regards
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 09:25 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.