LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 4 - Overfitting (http://book.caltech.edu/bookforum/forumdisplay.php?f=111)
-   -   Validaten and VC-Bound in practice (http://book.caltech.edu/bookforum/showthread.php?t=4713)

CountVonCount 11-10-2016 05:45 AM

Validaten and VC-Bound in practice

after reading the validation chapter of the book, I got some questions about the VC-bound.

I understand, that especially for cross-validation, the generalization theory is not completely applicable anymore. I think that's why your wrote the "model complexity penalty term" in all equations with the O(.) notation instead of using the exact bound from the second chapter (2.12).

However in a practical situation you must give an estimate about the E_out to the customer. And a constant factor like \sqrt8 would make a direct difference in the numbers given to the customer. With O(.) notation you lose those factors, that leads to a more optimistic estimation.

So in practice, when using the validation error to estimate Eout for the customer, do you simply apply the full formula for the VC-Bound or Hoeffding-Bound (in case of discrete models to select)?

Or does the validation error just gives an relative estimate and when you have to provide absolute numbers to customer you have to use a separated test-set and apply the simple Hoeffding-Bound to Etest?

Best regards

All times are GMT -7. The time now is 09:28 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.