LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 4 (http://book.caltech.edu/bookforum/forumdisplay.php?f=133)
-   -   Q2 - fixed point depending on the initial guess (http://book.caltech.edu/bookforum/showthread.php?t=3964)

jaroslawr 02-06-2013 04:26 AM

Q2 - fixed point depending on the initial guess
 
I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?

magdon 02-06-2013 01:44 PM

Re: Q2 - fixed point depending on the initial guess
 
Yes, you need to solve iteratively for \epsilon using for example a fixed point iteration. I did not experience any problems with this iteration. In general, the fixed point iteration does not converge to a unique point but in this case you know that the epsilon you are looking for is small, positive, less than about 2 for typical N. You can also start your iteration carefully, for example at the value of epsilon obtained using one of the explicit bounds.

To answer your other general questions:

When do fixed point iterations converge: in general such fixed point iterations converge to a unique limit point when the function on the RHS is a contraction mapping.

Where do these implicit bounds come from: These implicit bounds arise from corresponding probabilistic bounds. So, for example, the error bar in (c) corresponds to the bound:

P[|Ein-Eout|>\epsilon]\le 6 m(2N)e^{2\epsilon}e^{-N\epsilon^2}

(This probabilistic bound was proved by Parrondo/Van den Brock.) A similar probabilistic bound is proved by Devroye which led to (d). Compared to the VC bound which had 4m(2N), the Parrondo bound has 6m(2N)e^{2\epsilon}, which is worse. Where it gains is in how the exponent that depends on N, a factor of 8 better than the VC bound. This is a big saving because it is in the exponent.

Quote:

Originally Posted by jaroslawr (Post 9222)
I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?


jaroslawr 02-08-2013 03:45 AM

Re: Q2 - fixed point depending on the initial guess
 
Thanks for the great answer!


All times are GMT -7. The time now is 05:28 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.