Q2 - fixed point depending on the initial guess

[jaroslawr]

I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?

[magdon]

Yes, you need to solve iteratively for using for example a fixed point iteration. I did not experience any problems with this iteration. In general, the fixed point iteration does not converge to a unique point but in this case you know that the epsilon you are looking for is small, positive, less than about 2 for typical N. You can also start your iteration carefully, for example at the value of epsilon obtained using one of the explicit bounds.

To answer your other general questions:

When do fixed point iterations converge: in general such fixed point iterations converge to a unique limit point when the function on the RHS is a contraction mapping.

Where do these implicit bounds come from: These implicit bounds arise from corresponding probabilistic bounds. So, for example, the error bar in (c) corresponds to the bound:



(This probabilistic bound was proved by Parrondo/Van den Brock.) A similar probabilistic bound is proved by Devroye which led to (d). Compared to the VC bound which had , the Parrondo bound has , which is worse. Where it gains is in how the exponent that depends on , a factor of 8 better than the VC bound. This is a big saving because it is in the exponent.

Originally Posted by jaroslawr

I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?

[jaroslawr]

Thanks for the great answer!