LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 4

Reply
 
Thread Tools Display Modes
  #1  
Old 02-06-2013, 05:26 AM
jaroslawr jaroslawr is offline
Junior Member
 
Join Date: Jan 2013
Posts: 2
Default Q2 - fixed point depending on the initial guess

I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?
Reply With Quote
  #2  
Old 02-06-2013, 02:44 PM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Q2 - fixed point depending on the initial guess

Yes, you need to solve iteratively for \epsilon using for example a fixed point iteration. I did not experience any problems with this iteration. In general, the fixed point iteration does not converge to a unique point but in this case you know that the epsilon you are looking for is small, positive, less than about 2 for typical N. You can also start your iteration carefully, for example at the value of epsilon obtained using one of the explicit bounds.

To answer your other general questions:

When do fixed point iterations converge: in general such fixed point iterations converge to a unique limit point when the function on the RHS is a contraction mapping.

Where do these implicit bounds come from: These implicit bounds arise from corresponding probabilistic bounds. So, for example, the error bar in (c) corresponds to the bound:

P[|Ein-Eout|>\epsilon]\le 6 m(2N)e^{2\epsilon}e^{-N\epsilon^2}

(This probabilistic bound was proved by Parrondo/Van den Brock.) A similar probabilistic bound is proved by Devroye which led to (d). Compared to the VC bound which had 4m(2N), the Parrondo bound has 6m(2N)e^{2\epsilon}, which is worse. Where it gains is in how the exponent that depends on N, a factor of 8 better than the VC bound. This is a big saving because it is in the exponent.

Quote:
Originally Posted by jaroslawr View Post
I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?
__________________
Have faith in probability
Reply With Quote
  #3  
Old 02-08-2013, 04:45 AM
jaroslawr jaroslawr is offline
Junior Member
 
Join Date: Jan 2013
Posts: 2
Default Re: Q2 - fixed point depending on the initial guess

Thanks for the great answer!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:32 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.