LFD Book Forum Q2 - fixed point depending on the initial guess

#1
02-06-2013, 04:26 AM
 jaroslawr Junior Member Join Date: Jan 2013 Posts: 2
Q2 - fixed point depending on the initial guess

I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?
#2
02-06-2013, 01:44 PM
 magdon RPI Join Date: Aug 2009 Location: Troy, NY, USA. Posts: 595
Re: Q2 - fixed point depending on the initial guess

Yes, you need to solve iteratively for using for example a fixed point iteration. I did not experience any problems with this iteration. In general, the fixed point iteration does not converge to a unique point but in this case you know that the epsilon you are looking for is small, positive, less than about 2 for typical N. You can also start your iteration carefully, for example at the value of epsilon obtained using one of the explicit bounds.

When do fixed point iterations converge: in general such fixed point iterations converge to a unique limit point when the function on the RHS is a contraction mapping.

Where do these implicit bounds come from: These implicit bounds arise from corresponding probabilistic bounds. So, for example, the error bar in (c) corresponds to the bound:

(This probabilistic bound was proved by Parrondo/Van den Brock.) A similar probabilistic bound is proved by Devroye which led to (d). Compared to the VC bound which had , the Parrondo bound has , which is worse. Where it gains is in how the exponent that depends on , a factor of 8 better than the VC bound. This is a big saving because it is in the exponent.

Quote:
 Originally Posted by jaroslawr I tried to answer Q2 by calculating the values of the bounds via an iterative method similar to the one used for sample complexity bounds on page 57 of the book. Was I doing this wrong, is it possible to find an explicit form of those inequalities? In case of the Devroye bound it seems that different values of the initial guess converge to different fixed points in case of an iterative method... Where can one learn more about those implicit inequalities e.g. why and when does this iterative approach work?
__________________
Have faith in probability
#3
02-08-2013, 03:45 AM
 jaroslawr Junior Member Join Date: Jan 2013 Posts: 2
Re: Q2 - fixed point depending on the initial guess

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 06:09 AM.