Hoeffding's inequality with M
Hi,
I'm wondering why in Lecture 2  when we're trying to modify the right hand side of Hoeffding's inequality to take into account drawing from multiple bins (see page 16 of the slides)  we sum all the probabilities of samples being nonrepresentative, rather than taking 1  the product of all the probabilities of each sample being representative. In other words, (calling the right hand side of the original inequality p for simplicity) why doesn't the right hand side of the modified form become 1(1p)^M instead of M*p. Wouldn't that be a still correct but tighter bound? Thanks! 
Re: Hoeffding's inequality with M
Quote:

Re: Hoeffding's inequality with M
Oh, I see, that makes sense. The union bound just feels very loose to me, so I was hoping for a tighter one. But it sounds like maybe that's the best we can do without any additional assumptions about the hypothesis set  is that correct?
Also looking forward to finding out how infinite hypothesis sets are dealt with in future lectures... :) 
Re: Hoeffding's inequality with M
Quote:

All times are GMT 7. The time now is 09:03 PM. 
Powered by vBulletin® Version 3.8.3
Copyright ©2000  2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. AbuMostafa, Malik MagdonIsmail, and HsuanTien Lin, and participants in the Learning From Data MOOC by Yaser S. AbuMostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.