Quote:
Originally Posted by raghu
I did not grasp the reasoning behind the union bound for Hoffding's inequality.
If we have a bound for the error for a single hypothesis and we are able to calculate this, shouldnt this be the bound for any hypothesis?

Yes, for any
one hypothesis, but not necessarily of a bunch of them simultaneously. The probability that any particular person will suffer a car accident on a given day is very small, but the probability that
someone will suffer a car accident on that day is close to 100%. The bulk of Lecture 2 addresses this issue, so perhaps you can watch that part again (with the coins analogy).