Re: Hoeffding’s inequality
My point of view for this questions it's that we should take the Hoeffding’s inequality as a form of convergence guarantee. This is an obvious observation of course, but seeing things in this light lets you compare the averages to the true expected value and conclude that if some value is close enough (say with eps = 0.01), then it satisfy the inequality because the probability of something bad happening it's veeery close to zero.
On the subject of the coins experiments, it doesn't matter if the coin you select it's always different, in reality they should be indistinguishable from each other, they all are fair coins independently tossed.
