It seems a proof requires very strong probability background. But now I know at least that
![[ y_n \neq h(\mathbf{x}_n) ] [ y_n \neq h(\mathbf{x}_n) ]](/vblatex/img/bfbe1925ebcdfbc67d777936c40d677d-1.gif)
to be "fixed" before starting the proof. I will defer the proof until I will reach the required math level, and will refresh my probability knowledge reserving an eye on the guidelines you send. Before I write this reply, I've reviewed some probability books that I've possess. I Could find the Markov's inequality, and Chernoff bound in some of them. I could not find Hoeffding's bound in any of them. Thus, I especially interested if you have a probability book (tailored for ML

) you can recommend. The main difficulty in ML is the level of required math and I think it is never enough.