Hoeffding inequality and noisy targets

[v_venky]

I found the jump from learning a deterministic target function to learning a probability distribution a big jump. The treatment of this concept in the book was a bit too fast for me and not detailed. Also the "intuitive" justification of hoeffding in this case also was not clear to me at all -- Hoeffding seems to be a tricky concept in the sense that it's application is prone to error if one is not careful. Is there a more step-by-step explanation of this section somewhere?

One starter question in this regard is that in the basic hoeffding derivation, we have used a binary classifier i.e. the target function returns +/-1 (or possibly a multi-class classifier). In the noisy target case should the understanding be that it returns a number 'p' signifying the probability of +1 at x?

[htlin]

This lecture

https://www.csie.ntu.edu.tw/~htlin/m...08_handout.pdf

contains some materials related to noisy targets, though not a lot.

The assumption here is that we still get a deterministic classifier (to predict a noisy target). There are other settings that allow us to get a probabilistic classifier, but those are more complicated to analyze and not discussed in detail here.

Hope this helps.