Quote:
Originally Posted by ilya239
The VC dimension is single number that is a property of the hypothesis set.
But, what is "bias of a hypothesis set"? Bias seems to depend also on dataset size and the learning algorithm, since it depends on ; depends on the learning algorithm, and the set of datasets over which the expectation is taken depends on dataset size.

Your observation is correct that the biasvariance analysis is not as general as the VC analysis. The bias does depend on the learning algorithm. It also depends on the number of examples, usually slightly.
Good questions
. What you are saying would hold if we were using the best approximation of
in
as the vehicle for measuring the bias. We are not. We are using a "limited resource" version of it that is based on averaging hypotheses that we get from training on a finite set of data points. This version is often close to the best approximation so that's why we can take that liberty.