Quote:
Originally Posted by cabjoe
I have a question regarding degrees of freedom and their relation to the VC dimension
In Lecture 7, the professor states that the VC dimension of a hypothesis set is equal to the number of degrees of freedom and shows that this indeed holds for positive rays and positive intervals.
However in the case of a perceptron in R2, he has shown that the VC dimension is d+1, i.e. 3 but I can only see 2 degrees of freedom, the slope and intercept of the line.
|
Degrees of freedom are an abstraction of the effective number of parameters. The effective number is based on how many dichotomies one can get, rather than how many real-valued parameters are used. In the case of 2-dimensional perceptron, one can think of slope and intercept (plus a binary degree of freedom for which region goes to
), or one can think of 3 parameters
(though the weights can be simultaneously scaled up or down without affecting the resulting hypothesis). The degrees of freedom, however, are 3 because we have the flexibility to shatter 3 points, not because of one way or another of counting the number of parameters.