View Single Post
#3
04-23-2013, 09:48 AM
 Elroch Invited Guest Join Date: Mar 2013 Posts: 143
Re: d-dimensional Perceptrons and break points (related to Q4 of homework)

Quote:
 Originally Posted by jlaurentum Hello: In slide 9 of lecture 5 (minute 33:03), the Professor gives an example of 3 colinear points for which there can be no possible hypothesis. Still, "it doesn't bother us because we want the maximum bound of possible dichotomies", so k=3 is not considered as a breakpoint. My question is: In a d-dimensional perceptron, it appears we would not consider a set of points lying in a (d-1)-dimensional hyperplane as candidates for giving an "impossible" dichotomy. Why? Is it because the probability of picking such a set of points from the input space that all lie in a (d-1) dimensional space is zero? (As in the case of picking 3 collinear points in a plane).
It's worth observing that the set of -dimensional perceptrons, restricted to a -dimensional subspace, is simply , the set of -dimensional perceptrons on that subspace. hence, the capabilities of restricted to the subspace is the same as that of .

It turns out that the power of the hypothesis set comprising perceptrons increases as the dimension of their domain increases. The three points are a good example. If co-linear, they cannot be shattered, regardless of what dimension space they are in. If not co-linear, they can always be shattered: this requires the domain to be at least -dimensional.