Quote:
Originally Posted by cygnids
The PLA algorithm, eqn. 1.3, can be used to partition linearly separable data. What I'm curious is to what optimization criteria underlies eqn. 1.3? The figures on pp. 67 show that for a 2D case we have the algorithm converge to some straight line decision boundary, and it is also qualitatively clear that many different straightlines, would "work" equally well (ie give the same E_{in} error rate); however PLA converges to a specific solution. The PLA algorithm seems to provide both, an optimization criteria, and a method for solution too. The opt. criteria gives provides uniqueness. Can the optimization criteria underlying PLA (eqn 1.3) be spelled out explicitly? Thank you.

The optimization criterion for the PLA can be viewed as an application of Stochastic Gradient Descent to a particular error measure (Exercise 3.10). This is really just an artificial way of looking at it. A genuine optimization criterion based on margins leads to support vector machines.