Re: Support Vector Machines
Yes, SVM is simply a useful general learning algorithm of the sort we have seen in the PLA algorithm, logistic regression and linear regression (and even neural networks), but one which has advantages for many problems. The general idea of the algorithm seems intuitively comprehensible; the details may be more technical than algorithms for which we have every detail, but I'm not too concerned.
With a classification problem and separable sets, SVM can be thought of as starting where PLA ends. It doesn't just separate data, it separates it in the best possible way, ensuring (quantitatively) better generalisation. But as well as that, a modified version (using regularization) can be very good at dealing with nonseparable classification problems (I believe because it focuses on the critical subset of the data, not the subset that is no problem).
Next week I believe we will learn about how SVMs retain these advantages for nonlinear classification (and regression) problems by the use of kernels.
Another plus of SVMs is global optimums. NNs have local minima, so aren't guaranteed to get the global best answer, but this doesn't seem to cause many problems and NNs are also a genuinely useful tool.
My limited experience tells me that crossvalidation proves very useful with SVMs and avoids the necessity for precise theoretical results about their ability to generalise.
