View Single Post
Old 05-17-2013, 04:22 PM
Elroch Elroch is offline
Invited Guest
Join Date: Mar 2013
Posts: 143
Default Re: Support Vector Machines

Yes, SVM is simply a useful general learning algorithm of the sort we have seen in the PLA algorithm, logistic regression and linear regression (and even neural networks), but one which has advantages for many problems. The general idea of the algorithm seems intuitively comprehensible; the details may be more technical than algorithms for which we have every detail, but I'm not too concerned.

With a classification problem and separable sets, SVM can be thought of as starting where PLA ends. It doesn't just separate data, it separates it in the best possible way, ensuring (quantitatively) better generalisation. But as well as that, a modified version (using regularization) can be very good at dealing with non-separable classification problems (I believe because it focuses on the critical subset of the data, not the subset that is no problem).

Next week I believe we will learn about how SVMs retain these advantages for non-linear classification (and regression) problems by the use of kernels.

Another plus of SVMs is global optimums. NNs have local minima, so aren't guaranteed to get the global best answer, but this doesn't seem to cause many problems and NNs are also a genuinely useful tool.

My limited experience tells me that cross-validation proves very useful with SVMs and avoids the necessity for precise theoretical results about their ability to generalise.
Reply With Quote