Thread: SVMs versus NNs
View Single Post
Old 09-02-2012, 04:13 PM
htlin's Avatar
htlin htlin is offline
Join Date: Aug 2009
Location: Taipei, Taiwan
Posts: 601
Default Re: SVMs versus NNs

Originally Posted by Willem View Post
Hi, are there any guidelines to decide when to use a neural network in stead of a SVM? I guess neural networks are better for regression problems, but how about classification? I read somewhere on this forum that logistic regression can sometimes perform better than SVMs when the underlying problem is intrinsically probabilistic. Are there similar rules of thumb for when using neural networks? Or do SVMs generally perform better (or similar)? It would be nice to have some guidelines of when to try which model first (and why). Thanks!
In my experience, most of the well-tuned modern machine learning algorithms reach similar performance. The difference is how hard it is to make those algorithms "well-tuned" (without overfitting). I am more familiar with SVMs in my own experience, and I can tune them more easily than NNets --- but Yaser may tell you the other way around because of you-know-what reason. So in terms of performance, it is hard to give a guideline.

There are other issues, though. SVMs are notably harder to scale up to large data sets (except for the linear case); NNets can serve the need of "feature extraction" through the neurons (while the feature transformation of SVMs are just too implicit in the dual problem). Those issues often affect my own choice of algorithms.

And yes, SVMs are strong with classification but not so strong with regression. For regression, SVMs usually are not my first-hand choice.

Hope this helps.
When one teaches, two learn.
Reply With Quote