View Single Post
Old 07-31-2013, 01:37 AM
hsolo hsolo is offline
Join Date: Jul 2013
Posts: 12
Default Re: How many support vectors is too many?

Many thanks for your reply, Prof. Lin.

Originally Posted by htlin View Post
For the CV bound, the number of SVs include the "free SVs" (with alpha between 0 and C and are on the margin), and the "bounded SVs" (with alpha = C and violating the margin).

It sounds reasonable.

Numerical optimization is a difficult problem so it is difficult to define the principled way. In specific packages like LIBSVM, some careful implementations is used to carefully and stably mark at-bound alphas. In general packages for convex programming, this may not be the case.

If the convex programming package is working reasonably well, the b should be close to each other for the margin SVs. So that can act as a safety check.

For the multiclass case, there are theoretical results that analyze the performance of one-versus-one and one-versus-rest. The results are not as simple as counting the number of internal models and it is often difficult to theoretically decide which one to be the better choice. Practically people often favor one-versus-one SVMs for computational efficiency during training, because each SVM is trained with significantly less data than the whole set.

Hope this helps.
Reply With Quote