Quote:
Originally Posted by Willem
libsvm runs all problems in a few seconds or less, while cvxopt takes quite a bit longer. I wonder what's the reason for this, since using cvxopt is quite a basic routine and optimized for quadratic programming. The only reason I can think of is that for SVMs many solutions are equal to 0 and somehow this is built into the libsvm solver since it's targeted for SVMs. Does anybody know if this is the case or are there other tricks too? Thanks!

Designing efficient SVM solvers has been an important and nontrivial research problem after SVM has been proposed. Modern, specific purpose SVMsolvers carefully takes into account the special optimization and machine learning properties of the formulation. For instance, the "sparsity" (not too many support vectors) is one thing often used to shrink the optimization problem, the "simplicity" (only one equality constraint in the dual problem and other constraints are "box" constraints) is another, the fact that the kernel matrix can be computed online when needed instead of offline is yet another. Hope this helps.