Quadratic programming

[invis]

Yes, works fine
Green line is final hypothesis, black is

[rainbow]

The returned . What threshold is suggested to identify them as parameters for support vectors?

The alphas are never equal to 0 in my QP solver.

[yaser]

Originally Posted by rainbow

The returned . What threshold is suggested to identify them as parameters for support vectors?

The alphas are never equal to 0 in my QP solver.

Try to use other properties that a support vector should have.

[zifmia]

Any numerical algorithms can't be expected to return results that are exactly zero. I used a threshold of 1e-5, but in the cases I looked at a threshold of even 0.1 would have been fine.

[invis]

alpha =

     0.00000
   105.09327
     0.00000
   115.37459
     0.00000
     0.00000
    10.28131
    -0.00000
    -0.00000
     0.00000

Result of QP Octave, parameters on previous page of this thread. To be sure I use threshold of 1e-5.

[rainbow]

I use the QP solver in the R package kernlab. The solution (number of support vectors) is quite sensitive to the threshold (and why shouldn't it ).

The standard approach in professional applications seems to set it fairly small 1e-12. However, if I go for this choice I probably fail to answer question 10 since the numbers of alphas increase sharply.

I also ran into numerical issues when N = 100. I tried to distort the problem by adding a small diagonal term to the Q data matrix. That solved the numerical issues efficiently but that also changed the solution by returning many more alphas greater than "zero". Now I simply reiterate if the SVM run into numerical problems.

By other properties of SVs - is that related to distances to margin for example? In my "SVM world" almost everything is a result of the alphas.

[yaser]

Originally Posted by rainbow

By other properties of SVs - is that related to distances to margin for example? In my "SVM world" almost everything is a result of the alphas.

If we have the exact values of 's, then indeed that suffices. Looking for other properties is only meant to deal with numerical inaccuracies.

[melipone]

Originally Posted by jakvas

@invis

for the 100 points problem 400 iterations may be too little (i used 2000 iterations in my matlab code and in ~2% of the cases even that limit was exceeded but that still gives a decent accuracy you could use even more but don't go too far or you will never get results) also for the upper bound I used 10^5 and 10^10 without any significant change in results, 10^22 seems a bit much considering you are probably using single precision numbers

How do you specify the number of iterations in Octave's qp? I tried
options=optimset('MaxIter',1000);
[alphas]=qp([],H,Q,A,B.LB,UB,[],[],[],options) but got an error.

TIA

[Marc Zucker]

Any thoughts on R? I seem to be tweaking it and the program will not let me get my 1000 iterations. If I change the upper bound too low then either it gives me my iterations with a wrong answer (I submitted it and got the wrong answer), or it says that the matrix is not positive definite. It does not let me have an upper bound beyond about 1000. My threshold also has to be tights (about .01 or so, otherwise I get an answer but it is not correct). Is there a general way to deal with these issues when utilizing it practically? I have been using ipop in R.