LFD Book Forum Quadratic programming
 User Name Remember Me? Password
 FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#21
08-25-2012, 03:13 AM
 invis Senior Member Join Date: Jul 2012 Posts: 50

Yes, works fine
Green line is final hypothesis, black is
#22
08-25-2012, 04:58 AM
 rainbow Member Join Date: Jul 2012 Posts: 41

The returned . What threshold is suggested to identify them as parameters for support vectors?

The alphas are never equal to 0 in my QP solver.
#23
08-25-2012, 12:51 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477

Quote:
 Originally Posted by rainbow The returned . What threshold is suggested to identify them as parameters for support vectors? The alphas are never equal to 0 in my QP solver.
Try to use other properties that a support vector should have.
__________________
Where everyone thinks alike, no one thinks very much
#24
08-25-2012, 12:55 PM
 zifmia Junior Member Join Date: Jul 2012 Posts: 4

Any numerical algorithms can't be expected to return results that are exactly zero. I used a threshold of 1e-5, but in the cases I looked at a threshold of even 0.1 would have been fine.
#25
08-25-2012, 01:31 PM
 invis Senior Member Join Date: Jul 2012 Posts: 50

Code:
alpha =

0.00000
105.09327
0.00000
115.37459
0.00000
0.00000
10.28131
-0.00000
-0.00000
0.00000
Result of QP Octave, parameters on previous page of this thread. To be sure I use threshold of 1e-5.
#26
08-26-2012, 12:38 PM
 rainbow Member Join Date: Jul 2012 Posts: 41

I use the QP solver in the R package kernlab. The solution (number of support vectors) is quite sensitive to the threshold (and why shouldn't it ).

The standard approach in professional applications seems to set it fairly small 1e-12. However, if I go for this choice I probably fail to answer question 10 since the numbers of alphas increase sharply.

I also ran into numerical issues when N = 100. I tried to distort the problem by adding a small diagonal term to the Q data matrix. That solved the numerical issues efficiently but that also changed the solution by returning many more alphas greater than "zero". Now I simply reiterate if the SVM run into numerical problems.

By other properties of SVs - is that related to distances to margin for example? In my "SVM world" almost everything is a result of the alphas.
#27
08-26-2012, 01:15 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477

Quote:
 Originally Posted by rainbow By other properties of SVs - is that related to distances to margin for example? In my "SVM world" almost everything is a result of the alphas.
If we have the exact values of 's, then indeed that suffices. Looking for other properties is only meant to deal with numerical inaccuracies.
__________________
Where everyone thinks alike, no one thinks very much
#28
02-22-2013, 07:54 PM
 melipone Senior Member Join Date: Jan 2013 Posts: 72

Quote:
 Originally Posted by jakvas @invis for the 100 points problem 400 iterations may be too little (i used 2000 iterations in my matlab code and in ~2% of the cases even that limit was exceeded but that still gives a decent accuracy you could use even more but don't go too far or you will never get results) also for the upper bound I used 10^5 and 10^10 without any significant change in results, 10^22 seems a bit much considering you are probably using single precision numbers
How do you specify the number of iterations in Octave's qp? I tried
options=optimset('MaxIter',1000);
[alphas]=qp([],H,Q,A,B.LB,UB,[],[],[],options) but got an error.

TIA
#29
02-23-2013, 08:15 PM
 Marc Zucker Member Join Date: Jan 2013 Posts: 24

Any thoughts on R? I seem to be tweaking it and the program will not let me get my 1000 iterations. If I change the upper bound too low then either it gives me my iterations with a wrong answer (I submitted it and got the wrong answer), or it says that the matrix is not positive definite. It does not let me have an upper bound beyond about 1000. My threshold also has to be tights (about .01 or so, otherwise I get an answer but it is not correct). Is there a general way to deal with these issues when utilizing it practically? I have been using ipop in R.

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 11:53 AM.

 Contact Us - LFD Book - Top