LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 7

Reply
 
Thread Tools Display Modes
  #21  
Old 08-25-2012, 03:13 AM
invis invis is offline
Senior Member
 
Join Date: Jul 2012
Posts: 50
Default Re: Quadratic programming

Yes, works fine
Green line is final hypothesis, black is f
Reply With Quote
  #22  
Old 08-25-2012, 04:58 AM
rainbow rainbow is offline
Member
 
Join Date: Jul 2012
Posts: 41
Default Re: Quadratic programming

The returned \alpha_i. What threshold is suggested to identify them as parameters for support vectors?

The alphas are never equal to 0 in my QP solver.
Reply With Quote
  #23  
Old 08-25-2012, 12:51 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Quadratic programming

Quote:
Originally Posted by rainbow View Post
The returned \alpha_i. What threshold is suggested to identify them as parameters for support vectors?

The alphas are never equal to 0 in my QP solver.
Try to use other properties that a support vector should have.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #24  
Old 08-25-2012, 12:55 PM
zifmia zifmia is offline
Junior Member
 
Join Date: Jul 2012
Posts: 4
Default Re: Quadratic programming

Any numerical algorithms can't be expected to return results that are exactly zero. I used a threshold of 1e-5, but in the cases I looked at a threshold of even 0.1 would have been fine.
Reply With Quote
  #25  
Old 08-25-2012, 01:31 PM
invis invis is offline
Senior Member
 
Join Date: Jul 2012
Posts: 50
Default Re: Quadratic programming

Code:
alpha =

     0.00000
   105.09327
     0.00000
   115.37459
     0.00000
     0.00000
    10.28131
    -0.00000
    -0.00000
     0.00000
Result of QP Octave, parameters on previous page of this thread. To be sure I use threshold of 1e-5.
Reply With Quote
  #26  
Old 08-26-2012, 12:38 PM
rainbow rainbow is offline
Member
 
Join Date: Jul 2012
Posts: 41
Default Re: Quadratic programming

I use the QP solver in the R package kernlab. The solution (number of support vectors) is quite sensitive to the threshold (and why shouldn't it ).

The standard approach in professional applications seems to set it fairly small 1e-12. However, if I go for this choice I probably fail to answer question 10 since the numbers of alphas increase sharply.

I also ran into numerical issues when N = 100. I tried to distort the problem by adding a small diagonal term to the Q data matrix. That solved the numerical issues efficiently but that also changed the solution by returning many more alphas greater than "zero". Now I simply reiterate if the SVM run into numerical problems.

By other properties of SVs - is that related to distances to margin for example? In my "SVM world" almost everything is a result of the alphas.
Reply With Quote
  #27  
Old 08-26-2012, 01:15 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Quadratic programming

Quote:
Originally Posted by rainbow View Post
By other properties of SVs - is that related to distances to margin for example? In my "SVM world" almost everything is a result of the alphas.
If we have the exact values of \alpha's, then indeed that suffices. Looking for other properties is only meant to deal with numerical inaccuracies.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #28  
Old 02-22-2013, 07:54 PM
melipone melipone is offline
Senior Member
 
Join Date: Jan 2013
Posts: 72
Default Re: Quadratic programming

Quote:
Originally Posted by jakvas View Post
@invis

for the 100 points problem 400 iterations may be too little (i used 2000 iterations in my matlab code and in ~2% of the cases even that limit was exceeded but that still gives a decent accuracy you could use even more but don't go too far or you will never get results) also for the upper bound I used 10^5 and 10^10 without any significant change in results, 10^22 seems a bit much considering you are probably using single precision numbers
How do you specify the number of iterations in Octave's qp? I tried
options=optimset('MaxIter',1000);
[alphas]=qp([],H,Q,A,B.LB,UB,[],[],[],options) but got an error.

TIA
Reply With Quote
  #29  
Old 02-23-2013, 08:15 PM
Marc Zucker Marc Zucker is offline
Member
 
Join Date: Jan 2013
Posts: 24
Default Re: Quadratic programming

Any thoughts on R? I seem to be tweaking it and the program will not let me get my 1000 iterations. If I change the upper bound too low then either it gives me my iterations with a wrong answer (I submitted it and got the wrong answer), or it says that the matrix is not positive definite. It does not let me have an upper bound beyond about 1000. My threshold also has to be tights (about .01 or so, otherwise I get an answer but it is not correct). Is there a general way to deal with these issues when utilizing it practically? I have been using ipop in R.
Reply With Quote
Reply

Tags
quadratic programming

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 06:09 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.