#11




Re: Quadratic programming
as for the matlab this worked for me
UB(1:N,1)=10^5; options = optimset('LargeScale','off','MaxIter',2000); alpha=quadprog(Q,f,[],[],Class,0,zeros(N,1),UB,[],options); oh and for clarification Class is the +1/1 classification of the points and the empty matricies are unused restrictions I hope this helps a little and isn't frowned upon from the collaboration point of view after all you know what you are doing;P 
#12




Re: Quadratic programming
Can one use Solver in Excel for this problem? I've not used the other languages and platforms for quadratic programming packages and am desperately searching for a package that can be learned and used in a few days.

#13




Re: Quadratic programming
This was discussed extensively in the previous class. A user elkka found a solution to the nonterminating problem for Octave. The trick is to run qp on an altered version of H (using all the same other parameters) and then to use the output as the initial value alpha0 for qp on the original H. The second run seems to always terminate after one iteration.
The altered H is just to add a small amount to the diagonal of H (say 10^15). This makes its determinant nonzero, which is helpful for qp. So HH=H+eye(n)*10^15 should work. [ n is just the length of Y ] Also, the alpha0 I refer to goes first in the arg list for the second qp call. The results are much better, both in terms of termination and in producing a noticeably higher prediction accuracy (ie 10+% rise in better than PLA results). I posted about this here: http://book.caltech.edu/bookforum/showthread.php?t=1133 , but also there is the original thread: http://book.caltech.edu/bookforum/sh...p?t=513&page=5 
#14




Re: Quadratic programming
Wow !
I try it for n=10 on 400 iterations and have ZERO situations where QP cant solve ! Next I try to do the same iterations without altered version of H and have 164 problems without solving... Thanks very much ! 
#15




Re: Quadratic programming
I know. It's like magic, isn't it? And it appears to be a legitimate workaround for a problem with gp() on our problem.
One warning. I don't know if the answers we get are acceptable as far as the assignment goes (ie too good) or if we are simply fixing a problem in Octave. YMMV. I noticed that on both N=10 and N=100 that the average Eout for SVM drops by about 20% after using this technique. 
#16




Re: Quadratic programming
ML entirely is like magic too but its just a mathematics

#17




Re: Quadratic programming
I'm a matlab user and adding the diagonal terms results in no significant change for the results so it's probably something with the implementation of qp in Octave.
@patrickjtierney A change in Eout this big should be noticible when plotting the results (as in plotting the target function the PLA result and the SVM result with the training data set you should see that something is wrong?). @All Watch lecture 15 the part from 1:07:35 1:08:00 just to make us feel better 
#18




Re: Quadratic programming
Hmm, my SV looks pretty strange
Code:
options = optimset("MaxIter", 400); H = (Y*Y') .* (X*X'); A = Y'; q=1*ones(n,1); b=0; lb=zeros(n,1); ub=10^10*ones(n,1); alpha0 = qp ([], H+(eye(n)*10^15), q, A, b, lb, ub, options); alpha = qp (alpha0, H, q, A, b, lb, ub, options); Ok, look at this vectors (small black line is my ): Some of them pretty good, I place here only strange. Any ideas about behavior of this support vectors ? Or it is a part of homework ? Because I check with n=100 and vectors looks better, but some time some of them still strange 
#19




Re: Quadratic programming
I don't see anything odd with your support vectors. The solution may not be a great fit to your target f, but SVM doesn't see your f, it only sees 10 samples of f, and in all the cases you show, it looks like the support vectors could do a pretty good job of separating the given training points.
Try drawing the actual separating line found by SVM (from the w vector and b), then draw the parallels to this line through your support vectors to see the margin zone. I think you will find that SVM does a pretty good job of separating your points. Last edited by zifmia; 08242012 at 05:54 PM. Reason: correction 
#20




Re: Quadratic programming
@invis
The support vectors look fine to me. Could you draw the final hypothesis with the target function and the SVs? Then you should see more or less if the SVs support the final hypothesis like they should 
Tags 
quadratic programming 
Thread Tools  
Display Modes  

