Thread: Question 12
View Single Post
  #10  
Old 09-16-2012, 07:15 PM
fgpancorbo fgpancorbo is offline
Senior Member
 
Join Date: Jul 2012
Posts: 104
Default Re: Question 13

Quote:
Originally Posted by patrickjtierney View Post
This is the only question I got wrong on the final, and I would have got it right if I used my libsvm version of the answer rather than my hand-built version with qp (all in Octave). My qp (wrong!) answer was one less support vector than I got with libsvm and that might only be because I used 10e-012 as a threshhold. (If I had omitted the threshhold I would have gotten the same number of sv's as in libsvm ).

I got w = [-0.88889, 5.0e-016] and b = -1.6667 using qp, but strangely I get
w = [0.88869, 0] and b = 1.6663 using libsvm. They both have Ein=0 and on a thousand test runs of a million random points in [-3,3]^2 they agree on labels on average 99.999% of the cases. (For libsvm, I use svmpredict with all labels = +1 which is ~71% accurate to get the actual prediction labels.)

The difference in sign may not be significant. I got w and b for qp directly by following the class slides, but I got w = model.SVs'*model.sv_coef and b = - model.rho in the libsvm case (which may not be exactly correct).

The values of alpha (for qp) are different from model.sv_coef, and the qp version uses all but the last of the libsvm support vectors.

So I do agree that there may be 2 correct answers for this question, based on numerical issues and different ways qp and libsvm handle the calculations, but beyond the control of the student.

If required I can PM the alphas and the code I used to support the claim, or wait and post an **answer** after the deadline.
I haven't submitted my answers yet, but on this one I used libsvm; regarding how to get w and b, I found this on the website http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f804 ,

Code:
w = model.SVs' * model.sv_coef;
b = -model.rho;

if model.Label(1) == -1
  w = -w;
  b = -b;
end
The difference with what you did are the last 3 lines. Note that this would be good for problem 12 only. I get a different w though from what you get: w = [2 0] b = 1. svmpredict also gives me Ein =0. My options for 12 were '-s 0 -t 0 -q -h 0 -c 1e10'. For problem 13, I used libsvm as well and I get an answer that is amongst those suggested.
Reply With Quote