
#1




Q8Q10, computing b, Ein nonzero
I am finding that when I use N=100 I get a nonzero Ein value for about 6% of the trials (60 out of 1000). On those trials I get one misclassified training point which is very close the the boundary. I think it is due to the calculation of b. I am seeing a range of b values depending on which support vector I pick to compute it. As a result, I started using the mean b value. However, I suspect that is causing the misclassification. I am using Matlab's quadprog with the interior point method as recommended here:
http://courses.csail.mit.edu/6.867/w...pquadprog.pdf It is converging reliably. I am adding a very small amount to the diagonal of H. What is the best method for computing b? how much variability is expected? 
#2




Re: Q8Q10, computing b, Ein nonzero
Quote:
__________________
Where everyone thinks alike, no one thinks very much 
#3




Re: Q8Q10, computing b, Ein nonzero
Thank you for confirming that. I will keep looking.

#4




Re: Q8Q10, computing b, Ein nonzero
I found that the problem was in adding that small amount to the diagonal of H. Apparently, in Matlab quadprog() with Algorithm='interiorpointconvex' it is unnecessary and in fact, harmful.

#5




Re: Q8Q10, computing b, Ein nonzero
I'm having a similar issue and am using python cvxopt. First off the alphas are never identically 0. They are extremely close, but always just a little bit off. Is it supposed to give identically 0, or is what I'm experiencing typical?
As a result, in the calculation of w, I ignored alpha's that were not at least 10^5 away from 0. And then depending on which remaining SV I select, my b values differ very slightly. As a result, I imagine this is what is causing a small fraction of cases to have nonzero Ein (however in my case, its very rare. Only 2 out of 1000). I am quite unhappy with how things are turning out, as none of this feels right. But unfortunately I have no idea how to fix it  this seems to be internal to this QP solver. 
#6




Re: Q8Q10, computing b, Ein nonzero
Quote:
i.e. using all the alpha's, not just those with alpha > 1e5 to calculate w. I also seem to remember a thread which I can't find right now which suggests changing the tolerance parameters of cvxopt. This page has the relevant options: http://abel.ee.ucla.edu/cvxopt/userg...thmparameters 
Thread Tools  
Display Modes  

