Q13 - on using LIBSVM : C parameter?

[bargava]

I am trying to use LIBSVM to do this. And for using this, a value for C should be specified.

Should C be assigned an extremely large arbitrary value (say: 1e+10)? Would that be similar to implementing a hard margin SVM?

[sptripathi]

Originally Posted by bargava

I am trying to use LIBSVM to do this. And for using this, a value for C should be specified.

Should C be assigned an extremely large arbitrary value (say: 1e+10)? Would that be similar to implementing a hard margin SVM?

I used a high value for C, otherwise it regularizes. Observed that in libsvm case it regularized so much that E_in was never zero. So I suppressed regularization by choosing a high value of C.

From Prof's lecture, I recall that the high value of C ensures that slackness is penalized heavily, so as to achieve hard margin.

[marek]

Originally Posted by bargava

I am trying to use LIBSVM to do this. And for using this, a value for C should be specified.

Should C be assigned an extremely large arbitrary value (say: 1e+10)? Would that be similar to implementing a hard margin SVM?

I tried 10^3 first and then increased the exponent until the output stopped changing.

Edit: I seem to recall that 1000 was considered a good rule of thumb for a hard margin. But I found from practice that in this question it was still insufficient.

[Dorian]

Good catch, thanks! I had the same problem.

One might assume (wrongly as it turns out) that it does hard margin svm by default but of course it does soft margin by default as that is probably more useful in practice.

Also I guess, if you choose enough of a large C as you suggest, SVM should have no trouble to deal with such noiseless data, similar to the example we had in class.