Libsvm

[elkka]

The LIBSVM package is new to me, as I suppose it is for many of us. I also have very little experience with working with "black box" packages. I have difficulty trusting my results. I think it might be useful to have a thread where we could share our experience and help each other learn LIBSVM.

[elkka]

I currently struggle to understand how to use the build in cross-validation capability. I don't understand yet what exactly I don't understand, but I definitely don't understand something.

Specifically, when using 1-vs-1 classification on the digit set, I get some result for E_in error, that is close to E_out error. But whatever my parameters, the cross-validation accuracy on the problem gives me 99.8% accuracy, which is way higher than E_in or E_out. Any ideas?

	cva = svmtrain(Y, X, '-t 1 -d 2 -g 1 -r 1 -v 10  -c 0.01' );

[kkkkk]

I am running libsvm from the command line executables. The cross validation accuracy is shown in 4 decimal places for different values of C. I also used the command line tool to scale my input data, wonder if that mattered.

[alfansome]

Originally Posted by elkka

I currently struggle to understand how to use the build in cross-validation capability. I don't understand yet what exactly I don't understand, but I definitely don't understand something.

Specifically, when using 1-vs-1 classification on the digit set, I get some result for E_in error, that is close to E_out error. But whatever my parameters, the cross-validation accuracy on the problem gives me 99.8% accuracy, which is way higher than E_in or E_out. Any ideas?

	cva = svmtrain(Y, X, '-t 1 -d 2 -g 1 -r 1 -v 10  -c 0.01' );

I had this situation also; cv error is always the same. I also am saving the models that I generate (using sum_save_model function in the python script) but in looking at them, don't understand the values that show for the data points. Maybe scaling data would help as kkkkk did.

I have now applied data scaling (my own version) and this did result in discrete cva measures for the various C values, although cva does seem high still

[samirbajaj]

I'm new to libsvm (as I'm sure many other students), so my first question is how do I get E_in from svm_train?

I'm using the Java version.

Thanks for any pointers.

options:
-s svm_type : set type of SVM (default 0)
	0 -- C-SVC
	1 -- nu-SVC
	2 -- one-class SVM
	3 -- epsilon-SVR
	4 -- nu-SVR
-t kernel_type : set type of kernel function (default 2)
	0 -- linear: u'*v
	1 -- polynomial: (gamma*u'*v + coef0)^degree
	2 -- radial basis function: exp(-gamma*|u-v|^2)
	3 -- sigmoid: tanh(gamma*u'*v + coef0)
	4 -- precomputed kernel (kernel values in training_set_file)
-d degree : set degree in kernel function (default 3)
-g gamma : set gamma in kernel function (default 1/num_features)
-r coef0 : set coef0 in kernel function (default 0)
-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
-m cachesize : set cache memory size in MB (default 100)
-e epsilon : set tolerance of termination criterion (default 0.001)
-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)
-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)
-v n : n-fold cross validation mode
-q : quiet mode (no outputs)

[dvs79]

Originally Posted by elkka

I currently struggle to understand how to use the build in cross-validation capability. I don't understand yet what exactly I don't understand, but I definitely don't understand something.

Specifically, when using 1-vs-1 classification on the digit set, I get some result for E_in error, that is close to E_out error. But whatever my parameters, the cross-validation accuracy on the problem gives me 99.8% accuracy, which is way higher than E_in or E_out. Any ideas?

	cva = svmtrain(Y, X, '-t 1 -d 2 -g 1 -r 1 -v 10  -c 0.01' );

Hi,
did you mean that 0,2% (100-99,8) is higher than your E_in and E_out?
Because Accuracy = 100% - E, right?

[samirbajaj]

Maybe I should create a separate thread for this, but I'm still stuck at trying to figure out how to get libsvm to compute E_in.

Here is a typical output (I use the command-line Java tools):

optimization finished, #iter = NNNN
nu = XXXXXXX
obj = YYYYY, rho = ZZZZZ
nSV = A, nBSV = B
Total nSV = C

The alphas are not displayed. But I was really hoping that libsvm would compute E_in automatically (it has everything it needs to do so).

Is there a magical switch that I am missing?

Thanks.

-Samir

[apinde]

Tried to use libsvm in Windows as per the Youtube video. However when I try to run svm-train.exe on the training data a1a.train it comes back with the message "cannot open the input file".

This course was going great till HW6. After that, it has become a test of working with software packages and dealing with error messages.

[dvs79]

Originally Posted by samirbajaj

Maybe I should create a separate thread for this, but I'm still stuck at trying to figure out how to get libsvm to compute E_in.

Here is a typical output (I use the command-line Java tools):

optimization finished, #iter = NNNN
nu = XXXXXXX
obj = YYYYY, rho = ZZZZZ
nSV = A, nBSV = B
Total nSV = C

The alphas are not displayed. But I was really hoping that libsvm would compute E_in automatically (it has everything it needs to do so).

Is there a magical switch that I am missing?

Thanks.

-Samir

Samir,
you just have to set computed model to a variable:

MyModel = svmtrain(y,X,[options])

and then use this model to predict values for your test dataset:

[predicted_labels, accuracy] = svmpredict(y_test, X_test, MyModel)

To obtain E_in, use in svmpredict the same dataset that you have used for training.

PS I assume using Matlab/Octave, but for other interfaces procedures is the same, difference only in sintax, I believe.

[samirbajaj]

Thanks ... I'm using Java command-line version (hoping to get away without having to write any code in this homework).

I'll try to figure out how to do what you're suggesting in Java.

-Samir