LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 8

Reply
 
Thread Tools Display Modes
  #1  
Old 02-27-2013, 07:57 PM
Sendai Sendai is offline
Member
 
Join Date: Jan 2013
Location: Minnesota
Posts: 29
Lightbulb on the right track?

I thought it would be nice to have a way to check if we're on the right track with problems 2-5 without giving away the answers. I ran SVM (with the polynomial kernel) for a couple of cases and pasted the results below. Are others getting the same numbers?

0 vs 7 classifier, C=0.01, Q=2
number of support vectors = 861
E_{in} = 0.071778
E_{out} = 0.063241

2 vs 8 classifier, C=0.1, Q=3
number of support vectors = 721
E_{in} = 0.234878
E_{out} = 0.291209
Reply With Quote
  #2  
Old 02-28-2013, 01:00 AM
Anjoola Anjoola is offline
Junior Member
 
Join Date: Jan 2013
Posts: 3
Default Re: on the right track?

Hi, I got ALMOST the same numbers, except I got 860 support vectors for the first one instead of 861, and as a result my E_in is slightly different than yours. How did you choose your support vectors? Did you just check for \alpha > 0 or \ge or ?
Reply With Quote
  #3  
Old 02-28-2013, 12:25 PM
hemphill hemphill is offline
Member
 
Join Date: Jan 2013
Posts: 18
Default Re: on the right track?

I got exactly the same figures as the original poster. I'm using libsvm with the C programming language.
Reply With Quote
  #4  
Old 02-28-2013, 01:59 PM
Sendai Sendai is offline
Member
 
Join Date: Jan 2013
Location: Minnesota
Posts: 29
Default Re: on the right track?

Quote:
Originally Posted by Anjoola View Post
How did you choose your support vectors? Did you just check for \alpha > 0 or \ge or ?
I'm using libsvm via scikit-learn and Python, and it takes care of all of that for you.

For the previous week's homework, I looked for alpha greater than 10^{-5}.

Since we're all getting basically the same numbers, I have more confidence that I'm doing it right.
Reply With Quote
  #5  
Old 02-28-2013, 03:02 PM
ivankeller ivankeller is offline
Junior Member
 
Join Date: Jan 2013
Posts: 9
Default Re: on the right track?

Thanks Sendai, That was a good idea.
I'm using scikit-learn too, a pretty nice python module.

Your results helped me to figure out that I needed to set the parameters gamma and coef0 in sklearn.svm.SVC(...) to 1. These parameters don't appear in the lecture. Now I've got the same results.
Reply With Quote
  #6  
Old 03-01-2013, 04:54 PM
ilya239 ilya239 is offline
Senior Member
 
Join Date: Jul 2012
Posts: 58
Default Re: on the right track?

Quote:
Originally Posted by Sendai View Post
I thought it would be nice to have a way to check if we're on the right track with problems 2-5 without giving away the answers. I ran SVM (with the polynomial kernel) for a couple of cases and pasted the results below. Are others getting the same numbers?

0 vs 7 classifier, C=0.01, Q=2
number of support vectors = 861
E_{in} = 0.071778
E_{out} = 0.063241

2 vs 8 classifier, C=0.1, Q=3
number of support vectors = 721
E_{in} = 0.234878
E_{out} = 0.291209
Got similar numbers with python and cvxopt. As another check, got four margin support vectors for 0 vs 7, six for 2 vs 8.
Reply With Quote
  #7  
Old 03-02-2013, 05:02 PM
Sendai Sendai is offline
Member
 
Join Date: Jan 2013
Location: Minnesota
Posts: 29
Default Re: on the right track?

Quote:
Originally Posted by ilya239 View Post
Got similar numbers with python and cvxopt. As another check, got four margin support vectors for 0 vs 7, six for 2 vs 8.
I get three and five respectively using libsvm via Python and scikit-learn.
Reply With Quote
  #8  
Old 03-03-2013, 09:27 AM
Suhas Patil Suhas Patil is offline
Senior Member
 
Join Date: Dec 2012
Posts: 57
Default Re: on the right track?

I'm trying libsvm through C, with following parameters:

param.svm_type = C_SVC;
param.kernel_type = POLY;
param.degree = 2;
param.gamma = 1;
param.coef0 = 1;
param.nu = 0.5;
param.cache_size = 200;
param.C = 0.01;
param.eps = 1e-3;
param.p = 0.1;
param.shrinking = 1;
param.probability = 0;
param.nr_weight = 0;
param.weight_label = NULL;
param.weight = NULL;

but getting Ein as 0.350 with 0 versus 7 classification. Also unable to find good explaination of these parameters anywhere. Any help?

Thanks in advance.
Reply With Quote
  #9  
Old 03-03-2013, 10:23 AM
Suhas Patil Suhas Patil is offline
Senior Member
 
Join Date: Dec 2012
Posts: 57
Default Re: on the right track?

I found the issue...thanks for reply from buttterscotch. The problem was with the way I was initializing 'svm_node' structure after reading the training data.
Reply With Quote
  #10  
Old 03-03-2013, 10:25 AM
butterscotch butterscotch is offline
Caltech
 
Join Date: Jan 2013
Posts: 43
Default Re: on the right track?

Seems good to me. Are you getting the same number of support vectors with Sendai's post? You might want to verify how you calculate the error. The sv_coefficients are not just "alpha", but "y*alpha"
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 01:56 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.