LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 8

Reply
 
Thread Tools Display Modes
  #11  
Old 03-03-2013, 10:38 AM
Suhas Patil Suhas Patil is offline
Senior Member
 
Join Date: Dec 2012
Posts: 57
Default Re: on the right track?

I verified for 0 versus 7 case and I am getting exactly same number of support vectors. (and also Ein and Eout)
Haven't explored using sv coefficients for calculating the error. I am using this API:
double svm_predict(const struct svm_model *model, const struct svm_node *x); from 'svm.h'
that returns the predicted class value. I am calling this method in loop for all the test points and comparing against the ground truth (y array from svm_problem) to compute Ein or Eout.
Reply With Quote
  #12  
Old 03-04-2013, 10:25 AM
Anne Paulson Anne Paulson is offline
Senior Member
 
Join Date: Jan 2013
Location: Silicon Valley
Posts: 52
Default Re: on the right track?

Thanks, thanks, thanks!

Like Ivan Keller, I at first wasn't setting the gamma and coef0 parameters.

I know what gamma is for the radial kernel, but what does it mean for the polynomial kernel? And what is coef0? The bias? If so, why would the default be 0? Wouldn't you usually want an intercept?
Reply With Quote
  #13  
Old 03-04-2013, 10:33 AM
ilya239 ilya239 is offline
Senior Member
 
Join Date: Jul 2012
Posts: 58
Default Re: on the right track?

Quote:
Originally Posted by Anne Paulson View Post
I know what gamma is for the radial kernel, but what does it mean for the polynomial kernel? And what is coef0? The bias? If so, why would the default be 0? Wouldn't you usually want an intercept?
see http://scikit-learn.org/stable/modul...rnel-functions

also, from the python docs:
|
| kernel : string, optional (default='rbf')
| Specifies the kernel type to be used in the algorithm.
| It must be one of 'linear', 'poly', 'rbf', 'sigmoid', 'precomputed' or
| a callable.
| If none is given, 'rbf' will be used. If a callable is given it is
| used to precompute the kernel matrix.
|
| degree : int, optional (default=3)
| Degree of kernel function.
| It is significant only in 'poly' and 'sigmoid'.
|
| gamma : float, optional (default=0.0)
| Kernel coefficient for 'rbf' and 'poly'.
| If gamma is 0.0 then 1/n_features will be used instead.
|
| coef0 : float, optional (default=0.0)
| Independent term in kernel function.
| It is only significant in 'poly' and 'sigmoid'.
Reply With Quote
  #14  
Old 03-04-2013, 11:08 AM
kartikeya_t@yahoo.com kartikeya_t@yahoo.com is offline
Member
 
Join Date: Jul 2012
Posts: 17
Default Re: on the right track?

Quote:
Originally Posted by butterscotch View Post
Seems good to me. Are you getting the same number of support vectors with Sendai's post? You might want to verify how you calculate the error. The sv_coefficients are not just "alpha", but "y*alpha"
Thanks ButterScotch for pointing this out about the sv coefficients. I have been looking at getting the errors in the test data without relying on the svm-predict function. Once I run the training using svm-train, I take the resulting model file and extract the support vectors and their coefficients, taking care that the coefficients are "y*alpha".
If my understanding is correct, the support vectors are some points from the input data set (in particular, the points that are "supporting" the decision boundary.)
So I expect that the support vectors that are being reported in the model file should be found in the raw training data. But for some reason, I do not see that. None of the support vectors that the package calculates are in the raw data.
Am I missing something? How would one go about constructing the final hypothesis from the support vectors and coefficients that are reported in the model file?
Reply With Quote
  #15  
Old 03-04-2013, 11:20 AM
Anne Paulson Anne Paulson is offline
Senior Member
 
Join Date: Jan 2013
Location: Silicon Valley
Posts: 52
Default Re: on the right track?

Now I have a different problem (sorry to bug you all, thanks for your help). I'm getting the right (or at least, the same) results as the rest of you. But now I can't get answers to Q5 and Q6. I'm getting more than one statement being true, and numbers are not increasing/decreasing monotonically.

Suggestions? Hints?
Reply With Quote
  #16  
Old 03-04-2013, 11:24 AM
Anne Paulson Anne Paulson is offline
Senior Member
 
Join Date: Jan 2013
Location: Silicon Valley
Posts: 52
Default Re: on the right track?

Never mind: "goes down" is to be interpreted as "goes down monotonically".
Reply With Quote
  #17  
Old 03-04-2013, 11:46 AM
alternate alternate is offline
Member
 
Join Date: Jan 2013
Posts: 14
Default Re: on the right track?

As per another thread, when it says it goes up or goes down, it means it goes strictly, not monotonically.
Reply With Quote
  #18  
Old 03-04-2013, 11:59 AM
Anne Paulson Anne Paulson is offline
Senior Member
 
Join Date: Jan 2013
Location: Silicon Valley
Posts: 52
Default Re: on the right track?

Right. Strictly, that's what I meant to say.
Reply With Quote
  #19  
Old 03-05-2013, 07:07 AM
boulis boulis is offline
Member
 
Join Date: Feb 2013
Location: Sydney, Australia
Posts: 29
Default Re: on the right track?

Quote:
Originally Posted by Sendai View Post
I thought it would be nice to have a way to check if we're on the right track with problems 2-5 without giving away the answers. I ran SVM (with the polynomial kernel) for a couple of cases and pasted the results below. Are others getting the same numbers?

0 vs 7 classifier, C=0.01, Q=2
number of support vectors = 861
E_{in} = 0.071778
E_{out} = 0.063241

2 vs 8 classifier, C=0.1, Q=3
number of support vectors = 721
E_{in} = 0.234878
E_{out} = 0.291209
Very good idea. I am using LIBSVM with Python. I got the exact same results, with only slight difference that the numSV for second case was 722.

Code:
0 vs 7, Q=2, C=0.01 => Ein: 0.0717781402936 SV#: 861 Eout: 0.0632411067194
2 vs 8, Q=3, C=0.1 => Ein: 0.234878240377 SV#: 722 Eout: 0.291208791209
Reply With Quote
  #20  
Old 05-24-2013, 05:25 AM
alasdairj alasdairj is offline
Member
 
Join Date: Mar 2013
Posts: 12
Default Re: on the right track?

Quote:
Originally Posted by Suhas Patil View Post
I found the issue...thanks for reply from buttterscotch. The problem was with the way I was initializing 'svm_node' structure after reading the training data.
I too am getting Ein of 0.35 for 0-to-7 classification. What is this "sum_node" structure Suhas mentions?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 05:32 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.