LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 8

Reply
 
Thread Tools Display Modes
  #21  
Old 05-26-2013, 01:52 PM
marek marek is offline
Member
 
Join Date: Apr 2013
Posts: 31
Default Re: on the right track?

Quote:
Originally Posted by Anne Paulson View Post
Now I have a different problem (sorry to bug you all, thanks for your help). I'm getting the right (or at least, the same) results as the rest of you. But now I can't get answers to Q5 and Q6. I'm getting more than one statement being true, and numbers are not increasing/decreasing monotonically.

Suggestions? Hints?
I managed to reproduce the results earlier in this thread so I also had some confidence until I hit Q5 and Q6.

As I try to figure out what's going wrong, I guess I have one initial question. What are we supposed to do with -h, should we leave it at 1 as by default? -h 0 has no impact on the earlier questions but dramatically changes my answers for Q5 and Q6... and also takes incredibly long to compute.

Also regardless of which setting I choose, I always get the warning for hitting the max number of iterations... Any clues as to why that is or how I can prevent that?

Edit: Nevermind, after hours of trying to figure it out, minutes after I make a post I discover I had fat fingered -d 22 instead of -d 2. However, I am still curious as to what the effect of -h is if anyone knows.
Reply With Quote
  #22  
Old 05-26-2013, 05:29 PM
dlammerts dlammerts is offline
Junior Member
 
Join Date: Apr 2013
Location: San Mateo, CA
Posts: 6
Default Re: on the right track?

Quote:
Originally Posted by Sendai View Post
I thought it would be nice to have a way to check if we're on the right track with problems 2-5 without giving away the answers. I ran SVM (with the polynomial kernel) for a couple of cases and pasted the results below. Are others getting the same numbers?

0 vs 7 classifier, C=0.01, Q=2
number of support vectors = 861
E_{in} = 0.071778
E_{out} = 0.063241

2 vs 8 classifier, C=0.1, Q=3
number of support vectors = 721
E_{in} = 0.234878
E_{out} = 0.291209
Great idea. Got (almost) identical results using svm from the e1071 R plugin:

SVM model for 0-vs-7 classification with C = 0.01 and Q = 2: SVs = 861 Ein = 0.07177814 Eout = 0.06324111
SVM model for 2-vs-8 classification with C = 0.1 and Q = 3: SVs = 722 Ein = 0.2348782 Eout = 0.2912088
Reply With Quote
  #23  
Old 05-27-2013, 09:17 PM
mvellon mvellon is offline
Junior Member
 
Join Date: Apr 2013
Posts: 9
Default Re: on the right track?

I'm stuck half-way in this problem. I'm trying to use the C# version of libsvm and, I think it's working, but I can't corroborate the numbers I'm seeing here. Actually, I match on the # of support vectors, but my Ein and Eout numbers are significantly different.

For 0 vs. 7 with Q=2 and C=.01 I get 861 SVs but using Sign(svm_predict) and counting sign mismatches I get:

Ein=.060 and
Eout=.057

Looking at 2 vs 8 with C=.1 and Q=3, I get 721 SVs but errors are much worse:

Ein=.67
Eout=.63

Since I'm getting the right number of support vectors, I think things are somewhat ok, but I'm perplexed regarding the results from svm_predict.

One dumb question: I presume the right way to feed data into libsvm (using its data file reading capabilities) is to manually subset the data as well as to prep it for libsvm format. Is this correct? When processing 2 vs 8, for example, I'll generate a +1 for "2" data, a -1 for "8" data and then discard the rest. Is this the right approach?
Reply With Quote
  #24  
Old 05-27-2013, 10:11 PM
mvellon mvellon is offline
Junior Member
 
Join Date: Apr 2013
Posts: 9
Default Re: on the right track?

Hmm - running command-line versions of libsvm are corroborating numbers posted by others. I suspect the (mjohnson) .NET version has problems.
Reply With Quote
  #25  
Old 05-28-2013, 04:45 AM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: on the right track?

Quote:
Originally Posted by mvellon View Post
I'm stuck half-way in this problem. I'm trying to use the C# version of libsvm and, I think it's working, but I can't corroborate the numbers I'm seeing here. Actually, I match on the # of support vectors, but my Ein and Eout numbers are significantly different.

For 0 vs. 7 with Q=2 and C=.01 I get 861 SVs but using Sign(svm_predict) and counting sign mismatches I get:

Ein=.060 and
Eout=.057

Looking at 2 vs 8 with C=.1 and Q=3, I get 721 SVs but errors are much worse:

Ein=.67
Eout=.63

Since I'm getting the right number of support vectors, I think things are somewhat ok, but I'm perplexed regarding the results from svm_predict.

One dumb question: I presume the right way to feed data into libsvm (using its data file reading capabilities) is to manually subset the data as well as to prep it for libsvm format. Is this correct? When processing 2 vs 8, for example, I'll generate a +1 for "2" data, a -1 for "8" data and then discard the rest. Is this the right approach?
Yes, you need to extract just the data for those two digits, and your outputs are as instructed in the assignment.

Since several alternative interfaces to LIBSVM have got similar results (I used the R interface through the e1071 package myself), you might consider trying a different interface, if there is one that you could use in limited time. Other than that, the combination of right looking support vector count and wrong looking errors (behaving spectacularly different in the two test runs) is difficult to explain by something you have done.
Reply With Quote
  #26  
Old 05-28-2013, 06:29 AM
chiraz chiraz is offline
Member
 
Join Date: Apr 2013
Location: Tunis, Tunisia
Posts: 12
Default Re: on the right track?

I'm getting two possible cases (answers) for Q5. Randomisation did not help. Anyone have the same problem?

Not sure if there are any parameters to be tweaked that could help separate the cases...
Reply With Quote
  #27  
Old 05-28-2013, 07:49 AM
mluser mluser is offline
Junior Member
 
Join Date: Jan 2013
Posts: 5
Default Re: on the right track?

Was able to verify my numbers thanks to the original post. Got exactly the same results using libsvm with octave
Reply With Quote
  #28  
Old 05-28-2013, 08:40 AM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: on the right track?

Quote:
Originally Posted by chiraz View Post
I'm getting two possible cases (answers) for Q5. Randomisation did not help. Anyone have the same problem?

Not sure if there are any parameters to be tweaked that could help separate the cases...
It's safe to say exactly one of (a) to (e) is correct, as otherwise the question would have been fixed by now. It is not the case that people are generally finding two of the answers to be correct. Checking your individual results should get you there.

[EDIT: checking posts on the previous page, around #17, might also be helpful]

Last edited by Elroch; 05-28-2013 at 09:26 AM. Reason: Suggestion
Reply With Quote
  #29  
Old 05-28-2013, 10:56 AM
chiraz chiraz is offline
Member
 
Join Date: Apr 2013
Location: Tunis, Tunisia
Posts: 12
Default Re: on the right track?

Thanks for the hint!
Reply With Quote
  #30  
Old 05-28-2013, 11:02 AM
chiraz chiraz is offline
Member
 
Join Date: Apr 2013
Location: Tunis, Tunisia
Posts: 12
Default Re: on the right track?

so just what I thought: it boils down to interpreting "decreasing" as "strictly decreasing". C'mon, isn't that silly now
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 02:00 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.