LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 8

Reply
 
Thread Tools Display Modes
  #1  
Old 02-27-2013, 07:01 PM
jain.anand@tcs.com jain.anand@tcs.com is offline
Member
 
Join Date: Feb 2013
Location: Cleveland, OH
Posts: 11
Default SVM classifier One-vs-One and One-Vs-All clarification

I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.
Reply With Quote
  #2  
Old 02-27-2013, 09:06 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
Originally Posted by jain.anand@tcs.com View Post
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.
One-versus-one: Once class gets +1 and another class gets -1. Only data from these two classes are considered and the rest of the data is ignored. One has to specify both classes.

One-versus-all: One class gets +1 and all other classes get -1. Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Both methods can be used as building blocks in a bigger system that distinguishes more digits from each other.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 02-27-2013, 09:12 PM
jain.anand@tcs.com jain.anand@tcs.com is offline
Member
 
Join Date: Feb 2013
Location: Cleveland, OH
Posts: 11
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?
Reply With Quote
  #4  
Old 02-27-2013, 09:59 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
Originally Posted by jain.anand@tcs.com View Post
Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?
You are correct.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #5  
Old 02-28-2013, 11:21 AM
gah44 gah44 is offline
Invited Guest
 
Join Date: Jul 2012
Location: Seattle, WA
Posts: 153
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
Originally Posted by yaser View Post
(snip)

One-versus-all: One class gets +1 and all other classes get -1. Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.
Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with -1.
Reply With Quote
  #6  
Old 02-28-2013, 11:49 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
Originally Posted by gah44 View Post
Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with -1.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #7  
Old 02-28-2013, 03:33 PM
SeanV SeanV is offline
Junior Member
 
Join Date: Jan 2013
Posts: 8
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

I was also confused by the wording:
"Then how do we classify 4-vs-all etc.? Really appreciate some help here."

I took it to mean four digits versus all, rather than the digit 4 versus all the rest...( ie we 've just talked about one versus all and one versus one...)

[to explain my confusion - a divide and conquer strategy would make sense to me eg first classify into "straight" digits vs curved ..]
Reply With Quote
  #8  
Old 03-03-2013, 02:23 AM
foodcomazzz foodcomazzz is offline
Junior Member
 
Join Date: Jan 2013
Posts: 5
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
Originally Posted by jain.anand@tcs.com View Post
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.
So for the one-vs-one case, do we randomly pick 2 digits and try the classification, or do we need to average the classification error over all possible combination of 2 digits?
Reply With Quote
  #9  
Old 03-03-2013, 06:41 AM
butterscotch butterscotch is offline
Caltech
 
Join Date: Jan 2013
Posts: 43
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

The problems will specify which digits to label.
For example, problem 5 regards to "1 versus 5" classifier.
Reply With Quote
  #10  
Old 05-23-2013, 11:40 AM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: SVM classifier One-vs-One and One-Vs-All clarification

Ah!

So am I right in finally understanding that in Q2 and Q3 we have a boolean output corresponding to a single digit? And later questions are about detecting whether a digit is a "1" or a "5", given that it is one of these?

Amazingly, when I read these questions I got stuck on the idea it was about some sort of generalisation of "one versus the rest". For example, I imagined there would be 45 hypotheses for "2 versus all", each corresponding to a pair of digits!

Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 08:20 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.