LFD Book Forum SVM classifier One-vs-One and One-Vs-All clarification
 Register FAQ Calendar Mark Forums Read

#1
02-27-2013, 07:01 PM
 jain.anand@tcs.com Member Join Date: Feb 2013 Location: Cleveland, OH Posts: 11
SVM classifier One-vs-One and One-Vs-All clarification

I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.
#2
02-27-2013, 09:06 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by jain.anand@tcs.com I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1? Then how do we classify 4-vs-all etc.? Really appreciate some help here.
One-versus-one: Once class gets and another class gets . Only data from these two classes are considered and the rest of the data is ignored. One has to specify both classes.

One-versus-all: One class gets and all other classes get . Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Both methods can be used as building blocks in a bigger system that distinguishes more digits from each other.
__________________
Where everyone thinks alike, no one thinks very much
#3
02-27-2013, 09:12 PM
 jain.anand@tcs.com Member Join Date: Feb 2013 Location: Cleveland, OH Posts: 11
Re: SVM classifier One-vs-One and One-Vs-All clarification

Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?
#4
02-27-2013, 09:59 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by jain.anand@tcs.com Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?
You are correct.
__________________
Where everyone thinks alike, no one thinks very much
#5
02-28-2013, 11:21 AM
 gah44 Invited Guest Join Date: Jul 2012 Location: Seattle, WA Posts: 153
Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by yaser (snip) One-versus-all: One class gets and all other classes get . Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.
Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with .
#6
02-28-2013, 11:49 AM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by gah44 Not that I didn't figure it out, but it could be called One-versus-the-rest. Seems to me that many classifiers would work less well if you kept in the "one" class, also with .
__________________
Where everyone thinks alike, no one thinks very much
#7
02-28-2013, 03:33 PM
 SeanV Junior Member Join Date: Jan 2013 Posts: 8
Re: SVM classifier One-vs-One and One-Vs-All clarification

I was also confused by the wording:
"Then how do we classify 4-vs-all etc.? Really appreciate some help here."

I took it to mean four digits versus all, rather than the digit 4 versus all the rest...( ie we 've just talked about one versus all and one versus one...)

[to explain my confusion - a divide and conquer strategy would make sense to me eg first classify into "straight" digits vs curved ..]
#8
03-03-2013, 02:23 AM
 foodcomazzz Junior Member Join Date: Jan 2013 Posts: 5
Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by jain.anand@tcs.com I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1? Then how do we classify 4-vs-all etc.? Really appreciate some help here.
So for the one-vs-one case, do we randomly pick 2 digits and try the classification, or do we need to average the classification error over all possible combination of 2 digits?
#9
03-03-2013, 06:41 AM
 butterscotch Caltech Join Date: Jan 2013 Posts: 43
Re: SVM classifier One-vs-One and One-Vs-All clarification

The problems will specify which digits to label.
For example, problem 5 regards to "1 versus 5" classifier.
#10
05-23-2013, 11:40 AM
 Elroch Invited Guest Join Date: Mar 2013 Posts: 143
Re: SVM classifier One-vs-One and One-Vs-All clarification

Ah!

So am I right in finally understanding that in Q2 and Q3 we have a boolean output corresponding to a single digit? And later questions are about detecting whether a digit is a "1" or a "5", given that it is one of these?

Amazingly, when I read these questions I got stuck on the idea it was about some sort of generalisation of "one versus the rest". For example, I imagined there would be 45 hypotheses for "2 versus all", each corresponding to a pair of digits!

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 02:12 AM.