LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 8 (http://book.caltech.edu/bookforum/forumdisplay.php?f=137)
-   -   SVM classifier One-vs-One and One-Vs-All clarification (http://book.caltech.edu/bookforum/showthread.php?t=4043)

jain.anand@tcs.com 02-27-2013 06:01 PM

SVM classifier One-vs-One and One-Vs-All clarification
 
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

yaser 02-27-2013 08:06 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by jain.anand@tcs.com (Post 9580)
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

One-versus-one: Once class gets +1 and another class gets -1. Only data from these two classes are considered and the rest of the data is ignored. One has to specify both classes.

One-versus-all: One class gets +1 and all other classes get -1. Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Both methods can be used as building blocks in a bigger system that distinguishes more digits from each other.

jain.anand@tcs.com 02-27-2013 08:12 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?

yaser 02-27-2013 08:59 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by jain.anand@tcs.com (Post 9583)
Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?

You are correct.

gah44 02-28-2013 10:21 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by yaser (Post 9582)
(snip)

One-versus-all: One class gets +1 and all other classes get -1. Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with -1.

yaser 02-28-2013 10:49 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by gah44 (Post 9593)
Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with -1.

:)

SeanV 02-28-2013 02:33 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
I was also confused by the wording:
"Then how do we classify 4-vs-all etc.? Really appreciate some help here."

I took it to mean four digits versus all, rather than the digit 4 versus all the rest...( ie we 've just talked about one versus all and one versus one...)

[to explain my confusion - a divide and conquer strategy would make sense to me eg first classify into "straight" digits vs curved ..]

foodcomazzz 03-03-2013 01:23 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by jain.anand@tcs.com (Post 9580)
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

So for the one-vs-one case, do we randomly pick 2 digits and try the classification, or do we need to average the classification error over all possible combination of 2 digits?:clueless:

butterscotch 03-03-2013 05:41 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
The problems will specify which digits to label.
For example, problem 5 regards to "1 versus 5" classifier.

Elroch 05-23-2013 10:40 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Ah!

So am I right in finally understanding that in Q2 and Q3 we have a boolean output corresponding to a single digit? And later questions are about detecting whether a digit is a "1" or a "5", given that it is one of these?

Amazingly, when I read these questions I got stuck on the idea it was about some sort of generalisation of "one versus the rest". For example, I imagined there would be 45 hypotheses for "2 versus all", each corresponding to a pair of digits!

:)

yaser 05-23-2013 12:25 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by Elroch (Post 10936)
So am I right in finally understanding that in Q2 and Q3 we have a boolean output corresponding to a single digit? And later questions are about detecting whether a digit is a "1" or a "5", given that it is one of these?

Correct. Q2-4 are about a single digit versus the rest of the digits using data from all digits (commonly called one versus all) and after that we use data from two digits only, discriminating one versus the other (commonly called one versus one).


All times are GMT -7. The time now is 02:03 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.