LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 8 (http://book.caltech.edu/bookforum/forumdisplay.php?f=137)
-   -   SVM classifier One-vs-One and One-Vs-All clarification (http://book.caltech.edu/bookforum/showthread.php?t=4043)

 jain.anand@tcs.com 02-27-2013 07:01 PM

SVM classifier One-vs-One and One-Vs-All clarification

I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

 yaser 02-27-2013 09:06 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by jain.anand@tcs.com (Post 9580) I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1? Then how do we classify 4-vs-all etc.? Really appreciate some help here.
One-versus-one: Once class gets and another class gets . Only data from these two classes are considered and the rest of the data is ignored. One has to specify both classes.

One-versus-all: One class gets and all other classes get . Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Both methods can be used as building blocks in a bigger system that distinguishes more digits from each other.

 jain.anand@tcs.com 02-27-2013 09:12 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?

 yaser 02-27-2013 09:59 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by jain.anand@tcs.com (Post 9583) Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?
You are correct.

 gah44 02-28-2013 11:21 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by yaser (Post 9582) (snip) One-versus-all: One class gets and all other classes get . Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.
Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with .

 yaser 02-28-2013 11:49 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by gah44 (Post 9593) Not that I didn't figure it out, but it could be called One-versus-the-rest. Seems to me that many classifiers would work less well if you kept in the "one" class, also with .
:)

 SeanV 02-28-2013 03:33 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification

I was also confused by the wording:
"Then how do we classify 4-vs-all etc.? Really appreciate some help here."

I took it to mean four digits versus all, rather than the digit 4 versus all the rest...( ie we 've just talked about one versus all and one versus one...)

[to explain my confusion - a divide and conquer strategy would make sense to me eg first classify into "straight" digits vs curved ..]

 foodcomazzz 03-03-2013 02:23 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Quote:
 Originally Posted by jain.anand@tcs.com (Post 9580) I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1? Then how do we classify 4-vs-all etc.? Really appreciate some help here.
So for the one-vs-one case, do we randomly pick 2 digits and try the classification, or do we need to average the classification error over all possible combination of 2 digits?:clueless:

 butterscotch 03-03-2013 06:41 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification

The problems will specify which digits to label.
For example, problem 5 regards to "1 versus 5" classifier.

 Elroch 05-23-2013 11:40 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification

Ah!

So am I right in finally understanding that in Q2 and Q3 we have a boolean output corresponding to a single digit? And later questions are about detecting whether a digit is a "1" or a "5", given that it is one of these?

Amazingly, when I read these questions I got stuck on the idea it was about some sort of generalisation of "one versus the rest". For example, I imagined there would be 45 hypotheses for "2 versus all", each corresponding to a pair of digits!

:)

All times are GMT -7. The time now is 01:37 AM.