LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 8 (http://book.caltech.edu/bookforum/forumdisplay.php?f=137)
-   -   SVM classifier One-vs-One and One-Vs-All clarification (http://book.caltech.edu/bookforum/showthread.php?t=4043)

jain.anand@tcs.com 02-27-2013 07:01 PM

SVM classifier One-vs-One and One-Vs-All clarification
 
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

yaser 02-27-2013 09:06 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by jain.anand@tcs.com (Post 9580)
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

One-versus-one: Once class gets +1 and another class gets -1. Only data from these two classes are considered and the rest of the data is ignored. One has to specify both classes.

One-versus-all: One class gets +1 and all other classes get -1. Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Both methods can be used as building blocks in a bigger system that distinguishes more digits from each other.

jain.anand@tcs.com 02-27-2013 09:12 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?

yaser 02-27-2013 09:59 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by jain.anand@tcs.com (Post 9583)
Thank you professor for such quick response. I think now I understand correctly e.g. in Q 5 1 vs 5 classifier we should make all records of digit 1 as say +1 and all records of digit 5 as -1 and remove all other records from the training set and train our model. Is that right understanding?

You are correct.

gah44 02-28-2013 11:21 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by yaser (Post 9582)
(snip)

One-versus-all: One class gets +1 and all other classes get -1. Data from all classes are considered, and one needs only to specify the "one" class, e.g., 5-versus-all in the digits case.

Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with -1.

yaser 02-28-2013 11:49 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by gah44 (Post 9593)
Not that I didn't figure it out, but it could be called One-versus-the-rest.

Seems to me that many classifiers would work less well if you kept in the "one" class, also with -1.

:)

SeanV 02-28-2013 03:33 PM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
I was also confused by the wording:
"Then how do we classify 4-vs-all etc.? Really appreciate some help here."

I took it to mean four digits versus all, rather than the digit 4 versus all the rest...( ie we 've just talked about one versus all and one versus one...)

[to explain my confusion - a divide and conquer strategy would make sense to me eg first classify into "straight" digits vs curved ..]

foodcomazzz 03-03-2013 02:23 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Quote:

Originally Posted by jain.anand@tcs.com (Post 9580)
I don't understand the question correctly here. For one-vs-one should we pick 2 digits (e.g. 1 and 2) and make them +1 and -1 and rest of the data ignore and try the classification and repeat the exercise for 1,2 1,3 1,4 etc. That would be 10C2 combination. Also for one-vs-all should we do the same but instead of ignoring all the data should we make them all -1?

Then how do we classify 4-vs-all etc.? Really appreciate some help here.

So for the one-vs-one case, do we randomly pick 2 digits and try the classification, or do we need to average the classification error over all possible combination of 2 digits?:clueless:

butterscotch 03-03-2013 06:41 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
The problems will specify which digits to label.
For example, problem 5 regards to "1 versus 5" classifier.

Elroch 05-23-2013 11:40 AM

Re: SVM classifier One-vs-One and One-Vs-All clarification
 
Ah!

So am I right in finally understanding that in Q2 and Q3 we have a boolean output corresponding to a single digit? And later questions are about detecting whether a digit is a "1" or a "5", given that it is one of these?

Amazingly, when I read these questions I got stuck on the idea it was about some sort of generalisation of "one versus the rest". For example, I imagined there would be 45 hypotheses for "2 versus all", each corresponding to a pair of digits!

:)


All times are GMT -7. The time now is 01:37 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.