LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   General Discussion of Machine Learning (http://book.caltech.edu/bookforum/forumdisplay.php?f=105)
-   -   AUC Metric (http://book.caltech.edu/bookforum/showthread.php?t=617)

dbl001 06-05-2012 04:41 AM

AUC Metric
 
Can you comment on using the AUC metric for assessing the quality of a classifier?

Is this the best metric for assessing classifiers?
What is the mathematical basis for AUC?

Thanks!

biopsi 06-09-2012 04:57 AM

Re: AUC Metric
 
There is no such thing as "best", actually there is a jungle of validation metrics and curves out there which all have their merit.

What is often also used is the F1 score (+precision-recall-curves) aside from AUC and ROC. it's problem-dependent, ROC has the advantage/disadvantage of being invariant to class skew. The AUC can be directly computed using the Mann Whitney U statistic.

hth

htlin 06-11-2012 03:11 PM

Re: AUC Metric
 
Quote:

Originally Posted by dbl001 (Post 2791)
Can you comment on using the AUC metric for assessing the quality of a classifier?

Is this the best metric for assessing classifiers?
What is the mathematical basis for AUC?

Thanks!

The AUC can roughly be described as measuring the overall trade-off between false-positive and false-negative in binary classification.

Mathematically, the AUC is also equivalent to measuring the pairwise ranking accuracy introduced from the (decision value of the) classifier.

This paper http://www.icml-2011.org/papers/567_icmlpaper.pdf is a pretty recent study on the connection between AUC and other metrics (such as the usual 0/1 error).

Hope this helps.

CBrauer 07-10-2012 08:35 AM

Re: AUC Metric
 
The link to the paper is not valid. Please fix it.

admin 07-10-2012 09:17 AM

Re: AUC Metric
 
Quote:

Originally Posted by CBrauer (Post 3362)
The link to the paper is not valid. Please fix it.

Fixed.


All times are GMT -7. The time now is 02:03 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.