LFD Book Forum AUC Metric

#1
06-05-2012, 04:41 AM
 dbl001 Member Join Date: Apr 2012 Posts: 11
AUC Metric

Can you comment on using the AUC metric for assessing the quality of a classifier?

Is this the best metric for assessing classifiers?
What is the mathematical basis for AUC?

Thanks!
#2
06-09-2012, 04:57 AM
 biopsi Member Join Date: Apr 2012 Posts: 17
Re: AUC Metric

There is no such thing as "best", actually there is a jungle of validation metrics and curves out there which all have their merit.

What is often also used is the F1 score (+precision-recall-curves) aside from AUC and ROC. it's problem-dependent, ROC has the advantage/disadvantage of being invariant to class skew. The AUC can be directly computed using the Mann Whitney U statistic.

hth
#3
06-11-2012, 03:11 PM
 htlin NTU Join Date: Aug 2009 Location: Taipei, Taiwan Posts: 601
Re: AUC Metric

Quote:
 Originally Posted by dbl001 Can you comment on using the AUC metric for assessing the quality of a classifier? Is this the best metric for assessing classifiers? What is the mathematical basis for AUC? Thanks!
The AUC can roughly be described as measuring the overall trade-off between false-positive and false-negative in binary classification.

Mathematically, the AUC is also equivalent to measuring the pairwise ranking accuracy introduced from the (decision value of the) classifier.

This paper http://www.icml-2011.org/papers/567_icmlpaper.pdf is a pretty recent study on the connection between AUC and other metrics (such as the usual 0/1 error).

Hope this helps.
__________________
When one teaches, two learn.
#4
07-10-2012, 08:35 AM
 CBrauer Junior Member Join Date: Jul 2012 Posts: 1
Re: AUC Metric

#5
07-10-2012, 09:17 AM
Re: AUC Metric

Quote:
 Originally Posted by CBrauer The link to the paper is not valid. Please fix it.
Fixed.

 Tags auc metric, classifiers

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 11:37 AM.