Components of variance in ROC analysis of CADx classifier performance

Robert F. Wagner; Heang-Ping Chan; Joseph T. Mossoba; Berkman Sahiner; Nicholas Petrick

doi:10.1117/12.310896

24 June 1998 Components of variance in ROC analysis of CADx classifier performance

Robert F. Wagner, Heang-Ping Chan, Joseph T. Mossoba, Berkman Sahiner, Nicholas Petrick

Proceedings Volume 3338, Medical Imaging 1998: Image Processing; (1998) https://doi.org/10.1117/12.310896
Event: Medical Imaging '98, 1998, San Diego, CA, United States

Abstract

We analyze the contributions to the population variance of the area under the ROC curve in assessment of CADx classifier performance and consider a number of models for this variance. The models all contain a pure term or terms in the number of training samples, a pure term in the number of test samples, plus a term or terms representing their interaction. The subset of terms containing the number of test samples also provide a model for what we call the mean Wilcoxon variance based on a single data set. By this variance we mean a nonparametric estimate of the uncertainty in the ROC area obtainable from a single experiment. The remaining terms--i.e., the pure terms in the number of training samples--are not directly estimable without drawing additional training samples. We investigate whether they may be inferred indirectly using a resampling strategy. The current study is presented within the context of our previous work on finite-sample effects on classifier performance, and is related to recent work of others on Analysis of Variance in ROC analysis.

Citation Download Citation

Robert F. Wagner, Heang-Ping Chan, Joseph T. Mossoba, Berkman Sahiner, and Nicholas Petrick "Components of variance in ROC analysis of CADx classifier performance", Proc. SPIE 3338, Medical Imaging 1998: Image Processing, (24 June 1998); https://doi.org/10.1117/12.310896

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available