LAUSR

In diagnostic medicine the true disease status of a patient is often represented on an ordinal scale, for examples, cancer stage (0, I, II, III, and IV), and coronary artery disease severity using the CAD-RADs scale (none, minimal, mild, moderate, severe, and occluded). With advances in quantitation of diagnostic images and in Artificial Intelligence (AI), both supervised and unsupervised algorithms are being developed to help physicians correctly grade disease. Most of the diagnostic accuracy literature deals with binary disease status (disease present or absent); however, the assessment of tests diagnosing ordinal-scaled diseases should not be reduced to a binary status just to simplify diagnostic accuracy testing. In this paper the authors propose different characterizations of ordinal-scale accuracy for different clinical use scenarios, along with methods for comparing tests. In the simplest scenario, just the proportion of correct grades is considered; other scenarios address the magnitude and direction of mis-grading; and at the other extreme a weighted accuracy measure with weights based on the relative costs of different types of mis-grading is presented. The various scenarios are illustrated using a coronary artery disease example where the accuracy of AI algorithms to provide patients the correct CAD-RADs grade are compared.