LAUSR

OBJECTIVES To develop a classification system for urine cytology with artificial intelligence (AI) using a convolutional neural network algorithm that classifies urine cell images as negative (benign) or positive (atypical or malignant). PATIENTS AND METHODS We collected 195 urine cytology slides from consecutive patients with a histologically-confirmed diagnosis of urothelial cancer (between January 2016 and December 2017). Two certified cytotechnologists independently evaluated and labeled each slide; 4637 cell images with concordant diagnoses were selected, including 3128 benign cells (negative), 398 atypical cells, and 1111 cells that were malignant or suspicious for malignancy (positive). This pathologically-confirmed labeled dataset was used to represent the ground truth for AI training/validation/testing. Customized CutMix (CircleCut) and Refined Data Augmentation were used for image processing. The model architecture included EfficientNet B6 and Arcface. We used 80% of the data for training and validation (4:1 ratio) and 20% for testing. Model performance was evaluated with five-fold cross-validation. A receiver operating characteristic (ROC) analysis was used to evaluate the binary classification model. Bayesian posterior probabilities for the AI performance measure (Y) and cytotechnologist performance measure (X) were compared. RESULTS The area under the ROC curve was 0.99 (95% confidence interval [CI], 0.98-0.99), the highest accuracy was 95% (95%CI, 94-97%), sensitivity was 97% (95%CI, 95-99%), and specificity was 95% (95%CI, 93-97%). The accuracy of AI surpassed the highest level of cytotechnologists for the binary classification [Pr(Y>X) = 0.95]. AI achieved >90% accuracy for all cell subtypes. In the subgroup analysis based on the clinicopathological characteristics of patients who provided the test cells, the accuracy of AI ranged between 89 and 97%. CONCLUSION Our novel AI classification system for urine cytology successfully classified all cell subtypes with an accuracy of higher than 90%, and achieved superior diagnostic accuracy of malignancy to the highest level of cytotechnologists.