LAUSR

Background: Variation between radiologists when making recommendations for additional imaging and associated factors are unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose: Determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Material and Methods: This retrospective study analyzed 318,366 reports obtained from diagnostic imaging exams performed at a large urban quaternary care from 1/1/2016 to 12/31/2016, excluding breast and ultrasound reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training and validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318,366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed and intra-division, inter-radiologist variability quantified. Results: The machine learning algorithm classified 38,745 of 318,366 (12.2%) reports as containing follow-up recommendations. The average patient age was 59 years (SD ±17 years); 45.2% (143,767/318,366) of reports were from male patients. Among 65 radiologists, 56.9% (37/65) were male. In multivariable analysis, older patients had higher rates of follow-up recommendations (OR: 1.01 [1.01–1.01] for each additional year), male patients had lower rates (OR: 0.9 [0.9–1.0]), and follow-up recommendations were most common among CT studies (OR: 4.2 [4.0–4.4] compared to X-ray). Radiologist sex (p=0.54), presence of a trainee (p=0.45), and years in practice (p=0.49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold inter-radiologist variation. Conclusions: Substantial inter-radiologist variation exists in the probability of recommending a follow-up exam in a radiology report, after adjusting for patient, exam and radiologist factors.