LAUSR

OBJECTIVE The study sought to develop a criteria-based scoring tool for assessing drug-disease knowledge base content and creation of a subset and to implement the subset across multiple Kaiser Permanente (KP) regions. MATERIALS AND METHODS In Phase I, the scoring tool was developed, used to create a drug-disease alert subset, and validated by surveying physicians and pharmacists from KP Northern California. In Phase II, KP enabled the alert subset in July 2015 in silent mode to collect alert firing rates and confirmed that alert burden was adequately reduced. The alert subset was subsequently rolled out to users in KP Northern California. Alert data was collected September 2015 to August 2016 to monitor relevancy and override rates. RESULTS Drug-disease alert scoring identified 1211 of 4111 contraindicated drug-disease pairs for inclusion in the subset. The survey results showed clinician agreement with subset examples 92.3%-98.5% of the time. Postsurvey adjustments to the subset resulted in KP implementation of 1189 drug-disease alerts. The subset resulted in a decrease in monthly alerts from 32 045 to 1168. Postimplementation monthly physician alert acceptance rates ranged from 20.2% to 29.8%. DISCUSSION Our study shows that drug-disease alert scoring resulted in an alert subset that generated acceptable interruptive alerts while decreasing overall potential alert burden. Following the initial testing and implementation in its Northern California region, KP successfully implemented the disease interaction subset in 4 regions with additional regions planned. CONCLUSIONS Our approach could prevent undue alert burden when new alert categories are implemented, circumventing the need for trial live activations of full alert category knowledge bases.