Improving acceptance of computerized prescribing alerts in ambulatory care
Some Clinical Decision Support (CDS) systems use commercial knowledge bases that are highly inclusive and place inadequate emphasis on clinical relevancy. It is important for these systems to display alerts judiciously and to avoid over-alerting. Previous studies have suggested that, in the setting where the threshold for alerting is set too low, clinician acceptance of alerts declines (override rates increase).
The objective of this study was to demonstrate high clinician acceptance of drug alerts by designing a drug alert system that was more selective than commercially available systems, and by implementing an alert system in which only critical or high-severity alerts were allowed to interrupt the clinician.
The study involved the use of the Partners HealthCare System’s Longitudinal Medical Record (LMR) in an ambulatory setting. The LMR started with a commercial knowledge base which was then modified by using a physician-pharmacist expert panel. Drug-allergy alerts had already been implemented in the LMR. The panel determined appropriate duplicate drug, drug-disease, drug-drug, drug-lab and drug-pregnancy contraindications, and they placed all drug alerts into one of three clinical severity tiers: Level 1 alerts signaled a potentially fatal or life-threatening interaction, Level 2 alerts indicated an undesirable interaction with potential for serious injury, and Level 3 alerts referred to situations in which a drug should be used only with caution or with monitoring.
Additionally, the system was designed so that with Level 1 alerts, the clinician had to eliminate the contraindication in order to proceed. With Level 2 alerts, the clinician could proceed if they provided any override reason (chosen from preselected coded responses or entered as free-text). Level 3 alerts were displayed differently, at the top of the screen in red letters, designed to minimize interruption while still conveying the information. Thus, Levels 1 and 2 alerts were considered “interruptive”, and Level 3 alerts were considered “noninterruptive”.
The LMR modified knowledge base contained 1444 drug contraindication rules. Within this knowledge base system, alerts were distributed as follows: 2% Level 1, 63% Level 2, and 35% Level 3. During the study, 18115 alerts were generated: 29% were “interruptive” and 71% were “noninterruptive”. The majority (75%) of the “interruptive” alerts were in the duplicate drug category. The acceptance rate for “interruptive” drug alerts was 67% (= 33% override rate). Clinicians provided a specific reason for override 86% of the time.
The authors found high user acceptance of alerts, and they attributed this to the use of their selective knowledge base and to their attempts to minimize workflow interruptions by using tiered alerts. Their analysis of clinician overrides suggested that clinicians usually deviated from system recommendations for sound clinical reasons. They concluded by suggesting that their study should be used to influence future CDS system design and subsequent efforts to determine the best balance between under-alerting and over-alerting in these systems.
This is a valuable study which contributes not only to our understanding of clinician behavior in response to CDS alert systems, but also to our understanding of how best to design a CDS alert system. The impact of the study would have been strengthened by the presence of a control (i.e. a non-tiered alert system or an unmodified knowledge base), though it is acknowledged that building such controls into this type of system may not be practical. There are a number of studies of CDS systems in the literature describing relatively high alert override rates (69% – 91%). With the low override rate (33%) in this study, the authors have provided us with data supporting the importance and usefulness of two key components when designing drug alert CDS systems: a selective knowledge base and a tiered alert structure.