A July 2005 study by Mark Graber, Chief of Medical Service at the VA in Northport, NY, analyzed 100 cases of diagnosis error by physicians ². 93 cases involved injury, including 33 deaths. System related factors contributed to the diagnosis error in 65% of the cases and cognitive factors in 74%. Premature closure, i.e., the failure to continue considering reasonable alternatives after an initial diagnosis was reached, was the single most common cause.

According to Graber, although the act of making a diagnosis is a defining skill of a physician, we seldom think about how well we carry out this function. The value of a complete differential diagnosis is drilled into medical students but how well or consistently do we perform this fundamental task? Outside of autopsies or litigation, what systems do we have in place to assess the quality of the differential diagnosis? Graber has suggested that physicians need to improve their calibration by getting feedback on the diagnoses they make. Cognitive feedback improves judgment and decisions made under conditions of uncertainty ².

A decade ago Elstein suggested the value of compiling a complete differential diagnosis to combat the tendency to premature closure ³. The ability of a diagnosis decision support system to give clinicians an instant checklist of likely diagnoses thereby reminding them of reasonable and relevant diagnoses addresses the very issue of premature closure and has been shown to decrease the incidence of diagnosis error ^4,5.

As DDSS demonstrate their impact on patient safety, healthcare costs and efficiencies, an increasing number of healthcare institutions are adopting DDSS either as a standalone system or interfaced with EMR. An article by George J. Annas, J.D., M.P.H. in a recent issue of the NEJM, ‘The Patient's Right to Safety — Improving the Quality of Care through Litigation against Hospitals’ argues that courts could determine that a hospital's failure to adopt a new technology to prevent the injury of patients could subject the hospital to liability for injury in cases in which it could be demonstrated that adoption of the technology would not have been prohibitively expensive and would probably have prevented the injury ⁶.

In 2005 Charles Friedman looked at the alignment between the confidence physicians had in their diagnosis [subjective] and the correctness of the diagnosis [objective] ⁷. Residents & faculty (correctly diagnosed 44% and 50% of difficult cases, respectively) were overconfident, placing credence in a diagnosis that was in fact incorrect, in 15% and 12% of cases. The authors concluded "Even experienced clinicians may be unaware of the correctness of their diagnosis at the time they make them. DDSS designed to reduce diagnosis error, cannot rely exclusively on clinicians’ perceptions of their needs for such support". Interfacing DDSS with EMR is likely to lower the threshold for physicians to use these systems.

Isabel (Isabel Healthcare Inc, USA), a Web-based diagnosis reminder system is an award-winning, diagnosis decision support system designed by clinicians to enhance the quality of diagnosis decision making. For a given set of clinical features Isabel instantly provides a checklist of likely diagnoses including bio-terrorism conditions, related diagnoses and causative drugs

Isabel provides a simple interface for clinicians to rapidly search for a differential diagnosis by entering clinical features in free text natural language. Isabel differs from rules based DDSS like QMR, Iliad, Dxplain, DiagnosisPro, Problem Knowledge Couplers [PKC], Gideon in that Isabel uses statistical natural language processing [SNLP] software and is thus able to handle unstructured data – free text. A study of QMR and Iliad showed that each case took from 20 to 40 minutes to input ⁸. PKC handles one chief complaint at a time and is designed for data to be entered by the patient. A recent randomized outpatient trial of PKC showed that the average Coupler session took approximately 18 minutes of employee time to coordinate ⁹. Studies have shown that time taken to enter data into Isabel in free text and obtain a result is less than a minute [the time it takes to type the clinical features] ^4,5.

The ability to handle free text has enabled Isabel to be interfaced with several electronic medical records [EMR] systems. Clinical features are extracted from pre-assigned fields in the EMR and as result diagnosis decision support is provided instantly on a single click of an info button on the EMR.

Expert Systems vs. reminder systems

Dxplain and Quick Medical Reference (QMR) were developed two decades ago as 'expert systems' that aimed to solve clinical conundrums. They used a complex network of clinical findings and disease names within their database. Users entered a patient's clinical features through a controlled vocabulary. Diagnostic results were ranked by probability. In contrast, Isabel is not an 'expert system'; it simply aims to remind clinicians of key diagnostic possibilities. Results are arranged by body system. The database is merely a collection of textual disease descriptions. The user interface accepts natural language entries. Decision making is still left to the physician. Rules based systems like Dxplain give the user a ranked list of diagnoses. It’s understandable that clinicians balk, when presented with a system that seems to devalue their clinical education and experience.

Isabel has been independently evaluated in a number of studies – cases from the NEJM clinical pathological case series were used in a study by Graber et al in NY state to test the accuracy of the system. Case histories were pasted and inserted into the search field en bloc without any effort to summarise the clinical features into key phrases. Isabel presented the correct diagnosis in 74 % of the cases. When clinical features were extracted from CPC case Isabel prompted the consideration of the diagnosis in 96% of cases.

A study done at the University of Virginia assessed the impact of Isabel on residents’ diagnosis decision making. In 15 of the 150 cases completed (10%), Isabel caused the user to include a major diagnosis they had not considered and should have. For each of the six cases, the mean diagnostic quality score increased significantly after residents consulted the Isabel

3. Arthur S Elstein, Alan Schwarz, Clinical problem solving and diagnostic decision making: selective review of the cognitive literature.BMJ 2002;324:729-732.

8. Graber MA, VanScoy D. How well does decision support software perform in the emergency department? Emerg Med J 2003;20:426–428