NACB - Scientific Shorts
NACB - Scientific Shorts (formerly NACB Blog)
By Douglas F. Stickle, PhD

Many of you will have seen or heard of the recent appearance of IBM's Watson computer as a contestant on Jeopardy. The first show aired on February 14th, 2011 -- the 65th anniversary of the debut of ENIAC. The performance of Watson on Jeopardy was fantastic. The computational challenge was enormous -- Jeopardy was selected precisely because of its complex linguistic difficulties -- and Watson achieved great success. Watson crushed his human competitors. In fairness to humans, it appeared that Watson had a nearly unbeatable advantage to buzz in first. That aside, Watson had the right answer to almost every question. This was a breathtaking feat, demonstrating his ability to take in a verbal question and output the correct answer in very short time, using on-board data and programmed smarts. The IBM and computer science cheerleading that went along with this special Jeopardy program was, if anything, understated -- this type of interactive system will have enormous impact in many fields. Medicine is certainly one of the areas that would clearly benefit from such systems (1). Indeed, medical decision making is an area of emphasis that is being targeted by IBM (2,3). Ultimately, such systems would not only aid medical care, but direct it, with "sign off" capabilities.

Pathology and laboratory medicine are good examples of prime targets for deployment of automated expert systems. What expert systems exist now, and what will it take to bring Watson's progeny to the fore? Delta checks and reflex testing reflect low-level automated expert systems in widespread use. There are some clinical decision support systems (CDSSs) in laboratory medicine (4) and elsewhere (e.g., pharmacy (5), anesthesiology (6), radiology (7), general medicine (8), others?). However, these are not widely deployed or implemented. Nevertheless, there are many expert algorithms or protocols used routinely, albeit manually, in laboratory diagnosis (e.g., acid-base, endocrine, metabolic disorders). These systems do not typically involve ambiguities – there is but one pathway in following an "if-then-else" algorithm that is highly amenable to automated oversight, even without sophisticated language-interactive capability of a Watson-like system.

Similarly, Watson's Jeopardy encounter involved questions that had only one correct answer (referred to as "factoid" questions (9)), and Watson was highly optimized for Jeopardy with respect to data available to it (9,10). Most likely, Watson could easily be reprogrammed to do well on our board exams, which contain questions highly constrained to have one best answer. Essentially, exam questions all boil down to a simple request: "Recognize the canonical set of case conditions described below to choose the best answer." It is highly likely Watson would excel at that, if his database contained Tietz Textbook.

The difference between where Watson is now and where we would like him to go is that we want him to be able to ask the questions, not just answer them. The future Watson will be able to ask, in appropriate sequence, in real time, questions to efficiently elicit an optimized case treatment scheme. Importantly, he will be able to make rapid, highly informed first decisions about how best to proceed down a differential diagnosis tree of 10,000 possibilities based on initial case data. This is certainly feasible in the long run. In fact, IBM recently announced partnerships with software developers and universities to develop a "physician's assistant," a first step for Watson progeny along these lines (3).

Who would not wish to have the world's collected knowledge and expertise applied to their medical care? It is virtually certain that expert systems will ultimately play a pre-eminent role in the practice of medicine to provide an optimal harmonization of standards of care. Given government's financial interest in medical care, such systems will eventually involve formal government sanction and oversight. Professional organizations including NACB (with its future expert systems LMPGs) will likely play an essential role in development, validation and certification of widely deployed expert systems over the next few decades. One hundred years from now, Watson's descendents, in the form of indefatigable, unerringly pleasant robodocs (who will have spent entire milliseconds in medical school), will provide uniformly phenomenal, world-class expertise in diagnosis and treatment of human medical maladies, such as they then exist. Hasten the day!


Please log in to leave a comment!

Posted by
On 5/26/2011

It depends! IBM has bought SPSS and it has developed Predictive Analytics, while the world has not imagined what is possible. The genome is 10 years old, but the applied mathematics to move forward is really only 5 years mature. But it is correct to recognize the exciting path of the -Omics- revolution, leading to Translational Medicine. I don't think that the NACB LMPGs, and the current approach to evidence-based medicine based on mega-reviews of clinical trials is anywhere sufficient. Why? These are costly in slogging through the literature and dealing with confounders, and insufficient in POWER to deal with "subsets". The emergence of a truly productive tool is near, but it will require the integration of critical clinical information with the pertinent laboratory data. The minimum data set may be more than 2000 patients, depending on the number of predictors and the number of major disease classes. The actual probabilities come from the estqablished frequency within the entire data-base. The actual fit of the model is established by Akaike's Information Criterion and Bayes'Information Criterion. Ender the established principles of Kullback Entropy, the information present is measured by a drop in the total (unassociated) entropy (not the data entropy) in the system. Oddly enough, these principles were well defined by the work in microbial identification by Eugene Rypka, and essentially went unnoticed. The challenge then became how to take the huge amount of data generated for large populations that have varieties in race, age, gender, renal function reserve or loss, liver function, and evolved established risk factors, and create a human phenotypic taxonomy. This could not be handled by the standard parametric approaches that we use in analytical chemistry. In addition to a huge repository of data having a common structure, it is necessary to identify "high quality information", and then to establish a classification with "subsets" that are heterogeneous" within, but have in common a large distance between the major classes. Rypka referred to "Truth Tables". The whole science of Structural equations modeling, which includes bot nonparametric and mixed Latent Class Models relies on computation from tables, with Chi Squared a measure of separation. Is this possible at any time soon? Perhaps it is closer than appears. There are huge barriers that are likely to slow this development. The applied mathematics is essential, but not the whole story. Front-end Design Requirements for Medical Information System Compliance Broad Requirements: 1. Open architecture allows seamless access to data and applications 2. Ability to dictate in a structured format 3. Data capture of all key features used in clinical management, whether the data is numerical or categorical 4. Ability to format report to best present the information for medical decision-making, even if information is obtained from different locations 5. Ability to call up information for review for QI or study purposes 6. Ability to communicate results to physicians near or remote in a timely manner Specifications for Clinical Laboratory Processes: 1. Delta checking of 3 results in 24 hours 2. Application of rules to release of Quality Control runs 3. Application of enhanced CLIA options to meet test control requirements for each analyte (quality control limits are analyte specific and minimize the rate of PT failure) 4. Use of patient sample means (or medians) for QC and rejection of run (X-bar) 5. Application of criteria for acceptance or rejection of patient results (including hemolysis index, sampling error) 6. Ability to review turnaround times daily, by shift 7. Ability to determine bivariate reference ranges 8. Ability to determine odds ratios and probabilities for polychotomous data with nominal, ordered, and continuous predictor variables Vision. 1) Improve turnaround times by a system change that reduces human interaction in steps that can be automated, providing better service to meet the medical staff needs. 2) Reduce human error by building in six sigma quality – not permitting the release of absurd results. On 2 occasions in 3 weeks, 3 consecutive elevated troponin results were released in error. The probability that the results could be valid is less than 1 in 100,000. 3) Provide for continuous flow of work in a seamless manner. 4) Allow for integrated pre- and postanalytical treatment of reports Reflexive testing could be done by rules tied to insertion of add-on commands. Repeat testing could be by exception, and the amount of technologist interaction with normal results would be reduced 90%. Designed report formatting could be addressed. Larry H. Bernstein, MD

About the Author
Douglas F. Stickle, PhD
Douglas F. Stickle, PhD 

1. Demner-Fushman D, Chapman WW, McDonald CJ. What can natural language processing do for clinical decision support? J Biomed Inform 2009 Oct;42(5):760-72. Epub 2009 Aug 13. (Review)

2. IBM Watson

3. "Dr Watson, We Presume." Science 2011;331:992.

4. Main C, Moxham T, Wyatt JC, Kay J, Anderson R, Stein K. Computerized decision support systems in order communication for diagnostic, screening or monitoring test ordering: systematic reviews of the effects and cost-effectiveness of systems. Health Technol Assess 2010 Oct;14(48):1-227. (Review)

5. Robertson J, Walkom E, Pearson SA, Hains I, Williamsone M, Newby D. The impact of pharmacy computerised clinical decision support on prescribing, clinical and patient outcomes: a systematic review of the literature. Int J Pharm Pract 2010 Apr;18(2):69-87. (Review)

6. Hemmerling TM. Automated anesthesia. Curr Opin Anaesthesiol 2009 Dec;22(6):757-63. (Review)

7. Stivaros SM, Gledson A, Nenadic G, Zeng XJ, Keane J, Jackson A. Decision support systems for clinical radiological practice -- towards the next generation. Br J Radiol 2010 Nov;83(995):904-14. (Review)

8. Elkin PL, Liebow M, Bauer BA, Chaliki S, Wahner-Roedler D, Bundrick J, Lee M, Brown SH, Froehling D, Bailey K, Famiglietti K, Kim R, Hoffer E, Feldman M, Barnett GO. The introduction of a diagnostic decision support system (DXplain™) into the workflow of a teaching hospital service can decrease the cost of service for diagnostically challenging Diagnostic Related Groups (DRGs). Int J Med Inform 2010 Nov;79(11):772-7. Epub 2010 Oct 14. (Review)

9. "Schooling the Jeopardy! Champ: Far from Elementary." Science 2011;331:999.

10. Baker S. Final Jeopardy: Man vs. Machine and the Quest to Know Everything. Houghton Mifflin Harcourt Publishing, New York, 2011.