Listen to the Clinical Chemistry Podcast
Article
R. Arnaout. Elementary, My Dear Doctor Watson Clin Chem 2012; 58: 986-988.
Guest
Dr. Ramy Arnaout is a pathologist from the Beth Israel Deaconess Medical Center in Boston, Massachusetts.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry. I am Bob Barrett.
Watson, IBM's Jeopardy concurring super computer has become a national phenomenon for its ability to rapidly complete answers to questions utilizing database queries and voice recognition technology. The top competitors in Jeopardy's history were no match for the computer and were trounced in the television competition. But can the technology used in Watson be applied outside of game shows?
In the June 2012 issue of Clinical Chemistry, Dr. Ramy Arnaout, a pathologist from Beth Israel Deaconess Medical Center in Boston, Massachusetts tackled the implications of utilizing the machinery behind Watson in the laboratory setting. Dr. Arnaout is our guest in podcast. Well, doctor, tell us about this technology.
Ramy Arnaout:
Let me start with a word about Jeopardy. So for those who've never seen the show, it's a game show, of the question and answer variety and it's been on here forever and I used to watch it as a kid. The way it works is the host reads a clue from a category, and the contestants have five seconds to buzz in with a correct answer and it's got to be freeze in the form of a question. That's kind of the show's thing.
The thing about Jeopardy is the clues can be about just about anything and they're often tongue-in-cheek. So, a category might be called Royalty and the clue might be this Monarch was born in Mississippi in 1935, thank you very much. And for that, the answer would be "Who is Elvis Presley?" because he is the king of rock and roll.
So contestants suddenly have to know and offer a lot of information. They also have to get the questions and be quick on their feet in order to be able to answer correctly to win. So, you know, that's not the kind of behavior you really expect from a computer. I mean think of the PC on your desk get work. It works nothing like that.
So the challenge of figuring out how to get a computer to non-jeopardy is what IBM is interested in and after a bunch of years and a bunch of millions of dollars, they managed to do it and that's what was so exciting about Watson. It was a machine that was able to get those human questions and get them right. So the integration of huge amounts of data on the other hand which of course Watson has to do. Well, Google does that every time you do a search.
So, at least personally, that wasn't the very exciting part about Watson's accomplishment. What was exciting was that Watson ‘got’ the questions and understood and it returned not just the web page but an answer, the proper limit form of a question and that was what so exciting.
Bob Barrett:
Why has the IBM team targeted medicine?
Ramy Arnaout:
I think without putting too fine of a point on it, it's because in medicine, the questions and answers are life or death. I mean winning on Jeopardy was a nice proof of principle. The Watson team showed that a computer could be programmed to understand human language posed in a natural form and scour amounts of natural data pretty much instantly to find the right answer. But when you consider the big picture, there are an awful lot of situations other than TV game shows where people have to do just that and medicine is of course one of them. So, IBM wants to use Watson to answer questions in medicine because they're important, life or death.
Bob Barrett:
That sounds very general.
Ramy Arnaout:
We know it is and at this point it probably should be for two reasons. First, answering questions is a pretty general feature of medicine, everything from what's the result of this blood test to what's the best drug combination for my patient with this cancer. And of course, for a lot of questions in medicine, you don't need anything as complicated as Watson. But the second reason why it should sound general is it's not quite clear yet at least from outside of IBM where exactly IBM and the Watson team feel like they're going to have the greatest impact within medicine.
At least that's the conclusion from their public announcements. Now over the past several months, IBM or people collaborating with them have said Watson is being trained to answer to same kind of questions being asked to medical residents suggesting that United States Medical Licensing Exam Board kind of our capability. But they have also talked about putting it to work on cancer care and genome analysis and their Health Advisory Board is similarly ecumenical.
So, throw in the kitchen sink and you're done. And in fact, just a few days ago, the business lead on the Watson project announced that they are no longer even looking for new ideas for Dr. Watson. They said their plate is just absolutely full. So for now, you're right, the Watson team is trying out a bunch of specific things, but from the outside it's too early to make more than general conclusions.
Bob Barrett:
Now, what will Watson mean for lab medicine specifically?
Ramy Arnaout:
Well, clinical questions are all based on data, and the kind of questions that are most commonly asked in medicine often boil down to what is the pre-test probability of my patient having this particular diagnosis or what is the sensitivity of this particular test for that diagnosis being true in a patient like mine? What is the specificity? What's the positive and negative predictive value? When you consider that, most medical questions translate into those specific sorts of mathematical types of things.
You see where lab medicine comes in because lab medicine is all about the pretest probabilities, sensitivities, specificities and positive and negative predictive values of tests for particular diagnosis. Either you try to rule them in or you try to rule them out. The reason why that's especially important in laboratory medicine is that the biggest single problem, the biggest single source of error if you will introduce in laboratory medicine at the hospital is what you call "pre-pre and post-post analytical error" and the names are awful I agree.
Pre-pre analytical error just means "Did the doctor pick the right test?" And post-post analytical error is "Did the doctor interpret those results correctly?" And if you add it all up and say, "Okay, well where do things go wrong in the hospital when laboratory medicine is involved?" It isn't in doing the test. We do the test, just fine. It isn't in labeling. It isn't in reporting results. Again, we do those things just fine. It's making sure that the right test was ordered and that it was interpreted the right way and you pick those tests and you make your interpretations based on these positive predictive values, sensitivities and specificities and prior pretest probabilities and again, now going backward up to chain of translation, that's exactly the kind of question that Watson should be very suited to answer.
So the short story is, Watson, by being able to answer questions like “Which test should I order?” and “What test does this mean?” Watson should be able to impact the laboratory medicine directly by attacking those pre-pre and post-post analytical errors.
Bob Barrett:
So do you think Watson will really be taking over for what doctor's do?
Ramy Arnaout:
I think taking over is a strong term. I don't think Watson will take over any more than a calculator took over what mathematicians do. Now what people do is integrate data, and machines can do that more reliably. That frees us as people to do it in different ways and focus on problems and issues that we didn't have the luxury to perform. Instead of wondering or having to guess or looking to the greatest beard in the room to ask -- to tell you, "Well, what test should I order?" You can have Watson say, “You know, in situations like this, the following test gave a positive diagnosis this percentage of the time. This other test gave on a different percentage of the time.”
What we do with that information is up to us. Watson gives us that information, which was always out there. It's always latent in the data that we collect to the tune of 4.6 billion laboratory test results last year in the US alone. But now, we have the luxury of being able to look at hard data and help decide ourselves what to do with it. So Watson is a tool that will free us up to provide better service.
That said there are obstacles. Watson's results are only going to be as good as the data that goes into it and that data availability is an issue. It's an issue because most of this country at least still does not have its records electronic and beyond that, even when data is electronic, it’s not always made electronic in a form that's conducive for computers, Watson or any other to really understand. Beyond that, there is an awful lot of data which is of a different kind to what Watson has had to deal with so far on Jeopardy pretty much anything that's not text and that includes EKG's, images of various sorts including anything from radiology and actually a number of other examples like that.
So, putting all of that together and sort of making sure it works will take time and effort and finally, at the end of the day, well how it's actually going to work? It is going to be something that I speak into my smart phone and Watson will work kind of like theory? Will it be something that I as a community doc out in the rural part of the state will have access to or will it be something that only the biggest insurance companies will pay multimillions of dollars to IBM for to get at tertiary care centers? All those things are very practical, real world obstacles for Watson to realize it's immense promise at this area.
Bob Barrett:
So, doctor it sounds like you see this is part of a larger evolution in medicine?
Ramy Arnaout:
Exactly, although that's an idea that's been around for a while now and it's not mine. I think it was Ray Kurzweil, the futurist who is most famous for this idea that technological singularity that basically technology is getting so much faster at such a rate that at some point we will seize to be able to understand its progress. It was him who said that medicine is really becoming an information science.
It's the next step really in evidence-based practice, which is something that we have been taking more and more seriously in medicine especially over the last 10 or 20 years of having machines and systems to help us out with that evidence. And I think that that is something that is happening across medicine. You can see it everyday in papers published in medical journals, and I think the outcome is going to be a big win. Most importantly because it will help our patients live healthier and happier lives and that's why our medicine in the first place.
Bob Barrett:
Dr. Ramy Arnaout is a pathologist from the Beth Israel Deaconess Medical Center in Boston, Massachusetts. He's been our guest in this Podcast from Clinical Chemistry. I'm Bob Barrett. Thanks for listening.