Listen to the Clinical Chemistry Podcast
Article
C.D. Hawker, W. McCarthy, D. Cleveland, and B.L. Messinger. Invention and Validation of an Automated Camera System That Uses Optical Character Recognition to Identify Patient Name Mislabeled Specimens. Clin Chem 2013;60:463-470.
Guest
Dr. Charles Hawker is Scientific Director of Automation and Special Projects at ARUP, and an Adjunct Professor of Pathology at the University of Utah, School of Medicine.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
Although standardization of barcodes and label formats has lowered the number of mislabeled specimens in clinical laboratories, it remains a potential source of pre-analytical error. Published error rates of mislabeled specimens range up to as high as just over 1%. A paper in the March 2014 issue of Clinical Chemistry describes an automated device with four cameras used to photograph the outside of a specimen tube. This system then uses optical character recognition to look for discrepancies between the patient name in a lab information system compared to the patient name on the tube lid.
An author of that paper, Dr. Charles Hawker, is our guest in today's podcast. Dr. Hawker is Scientific Director of Automation and Special Projects at ARUP Laboratories, and an Adjunct Professor of Pathology at the University of Utah, School of Medicine. Dr. Hawker, the system described in your article seems impressive for its ability to rapidly separate specimens that are correctly labeled, from those that are mislabeled. What prompted you to develop this system?
Charles Hawker:
Well, there are two answers, first, our own laboratory which is a very large reference laboratory and we test many thousands of specimens each day, routinely monitors mislabeled specimens. Based on the published studies from the College of American Pathologists and other groups, we believed our error rate was much lower than the average rates in US laboratories; nevertheless, we thought it highly important to further improve our quality, especially in this critical area because for us the only acceptable labeling error rate is zero.
But there is a second part of the answer and that is about seven or eight years ago I was having a conversation with my former supervisor Dr. William Roberts, who unfortunately passed away in 2012. We were chatting about possible future automation projects and Bill said we ought to try to solve the problem of mislabeled specimens. And I'm not making this up, but I sort of scratched my head and replied that maybe some kind of high-speed camera system using OCR could do that, and then instantly I discounted my own idea, because I didn’t think it would be possible. But a seed had been planted and over the ensuing months I learned about some of these machine vision and camera systems and began exploring the idea with various manufacturers.
Then, even after we found the manufacturer with the leading systems, we still went down several different paths, tried several different prototypes before we were able to develop the system that’s described in the paper and then validate its performance.
Bob Barrett:
Well, what kind of a problem are we talking about, how serious are the labeling error rates in US labs?
Charles Hawker:
I think they are quite serious. The College of American Pathologists has published the results of several studies that they've done called Q-PROBES Studies. These are surveys that they sent to their participating laboratories which consist of questions to which the labs can choose whether to respond. It's purely voluntary.
In the published surveys about labeling errors a number of responding labs was typically 120 to a 150 laboratories which is really only a fraction of all the laboratories in the country but it’s still an appreciable number in terms of the significance of the findings, especially if one considers that the labs likely to respond might be those that routinely measure their error rate and think that their performance is reasonably good.
So at any rate in two of these published surveys of routine clinical laboratories, the published labeling error rates were 0.39 per 1000 specimens and 0.92 per 1000 specimens. Then even worse, however, there was another study conducted in 122 labs handling blood bank specimens for which the reported labeling error rate was 1.12%. This is a really scary number. In fact all of those numbers to me are scary, and I don’t often mention this when I am just talking to the general public, because it reflects rather poorly on the laboratory community. The laboratorians understand how hard it is to do this kind of work and have a better perspective on error rates than I think the public might have.
Bob Barrett:
What are some of the most importantly features of the system described in your article?
Charles Hawker:
Well, first the robotic system had to lift the tubes by their caps, because we didn’t want the grippers to cover the labels since we were trying to photograph the label and that would make the label not readable.
As a reference laboratory, we receive a very large assortment of specimen tubes of different sizes with different types of caps. So the robotic system has a smart camera and a backlight, and together these measure the height and the diameter of the tube and the shape and the diameter of the cap, so that the robot can precisely grasp the cap and lift the tube out of the automated track carrier. Then the vision system consists of four very powerful--there are five megapixel cameras and a strobe lighting system--in order to obtain very high resolution photographs. The system then stitches the four photographs together to make a single photograph, which is really a two-dimensional view of the entire exterior of the tube.
The software then orients the photographs so that our laboratory’s label and the client’s label are always in standard locations in this image. It then queries our Laboratory Information System, or LIS, to determine the patient name in our record, and then it uses what's called an OCR engine; that's Optical Character Recognition, an OCR engine to see if a matching character string is on the client’s label.
In addition, both the robot system and the vision system have to perform at a very high speed in order to keep up with our automation. Presently the total cycle time is about three seconds, but we're working to reduce that to two seconds. We hope to limit the total robotic time to about a second and the vision system time to about a second.
However, if one considers what we are doing with the present system, even three seconds is very impressive. Another critical element is that every one of our clients seem to have their own unique formats for their labels. The labels vary all over the place, in the location, format, and fonts that are used for the patient name. And then sometimes labs write on the labels, or circle the name, and this can interfere with the system. The technology that we are using was developed for high-speed pharmaceutical plants or bottling companies where every bottle ofaAspirin is identical and the OCR system is simply trying to read lot numbers, expiration dates, or similar data. It's always in the same location, format, and font as these bottles are very rapidly going past the cameras on a conveyor system.
When these engineers first saw the huge variety of tubes and labels that we handle, they weren't even sure that this was possible. The fact that this system is working so well is I think a tribute to the genius of our vision systems engineer.
Our system works best when the text on the client label reads from left to right with the tubes cap or stopper at the left end of the photograph. In other words, as though you were holding the tube in your left hand and you were able to read the label and it works best if the patient name is at the left end of the first line of the text.
However, we receive labels in which the patient name is anywhere from the first line to the fifth line of the label and it's not always on the left. Sometimes the patient name is split on two lines, or it's within a colored stripe, or it uses a reverse font in which the printing is white against the black background or in a black rectangle and sometimes the name is wrapped around the tube at a 90° angle to the length of the tube. All of these are issues that will prevent this system from working. So it’s a real challenge.
We also received specimens on which the label is affixed with the opposite orientation as though you would be holding the tube by its cap with your right hand, and then you would be able to read the label. However, our system can rotate those images by 180° to perform the OCR analysis.
Bob Barrett:
So is there any hope that some day the specimen labels for all of these laboratories in the country might be at least better standardized?
Charles Hawker:
Yes, I definitely have that expectation. In 2011, the Clinical and Laboratory Standards Institute, or CLSI, published a standard AUTO12-A, that is specifically focused on the content and formats of specimen labels.
Over the next couple of years, it is expected that the College of American Pathologists which inspects laboratories and the Joint Commission which inspects hospitals, will both reference this standard in their checklist questions and that this will push laboratories to adopt the standard for their own labels. Our laboratory is already working to adopt the standard, but so far it doesn’t appear that many of the laboratories who send their reference testing to us have yet adopted it.
So if our clients do implement this standard, it would obviously benefit our OCR system. But the real reason that CSLI developed the standard was specifically to reduce the labeling error rates that occur in the US laboratories and to improve patient safety.
Bob Barrett:
Has your laboratory seen a benefit in terms of improved patient safety?
Charles Hawker:
We definitely have, there is no question about that. Even though we just have the one existing prototype machine, we are running it 24 hours a day on our Production Automation System with routine specimens, because we want to gather more data and continue to make improvements on its performance. It is only run on one of our four automation tracks, only about a quarter of our daily workload is passing through the system, but because all of that volume is going to one large laboratory section, it's fairly easy for us to measure the impact of finding the mislabeled specimens prior to analysis, and thus to measure how many errors we are preventing.
Moreover, we aren't yet running it with a process of rapid review of the failed images that we plan to do when we go into full production with more advanced systems. In that setting our automation would route to an inspection station where the failed specimens would be instantly picked up and inspected by a technician.
However, we have seen a measurable decrease in corrected reports due to mislabeled specimens that we had to issue to clients, and as of this time we've run approximately 2.5 million specimens through this system. So the improvement in patient safety or the reduction of errors is quite evident.
Bob Barrett:
Well, finally Dr. Hawker, let’s look ahead, what do you think the future holds for this system?
Charles Hawker:
In our laboratory, we plan to build new production units with better robotic systems and faster vision systems that utilize more powerful computers. Our present vision system computer is more than four years old. So just based on Moore's Law, a new computer could shave perhaps a half second off the cycle time. But there are other refinements in both speed and pass rate that we think are possible.
So ultimately then we plan to implement these new units on our entire automation system. And once this occurs I believe that our error rate due to mislabeled specimens will improve significantly. Only about 85% of our total daily workload of our laboratory is on our automation system, but if there is a near zero error rate on that 85% of our volume, then that will be an incredible outcome in terms of improved patient safety.
In addition, I should mention that we have not patented this system. The components could be purchased from our vendor and designed and built to work on most of the major automation systems that laboratories purchase today. They're expensive, but for high-volume laboratories I think that cost will be justified by this possibility of near total elimination of mislabeled specimens and improved patient safety.
Bob Barrett:
Dr. Charles Hawker is Scientific Director of Automation and Special Projects at ARUP, and an Adjunct Professor of Pathology at the University of Utah, School of Medicine. He has been our guest in this podcast from Clinical Chemistry describing a system to lower error rates due to sample identification. I am Bob Barrett. Thanks for listening.