Listen to the Clinical Chemistry Podcast
Article
K. Imai, L.J. Kricka, P. Fortina. Concordance Study of 3 Direct-to-Consumer Genetic-Testing Services Clin Chem 2011; 57: 518-521.
Guest
Christopher deFilippi is an Associate Professor of Medicine at the University of Maryland School of Medicine and Cardiologist at the Maryland Heart Center.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry. I am Bob Barrett.
Recently several companies have started to offer consumers the opportunity to have genetic tests performed to evaluate their ancestry health and/or wellness. The result of consumers using these Direct-to-Consumer Genetic Testing Services is a massive scale testing of Single Nucleotide Polymorphisms or SNPs in a complex multi-step manner that could well result in errors. These errors could result in misclassification of risk that could in turn produce either a false sense of security or unnecessary anxiety. In a concordance study published in the March issue of Clinical Chemistry, Dr. Larry Kricka of the University of Pennsylvania and Paolo Fortina of Thomas Jefferson University evaluated concordance rates with four different Direct-to-Consumer Genetic Testing Services.
Dr. Kricka and Dr. Fortina are our guests in this podcast. Dr. Kricka, what is Direct-to-Consumer Testing and what sort of tests are available?
Larry Kricka:
Well, Direct Consumer Testing is a sort of test where the consumer themselves, a member of the general public can order a test. Normally with a clinical test, this is something that a doctor decides that you need, and then a doctor organizes that the test be done on you and the doctor looks at the results.
In a Direct-to-Consumer Test, the doctor is taken out of that equation, and so the member of the general public can go online or go to a pharmacy and buy a test collection kit, take a sample, send it in, pay for it, and then the results come back, and the member of the general public is then responsible for interpreting those results without the intermediacy of a doctor.
And the range of these tests is quite wide. The ones that have attracted a lot of attention have been obviously the genetic testing, that’s now available, but there are also other tests. There are serological tests for hepatitis. There are drug tests. There are tests for vitamins. All manner of tests are available.
Bob Barrett:
Now, I understand that you recently undertook a study of the reliability of this type of testing. What was the motivation for the study and tell us what you found?
Larry Kricka:
Well, the motivation for the study was, we were interested to see how reliable these tests were. Obviously a genetic test, the sort of genetic test that are being offered now by the various companies that have set up to do this are very involved and elaborate tests involving lots and lots of individual tests, hundreds of thousands of tests in fact.
And anytime you get into such large volumes of testing, the possibility of error is always there, and we wanted to see how someone would fare if they had their testing done by different services, whether they would get the same results.
Bob Barrett:
So how did you do this test? What did you do to get these services to give you data?
Larry Kricka:
In some cases, you can have access to the data. In other cases, we actually requested that we could have the data in a form that we could analyze, so that we could have an electronic copy. And then as part of the study, we actually have to work out how to do the comparison and how to have the data in a format that we could actually analyze easily. So that took a little bit of collaboration with the companies involved.
Bob Barrett:
What did the testing reveal? Did each of the companies report the same results or were there differences?
Larry Kricka:
Generally, the results agreed. So when you look at this, the concordance between the genetic results was greater than 99.6%, which sounds very, very good, but this involves hundreds of thousands of tests. So once you have this sort of level of concordance, the fact that it isn’t a 100% makes a difference.
So for example, in one instance we compared half a million SNPs that had been tested by each of several companies, and based on those, even though the concordance was 99.946%, what that meant over the half a million SNPs was there are about 300 SNPs, where the two companies disagreed in the call for the SNP, and the question is, what is the significance of that?
And I think -- we would think -- that a 100% is what you need, because some of those SNPs could be vital SNPs involved in disease risk calculation, and the fact that they were different from different companies may be important.
Now, the other thing that we found is that was just at the basic analytical level, looking at the analytical results. The other thing, the companies provide estimates of relative disease risk, and so when we looked at these between different companies, we found that they were different.
So for example, you could send your sample into several companies. The first company might tell you have a high risk, the next company might tell you have a low-ish risk, another company might tell you that you are protected. So the significance of this is something that we were and still are interested in, and part of these differences comes from the reference population that’s being used.
So if you’re a Caucasian, you are probably in luck that the risk data has been derived in a Caucasian population. If you are say an Asian, then the data that is going to be used is from the wrong ethnic group and that’s an issue.
Bob Barrett:
These SNP based genetic tests also provide estimates of relative disease risk. Did you see differences in these estimates between the different direct-to-consumer companies offering the SNP testing?
Larry Kricka:
Well, yes, the answer to that was yes, and the work that we published on this, when we published our results, we show some of these differences.
So a good example was for rheumatoid arthritis, where one company had a relative risk for this particular person of 0.9, which is less than one and it's equivalent to protective. Another one had a value of 1.3, which is increased risk, another one of 1.9, further increased risk, and the issue is, what is the accuracy of these numbers? Is 1.3 different from 1.9? Is 1.3 different from 1, which will be no increased risk?
There aren’t confidence limits given with these numbers. So the average member of the general public, do they really understand what the significance of these numbers are, and that’s something -- that’s a completely different question, and one we really didn't attempt to try and answer.
Bob Barrett:
Are there any ethical issues to this sort of testing?
Larry Kricka:
Well, I think people are concerned about this in different ways. One of the biggest concerns here I think is the fact that a physician is not necessarily involved in this process, and so members of the general public can receive these results and have no professional guidance as to the meaning of these results. So I think that’s one ethical consideration, and one I think that people are particularly concerned about.
And then there are other issues beyond that. The idea that perhaps you could steal someone's sample and send it off yourself to a genetic testing company and have the tests done under a false name, and then use that information in some way which was adverse to the person from whom you stole the sample. I mean, these are the sorts of things that I think people worry about.
Bob Barrett:
Now, Dr. Fortina, what do you think the allowable error should be for SNP testing?
Paolo Fortina:
I don’t have a straight answer about the allowable error. I think these platforms were developed for research rather than diagnostics, and they are being used widely for genome-based analysis.
We don’t have in place federal regulations about the use of this platform. At the same time, I must say that some of these platforms that are used in clinical setting, however, and I will make an example like copy number variation that recently is becoming a widely used test. Whenever pathogenic variation is found, that is a validation using other means and more traditional effort.
Bob Barrett:
What are the possible sources of errors than the different genotyping methods employed by the Direct-to-Consumer genetic testing companies?
Paolo Fortina:
So definitely the quality of the safety material of the sample that the company is receiving, and the way it's processed is a key factor that may generate error. The life shelf of the arrays, whatever the platform that is used, and also the quality of the reagents may jeopardize the outcome of the test.
Bob Barrett:
Now, how do you think the field of Direct-to-Consumer testing will develop in the future?
Paolo Fortina:
I think it will develop further. Technology is improving. More information is becoming available on different population[s]. I think recently there have been generated database on African population[s] and so it's still limited.
I know that Navigenics quite recently has implemented the option that the results are provided with a genetic counselor as a post-test service, and I am confident that there will be on both sides of the Atlantic rules about the implementation of this test for general population at the federal level.
So I think there is an area that it will improve and it will become more regulated, but at the same time with results that will have more meaning.
Larry Kricka:
Just to add to that, one other source of error is mix up within laboratories, which happens occasionally, and in this area there has already been one example of that, where a large number of samples were basically mislabeled, and so the wrong results went out to people. And that was only picked up, I think, because one person had sent out, not only their sample, but other members of their family, and when looking through the results from their tests and their families tests realized that things didn't match up in a relationship type of way.
That I think at least alerted that particular user that there was perhaps something strange here. That’s also always another issue. With these sorts of tests, they are so complicated that no one is going to look at a million SNPs and perhaps recognize that there might be a problem.
It's not like it if you have your cholesterol done regularly and you regularly run around 175 and suddenly you get a 500, it would make you think that perhaps something had been done wrongly.
Bob Barrett:
Dr. Fortina was talking about possible regulations being added in the future. What is the FDA involvement on this now? Is there oversight?
Larry Kricka:
Well, I think here in terms of testing, first of all, the labs that do the testing have to be Clear certified, and then beyond that it's a state-by-state basis as to whether you can have these tests done without the intermediacy of a physician.
So I think this is an area that’s evolving. I think that is the best way to describe it at the moment.
Bob Barrett:
Dr. Larry J. Kricka is a Professor of Pathology and Laboratory Medicine at the University of Pennsylvania in Philadelphia. Dr. Paolo Fortina is a Professor at the Kimmel Cancer Center at Thomas Jefferson University in Philadelphia. They have been our guests in this podcast from Clinical Chemistry. I am Bob Barrett. Thanks for listening.