In this Issue...
Pushing Predictive Power
Framingham Framingham Study Concludes that Multiple Biomarkers Add Only Modest Value
by Julie McDowell
Investigators with the landmark Framingham Heart Study recently evaluated how well 10 biomarkers performed in predicting both major cardiovascular events, as well as death from any cause. After comparing data on the multiple biomarkers, including C-reactive protein (CRP) and B-type natriuretic peptide levels (BNP), the researchers found that the markers added only moderate predictive value compared to standard risk factors like diabetes and blood pressure. This issue of Strategies analyzes what these findings represent for the role and value of biomarker testing.
Since 1948, the Framingham Heart Study has analyzed thousands of patients, searching for the root causes of cardiovascular disease, as well as better ways to prevent, diagnose, and treat the disease. Findings of this ongoing study have resulted in evidence-based medical milestones, including the link between cigarette smoking and heart disease and the association between high blood pressure and risk of stroke.
Even today, the Framingham data continues to be mined for diagnostic research nuggets. In recent analysis, researchers measured 10 biomarkers in 3,209 participants attending a routine examination between 1995 and 1998. In addition to the well studied markers, CRP and BNP, they also evaluated N-terminal pro-atrial natriuretic peptide, aldosterone, renin, fibrinogen, D-dimer, plasminogen-activator inhibitor type 1, homocysteine, and the urinary albumin-to-creatinine ratio. During the examination, cardiovascular risk factors—such as smoking, body-mass index, total cholesterol levels, and high-density lipoprotein (HDL) cholesterol levels—were also evaluated.
During seven years of follow-up, 207 participants died and 169 had initial major cardiovascular events. The study’s findings were reported in the Dec. 21, 2006, issue of the New England Journal of Medicine (355:2631-9; http://content.nejm.org/cgi/content/full/355/25/2631). “With regard to death, we found that five out of the 10 markers were significant predictors in our analysis,” said Thomas Wang, MD, lead investigator and an Assistant Professor at Massachusetts General Hospital and Harvard Medical School in Boston, adding that these five included BNP, CRP, homocysteine, renin, and urinary albumin. For cardiovascular events, two out of the 10 markers—BNP and urinary albumin—were significant predictors.
“But even when we put all of these markers in our statistical models [see below], at least one standard index of performance in terms of ability to predict risk—which is the C statistic or area under the curve—did not change a lot by adding these markers,” explained Wang. These findings are somewhat discouraging, but Wang is quick to point out that the conclusions underscore the effectiveness of evaluating traditional risk factors. “Once you have the traditional risk factors in there, it can be hard to improve upon them,” he explained. “While you may improve upon them, the improvement may not be huge.”
N Engl J Med 2006;355:2631-9. Copyright © 2006 Massaschusetts Medical Society.
At Risk vs. Well-Patient Populations
When reflecting on these findings, Wang emphasized that the Framingham patients were a broad and heterogeneous group, also known as a “well-patient” population. “These findings do not exclude the possibility that in some subsets of people, these biomarkers will be helpful in predicting risk,” he explained. “It’s possible that if you look at traditional risk factors in a narrower population or patients who fall into the gray zone in terms of their predictive risk, the biomarkers in these people might add the most clinical information.”
When assessing the value of biomarker testing, it’s all about the patients being evaluated and their cardiovascular disease risk, emphasized James Januzzi, Jr., MD, Director of the Coronary Care Unit at Massachusetts General Hospital in Boston, who has published extensively on biomarkers, including BNP. He referred to a recent study in the Journal of the American Medical Association, where results showed that an increasing quartile of NT-proBNP levels in patients with coronary heart disease was associated with a greater risk of cardiovascular events or death (2007;297:131; http://jama.ama-assn.org/cgi/content/abstract/297/2/169).
“It does not surprise me, on the surface, that biomarker testing was less useful for stratifying risk in an apparently well-patient population because the baseline level of risk is so very low in these patients,” said Januzzi. “It takes a long time for heart disease to manifest itself in a way that would be clinically evident, including death. In addition, while BNP was in fact useful for predicting events, it wasn’t incrementally useful to using other risk factors like age, which is obviously a lot less expensive to evaluate.”
But Januzzi added that the statistical model used by the Framingham investigators should be scrutinized. This method takes independent hazard factors and adds one upon another, examining the change under the curve, or the C statistic. “If something adds to the area under the curve, that means it’s a good thing, and if it doesn’t add anything to the area under the curve, it means it’s not useful,” he explained. However, many believe that this is not the appropriate way to evaluate biomarkers, because a number of traditional variables—such as LDL and HDL cholesterol and blood pressure—will not appear useful in a C statistic curve, even though the evidence is clear about the association between these variables and cardiovascular events and death. “This approach has been both embraced as well as challenged because it is somewhat superficial in the sense that we are selecting a number of variables for the analysis while we are also neglecting a number of traditional variables that we know are important that just didn’t prove useful in this statistical analysis.”
In addition to the concerns about the C statistic model, Januzzi also believes that results involving BNP might have been impacted by the study’s collection methods. The specimens used in this study were originally collected during 1995-1998, and then frozen for at least seven years before undergoing this recent analysis. Because BNP is sensitive to temperature—and can become unstable when frozen—it’s likely measurements of this protein were impacted by these environmental factors. “Perhaps if they had looked at NT-pro BNP, which is not temperature sensitive, the results would have come out different,” Januzzi explained.
More Biomarkers Needed
In terms of future directions for multiple biomarker research, Wang believes that his study’s results indicate that research needs to focus on finding more biomarkers with the potential to predict heart disease. “These results also suggest that the biomarkers that we currently have are not likely to be enough and we need to identify additional biomarkers in the future through newer methods of discovery to assist us in the ability to predict risk,” he explained. “There’s no reason why we couldn’t find those biomarkers. In the grand scheme of things, 10 molecules is a drop in the bucket in terms of what is measurable in the blood.”
Julie McDowell is the Editor of Strategies. She can be reached by email.