Email updates

Keep up to date with the latest news and content from BMC Medicine and BioMed Central.

Journal App

google play app store

This article is part of the series Personalized medicine: genes, biomarkers and tailored treatment.

Open Access Research article

Support of personalized medicine through risk-stratified treatment recommendations - an environmental scan of clinical practice guidelines

Tsung Yu1, Daniela Vollenweider2, Ravi Varadhan3, Tianjing Li1, Cynthia Boyd3 and Milo A Puhan1*

Author Affiliations

1 Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205, USA

2 Department of Medicine, Division of General Internal Medicine, Johns Hopkins School of Medicine, Baltimore, MD 21205, USA

3 Department of Medicine, Division of Geriatric Medicine and Gerontology, Johns Hopkins School of Medicine, Baltimore, MD 21205, USA

For all author emails, please log on.

BMC Medicine 2013, 11:7  doi:10.1186/1741-7015-11-7

The electronic version of this article is the complete one and can be found online at:

Received:3 July 2012
Accepted:9 January 2013
Published:9 January 2013

© 2013 Yu et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.



Risk-stratified treatment recommendations facilitate treatment decision-making that balances patient-specific risks and preferences. It is unclear if and how such recommendations are developed in clinical practice guidelines (CPGs). Our aim was to assess if and how CPGs develop risk-stratified treatment recommendations for the prevention or treatment of common chronic diseases.


We searched the United States National Guideline Clearinghouse for US, Canadian and National Institute for Health and Clinical Excellence (United Kingdom) CPGs for heart disease, stroke, cancer, chronic obstructive pulmonary disease and diabetes that make risk-stratified treatment recommendations. We included only those CPGs that made risk-stratified treatment recommendations based on risk assessment tools. Two reviewers independently identified CPGs and extracted information on recommended risk assessment tools; type of evidence about treatment benefits and harms; methods for linking risk estimates to treatment evidence and for developing treatment thresholds; and consideration of patient preferences.


We identified 20 CPGs that made risk-stratified treatment recommendations out of 133 CPGs that made any type of treatment recommendations for the chronic diseases considered in this study. Of the included 20 CPGs, 16 (80%) used evidence about treatment benefits from randomized controlled trials, meta-analyses or other guidelines, and the source of evidence was unclear in the remaining four (20%) CPGs. Nine CPGs (45%) used evidence on harms from randomized controlled trials or observational studies, while 11 CPGs (55%) did not clearly refer to harms. Nine CPGs (45%) explained how risk prediction and evidence about treatments effects were linked (for example, applying estimates of relative risk reductions to absolute risks), but only one CPG (5%) assessed benefit and harm quantitatively and three CPGs (15%) explicitly reported consideration of patient preferences.


Only a small proportion of CPGs for chronic diseases make risk-stratified treatment recommendations with a focus on heart disease and stroke prevention, diabetes and breast cancer. For most CPGs it is unclear how risk-stratified treatment recommendations were developed. As a consequence, it is uncertain if CPGs support patients and physicians in finding an acceptable benefit- harm balance that reflects both profile-specific outcome risks and preferences.

Cancer; cardiovascular disease; chronic disease; COPD; diabetes; guidelines; randomized trials; risk assessment; stroke; treatment


An important goal of evidence-based health care is to maximize benefits and minimize harms from medical treatments. To achieve an optimal balance, patients' individual profiles and preferences need to be considered [1]. For example, inhaled corticosteroids are used to prevent exacerbations in patients with chronic obstructive pulmonary disease (COPD) [2-4], but these drugs are associated with an increased risk for pneumonia and fractures [5,6]. In patients at high risk for exacerbations, the potential benefits (preventing exacerbations) are likely to be larger than harms, while patients at low risk for exacerbations may experience more harms from inhaled corticosteroids than benefits.

Risk-stratified treatment recommendations are potentially useful to support personalized medicine. Personalized medicine aims at optimizing the benefit-harm balance by considering patient profiles (combination of characteristics) and preferences [7]. For the prevention and treatment of chronic disease, most health care decisions are sensitive to patient profiles and preferences [8]. Risk-stratified treatment recommendations suggest different treatment regimens for patients who are at different risks for outcomes [9]. For example, in the Third Report of the National Cholesterol Education Program's Adult Treatment Panel treatment algorithm [10], the recommendation for primary prevention of coronary heart disease is based on the Framingham Risk Score. According to different risk categories predicted by the Framingham Risk Score, individuals with higher predicted absolute risk (10-year risk > 20%) are recommended for more intensive treatments (such as combined pharmacological and non-pharmacological treatments) than those with lower predicted risk (10-year risk < 10%). There is evidence that using risk-stratified treatments is superior to treatments that are not informed by a risk assessment tool [11-13].

Risk-stratified treatment recommendations only serve their purpose of supporting personalized medicine if valid methods were used to develop them. Because it is not known what proportion of clinical practice guidelines (CPGs) make risk-stratified treatment recommendations and what methods were used to develop them, our aim was to assess the methods CPGs applied in developing risk-stratified treatment recommendations for the prevention or treatment of selected common chronic diseases.


Framework for developing risk-stratified treatment recommendations

We started by forming a framework for developing risk-stratified treatment recommendations. Figure 1 outlines the major steps for developing risk-stratified treatment recommendations, each of which requires high quality evidence from observational studies (development and validation of risk assessment tools), randomized trials (evidence about treatment effects) and studies to elicit patient preferences (using various study designs, for example, discrete choice experiments). It is well known for all guidelines that evidence about treatment effects on benefit and harm outcomes must be available. In addition, a risk assessment tool should be available that allows the assigning of patients to different risk categories. A method is required to estimate how treatment evidence applies to patients at different risks and how the benefits compare to the harms in patients at different risks. As a result of such a benefit-harm assessment, treatment thresholds can be defined for patients with different risk profiles that maximize the chance for benefits while minimizing harms. In addition, patient preferences for outcomes would ideally be explicitly considered for the development of risk-stratified treatment recommendations or their application in practice.

thumbnailFigure 1. Important elements for the development of risk-stratified treatment recommendations.

Environmental scan of clinical practice guidelines

We performed an environmental scan of CPGs, which included a limited literature search (described below) but not a comprehensive, systematic review of all CPGs. We focused on CPGs for major chronic diseases and from the United States (US), Canada, or the United Kingdom (UK) National Institute for Health and Clinical Excellence (NICE). The completed PRISMA checklist is available as Additional file 1.

Additional file 1. PRISMA checklist. The additional file shows the completed PRISMA checklist and shows where the checklist items are reported.

Format: DOC Size: 63KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Data sources and searches

We searched the US National Guideline Clearinghouse (NGC) database on February 5 2011 for CPGs with treatment recommendations for five major chronic diseases. The top five chronic diseases in the US are heart disease, cancer, stroke, COPD and diabetes, accounting for more than two-thirds of all deaths [14]. In the NGC database, guidelines were categorized by disease topics that were linked to a specific term derived from the US National Library of Medicine's Medical Subject Headings classification.

For heart disease and stroke, we performed our search within the Cardiovascular Diseases section of the database (n = 442) and considered CPGs specific for primary prevention of heart disease and stroke, that is, the prevention of an event in persons free of established cardiovascular diseases. For cancer, we chose to examine three types of cancer with the highest mortality rates in the US (lung cancer, prostate cancer and breast cancer) [15]. We searched for CPGs within the Lung Neoplasms (n = 53), Prostatic Neoplasms (n = 26) and Breast Neoplasms (n = 52) sections, respectively. For COPD, we considered CPGs specific for COPD within the Respiratory Tract Diseases section (n = 102). For diabetes mellitus, we considered CPGs for type II diabetes within the Diabetes Mellitus, Type 2 section (n = 44).

Eligibility criteria for guidelines

We included CPGs that recommended using risk assessment tools to inform treatment decisions. Risk assessment tools are tools to calculate the probability of developing an event or a disease based on a prediction model (binary outcome), or tools that make projections about the course of disease measured by patient-reported or other continuous outcomes (for example, decline of functional status over time). We excluded CPGs if they were not from the US, Canada or NICE (UK); focused on childhood diseases; made recommendations on screening, genetic counseling or diagnostic work-up alone; or did not use any risk assessment tools to inform risk-stratified treatment decisions. This latter excluded category involved guidelines that recommended treatments according to diagnostic criteria, as for example based on pathological staging, rather than according to prognostic information (for example, the risk stratification scheme proposed by D'Amico et al. in prostate cancer guidelines [16]).

Guideline selection

Two reviewers (TY and DV) independently reviewed the Guideline Summary section of each CPG on the NGC website to assess its potential eligibility. We excluded the CPGs labeled ineligible by both reviewers. For the other CPGs, we retrieved and examined the full text and resolved any discrepancies in eligibility through discussion or arbitration by a third reviewer (MP).

Data extraction and synthesis

We developed a standardized form to extract data from the included CPGs and the background documents detailing the methods used in developing CPGs when available. We extracted general items such as guideline title, bibliographic source, date released and guideline developer. We then extracted information related to five key components for developing risk-stratified treatment recommendations (Figure 1). We extracted the following information on risk assessment tools: the name of the prediction model, the outcome and the timeframe (for example, 10 years) used in the model, and whether validation of the model (for example, assessment of discrimination and/or calibration) was discussed in the CPGs. We extracted information on the type of evidence used to determine the effects of treatments on benefit and harm outcomes (observational studies, single or several randomized controlled trials (RCTs), or meta-analyses). We recorded the methods to link risk prediction and evidence on treatment effects (for example, applying relative risk reductions to different absolute risks calculated from the risk assessment tool). We recorded the way the treatment benefits and harms were assessed and how treatment thresholds (based on risk assessment tools) were determined. We also extracted information on assumptions made for linking risk prediction and treatment evidence (for example, assumption of constant relative risk reductions across the risk spectrum) and on assumptions made for the assessment of benefits and harms (for example, assumption that benefit and harm outcomes can be put on a single scale and the overall net benefit expressed as a single number indicating benefit or harm). Finally, we noted whether patient preferences (for example, relative importance of different benefit and harm outcomes) were considered for developing risk-stratified treatment recommendations. Because some CPGs were very brief, without detailing the development process but referring to other documents, we considered those documents for data extraction to avoid underestimating the rigor of the development process of a CPG. Two reviewers (TY and DV) independently extracted all relevant information from each CPG and the discrepancies were resolved by discussion or third-party (MP) arbitration. We constructed a table for comparison of recommendations from each of the included CPGs.


Most CPGs that we excluded (Figure 2) were on topics not related to our study question or because they were not from the US, Canada or NICE (UK). We excluded 60 CPGs based on NGC website review and 49 CPGs based on the full text because they did not recommend using a risk assessment tool. We excluded four additional CPGs because they recommended using a risk assessment tool but did not make any link to treatments (Figure 2). Thus out of 133 CPGs that made treatment recommendations for the chronic diseases of interest (60 + 49 + 4 + 20 = 133 CPGs), 20 made risk-stratified treatment recommendations (15%) for heart disease, stroke, type II diabetes or breast cancer (Figure 2) [10,17-36]. The characteristics of the 20 included CPGs are summarized in Table 1.

thumbnailFigure 2. Guideline search and review process. 1For cardiovascular diseases, we excluded guidelines not focused on primary prevention. 2Two citations in Diabetes Mellitus, Type 2 were the same guidelines as in Cardiovascular Diseases. NGC: US National Guideline Clearinghouse; NICE: UK National Institute for Health and Clinical Excellence.

Table 1. Characteristics of the included guidelines.

Risk assessment tools used to estimate baseline risk for outcome of interest

A large proportion of CPGs (16 out of 20, 80%) were on type II diabetes or on primary prevention of heart disease and stroke [10,18-32], and the remaining four CPGs were on breast cancer management [33-36]. All 16 CPGs on diabetes or cardiovascular disease recommended one or several risk assessment tools to assess 10-year cardiovascular disease risk. The Framingham Risk Score was explicitly suggested in 12 CPGs [10,19-29]; the UK Prospective Diabetes Study Risk Engine was suggested in four CPGs [21,24,30,31] for patients with diabetes, and the Prospective Cardiovascular Münster and the Systematic Coronary Risk Evaluation risk tools were used in one CPG [29]. Of the 16 CPGs on diabetes or cardiovascular disease, two (13%) did not clearly specify the prediction model used for calculating 10-year cardiovascular disease risk [18,32]. Among the four CPGs on breast cancer [33-36], two recommended using the National Cancer Institute breast cancer risk assessment tool to calculate 5-year risk of invasive breast cancer [33,35]; one recommended using the Nottingham Prognostic Index to calculate 10-year survival [36]; one mentioned different risk assessment tools but did not clearly define risk categories [34]. Information on the validation of the risk models was reported in seven (35%) of the 20 included CPGs (Table 2) [10,19,20,22,29,30,35].

Table 2. Risk-stratified treatment recommendations of the included guidelines.

Treatments recommended and evidence of treatment benefits and harms

Of the 16 CPGs for type II diabetes and primary prevention of heart disease and stroke, nine (56%) suggested specific target lipid levels for each risk category when making recommendations about lifestyle management or pharmacotherapy (for example, aspirin, statins and antihypertensive drugs) [10,19-21,26-29,31]. The four CPGs on breast cancer [33-36] provided recommendations on surgery or pharmacotherapy (for example, tamoxifen, raloxifene and aromatase inhibitors) according to risk levels.

Sixteen (80%) of the 20 CPGs reported using evidence on treatment benefits from RCTs or meta-analyses or other guidelines [10,18-25,27,29-32,35,36]. One CPG (5%) did not report quantitative information on treatment benefits [28] and three (15%) did not clearly specify the type of studies considered [26,33,34]. Treatment harms were only reported in 13 of the 20 CPGs (65%) [10,18,22,23,25-27,29,30,32,33,35,36]. The source of evidence on harms was specified in nine out of these 13 CPGs (69%) and included observational studies, RCTs and meta-analyses [10,18,32,23,25,27,30,35,36]. Heterogeneity of treatment effects were assessed in eight (40%) of the included 20 CPGs (Table 2) [10,18,22,23,25,26,32,35].

Linking treatment effects to baseline risks

In reviewing how CPGs made the link between risk prediction and treatment effects, we found fewer than half of the CPGs (eight out of 20, 40%) explicitly or implicitly stated that they applied evidence of relative risk reductions from RCTs and/or meta-analyses to different absolute risks [10,18,23-27,32]. For instance, the U.S. Preventive Services Task Force (USPSTF) guideline [25] applied a 32% risk reduction of myocardial infarction (in men) and a 17% risk reduction of strokes (in women) with regular aspirin use to absolute outcome risks and assumed that the effects were constant across risk levels and age categories. One (5%) of the 20 included CPGs [33], instead of applying treatment evidence to all risk levels, used the evidence from RCTs with the same risk profile (high breast cancer risk) population for which the recommendation was made. Eleven (55%) of the included CPGs did not report the way in which they linked risk prediction to treatment effects (Table 2) [19-22,28-31,34-36].

Benefit-harm assessment to define treatment thresholds and consideration of patient preferences

Only a small proportion (two out of 20, 10% [25,35]) of CPGs explicitly stated that they planned to perform benefit and harm assessment as the basis for making risk-stratified treatment recommendations. To define treatment thresholds, only the USPSTF guideline quantitatively weighed treatment benefits and harms by putting the expected benefit and harm outcomes on the same scale (events per 1,000 persons treated over 10 years). The USPSTF guideline recommended using aspirin when the treatment benefits (number of myocardial infarctions or strokes prevented per 1,000 persons treated over 10 years) outweigh the treatment harms (number of gastrointestinal bleedings or hemorrhagic strokes per 1,000 persons treated over 10 years). For example, the expected number of myocardial infarctions prevented by aspirin was estimated to be 16 per 1,000 men of age 60 to 69 years if men had a 10-year risk for myocardial infarction of 5%, while the expected number of excess gastrointestinal bleedings was 24 and hemorrhagic strokes was one. Because the number of excess events exceeded the number of prevented myocardial infarctions, the USPSTF recommended against the use of aspirin in men at 5% risk for myocardial infarction and an age of 60 to 69 years. Based on observational studies, the USPSTF assumed different risks for gastrointestinal bleeding with aspirin according to age. Finally, the USPSTF presented their benefit-harm assessment and the resulting treatment thresholds as a matrix table with categories for age and risk for myocardial infarction defining each cell.

Three (15%) of the 20 CPGs qualitatively weighed the treatment benefits and harms [23,29,32]. Nine (45%) of the 20 CPGs made the recommendation on thresholds based on expert consensus or referred to other guidelines [18,19,21,22,26-28,33,33]. Seven (35%) of the 20 CPGs did not report how they determined the treatment thresholds when making recommendations [10,20,24,30,31,34,36]. With regard to involving patient preferences when developing treatment recommendations, only three (15%) of the 20 CPGs explicitly reported that they considered patient preferences in the process (Table 2) [25,30,36]. For example, the USPSTF focused on major benefit (myocardial infarction) and harm events (gastrointestinal bleeding and hemorrhagic stroke) and assumed equal preferences (that is, importance) for those outcomes.


We found a rather small proportion of CPGs for heart disease, cancer, stroke, COPD and diabetes that made risk-stratified treatment recommendations using risk assessment tools. Most of these CPGs recommend risk assessment tools that had been shown to accurately predict outcome risk in the target population of the CPGs and most of the treatment evidence is based on RCTs and meta-analyses. For the majority of the CPGs, however, it was not explicitly explained how treatment effects on benefit and harm outcomes were estimated for patients at different risks. Perhaps most importantly, it was unclear for all but one CPG how treatment thresholds were determined to generate risk-stratified treatment recommendations.

We formed a framework for the development of risk-stratified treatment recommendations (Figure 1) to systematically identify the strengths and weaknesses of current CPGs. Our findings suggest that risk assessment tools were carefully appraised and selected during the development of CPGs. For example, some CPG developers critically appraised validation studies of risk tools to judge their calibration (agreement between predicted and observed risk) and discrimination (probability that those with an event receive higher risk predictions that those without an event) [10,30]. Minimizing misclassification of outcome risks is important to avoid over- or under-treatment [37-39]. While some CPGs recommended specific risk assessment tools, one CPG suggested using the risk assessment tool that is most likely to be accurate in the specific population of interest [30]. However, the set of CPGs selected in this study may give an overoptimistic picture of risk assessment tools proposed by guidelines. For many diseases and geographical locations other than the US, Canada and the UK, calibrated and discriminative risk assessment tools may not exist. A strength of existing CPGs is that the majority of them relied on RCTs and meta-analyses of RCTs for intervention effectiveness. The CPG developers recognized limitations within this body of evidence, including insufficient evidence on treatment heterogeneity (that is, subgroup effects) and scarcity of data on harm outcomes.

We discovered a number of major limitations in how CPGs develop risk-stratified treatment recommendations. It should be noted that some limitations propagated from single, prominent CPG (for example, National Cholesterol Education Program) to other CPGs that adopted the approach or even the recommendations. For example, it was often unclear how the benefit and harm outcomes were estimated for different risk profiles. Some CPGs applied estimates on relative risk reduction to absolute risks. This approach relies on the assumption of constant (relative) effects across the risk spectrum. This assumption of constant relative treatment effects may be justifiable in many instances but it is usually difficult to verify. No alternative approaches for linking the absolute risk with treatment evidence were used. Additional sensitivity analyses may sometimes be appropriate to explore the assumption of relative treatment effects. For example, one could obtain risk-specific treatment estimates from large trials using individual patient data [12]. Or, one could employ simulation studies to estimate the probability of outcomes in the population of interest by combining observational data and treatment effects from randomized trials. It is currently unclear what the most appropriate approach is to link risk predictions with evidence from randomized trials. Nevertheless, we believe CPGs should be explicit about the method they use and acknowledge the associated advantages and limitations (for example, assumption of constant relative risk reduction).

In our view, the greatest limitation of current CPGs is that it is unclear how treatment thresholds were developed for most of them. Some CPGs stated that the thresholds were determined by experts. The USPSTF guideline on aspirin [25] was the only guideline that conducted a formal quantitative assessment by comparing the expected number of benefit and harm events for patients at different risk for myocardial infarction and major gastrointestinal bleeding. We believe that transparency will be enhanced by conducting quantitative benefit-harm assessments alongside more qualitative approaches, such as using expert consensus about treatment thresholds.

Treatment thresholds are important because medical decision-making is discrete (to treat the patient or not). It is challenging to determine thresholds because clear cuts on the (commonly) continuous benefit-harm scale may not exist. In addition, there may often be substantial uncertainty about harms and heterogeneity of treatment effects as a consequence of poor reporting or a lack of evidence from primary studies. However, this should, in our view, not prevent CPG developers from making risk-stratified recommendations because health care providers need evidence-based guidance nevertheless and because variability in delivering health care may be unacceptably high in the absence of guidance. Quanstrum and Hayward [40] recently suggested an approach that acknowledges uncertainty about treatment decision thresholds and proposed two thresholds instead of one: one above which physicians should recommend treatments (benefits outweighing harms irrespective of patient preferences and uncertainties about evidence base) and one below which physicians should recommend against treatments (harms outweighing benefits). The interval between the two thresholds represents an area where treatment could provide small benefits or harms depending on patient preferences but also where uncertainty about the evidence precludes CPG developers from making recommendations. Alternatively, CPG developers could frame strong recommendations for or against treatment for patients at outcomes risks above or below the two thresholds, respectively, and weak recommendations for patients at outcome risks between the two thresholds [41].

One may criticize the approach used by the USPSTF, assigning equal weight to benefit and harm outcomes to calculate events expected per 1,000 people treated over 10 years, because empirical evidence suggests that patients, on average, assign different importance to myocardial infarction, major gastrointestinal bleeding and major stroke, the major drivers of the benefit-harm balance of aspirin [42]. Nevertheless, such transparency about the relative importance of outcomes comes with several important advantages. Users of CPGs can understand and replicate how the treatment thresholds were derived and, if they do not agree with certain assumptions (for example, equal importance of myocardial infarction and major gastrointestinal bleeding), they can adjust the result to derive thresholds that would suit their settings (for example, myocardial infarction considered twice as important as major gastrointestinal bleeding). This would also allow the guideline to be interpreted for an individual patient, who may weigh the various outcomes differently than those preferences assumed in the CPG.

The framework for developing risk-stratified treatment recommendation we proposed may be useful for those developing CPGs and to stimulate further research. While much research has been done on how to select and appraise evidence on treatment benefits and harms [43,44] and how to judge the validity of prediction models [37-39], it is less clear how to link risk prediction and treatment evidence, how to select a method for benefit-harm assessment to develop treatment thresholds, and how to include patient preferences. It would be useful to have empirical evidence on how the results of different approaches for linking risk prediction and treatment evidence and for defining treatment thresholds differ and how sensitive they are to assumptions [45]. As for patient preferences, little research has been done to find ways to include stakeholders in the process of selecting important outcomes, or a benefit-harm assessment method that provides the information patients need in order to make decisions [46-48]. The newly founded Patient-Centered Outcomes Research Institute is likely to contribute substantially to the questions raised.

Our study has some weaknesses. We selected guidelines from five major disease categories and from one database and focused on CPGs from the US, Canada and NICE (UK). Thus our results may not be generalizable, but provide an optimistic assessment of CPGs because we included some of the most prominent guidelines in medicine. For the fields of cardiovascular medicine and diabetes, guideline developers have a long tradition of making risk-stratified treatment recommendations. We relied on published reports, which may not reflect the true underlying development process for CPGs. We considered all background documents that were openly accessible but we may have missed some information on the development of risk-stratified treatment recommendations.


We found that the methods for linking risk prediction with treatment evidence are often not reported and that it was unclear for all but one CPG how treatment thresholds were developed. Therefore, current CPGs for major chronic diseases may not support patients and physicians in finding an acceptable benefit-harm balance that reflects profile-specific outcome risks and preferences.


COPD: chronic obstructive pulmonary disease; CPG: clinical practice guideline; NGC: National Guideline Clearinghouse; NICE: National Institute for Health and Clinical Excellence; RCT: randomized controlled trial; USPSTF: U.S. Preventive Services Task Force.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

All authors contributed to the conception of the study; TY, DV and MP collected and analyzed the data; all authors interpreted the data; TY and MP drafted the article; all authors revised it critically for important intellectual content; and all authors read and approved the final manuscript.


This publication was made possible by Grant Number UL1 RR 025005 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. CB's effort was supported by the Paul Beeson Career Development Award Program (NIA K23 AG032910, AFAR, The John A. Hartford Foundation, The Atlantic Philanthropies, The Starr Foundation and an anonymous donor). MP had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.


  1. Kent DM, Hayward RA: Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification.

    JAMA 2007, 298:1209-1212. PubMed Abstract | Publisher Full Text OpenURL

  2. Falk JA, Minai OA, Mosenifar Z: Inhaled and systemic corticosteroids in chronic obstructive pulmonary disease.

    Proc Am Thorac Soc 2008, 5:506-512. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  3. Puhan MA, Bachmann LM, Kleijnen J, Ter Riet G, Kessels AG: Inhaled drugs to reduce exacerbations in patients with chronic obstructive pulmonary disease: a network meta-analysis.

    BMC Med 2009, 7:2. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  4. The Global Initiative for Chronic Obstructive Lung Disease (GOLD): Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease. [] webcite

    Bethesda (MD): Global Initiative for Chronic Obstructive Lung Disease (GOLD); 2011.

  5. Loke YK, Cavallazzi R, Singh S: Risk of fractures with inhaled corticosteroids in COPD: systematic review and meta-analysis of randomised controlled trials and observational studies.

    Thorax 2011, 66:699-708. PubMed Abstract | Publisher Full Text OpenURL

  6. Singh S, Amin AV, Loke YK: Long-term use of inhaled corticosteroids and the risk of pneumonia in chronic obstructive pulmonary disease: a meta-analysis.

    Arch Intern Med 2009, 169:219-229. PubMed Abstract | Publisher Full Text OpenURL

  7. Eddy DM, Adler J, Patterson B, Lucas D, Smith KA, Morris M: Individualized guidelines: the potential for increasing quality and reducing costs.

    Ann Intern Med 2011, 154:627-634. PubMed Abstract | Publisher Full Text OpenURL

  8. Wennberg JE: Tracking Medicine: A Researcher's Quest to Understand Health Care. Ne York, USA: Oxford University Press; 2010. PubMed Abstract OpenURL

  9. 27th Bethesda conference. Matching the intensity of risk factor management with the hazard for coronary disease events. September 14-15, 1995

    J Am Coll Cardiol 1996, 27:957-1047. PubMed Abstract OpenURL

  10. National Cholesterol Education Program (NCEP) Expert Panel on Detection, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III): Third report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment anel III) final report.

    Circulation 2002, 106:3143-3421. PubMed Abstract | Publisher Full Text OpenURL

  11. Dorresteijn JA, Visseren FL, Ridker PM, Paynter NP, Wassink AM, Buring JE, van der Graaf Y, Cook NR: Aspirin for primary prevention of vascular events in women: individualized prediction of treatment effects.

    Eur Heart J 2011, 32:2962-2969. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  12. Dorresteijn JA, Visseren FL, Ridker PM, Wassink AM, Paynter NP, van der Graaf Y, Cook NR: Estimating treatment effects for individual patients based on the results of randomised clinical trials.

    BMJ 2011, 343:d5888. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  13. Hill JC, Whitehurst DG, Lewis M, Bryan S, Dunn KM, Foster NE, Konstantinou K, Main CJ, Mason E, Somerville S, Sowden G, Vohora K, Hay EM: Comparison of stratified primary care management for low back pain with current best practice (STarT Back): a randomised controlled trial.

    Lancet 2011, 378:1560-1571. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  14. Centers for Disease Control and Prevention: The Burden of Chronic Diseases and Their Risk Factors: National and State Perspectives 2004. Atlanta (GA): Centers for Disease Control and Prevention; 2005. OpenURL

  15. US Cancer Statistics Working Group: [] webcite

    United States Cancer Statistics: 1999-2007 Incidence and Mortality Web-based Report. Atlanta (GA): Department of Health and Human Services, Centers for Disease Control and Prevention, and National Cancer Institute; 2010. OpenURL

  16. D'Amico AV, Whittington R, Malkowicz SB, Schultz D, Blank K, Broderick GA, Tomaszewski JE, Renshaw AA, Kaplan I, Beard CJ, Wein A: Biochemical outcome after radical prostatectomy, external beam radiation therapy, or interstitial radiation therapy for clinically localized prostate cancer.

    JAMA 1998, 280:969-974. PubMed Abstract | Publisher Full Text OpenURL

  17. Grundy SM, Cleeman JI, Merz CN, Brewer HB Jr, Clark LT, Hunninghake DB, Pasternak RC, Smith SC Jr, Stone NJ, National Heart, Lung, and Blood Institute, American College of Cardiology Foundation, American Heart Association: Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines.

    Circulation 2004, 110:227-239. PubMed Abstract | Publisher Full Text OpenURL

  18. National Institute for Health and Clinical Excellence (NICE): [] webcite

    Statins for the Prevention of Cardiovascular Events. London (UK): National Institute for Health and Clinical Excellence (NICE); 2006. OpenURL

  19. Goldstein LB, Adams R, Alberts MJ, Goldstein LB, Adams R, Alberts MJ, Appel LJ, Brass LM, Bushnell CD, Culebras A, DeGraba TJ, Gorelick PB, Guyton JR, Hart RG, Howard G, Kelly-Hayes M, Nixon JV, Sacco RL: Primary prevention of ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council: cosponsored by the Atherosclerotic Peripheral Vascular Disease Interdisciplinary Working Group; Cardiovascular Nursing Council; Clinical Cardiology Council; Nutrition, Physical Activity, and Metabolism Council; and the Quality of Care and Outcomes Research Interdisciplinary Working Group.

    Circulation 2006, 113:e873-923. PubMed Abstract | Publisher Full Text OpenURL

  20. Mosca L, Banka CL, Benjamin EJ, Berra K, Bushnell C, Dolor RJ, Ganiats TG, Gomes AS, Gornik HL, Gracia C, Gulati M, Haan CK, Judelson DR, Keenan N, Kelepouris E, Michos ED, Newby LK, Oparil S, Ouyang P, Oz MC, Petitti D, Pinn VW, Redberg RF, Scott R, Sherif K, Smith SC Jr, Sopko G, Steinhorn RH, Stone NJ, Taubert KA, et al.: Evidence-based guidelines for cardiovascular disease prevention in women: 2007 update.

    Circulation 2007, 115:1481-1501. PubMed Abstract | Publisher Full Text OpenURL

  21. Medical Services Commission: [] webcite

    Cardiovascular Disease - Primary Prevention. Victoria (BC): British Columbia Medical Services Commission; 2008. OpenURL

  22. National Collaborating Centre for Primary Care: [] webcite

    Lipid modification. Cardiovascular Risk Assessment and the Modification of Blood Lipids for the Primary and Secondary Prevention of Cardiovascular Disease. London (UK): National Institute for Health and Clinical Excellence (NICE); 2008. OpenURL

  23. Becker RC, Meade TW, Berger PB, Ezekowitz M, O'Connor CM, Vorchheimer DA, Guyatt GH, Mark DB, Harrington RA, American College of Chest Physicians: The primary and secondary prevention of coronary artery disease: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition).

    Chest 2008, 133:776S-814S. PubMed Abstract | Publisher Full Text OpenURL

  24. Medical Services Commission: [] webcite

    Hypertension - Detection, Diagnosis and Management. Victoria (BC): British Columbia Medical Services Commission; 2008. OpenURL

  25. U.S. Preventive Services Task Force: Aspirin for the prevention of cardiovascular disease: U.S. Preventive Services Task Force recommendation statement.

    Ann Intern Med 2009, 150:396-404. PubMed Abstract | Publisher Full Text OpenURL

  26. University of Michigan Health System: [] webcite

    Screening and Management of Lipids. Ann Arbor (MI): University of Michigan Health System; 2009. OpenURL

  27. Institute for Clinical Systems Improvement (ICSI): [http:/ / guidelines_and_more/ gl_os_prot/ cardiovascular/ lipid_management_3/ lipid_management_in_adults__4.html] webcite

    Lipid Management in Adults. Bloomington (MN): Institute for Clinical Systems Improvement (ICSI); 2009. OpenURL

  28. Michigan Quality Improvement Consortium: [http:/ / pdf/ mqic_screening_and_management_of_hy percholesterolemia_cpg.pdf] webcite

    Screening and Management of Hypercholesterolemia. Southfield (MI): Michigan Quality Improvement Consortium; 2009. OpenURL

  29. Rosenzweig JL, Ferrannini E, Grundy SM, Haffner SM, Heine RJ, Horton ES, Kawamori R, Endocrine Society: Primary prevention of cardiovascular disease and type 2 diabetes in patients at metabolic risk: an Endocrine Society Clinical Practice Guideline.

    J Clin Endocrinol Metab 2008, 93:3671-3689. PubMed Abstract | Publisher Full Text OpenURL

  30. National Collaborating Centre for Chronic Conditions: [] webcite

    Type 2 Diabetes: National Clinical Guideline for Management in Primary and Secondary Care (update). London (UK): National Institute for Health and Clinical Excellence (NICE); 2008. OpenURL

  31. Medical Services Commission: [] webcite

    Diabetes Care. Victoria (BC): British Columbia Medical Services Commission; 2010. OpenURL

  32. American Diabetes Association: Standards of medical care in diabetes--2011.

    Diabetes Care 2011, 34:S11-S61. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  33. University of Michigan Health System: Common Breast Problems. [] webcite

    Ann Arbor (MI): University of Michigan Health System; 2007.

  34. Berliner JL, Fay AM, Practice Issues Subcommittee of the National Society of Genetic Counselors' Familial Cancer Risk Counseling Special Interest Group: Risk assessment and genetic counseling for hereditary breast and ovarian cancer: recommendations of the National Society of Genetic Counselors.

    J Genet Couns 2007, 16:241-260. PubMed Abstract | Publisher Full Text OpenURL

  35. Visvanathan K, Chlebowski R, Hurley P, Col NF, Ropka M, Collyar D, Morrow M, Runowicz C, Pritchard KI, Hagerty K, Arun B, Garber J, Vogel V, Wade JL, Brown P, Cuzick J, Kramer BS, Lippman SM: American Society of Clinical Oncology Clinical Practice Guideline update on the use of pharmacologic interventions including tamoxifen, raloxifene, and aromatase inhibition for breast cancer risk reduction.

    J Clin Oncol 2009, 27:3235-3258. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  36. National Collaborating Centre for Cancer: [] webcite

    Early and Locally Advanced Breast Cancer: Diagnosis and Treatment. London (UK): National Institute for Health and Clinical Excellence (NICE); 2009. OpenURL

  37. Altman DG, Royston P: What do we mean by validating a prognostic model?

    Stat Med 2000, 19:453-473. PubMed Abstract | Publisher Full Text OpenURL

  38. Altman DG, Vergouwe Y, Royston P, Moons KG: Prognosis and prognostic research: validating a prognostic model.

    BMJ 2009, 338:b605. PubMed Abstract | Publisher Full Text OpenURL

  39. Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, Moons KG: External validation is necessary in prediction research: a clinical example.

    J Clin Epidemiol 2003, 56:826-832. PubMed Abstract | Publisher Full Text OpenURL

  40. Quanstrum KH, Hayward RA: Lessons from the mammography wars.

    N Engl J Med 2010, 363:1076-1079. PubMed Abstract | Publisher Full Text OpenURL

  41. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, Schünemann HJ, GRADE Working Group: Going from evidence to recommendations.

    BMJ 2008, 336:1049-1051. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  42. Pignone M, Earnshaw S, Pletcher MJ, Tice JA: Aspirin for the primary prevention of cardiovascular disease in women: a cost-utility analysis.

    Arch Intern Med 2007, 167:290-295. PubMed Abstract | Publisher Full Text OpenURL

  43. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH: GRADE guidelines: 3. Rating the quality of evidence.

    J Clin Epidemiol 2011, 64:401-406. PubMed Abstract | Publisher Full Text OpenURL

  44. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, Montori V, Akl EA, Djulbegovic B, Falck-Ytter Y, Norris SL, Williams JW Jr, Atkins D, Meerpohl J, Schünemann HJ: GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias).

    J Clin Epidemiol 2011, 64:407-415. PubMed Abstract | Publisher Full Text OpenURL

  45. Guo JJ, Pandey S, Doyle J, Bian B, Lis Y, Raisch DW: A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy - report of the ISPOR risk-benefit management working group.

    Value Health 2010, 13:657-666. PubMed Abstract | Publisher Full Text OpenURL

  46. Institute of Medicine of the National Academies: [] webcite

    Finding What Works in Health Care. Standards for Systematic Reviews. Washington, DC: The National Academies Press; 2011. OpenURL

  47. Kreis J, Puhan MA, Schünemann HJ, Dickersin K: Consumer involvement in systematic reviews of comparative effectiveness research.

    Health Expect 2012, in press. OpenURL

  48. Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD: Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material.

    Cochrane Database Syst Rev 2006, 3:CD004563. PubMed Abstract | Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here: