Voltar
Artigo Revisado por pares
BIBTEX RIS

Development and Validation of a Cardiovascular Risk Assessment Model in Patients With Established Coronary Artery Disease

2013; Elsevier BV; Volume: 112; Issue: 1 Linguagem: Inglês

10.1016/j.amjcard.2013.02.049

ISSN

1879-1913

Autores

Linda C. Battes, Rogier Barendse, Ewout W. Steyerberg, Maarten L. Simoons, Jaap W. Deckers, Daan Nieboer, Michel E. Bertrand, Roberto Ferrari, Willem J. Remme, Keith A.A. Fox, Johanna J.M. Takkenberg, Eric Boersma, Isabella Kardys,

Tópico(s)

Cardiovascular Function and Risk Factors

Resumo

Appropriate risk stratification of patients with established, stable coronary artery disease could contribute to the prevention of recurrent cardiovascular events. The purpose of the present study was to develop and validate risk prediction models for various cardiovascular end points in the EURopean trial On reduction of cardiac events with Perindopril in stable coronary Artery disease (EUROPA) database, consisting of 12,218 patients with established coronary artery disease, with a median follow-up of 4.1 years. Cox proportional hazards models were used for model development. The end points examined were cardiovascular mortality, noncardiovascular mortality, nonfatal myocardial infarction, coronary artery bypass grafting, percutaneous coronary intervention, resuscitated cardiac arrest, and combinations of these end points. The performance measures included Nagelkerke's R2, time-dependent area under the receiver operating characteristic curves, and calibration plots. Backward selection resulted in a prediction model for cardiovascular mortality (464 events) containing age, current smoking, diabetes mellitus, total cholesterol, body mass index, previous myocardial infarction, history of congestive heart failure, peripheral vessel disease, previous revascularization, and previous stroke. The model performance was adequate for this end point, with a Nagelkerke R2 of 12%, and an area under the receiver operating characteristic curve of 0.73. However, the performance of models constructed for nonfatal and combined end points was considerably worse, with an area under the receiver operating characteristic curve of about 0.6. In conclusion, in patients with established coronary artery disease, the risk of cardiovascular mortality during longer term follow-up can be adequately predicted using the clinical characteristics available at baseline. However, the prediction of nonfatal outcomes, both separately and combined with fatal outcomes, poses major challenges for clinicians and model developers. Appropriate risk stratification of patients with established, stable coronary artery disease could contribute to the prevention of recurrent cardiovascular events. The purpose of the present study was to develop and validate risk prediction models for various cardiovascular end points in the EURopean trial On reduction of cardiac events with Perindopril in stable coronary Artery disease (EUROPA) database, consisting of 12,218 patients with established coronary artery disease, with a median follow-up of 4.1 years. Cox proportional hazards models were used for model development. The end points examined were cardiovascular mortality, noncardiovascular mortality, nonfatal myocardial infarction, coronary artery bypass grafting, percutaneous coronary intervention, resuscitated cardiac arrest, and combinations of these end points. The performance measures included Nagelkerke's R2, time-dependent area under the receiver operating characteristic curves, and calibration plots. Backward selection resulted in a prediction model for cardiovascular mortality (464 events) containing age, current smoking, diabetes mellitus, total cholesterol, body mass index, previous myocardial infarction, history of congestive heart failure, peripheral vessel disease, previous revascularization, and previous stroke. The model performance was adequate for this end point, with a Nagelkerke R2 of 12%, and an area under the receiver operating characteristic curve of 0.73. However, the performance of models constructed for nonfatal and combined end points was considerably worse, with an area under the receiver operating characteristic curve of about 0.6. In conclusion, in patients with established coronary artery disease, the risk of cardiovascular mortality during longer term follow-up can be adequately predicted using the clinical characteristics available at baseline. However, the prediction of nonfatal outcomes, both separately and combined with fatal outcomes, poses major challenges for clinicians and model developers. Several cardiovascular risk stratification models are currently available for primary prevention setting, such as the Framingham risk score, the SCORE project, Prospective Cardiovascular Münster Study (PROCAM) and QRISK.1Conroy R.M. Pyorala K. Fitzgerald A.P. Sans S. Menotti A. De Backer G. De Bacquer D. Ducimetiere P. Jousilahti P. Keil U. Njolstad I. Oganov R.G. Thomsen T. Tunstall-Pedoe H. Tverdal A. Wedel H. Whincup P. Wilhelmsen L. Graham I.M. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project.Eur Heart J. 2003; 24: 987-1003Crossref PubMed Scopus (4248) Google Scholar, 2Voss R. Cullen P. Schulte H. Assmann G. Prediction of risk of coronary events in middle-aged men in the Prospective Cardiovascular Münster Study (PROCAM) using neural networks.Int J Epidemiol. 2002; 31 (discussion 1262–1264): 1253-1262Crossref PubMed Scopus (115) Google Scholar, 3Kannel W.B. McGee D. Gordon T. A general cardiovascular risk profile: the Framingham study.Am J Cardiol. 1976; 38: 46-51Abstract Full Text PDF PubMed Scopus (881) Google Scholar, 4Hippisley-Cox J. Coupland C. Vinogradova Y. Robson J. May M. Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study.BMJ. 2007; 335: 136Crossref PubMed Scopus (717) Google Scholar However, the risk stratification models for patients with established coronary artery disease (CAD) are less abundant and have several limitations.5Clayton T.C. Lubsen J. Pocock S.J. Voko Z. Kirwan B.A. Fox K.A. Poole-Wilson P.A. Risk score for predicting death, myocardial infarction, and stroke in patients with stable angina, based on a large randomised trial cohort of patients.BMJ. 2005; 331: 869Crossref PubMed Scopus (113) Google Scholar, 6Prugger C. Wellmann J. Heidrich J. Brand-Herrmann S.M. Keil U. Cardiovascular risk factors and mortality in patients with coronary heart disease.Eur J Epidemiol. 2008; 23: 731-737Crossref PubMed Scopus (30) Google Scholar, 7Haverkate F. Thompson S.G. Pyke S.D. Gallimore J.R. Pepys M.B. Production of C-reactive protein and risk of coronary events in stable and unstable angina: European Concerted Action on Thrombosis and Disabilities Angina Pectoris Study Group.Lancet. 1997; 349: 462-466Abstract Full Text Full Text PDF PubMed Scopus (1421) Google Scholar, 8Marschner I.C. Colquhoun D. Simes R.J. Glasziou P. Harris P. Singh B.B. Friedlander D. White H. Thompson P. Tonkin A. Long-term risk stratification for survivors of acute coronary syndromes: results from the Long-term Intervention with Pravastatin in Ischemic Disease (LIPID) Study. LIPID Study Investigators.J Am Coll Cardiol. 2001; 38: 56-63Abstract Full Text Full Text PDF PubMed Scopus (108) Google Scholar, 9Unal B. Capewell S. Critchley J.A. Coronary heart disease policy models: a systematic review.BMC Public Health. 2006; 6: 213Crossref PubMed Scopus (64) Google Scholar, 10Marchioli R. Avanzini F. Barzi F. Chieffo C. Di Castelnuovo A. Franzosi M.G. Geraci E. Maggioni A.P. Marfisi R.M. Mininni N. Nicolosi G.L. Santini M. Schweiger C. Tavazzi L. Tognoni G. Valagussa F. Assessment of absolute risk of death after myocardial infarction by use of multiple-risk-factor assessment equations: GISSI-Prevenzione mortality risk chart.Eur Heart J. 2001; 22: 2085-2103Crossref PubMed Scopus (128) Google Scholar These limitations include a retrospective study design,6Prugger C. Wellmann J. Heidrich J. Brand-Herrmann S.M. Keil U. Cardiovascular risk factors and mortality in patients with coronary heart disease.Eur J Epidemiol. 2008; 23: 731-737Crossref PubMed Scopus (30) Google Scholar a lack of validation,9Unal B. Capewell S. Critchley J.A. Coronary heart disease policy models: a systematic review.BMC Public Health. 2006; 6: 213Crossref PubMed Scopus (64) Google Scholar a lack of uniformity in baseline characteristics (because of a long inclusion period during which changes in treatment recommendations have occurred),7Haverkate F. Thompson S.G. Pyke S.D. Gallimore J.R. Pepys M.B. Production of C-reactive protein and risk of coronary events in stable and unstable angina: European Concerted Action on Thrombosis and Disabilities Angina Pectoris Study Group.Lancet. 1997; 349: 462-466Abstract Full Text Full Text PDF PubMed Scopus (1421) Google Scholar and a focus on specific ethnic groups.10Marchioli R. Avanzini F. Barzi F. Chieffo C. Di Castelnuovo A. Franzosi M.G. Geraci E. Maggioni A.P. Marfisi R.M. Mininni N. Nicolosi G.L. Santini M. Schweiger C. Tavazzi L. Tognoni G. Valagussa F. Assessment of absolute risk of death after myocardial infarction by use of multiple-risk-factor assessment equations: GISSI-Prevenzione mortality risk chart.Eur Heart J. 2001; 22: 2085-2103Crossref PubMed Scopus (128) Google Scholar In the present study, we set out to develop and validate a series of risk prediction models for different end points in a prospective cohort of European patients with established CAD. Our cohort consisted of >12,000 patients, making this the largest study to date to develop such a model. The end points examined included cardiovascular mortality, noncardiovascular mortality, nonfatal myocardial infarction (MI), coronary artery bypass grafting (CABG), percutaneous coronary intervention (PCI), and resuscitated cardiac arrest, and combinations of these end points. The design of the EURopean trial On reduction of cardiac events with Perindopril in stable coronary Artery disease (EUROPA) study has been previously reported.11Fox K.M. EURopean trial On reduction of cardiac events with Perindopril in stable coronary Artery disease Investigators Efficacy of perindopril in reduction of cardiovascular events among patients with stable coronary artery disease: randomised, double-blind, placebo-controlled, multicentre trial (the EUROPA study).Lancet. 2003; 362: 782-788Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar In brief, this randomized, double-blind, placebo-controlled trial investigated the efficacy of perindopril in the reduction of cardiovascular events in 12,218 patients. Each patient provided informed consent, and the study protocol conformed to the ethical guidelines of the Declaration of Helsinki. The study participants consisted of men and women aged ≥18 years, with evidence of coronary heart disease documented by previous MI (>3 months before screening), percutaneous or surgical coronary revascularization (>6 months before screening), angiographic evidence of ≥70% narrowing of ≥1 major coronary artery, or, in men, a history of typical chest pain with abnormal stress test findings. All patients provided informed consent. The exclusion criteria included clinically evident (New York Heart Association class II or greater) heart failure, planned revascularization procedure, hypotension (sitting systolic blood pressure 180 mm Hg and/or diastolic blood pressure >100 mm Hg), use of angiotensin-converting enzyme inhibitors or angiotensin 2 receptor blockers in the past month, renal insufficiency (serum creatinine >150 μmol/L or 1.5 mg/dl), and serum potassium >5.5 mmol/L. The patients were randomly assigned to perindopril 8 mg or placebo once daily for ≥3 years. The first patient was enrolled in October 1997. At baseline, exposure data were collected for age, current smoking (patients who were current smokers or had smoked in the previous month), diastolic blood pressure, systolic blood pressure, heart rate, diabetes (known history of diabetes or the use of antidiabetic agents), total cholesterol, body mass index, family history of CAD, history of congestive heart failure, history of peripheral vessel disease, history of previous MI, history of previous revascularization, and previous stroke. The patients were followed up for cardiovascular mortality, noncardiovascular mortality, MI, CABG, PCI, and resuscitated cardiac arrest until March 2003. Intensive monitoring and end point validation was done by a clinical event committee.11Fox K.M. EURopean trial On reduction of cardiac events with Perindopril in stable coronary Artery disease Investigators Efficacy of perindopril in reduction of cardiovascular events among patients with stable coronary artery disease: randomised, double-blind, placebo-controlled, multicentre trial (the EUROPA study).Lancet. 2003; 362: 782-788Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar The median follow-up period was 4.1 years (interquartile range 4.0 to 4.5). For the development of risk prediction models with subsequent internal validation, several techniques are available, including the split-sample and bootstrap methods. According to the split-sample method, the original sample is randomly divided into 2 parts: a training set for model development and a testing set for model validation.12Steyerberg E.W. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. Springer, New York2008Google Scholar Bootstrap replicates the process of sample generation from an underlying population by drawing "bootstrap samples," with replacement from the original sample. The bootstrap samples usually have the same size as the original.13Visser M. [Roaming through methodology. XXXIV. Limitations of predictive models].Ned Tijdschr Geneeskd. 2001; 145: 1109-1112PubMed Google Scholar According to the bootstrap method, risk prediction models are developed on the original sample and validated in the set of bootstrap samples. The bootstrap method is preferred for small data sets. Our original sample was large; thus, we were confident to apply the split-sample method. The original sample was randomly divided into a training set of 8,144 patients (2/3 of the original sample) and a testing set of 4,074 patients. To develop the optimal risk prediction model, multivariate Cox proportional hazards analysis was applied in the training set. All available predictors were considered as potential determinants of the outcomes we studied. Backward stepwise selection was used for variable selection, because this has been argued to render reliable predictors.12Steyerberg E.W. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. Springer, New York2008Google Scholar Variable exclusion was performed using a 5% significance level as a stopping criterion. We examined the end points of cardiovascular mortality, noncardiovascular mortality, nonfatal MI, CABG, and PCI. Moreover, we examined the combination of cardiovascular mortality, nonfatal MI, and resuscitated cardiac arrest, which was originally the primary end point of the EUROPA study14Deckers J.W. Goedhart D.M. Boersma E. Briggs A. Bertrand M. Ferrari R. Remme W.J. Fox K. Simoons M.L. Treatment benefit by perindopril in patients with stable coronary artery disease at different levels of risk.Eur Heart J. 2006; 27: 796-801Crossref PubMed Scopus (58) Google Scholar (combined end point 1), and the combination of cardiovascular mortality, noncardiovascular mortality, nonfatal MI, CABG, PCI, and resuscitated cardiac arrest (combined end point 2). In the analysis of the combined end points, we applied censoring at the first moment that any 1 of the end point components occurred in a patient. In the analysis of the separate outcomes, we used time-dependent covariates to account for other nonfatal end points. For instance, when CABG was the end point of interest, we used nonfatal MI and PCI as time-dependent variables. Complete information was available for most variables. The values for total cholesterol, heart rate, history of MI and revascularization, and body mass index were missing in <5% of participants. Missing values were managed using expectation maximization,15Dempster A.P. Laird N.M. Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm.J Royal Stat Soc Series B (Stat Method). 1977; 39: 1-38Google Scholar an iterative method for finding maximum likelihood estimates of parameters in statistical models, in which the model depends on unobserved latent variables. The Statistical Package for Social Sciences, version 17.0, for Windows (SPSS, Chicago, Illinois) was used for these analyses. After deriving the models in the training set, we assessed their performance in the testing set. We used Nagelkerke's R2 to assess global model performance.12Steyerberg E.W. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. Springer, New York2008Google Scholar R2 is a likelihood-based measure that provides information about the goodness of fit of the model (i.e., how well the regression line estimates the real survival). There are several different definitions of R2.12Steyerberg E.W. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. Springer, New York2008Google Scholar The definition proposed by Nagelkerke can be readily applied to survival outcomes and has the advantage of being scaled from 1% to 100%. Of note is that the value of R2 depends in part on incidence of the outcome. A lower incidence results in lower values of R2, which thus should be interpreted in their appropriate context.16Chambless L.E. Cummiskey C.P. Cui G. Several methods to assess improvement in risk prediction models: extension to survival analysis.Stat Med. 2011; 30: 22-38Crossref PubMed Scopus (151) Google Scholar Subsequently, we assessed model discrimination for every end point by calculating area under the receiver operating characteristic curve (AUC). Model discrimination is the ability of the model to rank persons appropriately, from low to high risk. We calculated time-dependent AUCs using the statistical program R.17Available at: http://cran.r-project.org/web/packages/survivalROC/survivalROC.pdf. Accessed on December 20, 2012.Google Scholar This approach takes into account the follow-up time until event occurrence. Standard errors were calculated by bootstrapping. To assess differences in model discrimination between the training and testing sets, we compared the AUCs using chi-square tests.18Zweig M.H. Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.Clin Chem. 1993; 39: 561-577Crossref PubMed Scopus (5424) Google Scholar, 19Gonen M. Analyzing Receiver Operating Characteristic Curves with SAS. SAS Institute, Cary, North Carolina2007Google Scholar A 2-tailed probability <0.05 was considered a statistically significant result. Moreover, we investigated calibration, or how closely the predicted probabilities reflected actual risk. For this purpose, we compared observed survival, derived from Kaplan-Meier curves, with predicted survival, calculated from the Cox proportional hazards models. We constructed calibration plots according to the categories defined by deciles of predicted risk. The baseline characteristics are summarized in Table 1. The mean age was 60 years, and 85% were men. No significant differences were present between the training and testing sets. The incidence of the end points is listed in Table 2. The incidence of cardiovascular mortality was 9.6/1,000 person-years in the training set.Table 1Patient baseline characteristics stratified by training set and testing setVariableTraining Set (n = 8,144)Testing Set (n = 4,074)p ValueAge (yrs)60 ± 9.360 ± 9.40.89Men6,965 (86)3,474 (85)0.71Current smoker1,250 (15)612 (15)0.64Diastolic blood pressure (mm Hg)82 ± 882 ± 80.50Systolic blood pressure (mm Hg)137 ± 16137 ± 150.81Total cholesterol mmol/L5.4 ± 15.4 ± 10.66 mg/dl207.8 ± 40207.5 ± 400.66Diabetes mellitus1,021 (13)481 (12)0.25Body mass index (kg/m2)27 ± 3.527 ± 3.50.72Estimated glomerular filtration rate (ml/min/1.73 m2)75 ± 2075 ± 200.71Heart rate (beats/min)68 ± 1068 ± 100.18Peripheral vessel disease581 (7)302 (7)0.58Family history of coronary artery disease2,173 (27)1,155 (28)0.05Congestive heart failure105 (1)48 (1%)0.60Previous stroke154 (2)68 (2%)0.39Previous MI5,267 (65)2,643 (65)0.81Previous revascularization4,454 (55)2,255 (55)0.49Angiographic evidence of coronary artery disease∗Angiographic evidence of ≥70% narrowing of ≥1 major coronary artery.4,956 (61)2,433 (60)0.23History of typical chest pain†A history of typical chest pain with abnormal stress test findings (in men).916 (23)1,886 (23)0.40Data are presented as mean ± SD or n (%).∗ Angiographic evidence of ≥70% narrowing of ≥1 major coronary artery.† A history of typical chest pain with abnormal stress test findings (in men). Open table in a new tab Table 2Event rates of end point in training and testing setsOutcomeTotal Events (n)Event Ratep Value∗p Value for comparison of cumulative incidence for 4 years of follow-up.Training Set (per 1,000 Person-Yrs)Testing Set (per 1,000 Person-Yrs)Cardiovascular mortality4649.68.10.12Noncardiovascular mortality3236.16.80.32Nonfatal MI67313.612.40.26PCI67113.412.70.51CABG56411.610.00.12Combined end point 1†Combined end point 1 included cardiovascular mortality, nonfatal MI, and resuscitated cardiac arrest.1,09122.920.20.06Combined end point 2‡Combined end point 2 included cardiovascular mortality, noncardiovascular mortality, MI, CABG, PCI, and resuscitated cardiac arrest.2,18843.647.10.09∗ p Value for comparison of cumulative incidence for 4 years of follow-up.† Combined end point 1 included cardiovascular mortality, nonfatal MI, and resuscitated cardiac arrest.‡ Combined end point 2 included cardiovascular mortality, noncardiovascular mortality, MI, CABG, PCI, and resuscitated cardiac arrest. Open table in a new tab Data are presented as mean ± SD or n (%). In the training set, 16 potential variables were evaluated for model inclusion. The variables included in the best-fitting prediction model for cardiovascular mortality after backward selection were age, current smoking, diabetes mellitus, total cholesterol, body mass index, previous MI, a history of congestive heart failure, peripheral vessel disease, previous revascularization, and previous stroke (Table 3). The hazard ratios for the variables in this model are listed in Table 4. A smaller number of variables were included in the predictive model for noncardiovascular mortality after backward selection (i.e., age, current smoking, and heart rate). The variables included in the prediction models for MI, CABG, PCI, and the combined end points are also listed in Table 3.Table 3Prognostic models resulting from backward stepwise selection, with corresponding area under receiver operating characteristic curve (AUC) and Nagelkerke's R2 in training and testing setsEnd PointFull Model from Backward Stepwise SelectionTraining SetTesting Setp Value for Difference in AUC Between Training and Testing SetsAUC (95% CI)Nagelkerke R2 (%)AUC (95% CI)Nagelkerke R2 (%)Cardiovascular mortalityAge, smoking, DM, cholesterol, BMI, HF, previous MI, PVD, revascularization, stroke0.70 (0.69–0.71)100.73 (0.70–0.77)120.12Noncardiovascular mortalityAge, smoking, and heart rate0.69 (0.67–0.71)50.71 (0.69–0.73)8<0.001Nonfatal MIAge, smoking, DM, cholesterol, previous MI, family history of CAD, PVD0.60 (0.58–0.62)20.59 (0.56–0.62)20.42CABGAge, gender, DM, cholesterol, BMI, previous MI, family history of CAD, revascularization0.67 (0.64–0.68)50.65 (0.62–0.68)40.12PCIDM, renal function, revascularization0.55 (0.54–0.56)10.57 (0.53–0.59)10.08Combined end point 1∗Combined end point 1 included cardiovascular mortality, nonfatal MI, and resuscitated cardiac arrest.Age, gender, smoking, DM, cholesterol, DBP, renal function, previous MI, PVD, revascularization, stroke0.64 (0.63–0.66)50.63 (0.61–0.66)60.36Combined end point 2†Combined end point 2 included cardiovascular mortality, noncardiovascular mortality, MI, CABG, PCI, and resuscitated cardiac arrest.Age, gender, smoking, DM, cholesterol, family history of CAD, PVD, revascularization, stroke0.62 (0.60–0.63)40.61 (0.59–0.63)40.56BMI = body mass index; CI = confidence interval; DBP = diastolic blood pressure; DM = diabetes mellitus; HF = history of congestive heart failure; PVD = peripheral vessel disease.∗ Combined end point 1 included cardiovascular mortality, nonfatal MI, and resuscitated cardiac arrest.† Combined end point 2 included cardiovascular mortality, noncardiovascular mortality, MI, CABG, PCI, and resuscitated cardiac arrest. Open table in a new tab Table 4Hazard ratios for cardiovascular mortality for baseline variables included in prediction modelVariableHR (95% CI)p ValueAge (yrs)1.06 (1.04–1.07)<0.001Current smoker1.81 (1.37–2.38)<0.001Diabetes mellitus1.77 (1.36–2.30)<0.001Total cholesterol1.23 (1.12–1.36)<0.001Body mass index1.03 (1.00–1.07)0.04Previous MI1.74 (1.33–2.29)<0.001History of heart failure2.33 (1.38–3.94)0.002Peripheral vessel disease2.01 (1.49–2.71)<0.001Previous revascularization0.66 (0.53–0.83)<0.001Previous stroke2.39 (1.52–3.75)<0.001CI = confidence interval; HR = hazard ratio. Open table in a new tab BMI = body mass index; CI = confidence interval; DBP = diastolic blood pressure; DM = diabetes mellitus; HF = history of congestive heart failure; PVD = peripheral vessel disease. CI = confidence interval; HR = hazard ratio. Nagelkerke's R2 and time-dependent AUCs were calculated for each end point in the training set (Table 3). The overall performance, assessed with Nagelkerke's R2, was 10% for cardiovascular mortality. The overall performance was worse for the other outcomes. The model for cardiovascular mortality risk prediction displayed the best discrimination, with an AUC of 0.70. The prediction models for noncardiovascular mortality, MI, CABG, PCI, and combined end points 1 and 2 resulted in a lower AUC of 0.69, 0.60, 0.67, 0.55, 0.64, and 0.62, respectively. The models were subsequently validated in the testing set. Again, the values for Nagelkerke's R2 indicated good overall performance for cardiovascular mortality (12%; Table 3). For the other outcomes, the overall performance was worse, in accordance with the training set findings. Discrimination was comparable to the training set, including an AUC of 0.73 for cardiovascular mortality. The model for noncardiovascular mortality displayed slightly better discrimination in the testing set than in the training set. The mean and 95% confidence intervals of the AUCs for both sets are also listed in Table 3. Figure 1 shows the calibration plots for cardiovascular mortality, nonfatal MI, and combined end points 1 and 2 in the testing set. Because of the relatively low event rate in this study population (9.6/1,000 person-years for cardiovascular mortality and 13.6/1,000 person-years for nonfatal MI), the observed fraction of the population that remained event-free ranged from 0.8 to 1.0 for each decile of predicted probability of being free from cardiovascular mortality. These deciles of predicted probability of being event free also ranged from 0.7 upward. As such, the calibration plot only displayed values in the upper range of both the x- and y-axis. A similar situation occurred during evaluation of the prediction model for nonfatal MI and the models for the combined end points. It was not possible to construct calibration plots for noncardiovascular mortality, PCI, and CABG, because survival was even closer to 1.0 for these end points. As such, these calibration plots were not informative. In the present study, we developed and validated a comprehensive set of prediction models in patients with established, stable CAD to estimate the risk of cardiovascular mortality, noncardiovascular mortality, nonfatal MI, CABG, or PCI. The model for the prediction of cardiovascular mortality included age, current smoking, diabetes mellitus, total cholesterol, body mass index, previous MI, a history of congestive heart failure, peripheral vessel disease, previous revascularization, and previous stroke. This model displayed good overall performance, with a Nagelkerke R2 of 12%, and good discrimination, with an AUC of 0.73. We showed that cardiovascular mortality in patients with CAD can to a large extent be predicted by "established" clinical risk factors. These risk factors are easily obtainable during patient assessment. Earlier studies by St. John Sutton et al20St. John Sutton M. Pfeffer M.A. Moye L. Plappert T. Rouleau J.L. Lamas G. Rouleau J. Parker J.O. Arnold M.O. Sussex B. Braunwald E. Cardiovascular death and left ventricular remodeling two years after myocardial infarction: baseline predictors and impact of long-term use of captopril: information from the Survival and Ventricular Enlargement (SAVE) trial.Circulation. 1997; 96: 3294-3299Crossref PubMed Scopus (409) Google Scholar and Maas et al21Maas A.C. van Domburg R.T. Deckers J.W. Vermeer F. Remme W.J. Kamp O. Manger Cats V. Simoons M.L. Sustained benefit at 10-14 years follow-up after thrombolytic therapy in myocardial infarction.Eur Heart J. 1999; 20: 819-826Crossref PubMed Scopus (10) Google Scholar have implicated established risk factors in long-term survival after MI. These studies examined demographic variables, co-morbid conditions, and procedural variables and showed that age,20St. John Sutton M. Pfeffer M.A. Moye L. Plappert T. Rouleau J.L. Lamas G. Rouleau J. Parker J.O. Arnold M.O. Sussex B. Braunwald E. Cardiovascular death and left ventricular remodeling two years after myocardial infarction: baseline predictors and impact of long-term use of captopril: information from the Survival and Ventricular Enlargement (SAVE) trial.Circulation. 1997; 96: 3294-3299Crossref PubMed Scopus (409) Google Scholar, 21Maas A.C. van Domburg R.T. Deckers J.W. Vermeer F. Remme W.J. Kamp O. Manger Cats V. Simoons M.L. Sustained benefit at 10-14 years follow-up after thrombolytic therapy in myocardial infarction.Eur Heart J. 1999; 20: 819-826Crossref PubMed Scopus (10) Google Scholar a history of previous MI,20St. John Sutton M. Pfeffer M.A. Moye L. Plappert T. Rouleau J.L. Lamas G. Rouleau J. Parker J.O. Arnold M.O. Sussex B. Braunwald E. Cardiovascular death and left ventricular remodeling two years after myocardial infarction: baseline predictors and impact of long-term use of captopril: information from the Survival and Ventricular Enlargement (SAVE) trial.Circulation. 1997; 96: 3294-3299Crossref PubMed Scopus (409) Google Scholar, 21Maas A.C. van Domburg R.T. Deckers J.W. Vermeer F. Remme W.J. Kamp O. Manger Cats V. Simoons M.L. Sustained benefit at 10-14 years follow-up after thrombolytic therapy in myocardial infarction.Eur Heart J. 1999; 20: 819-826Crossref PubMed Scopus (10) Google Scholar congestive heart failure,20St. John Sutton M. Pfeffer M.A. Moye L. Plappert T. Rouleau J.L. Lamas G. Rouleau J. Parker J.O. Arnold M.O. Sussex B. Braunwald E. Cardiovascular death and left ventricular remodeling two years after myocardial infarction: baseline predictors and impact of long-term use of captopril: information from the Survival and Ventricular Enlargement (SAVE) trial.Circulation. 1997; 96: 3294-3299Crossref PubMed Scopus (409) Google Scholar left ventricular function,20St. John Sutton M. Pfeffer M.A. Moye L. Plappert T. Rouleau J.L. Lamas G. Rouleau J. Parker J.O. Arnold M.O. Sussex B. Braunwald E. Cardiovascular death and left ventricular remodeling two years after myocardial infarction: baseline predictors and impact of long-term use of captopril: information from the Survival and Ventricular Enlargement (SAVE) trial.Circulation. 1997; 96: 3294-3299Crossref PubMed Scopus (409) Google Scholar, 21Maas A.C. van Domburg R.T. Deckers J.W. Vermeer F. Remme W.J. Kamp O. Manger Cats V. Simoons M.L. Sustained benefit at 10-14 years follow-up after thrombolytic therapy in myocardial infarction.Eur Heart J. 1999; 20: 819-826Crossref PubMed Scopus (10) Google Scholar and multivessel disease21Maas A.C. van Domburg R.T. Deckers J.W. Vermeer F. Remme W.J. Kamp O. Manger Cats V. Simoons M.L. Sustained benefit at 10-14 years follow-up after thrombolytic therapy in myocardial infarction.Eur Heart J. 1999; 20: 819-826Crossref PubMed Scopus (10) Google Scholar were independent predictors of long-term mortality. Deckers et al14Deckers J.W. Goedhart D.M. Boersma E. Briggs A. Bertrand M. Ferrari R. Remme W.J. Fox K. Simoons M.L. Treatment benefit by perindopril in patients with stable coronary artery disease at different levels of risk.Eur Heart J. 2006; 27: 796-801Crossref PubMed Scopus (58) Google Scholar previously examined risk factors for cardiovascular events within the EUROPA study, using a combined end point. In the present report, we examined additional outcomes and explored aspects of model performance further. Although blood pressure was part of our prediction model for the combined end point, it did not predict cardiovascular mortality in our data. These seemingly discrepant findings could have resulted in part because a large part of the study population was using angiotensin-converting enzyme inhibitors and β blockers. We chose not to enter medical therapy into the models because their estimates are prone to confounding by indication.22Walker A.M. Confounding by indication.Epidemiology. 1996; 7: 335-336PubMed Google Scholar When we added medical therapy (β blockers, lipid-lowering agents, angiotensin-converting enzyme inhibitors, and platelet inhibitors) to the final models in a sensitivity analysis, the models remained materially the same; the estimates of the predictors did not change essentially, nor did the level of statistical significance. Although, as described in the previous paragraphs, several studies have examined the associations of risk factors with recurrent coronary events, the number of actual risk prediction models that have been developed for patients with established CAD is limited.5Clayton T.C. Lubsen J. Pocock S.J. Voko Z. Kirwan B.A. Fox K.A. Poole-Wilson P.A. Risk score for predicting death, myocardial infarction, and stroke in patients with stable angina, based on a large randomised trial cohort of patients.BMJ. 2005; 331: 869Crossref PubMed Scopus (113) Google Scholar, 6Prugger C. Wellmann J. Heidrich J. Brand-Herrmann S.M. Keil U. Cardiovascular risk factors and mortality in patients with coronary heart disease.Eur J Epidemiol. 2008; 23: 731-737Crossref PubMed Scopus (30) Google Scholar, 8Marschner I.C. Colquhoun D. Simes R.J. Glasziou P. Harris P. Singh B.B. Friedlander D. White H. Thompson P. Tonkin A. Long-term risk stratification for survivors of acute coronary syndromes: results from the Long-term Intervention with Pravastatin in Ischemic Disease (LIPID) Study. LIPID Study Investigators.J Am Coll Cardiol. 2001; 38: 56-63Abstract Full Text Full Text PDF PubMed Scopus (108) Google Scholar, 10Marchioli R. Avanzini F. Barzi F. Chieffo C. Di Castelnuovo A. Franzosi M.G. Geraci E. Maggioni A.P. Marfisi R.M. Mininni N. Nicolosi G.L. Santini M. Schweiger C. Tavazzi L. Tognoni G. Valagussa F. Assessment of absolute risk of death after myocardial infarction by use of multiple-risk-factor assessment equations: GISSI-Prevenzione mortality risk chart.Eur Heart J. 2001; 22: 2085-2103Crossref PubMed Scopus (128) Google Scholar, 23Blankenberg S. McQueen M.J. Smieja M. Pogue J. Balion C. Lonn E. Rupprecht H.J. Bickel C. Tiret L. Cambien F. Gerstein H. Munzel T. Yusuf S. Comparative impact of multiple biomarkers and N-terminal pro-brain natriuretic peptide in the context of conventional risk factors for the prediction of recurrent cardiovascular events in the Heart Outcomes Prevention Evaluation (HOPE) study.Circulation. 2006; 114: 201-208Crossref PubMed Scopus (233) Google Scholar, 24Singh M. Holmes D.R. Lennon R.J. Rihal C.S. Development and validation of risk adjustment models for long-term mortality and myocardial infarction following percutaneous coronary interventions.Circ Cardiovasc Interv. 2010; 3: 423-430Crossref Scopus (39) Google Scholar The demographic variables and co-morbid conditions used in these models have largely concurred with the variables in our models.5Clayton T.C. Lubsen J. Pocock S.J. Voko Z. Kirwan B.A. Fox K.A. Poole-Wilson P.A. Risk score for predicting death, myocardial infarction, and stroke in patients with stable angina, based on a large randomised trial cohort of patients.BMJ. 2005; 331: 869Crossref PubMed Scopus (113) Google Scholar, 6Prugger C. Wellmann J. Heidrich J. Brand-Herrmann S.M. Keil U. Cardiovascular risk factors and mortality in patients with coronary heart disease.Eur J Epidemiol. 2008; 23: 731-737Crossref PubMed Scopus (30) Google Scholar, 23Blankenberg S. McQueen M.J. Smieja M. Pogue J. Balion C. Lonn E. Rupprecht H.J. Bickel C. Tiret L. Cambien F. Gerstein H. Munzel T. Yusuf S. Comparative impact of multiple biomarkers and N-terminal pro-brain natriuretic peptide in the context of conventional risk factors for the prediction of recurrent cardiovascular events in the Heart Outcomes Prevention Evaluation (HOPE) study.Circulation. 2006; 114: 201-208Crossref PubMed Scopus (233) Google Scholar, 24Singh M. Holmes D.R. Lennon R.J. Rihal C.S. Development and validation of risk adjustment models for long-term mortality and myocardial infarction following percutaneous coronary interventions.Circ Cardiovasc Interv. 2010; 3: 423-430Crossref Scopus (39) Google Scholar The limitations of the existing models include a lack of performance assessment,5Clayton T.C. Lubsen J. Pocock S.J. Voko Z. Kirwan B.A. Fox K.A. Poole-Wilson P.A. Risk score for predicting death, myocardial infarction, and stroke in patients with stable angina, based on a large randomised trial cohort of patients.BMJ. 2005; 331: 869Crossref PubMed Scopus (113) Google Scholar, 8Marschner I.C. Colquhoun D. Simes R.J. Glasziou P. Harris P. Singh B.B. Friedlander D. White H. Thompson P. Tonkin A. Long-term risk stratification for survivors of acute coronary syndromes: results from the Long-term Intervention with Pravastatin in Ischemic Disease (LIPID) Study. LIPID Study Investigators.J Am Coll Cardiol. 2001; 38: 56-63Abstract Full Text Full Text PDF PubMed Scopus (108) Google Scholar a retrospective study design,6Prugger C. Wellmann J. Heidrich J. Brand-Herrmann S.M. Keil U. Cardiovascular risk factors and mortality in patients with coronary heart disease.Eur J Epidemiol. 2008; 23: 731-737Crossref PubMed Scopus (30) Google Scholar a lack of uniformity in the baseline characteristics because of a long inclusion period of 7 years (during which changes in treatment recommendations have occurred),24Singh M. Holmes D.R. Lennon R.J. Rihal C.S. Development and validation of risk adjustment models for long-term mortality and myocardial infarction following percutaneous coronary interventions.Circ Cardiovasc Interv. 2010; 3: 423-430Crossref Scopus (39) Google Scholar a focus on specific ethnic groups,10Marchioli R. Avanzini F. Barzi F. Chieffo C. Di Castelnuovo A. Franzosi M.G. Geraci E. Maggioni A.P. Marfisi R.M. Mininni N. Nicolosi G.L. Santini M. Schweiger C. Tavazzi L. Tognoni G. Valagussa F. Assessment of absolute risk of death after myocardial infarction by use of multiple-risk-factor assessment equations: GISSI-Prevenzione mortality risk chart.Eur Heart J. 2001; 22: 2085-2103Crossref PubMed Scopus (128) Google Scholar and the inclusion of patients directly after the occurrence of MI, although such patients are known to have a greater risk of recurrent events in the first months after their index event. A comprehensive review of the strengths and limitations of the CAD policy models for patients with CAD developed to date has been reported by Unal et al.9Unal B. Capewell S. Critchley J.A. Coronary heart disease policy models: a systematic review.BMC Public Health. 2006; 6: 213Crossref PubMed Scopus (64) Google Scholar Most studies that have developed prediction models for patients with CAD did not include PCI and CABG as outcomes, nor did they examine these outcomes separately. When we examined nonfatal MI, CABG, and PCI separately, the model performance and discrimination were notably worse than those for cardiovascular mortality. For nonfatal MI, this could in part be explained because this end point is not a pathophysiologic entity, in that some patients who experience MI die immediately. For PCI and CABG, it could in part be explained because undergoing revascularization is heavily influenced by the opinion of the treating physician.25Lenfant C. Shattuck lecture—clinical research to clinical practice—lost in translation?.N Engl J Med. 2003; 349: 868-874Crossref PubMed Scopus (765) Google Scholar Because of the relatively low event rate in our study population, the calibration plots only displayed values in the upper range of both the x- and y-axis. To further assess the capacity of the models to stratify risk in populations that contain more patients with greater absolute risk, external validation in other populations would be helpful. Furthermore, the incidence rates for the nonfatal end points were so low, the calibration plots for these end points were not informative. This also influenced model calibration for the combined end points. The calibration plot for combined end point 2 showed discrepancies in the highest decile of predicted probability of being event free. This was partly because the follow-up interval until the occurrence of nonfatal events was generally shorter than that until the occurrence of fatal events, and the number of nonfatal events was much greater than the number of fatal events. These results suggest that when modeling prognosis in patients with stable CAD, the study end points should be chosen with care, because pooling several outcomes might influence model performance to a great extent. The strengths of our study included the availability of a large, prospective CAD patient cohort for model development and the availability of a wide range of risk factors, whose importance we were able to explore for the prediction of nonfatal and fatal cardiovascular outcomes. Because our study consisted of a multicenter, multinational population, it is likely that the results apply to a broad range of clinical practices. Heterogeneity in center-specific policies was minimized by a uniform study protocol. We performed an extensive assessment of model performance, which is key for model appraisal.9Unal B. Capewell S. Critchley J.A. Coronary heart disease policy models: a systematic review.BMC Public Health. 2006; 6: 213Crossref PubMed Scopus (64) Google Scholar We applied multiple methods to do so, which has been recommended.26Ash A. Shwartz M. R2: a useful measure of model performance when predicting a dichotomous outcome.Stat Med. 1999; 18: 375-384Crossref PubMed Scopus (87) Google Scholar We used time-dependent receiver operating characteristic curves, thus taking the follow-up time into account. Still, several aspects of our study warrant additional consideration. Although the absolute number of events was high (combined end point 2, n = 2,188), the incidence rate was relatively low, which posed challenges for constructing calibration plots for the nonfatal end points. The follow-up period was limited to 4 years, hampering longer term prediction. Also, the EUROPA study was a clinical trial with strict inclusion and exclusion criteria, resulting in a rather specific study population. We did not have biomarker information available, which could have further improved model performance. In conclusion, future studies developing risk stratification models for patients with CAD should give appropriate attention to model performance assessment and should use caution when examining nonfatal end points or combined outcomes that include such end points. The authors have no conflicts of interest to disclose.

Referência(s)