Portal de Periódicos da CAPES

From effect size into number needed to treat

1999; Elsevier BV; Volume: 354; Issue: 9178 Linguagem: Inglês

10.1016/s0140-6736(05)77952-6

1474-547X

Rob Scholten, Edwin de Beurs, L.M. Bouter,

Sensory Analysis and Statistical Methods

Sir—In meta-analysis, effect sizes are used for combining results of individual studies in which the same construct (eg, depression) is measured with different instruments or scales. The effect size (the difference of the mean effect of treatment and control group over the pooled SD) expresses the treatment effect in standard units (instead of original units) and the results of all studies can be combined by calculating a pooled effect size.Interpretation of effect sizes, however, is difficult. Toshi Furukawa1Furukawa TA From effect size into number needed to treat..Lancet. 1999; 353: 1680Summary Full Text Full Text PDF PubMed Scopus (70) Google Scholar presents a table to convert effect sizes into numbers needed to treat, which are easier to understand. Continuous data, however, will then be degraded to dichotomous data (with potential loss of information) and assumptions must be made regarding the cut-off value for the calculation of the response rate of, for example, the control group, which may be arbitrary. Cohen's guideline that effect sizes of 0·2, 0·5, and 0·8 correspond to a small, medium, and large effect, respectively, is also arbitrary.1Furukawa TA From effect size into number needed to treat..Lancet. 1999; 353: 1680Summary Full Text Full Text PDF PubMed Scopus (70) Google Scholar, 2Cohen J Statistical power analysis in the behavioral sciences. Erlbaum, Hillsdale, N J1988Google Scholar Moreover, the clinical relevance of a treatment effect cannot be deduced from Cohen's interpretation.A third means of facilitating the interpretation of effect sizes is to back-transform the pooled effect size by multiplying it with a typical SD of one of the instruments of interest.3Gotzsche PC Hammarquist C Burr M House dust mite control measures in the management of asthma: meta-analysis..BMJ. 1998; 317: 1105-1110Crossref PubMed Scopus (208) Google Scholar In this way the treatment effect can be interpreted in the units of that instrument. The result of this back-transformation, however, depends on which SD is judged typical. The table shows an example of three meta-analyses (by the fixed effects method)4Whitehead A Whitehead J A general parametric approach to the meta-analysis of randomized clinical trials..Stat Med. 1991; 10: 1665-1677Crossref PubMed Scopus (617) Google Scholar of studies with fictitious data comparing the effect of drug A and B for the treatment of high blood pressure. Let us assume that each study involved two groups of 100 patients and that drug B is more effective than drug A, leading to a 5 mm Hg lower mean blood pressure after treatment. Study 1 and 2 investigated homogeneous study populations (eg, with respect to age), indicated by small SDs, whereas studies 3 and 4 represent more heterogeneous study populations (with larger SDs). The pooled effect sizes would be 1·0, 0·3, and 0·6, respectively. Note that these figures roughly correspond with a large, small, and medium effect, respectively,2Cohen J Statistical power analysis in the behavioral sciences. Erlbaum, Hillsdale, N J1988Google Scholar although a constant mean difference of 5 mm Hg was present. In the third meta-analysis (pertaining to all studies with varying SDs), back-transformation of the pooled effect size with various typical SDs produces treatment effects ranging from 3·2 mm Hg to 9·7 mm Hg (table). Only an SD of 7·7 would have given a correct back-transformed mean difference of 5 mm Hg, but there is no way of finding this required SD post hoc. Because methods for deriving a correct typical SD are lacking, back-transformation of effect sizes produces incorrect and confusing results.TableMeta-analyses of various sets of studiesStudyMean (SD)Mean differenceEffect sizeTypical SDBack-transformed mean differenceDrug ADrug B195 (5·0)90 (5·0)5·01·02105 (5·0)100 (5·0)5·01·01+25·01·005·05·03100 (15·0)95 (15·0)5·00·34110 (15·0)105 (15·0)5·00·33+45·00·3315·05·05·03·2Pooled results of studies1–45·00·6510·06·515·09·7All data (except effect size) are mm Hg. (data are fictitious). Open table in a new tab Translation of effect sizes into clinically meaningful units is a hazardous endeavour. Assessment of the clinical relevance of a treatment effect, based on effect size values only, continues to be a challenging undertaking. Sir—In meta-analysis, effect sizes are used for combining results of individual studies in which the same construct (eg, depression) is measured with different instruments or scales. The effect size (the difference of the mean effect of treatment and control group over the pooled SD) expresses the treatment effect in standard units (instead of original units) and the results of all studies can be combined by calculating a pooled effect size. Interpretation of effect sizes, however, is difficult. Toshi Furukawa1Furukawa TA From effect size into number needed to treat..Lancet. 1999; 353: 1680Summary Full Text Full Text PDF PubMed Scopus (70) Google Scholar presents a table to convert effect sizes into numbers needed to treat, which are easier to understand. Continuous data, however, will then be degraded to dichotomous data (with potential loss of information) and assumptions must be made regarding the cut-off value for the calculation of the response rate of, for example, the control group, which may be arbitrary. Cohen's guideline that effect sizes of 0·2, 0·5, and 0·8 correspond to a small, medium, and large effect, respectively, is also arbitrary.1Furukawa TA From effect size into number needed to treat..Lancet. 1999; 353: 1680Summary Full Text Full Text PDF PubMed Scopus (70) Google Scholar, 2Cohen J Statistical power analysis in the behavioral sciences. Erlbaum, Hillsdale, N J1988Google Scholar Moreover, the clinical relevance of a treatment effect cannot be deduced from Cohen's interpretation. A third means of facilitating the interpretation of effect sizes is to back-transform the pooled effect size by multiplying it with a typical SD of one of the instruments of interest.3Gotzsche PC Hammarquist C Burr M House dust mite control measures in the management of asthma: meta-analysis..BMJ. 1998; 317: 1105-1110Crossref PubMed Scopus (208) Google Scholar In this way the treatment effect can be interpreted in the units of that instrument. The result of this back-transformation, however, depends on which SD is judged typical. The table shows an example of three meta-analyses (by the fixed effects method)4Whitehead A Whitehead J A general parametric approach to the meta-analysis of randomized clinical trials..Stat Med. 1991; 10: 1665-1677Crossref PubMed Scopus (617) Google Scholar of studies with fictitious data comparing the effect of drug A and B for the treatment of high blood pressure. Let us assume that each study involved two groups of 100 patients and that drug B is more effective than drug A, leading to a 5 mm Hg lower mean blood pressure after treatment. Study 1 and 2 investigated homogeneous study populations (eg, with respect to age), indicated by small SDs, whereas studies 3 and 4 represent more heterogeneous study populations (with larger SDs). The pooled effect sizes would be 1·0, 0·3, and 0·6, respectively. Note that these figures roughly correspond with a large, small, and medium effect, respectively,2Cohen J Statistical power analysis in the behavioral sciences. Erlbaum, Hillsdale, N J1988Google Scholar although a constant mean difference of 5 mm Hg was present. In the third meta-analysis (pertaining to all studies with varying SDs), back-transformation of the pooled effect size with various typical SDs produces treatment effects ranging from 3·2 mm Hg to 9·7 mm Hg (table). Only an SD of 7·7 would have given a correct back-transformed mean difference of 5 mm Hg, but there is no way of finding this required SD post hoc. Because methods for deriving a correct typical SD are lacking, back-transformation of effect sizes produces incorrect and confusing results. All data (except effect size) are mm Hg. (data are fictitious). Translation of effect sizes into clinically meaningful units is a hazardous endeavour. Assessment of the clinical relevance of a treatment effect, based on effect size values only, continues to be a challenging undertaking.