Portal de Periódicos da CAPES

Got Bias? The Authors Reply

2004; Elsevier BV; Volume: 75; Issue: 4 Linguagem: Inglês

10.1086/424758

1537-6605

Nicholas J. Schork, Tiffany A. Greenwood,

Cancer-related molecular mechanisms research

To the Editor: We are happy to see that our colleagues have taken seriously the issue we raised in our article (Schork and Greenwood Schork and Greenwood, 2004Schork NJ Greenwood TA Inherent bias toward the null hypothesis in conventional multipoint nonparametric linkage analysis.Am J Hum Genet. 2004; 74: 306-316Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar), and, in essence, we do not disagree with much of the factual content of their letters (Abecasis et al. Abecasis et al., 2004Abecasis G Cox N Daly MJ Kruglyak L Laird N Markianos K Patterson N No bias in linkage analysis.Am J Hum Genet. 2004; 75 (XXX–XXX (in this issue))Abstract Full Text Full Text PDF PubMed Scopus (2) Google Scholar; Mukhopadhyay et al. Mukhopadhyay et al., 2004Mukhopadhyay I Feingold E Weeks DE No "bias" toward the null hypothesis in most conventional multipoint nonparametric linkage analysis.Am J Hum Genet. 2004; 75 (XXX–XXX (in this issue))Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar; Visscher and Wray Visscher and Wray, 2004Visscher PM Wray NR Conventional multipoint nonparametric linkage analysis is not necessarily inherently biased.Am J Hum Genet. 2004; 75 (XXX–XXX (in this issue))Abstract Full Text Full Text PDF Scopus (2) Google Scholar [all in this issue]). However, we strongly disagree with aspects of their commentaries and will concentrate on four related issues in our response: (1) the use of the word "bias" to characterize the effects of the treatment of non–fully informative observations as though they were fully informative, in a nonparametric linkage analysis setting; (2) the prevalence and pervasiveness of the inappropriate treatment of non–completely informative observations, in nonparametric linkage analyses; (3) the use of both simulation studies and published "guidelines" for the interpretation of linkage test statistics in the face of inappropriate treatment of non–fully informative observations; and (4) the difference between, and need for refinements in, parametric and nonparametric linkage–based gene-discovery strategies. First, our commentators generally take offense to the use of the word "bias" in our description of what happens in a nonparametric linkage analysis when uninformative or partially informative observations (e.g., affected sibling pairs with non–completely informative marker-genotype data) are treated on equal footing with those that are fully informative. We are in no way wedded to the term "bias" and do not actually care how one refers to the issue we raised in our paper, whether as a "conservative handling" of partially informative observations or as a "power loss" due to the treatment of partially informative observations as though they were fully informative. We do want to emphasize that, as shown by Cordell (Cordell, 2004Cordell HJ Bias toward the null hypothesis in model-free linkage analysis is highly dependent on the test statistic used.Am J Hum Genet. 2004; 74: 1294-1302Abstract Full Text Full Text PDF PubMed Scopus (27) Google Scholar), the treatment of non–completely informative observations as though they were informative does, on average, deflate the test statistic toward values more consistent with the null hypothesis—as we showed in our simple and contrived example involving affected sibling pairs and the coin-flip example—and thus suggests that this phenomenon induces a tendency or "bias" (in a general sense) toward test-statistic values closer to the null hypothesis. Second, our commentators dwell on the elegant work of Kong and Cox (Kong and Cox, 1997Kong A Cox NJ Allele sharing models: LOD scores and accurate linkage tests.Am J Hum Genet. 1997; 61: 1179-1188Abstract Full Text Full Text PDF PubMed Scopus (836) Google Scholar), which considers the issue we describe in the context of affected–sibling-pair analyses. Kong and Cox (Kong and Cox, 1997Kong A Cox NJ Allele sharing models: LOD scores and accurate linkage tests.Am J Hum Genet. 1997; 61: 1179-1188Abstract Full Text Full Text PDF PubMed Scopus (836) Google Scholar) provide a test statistic that appropriately combines uninformative and informative observations into a test statistic based on marker information. However, not all statistics currently in use exploit the principles described by Kong and Cox (Kong and Cox, 1997Kong A Cox NJ Allele sharing models: LOD scores and accurate linkage tests.Am J Hum Genet. 1997; 61: 1179-1188Abstract Full Text Full Text PDF PubMed Scopus (836) Google Scholar). For example, a very recent survey by Cordell (Cordell, 2004Cordell HJ Bias toward the null hypothesis in model-free linkage analysis is highly dependent on the test statistic used.Am J Hum Genet. 2004; 74: 1294-1302Abstract Full Text Full Text PDF PubMed Scopus (27) Google Scholar) suggests that, indeed, statistics do exist that inappropriately treat non–completely informative observations as though they were fully informative, although the degree to which this phenomenon affects various linkage test statistics is context dependent. Thus, for example, of the six statistics for quantitative-trait analysis that Cordell examined, only one—the statistic implemented in the Merlin "Regress" software module—did not show the effects of this phenomenon. In this context, it could be said that perhaps the message of Kong and Cox (Kong and Cox, 1997Kong A Cox NJ Allele sharing models: LOD scores and accurate linkage tests.Am J Hum Genet. 1997; 61: 1179-1188Abstract Full Text Full Text PDF PubMed Scopus (836) Google Scholar) simply has not reached the broader genetics community in the way our commentators would like. It should also be noted that Cordell's study was not exhaustive, suggesting that more research investigating other statistics is needed. Since the SOLAR analysis program (Almasy and Blangero Almasy and Blangero, 1998Almasy L Blangero J Multipoint quantitative-trait linkage analysis in general pedigrees.Am J Hum Genet. 1998; 62: 1198-1211Abstract Full Text Full Text PDF PubMed Scopus (2568) Google Scholar) was not considered by Cordell (Cordell, 2004Cordell HJ Bias toward the null hypothesis in model-free linkage analysis is highly dependent on the test statistic used.Am J Hum Genet. 2004; 74: 1294-1302Abstract Full Text Full Text PDF PubMed Scopus (27) Google Scholar), we explored its handling of uninformative families in a simple study meant to showcase the issue of concern in a practical example. We want to emphasize that we believe SOLAR provides an excellent suite of genetic analysis tools despite the issue we expose (which is a result of the potential complexity of its handling in the setting of variance-components models). We used a subset of the data investigating the genetic determinants of a molecular phenotype in genotyped three-generation CEPH pedigrees (Greenwood et al. Greenwood et al., 2004Greenwood TA Cadman PE Stridsberg M Nguyen S Taupenot L Schork NJ O'Connor DT Genome-wide linkage analysis of chromogranin B expression in the CEPH pedigrees: implications for exocytotic sympathochromaffin secretion in humans.Physiol Genomics. 2004; 18: 119-127Crossref PubMed Scopus (11) Google Scholar; data available on request). We analyzed 12 CEPH pedigrees together and then forced 5 of them to be completely uninformative by three different methods. We then compared the results, which are presented in table 1, as per-family and overall LOD scores. Families with blank data in the second column not only contributed negative evidence for linkage to the overall original linkage signal (column 1 of table 1) but were forced to be completely uninformative in subsequent analyses—a phenomenon which, if accounted for properly (i.e., by not considering the contribution of the uninformative families to the linkage signal), should increase evidence for linkage, via the LOD score.Table 1SOLAR Variance-Components Analysis of a Quantitative Trait in which Some Families Are Forced to Be UninformativeLOD ScoresFamily IDOriginal DataaPer-family scores produced by SOLAR with the original data.LinkedbScores computed on the basis of a reanalysis of only the families showing linkage in the original data.Data RemovedcPer-family scores computed from an analysis in which the families contributing negative evidence for linkage in the original analysis were made uninformative at the marker locus by removing their genotype data (column 4), by making them homozygous for the same allele (column 5), or by making them heterozygous for the same alleles (column 6).HomozygouscPer-family scores computed from an analysis in which the families contributing negative evidence for linkage in the original analysis were made uninformative at the marker locus by removing their genotype data (column 4), by making them homozygous for the same allele (column 5), or by making them heterozygous for the same alleles (column 6).HeterozygouscPer-family scores computed from an analysis in which the families contributing negative evidence for linkage in the original analysis were made uninformative at the marker locus by removing their genotype data (column 4), by making them homozygous for the same allele (column 5), or by making them heterozygous for the same alleles (column 6).1334−.2215…−.0949−.0949−.09491345−.2390…−.1149−.1149−.114913462.04632.12241.71081.71081.71081349.1005−.1837.2588.2588.25881350−.1488…−.2615−.2615−.26151358.6113….3778.3778.37781362−.0085…−.4964−.4964−.49641377.84641.4597.6918.6918.69181408.75181.19831.19691.19691.19691418−.4073…−.4245−.4245−.42451421.7240.99211.12421.12421.12421424.3491.1458.3342.3342.3342 Total LOD4.40435.73464.30234.30234.3023a Per-family scores produced by SOLAR with the original data.b Scores computed on the basis of a reanalysis of only the families showing linkage in the original data.c Per-family scores computed from an analysis in which the families contributing negative evidence for linkage in the original analysis were made uninformative at the marker locus by removing their genotype data (column 4), by making them homozygous for the same allele (column 5), or by making them heterozygous for the same alleles (column 6). Open table in a new tab From table 1, it can be seen that the LOD score actually decreases (from 4.4 to 4.3) when the families providing negative evidence for linkage are made completely uninformative, which suggests that informativeness is not accounted for in this analysis. In addition, family 1358 was uninformative in the original data set yet contributed substantial positive evidence for linkage, which, again, is consistent with the potential for the inclusion of uninformative families to increase the value of the linkage statistic because of stochastic effects, as discussed in our article (Schork and Greenwood Schork and Greenwood, 2004Schork NJ Greenwood TA Inherent bias toward the null hypothesis in conventional multipoint nonparametric linkage analysis.Am J Hum Genet. 2004; 74: 306-316Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar) and Cordell's (Cordell, 2004Cordell HJ Bias toward the null hypothesis in model-free linkage analysis is highly dependent on the test statistic used.Am J Hum Genet. 2004; 74: 1294-1302Abstract Full Text Full Text PDF PubMed Scopus (27) Google Scholar). (Indeed, the individual informative and uninformative family LODs are simply summed without weighting, to give the total LOD, thus allowing the uninformative families to contribute to the LOD score for the sample.) We also found that the variance-components statistic implemented in the Merlin software package provided exactly the same overall LOD scores for these families as SOLAR did in each context, suggesting that Merlin is computing statistics in the same way as SOLAR. Third, although simulation-based tests could be of value in helping determine the impact of the use of statistics that inappropriately treat non–completely informative observations as though they were informative in actual linkage studies (i.e., by simulating the process of including non–completely informative families in data sets and then estimating P values for observed statistics from these simulations), such practices can be problematic for a number of reasons: 1.Resorting to simulation studies merely reinforces the need to accommodate inappropriate handling of non–completely informative observations in the construction of a test statistic.2.One would have to simulate in accordance with the exact mechanism creating the lack of informativeness (partial missing genotype data, marker informativeness, etc.), although the use of permutation tests of allele-sharing information in certain settings may ease this problem (note that not all computer programs provide, by default, P values for statistics based on simulation studies)—in addition, this would have to be pursued on a locus-by-locus basis to accommodate the marker information (and/or lack thereof) at each locus.3.Point estimates of relevant parameters (sibling risk, variance explained, etc.) would not be as reliable as those obtained in a comparable sample of informative observations (as described in our analogy to flipping a coin).4.Analyses that require simulation studies would produce actual test statistics that are highly context dependent (e.g., a low LOD score on one chromosome may have a low P value as a result of the reductions in the test statistic that arise from the inclusion of non–completely informative families as though they were completely informative, whereas a high LOD score on a different chromosome may have a high P value for the same reason), which would undermine conventional "guidelines" for assessment of linkage evidence based on test-statistic values—for example, to convey the value of a LOD score as an indication of linkage strength (Lander and Kruglyak 1995)5.Because of the nonmonotonic relationships between test-statistic values that require simulations to assess significance, total sample size (i.e., a sample that is not adjusted for informativeness), and P values (from the simulations), one would have to be conscious not only of test statistics conveying linkage with artificially low values through these simulation studies but also of test statistics with artificially high values for the same reason—especially for statistics, such as variance-components statistics, that show wide variation in values when constructed without appropriate weighting for marker informativeness (Cordell Cordell, 2004Cordell HJ Bias toward the null hypothesis in model-free linkage analysis is highly dependent on the test statistic used.Am J Hum Genet. 2004; 74: 1294-1302Abstract Full Text Full Text PDF PubMed Scopus (27) Google Scholar).It is thus arguably better to use statistics that are designed to account for marker informativeness. In this context, however, studies that have not used, for example, locus-by-locus simulation studies to investigate the effect of the inclusion of non–completely informative observations on test-statistic values obtained throughout the genome might benefit from such studies, since interpretation of the statistical significance of their results is in doubt (see, e.g., the otherwise comprehensive and excellent studies by Panhuysen et al. [Panhuysen et al., 2003Panhuysen CIM Cupples LA Wilson PWF Herbert AG Myers RH Meigs JB A genome scan for loci linked to quantitative insulin traits in persons without diabetes: the Framingham offspring study.Diabetologia. 2003; 46: 579-587PubMed Google Scholar] and Arya et al. [Arya et al., 2004Arya R Duggirala R Jenkinson CP Almasy L Blangero J O'Connell P Stern MP Evidence of a novel quantitative-trait locus for obesity on chromosome 4p in Mexican Americans.Am J Hum Genet. 2004; 74: 272-283Abstract Full Text Full Text PDF PubMed Scopus (79) Google Scholar])—a practice entirely consistent with the advice given in our article (Schork and Greenwood Schork and Greenwood, 2004Schork NJ Greenwood TA Inherent bias toward the null hypothesis in conventional multipoint nonparametric linkage analysis.Am J Hum Genet. 2004; 74: 306-316Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar). Fourth, the problem of the inappropriate handling of non–completely informative observations is unique to nonparametric, as opposed to parametric, linkage analysis, since many conventional nonparametric linkage test statistics make use of assigned or imputed allele-sharing values in their construction from available marker information. Thus, the inappropriate treatment of allele-sharing values assigned to observations that do not have informative marker data creates problems. This simply is not the case in conventional parametric linkage analysis, where, for example, uninformative observations simply do not contribute to a linkage statistic (i.e., they do not contribute positively or negatively to the signal but contribute a value of 0.0 to the overall LOD score, as though they were simply removed from the analysis). To combat the issue we exposed, we suggest the following actions, all of which are consistent with our commentators' considerations: (1) software documentation should inform the user about (appropriate) potential problems in interpreting test statistics implemented in that software at face value (e.g., on the basis of the guidelines published by Lander and Kruglyak [Lander and Kruglyak, 1995Lander E Kruglyak L Genetic dissection of complex traits: guidelines for interpreting and reporting linkage results.Nat Genet. 1995; 11: 241-247Crossref PubMed Scopus (4470) Google Scholar] that focus on actual test-statistic values, such as LOD scores or t statistics); (2) simulation-based P values should be provided by default for problematic test statistics; and (3) greater emphasis should be placed on the derivation and use of statistics that, like the statistic in Kong and Cox (Kong and Cox, 1997Kong A Cox NJ Allele sharing models: LOD scores and accurate linkage tests.Am J Hum Genet. 1997; 61: 1179-1188Abstract Full Text Full Text PDF PubMed Scopus (836) Google Scholar), are based on sound statistical principles for the treatment of non–completely informative observations. The problems plaguing the reconciliation of multiple nonparametric linkage analysis results—in, for example, the combination of evidence to guide a positional cloning effort—are both numerous and vexing. Consider a recent example in which a LOD score of 11.68 implicating a susceptibility locus for myocardial infarction was reported (Wang et al. Wang et al., 2004aWang Q Rao S Shen GQ Li L Moliterno DJ Newby LK Rogers WJ Cannata R Zirzow E Elston RC Topol EJ Premature myocardial infarction novel susceptibility locus on chromosome 1P34–36 identified by genomewide linkage analysis.Am J Hum Genet. 2004a; 74 (erratum 74:1080): 262-271Abstract Full Text Full Text PDF PubMed Scopus (175) Google Scholar; see also the correspondence of Newton-Cheh et al. [Newton-Cheh et al., 2004Newton-Cheh C Larson M Kathiresan S O'Donnell C On the significance of linkage studies of complex traits.Am J Hum Genet. 2004; 75: 151-152Abstract Full Text Full Text PDF PubMed Google Scholar] and Wang et al. [Wang et al., 2004bWang Q Rao S Topol EJ Reply to Newton-Cheh et al.Am J Hum Genet. 2004b; 75: 152-154Abstract Full Text Full Text PDF Google Scholar]). On the basis of conventional guidelines, this LOD score should have (and was reported to have) an associated nominal P value of ∼.00000000001, making it one of the (if not the single) most significant linkages ever reported for a complex trait. However, after simulation studies, this LOD score was found to have a P value of .0001 (still impressive but much less so). Although it is unclear if the statistic used to produce the LOD score of 11.68 was plagued by the stochastic effects of treating of non–fully informative observations as though they were informative, our article (Schork and Greenwood Schork and Greenwood, 2004Schork NJ Greenwood TA Inherent bias toward the null hypothesis in conventional multipoint nonparametric linkage analysis.Am J Hum Genet. 2004; 74: 306-316Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar) (and Cordell's [Cordell, 2004Cordell HJ Bias toward the null hypothesis in model-free linkage analysis is highly dependent on the test statistic used.Am J Hum Genet. 2004; 74: 1294-1302Abstract Full Text Full Text PDF PubMed Scopus (27) Google Scholar]) suggests that some statistics could (and, in fact, do) treat them this way and hence could lead to interpretive difficulties and discrepancies of this type. It is in this context that we provided the conclusion in our article, which we restate here with minor parenthetical qualifications (in brackets): "…researchers who have actually conducted relevant linkage studies (without completely informative data) in the past and ignored, or were not aware of, [the allele-sharing information] problem [i.e., by, e.g., knowingly or unknowingly using an available, though problematic, statistic without adjustment via, e.g., extensive locus-by-locus simulation studies] should go back and revisit their analyses" (Schork and Greenwood Schork and Greenwood, 2004Schork NJ Greenwood TA Inherent bias toward the null hypothesis in conventional multipoint nonparametric linkage analysis.Am J Hum Genet. 2004; 74: 306-316Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar, p. 316). This work was supported by the following large-scale human genetics research programs: the National Heart, Lung, and Blood Institute (NHLBI) Family Blood Pressure Program (HL64777-01), the NHLBI hypertension SCOR program (HL54998), the National Institutes of Health Pharmacogenetics Network (HL69758-01), and the National Institute of Medical Health Consortium on the Genetics of Schizophrenia (1 R01 MH06557-01A1). The authors would like to thank Dr. Heather Cordell, for critical discussions and the opportunity to review her work in progress.