Portal de Periódicos da CAPES

Data sharing requirements: perspectives from three authors

2018; Elsevier BV; Volume: 109; Issue: 1 Linguagem: Inglês

10.1016/j.fertnstert.2017.11.034

1556-5653

Kurt T. Barnhart, Richard S. Legro, Richard T. Scott,

Health Systems, Economic Evaluations, Quality of Life

The International Committee of Medical Journal Editors (ICMJE) believes there is an ethical obligation to responsibility share data generated by interventional clinical trials (1Taichman D. Sahni P. Pinborg A. Peiperl L. Laine C. James A. et al.Data sharing statements for clinical trials: a requirement of the International Committee of Medical Journal Editors.JAMA. 2017; 317: 2491-2492Crossref PubMed Scopus (55) Google Scholar). The goal of this opinion is that sharing of de-identified individual participant data will become the norm. The World Health Organization has articulated that best practice for clinical trials include: universal prospective registration, public disclosure of results, and data sharing. Achieving these goals will maximize generation of high quality data resulting in the practice of evidence based medicine, while concomitantly maximizing the ethical obligation to trial participants who put themselves at risk. The research community has started to adopt prospective clinical trail registration, while uptake is yet to be universal. Achieving the other goals, including data sharing, is both good and bad, and assuredly will be ugly. There are obvious benefits to data sharing. The availability of scrutiny of data by peers, or publicly, will increase transparency. Transparency will minimize false or overreaching conclusions, as well as potential of suppression of “undesirable” findings. Sharing data will also maximize use of data allowing secondary analysis, addressing additional questions, generating additional hypotheses and aggregation with other data. There is clearly inherent value in pooling data, when possible, allowing potential increase in power, assessment of subgroups, or validation of findings. Drawbacks in data sharing include how to appropriately give scholarly credit to those who generate and share the data, as well as those who use the data for secondary purposes. The conception, conduct, and analysis of a clinical trial is very expensive and time consuming. Despite this effort, it is often difficult to distinguish impact of manuscripts from the primary data from manuscripts based on reanalysis or aggregation. If academic productivity is measured by the number of scholar manuscripts (as it often is), it will be far easier to develop a career in secondary analysis compared to primary data generation. The logistics of data sharing will be ugly. Many aspects will have to be resolved. Examples include how to ensure transparent data request, approval and prioritization. How will data be archived and at whose expense? What data security is necessary to protect data? While data will be de-identified, there is always the potential of unblinding a subjects' identity based on data. This is especially true if genetic data is included. There have been instances when individuals have claimed they were identifiable based on public sharing of genetic data. While these important issues are debated and practice evolves, clinical trialists should start to consider data sharing plans now. While there is consensus that data should be shared, there is no consensus on how much data should be shared or how. A data sharing plan should indicate what particular data will be shared (i.e. individual patient data, data dictionary, protocol, data analysis plan, primary and secondary analysis, genetic data), with whom it can be shared, when it will be available, for how long and what is the mechanism. A data sharing plan is already a requirement of National Institutes of Health (NIH) grant application. Data sharing statements may soon be required at the time of manuscript submission. While data sharing is not yet mandated, in the wild west that is publication peer review data, consideration of data sharing statements are starting to influence editorial decisions. When Bob Dylan came out with his eponymous album of this section's title in 1964, did anyone see him one day winning a Nobel Prize? Similarly when the ICMJE, to which Fertility and Sterility and most other high impact journals belong, announced in 2004 (2De Angelis C. Drazen J.M. Frizelle F.A. Haug C. Hoey J. Horton R. et al.Clinical trial registration: a statement from the International Committee of Medical Journal Editors.New Engl J Med. 2004; 351: 1250-1251Crossref PubMed Scopus (686) Google Scholar) the mandatory registration of all clinical trials prior to commencement, did anyone foresee the radical transformation in clinical trial reporting and oversight that would follow. Now the most recent 2017 ICMJE announcement regarding clinical trials requires a report in the manuscript of a data sharing plan (1Taichman D. Sahni P. Pinborg A. Peiperl L. Laine C. James A. et al.Data sharing statements for clinical trials: a requirement of the International Committee of Medical Journal Editors.JAMA. 2017; 317: 2491-2492Crossref PubMed Scopus (55) Google Scholar). This plan should include if and how data will be shared, though it does not (yet) mandate data sharing. Let me, in this Inkling, review the impact of the initial 2004 ICMJE mandate, discuss the implementation of the 2017 mandate and speculate about the future of data sharing. The initial 2004 trial registration mandate resulted from the fact that there was selective reporting of clinical trial results. Trials in which a drug or interventions succeeded were reported, but trials with negative or harmful results never saw the light of day. While it is popular to identify Big Pharma as the primary culprit, there are plenty of instances of investigator-initiated and NIH funded trials where only the positive survived to publication and the negative were buried (3Ioannidis J.P. Why most published research findings are false.PLOS Med. 2005; 2: e124Crossref PubMed Scopus (5645) Google Scholar). Without transparency of clinical trial registration prior to initiation that identifies among other criteria, the primary and secondary outcomes, there was also post hoc cherry picking of outcomes. Surrogate markers that were an afterthought of the trial were elevated to primary outcomes if no major health benefits were noted. I think metformin has been highlighted in the title of multiple manuscripts to improve just about every surrogate marker of inflammation in women with polycystic ovary syndrome. My response: why would anyone do such an intensive expensive trial for such a clinically irrelevant outcome as change in circulating PAI-1, CRP, IL-6, etc.? Failure to account for multiple hypothesis testing leads to an increased incidence of type 1 errors. While the uptake of clinical trial registration was gradual, it is now rare, if not exceptional to see a clinical trial reported that was not registered a priori. As an Associate Editor of this journal, I can say that while submission of such a manuscript still occurs, they are not sent out for review as they do not meet the 2004 ICMJE mandate for clinical trial registration. The times they are a-changin.' Although initially the impetus to trial registration was primarily a carrot, i.e. possible publication of your work in a high impact journal, the failure to register and timely report the results of a clinical trial on Clinicaltrials.gov now carries substantial penalties through the Department of Health and Human Services Final Rule, implemented in January of 2017 (with a similar NIH policy at the same time). The Final Rule requires a responsible party to both register the trial at Clinicaltrials.gov and to submit summary results information to ClinicalTrials.gov for any applicable clinical trial (within one year of completion), regardless of whether the drug, biological, or device products under study have been approved, licensed, or cleared for marketing by the Food and Drug Administration. Noncompliance can be noted on the clinical trial record at Clinicaltrials.gov and can in certain instances result in significant monetary penalties (on a daily basis until corrected) to the sponsor. Our academic health center has responded by requiring all registered clinical trials, whether NIH, industry or investigator-initiated to comply with the final rule for registration and reporting. The times they are a-changin.' The impact of publishing a statement requiring a data sharing plan in a manuscript, especially when the statement can read, “Data will not be shared,” seems minimal at first, a la the 2004 ICMJE announcement. But how will those investigators who refuse to share data be viewed by their peers and eventually by their peer reviewers and editors? If most or the best investigative teams are sharing their data, will not eventually peer pressure force the others to come on board? The NIH requires a formal data sharing plan (without the option of an opt out) for all clinical trial applications with direct costs over $500,000 per year. Industry and the NIH are making de-identified clinical trial data available right now. As an example, The Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) has created a Date and Specimen Repository (DASH; https://dash.nichd.nih.gov) for clinical studies, where these can be accessed by investigators throughout the world. NICHD-funded clinical trial networks, including the Reproductive Medicine Network, have already posted their data from specific trials for accession. To date there have been over 10,000 queries requesting data and/or specimens at DASH with 10% originating from abroad (Personal Communication, Diana Bianchi, Director NICHD, November 13, 2017). The times they are a-changin.' The need for clinical trial data sharing is increasing in a world where emerging evidence based medicine data synthesis methods above and beyond conventional meta-analysis can weave together disparate clinical trial data to prioritize research and guide personalized patient care. These include network meta-analysis, a tool to rank the efficacy of various interventions where head to head comparisons are lacking. For instance a recent network meta-analysis we performed suggested the next trial of ovulation induction in women with polycystic ovary syndrome should be letrozole vs clomiphene plus metformin (4Wang R. Kim B.V. van Wely M. Johnson N.P. Costello M.F. Zhang H. et al.Treatment strategies for women with WHO group II anovulation: systematic review and network meta-analysis.BMJ. 2017; 356: j138Crossref PubMed Scopus (67) Google Scholar). Another emerging method is individual patient data (IPD) meta-analysis which uses original (de-identified) patient data to better identify responders and non-responders to treatments (as well as those experiencing adverse events) especially timely in this age of personalized medicine. Thus it does not take too much speculation to see the next steps in data sharing, i.e. accessible data through a clear mechanism, and more likely the immediate availability upon publication to all interested parties of de-identified clinical trial databases relating to the reported primary and secondary outcomes of clinical trials. This will likely first be piloted by the highest impact journals and then trickle down the hierarchy of journals. I hope that Fertility and Sterility will be high up the water chain. Clinical trial protocols, once highly protected documents, the intellectual property of clinical trialists, are now available as supplementary materials at many high impact journals, allowing assessment of rigor and easy replication of results. Replication is the hallmark of science. The times they are a-changin.' The requirement to share all primary data from a clinical trial will ultimately reward those investigators, independent of name or nation or funding body from any categorical discrimination (think of investigators from the BRICS, the up and coming economies of the world (Brazil, Russia, India, China, and South Africa) and beyond and the more rigorous scrutiny their trials evoke). When the data are shared, the data will speak for itself. We are fortunately in a time of greater transparency of research coupled with increasing concerns about the lack of reproducibility of our research. One of the reasons for lack of reproducibility is the increasing recognition of fraudulent trials, where data are invented or massaged to reach a pre-determined outcome. Unfortunately according to the website Retraction Watch (http://retractionwatch.com/the-retraction-watch-leaderboard/), the largest mass perpetrators of fraudulent published data resulting in manuscript retraction are clinical trialists. Yoshitaka Fujii, a Japanese Anesthesiologist, has had to date 183 manuscripts retracted, followed by a German anesthesiologist Joachim Boldt with 96 (and yes, there are U.S. clinical investigators on the list and 30 of the top 31 are men!). Most, if not all of these retracted clinical trials by these investigators were never performed and the results were simply made up. These fraudulent trials are also present in our field of infertility research and one can argue that the double barrels of academic pressure and financial rewards for developing interventions that improve pregnancy rates is fertile soil for potential fraud and abuse of data. These trials in our field, if fraudulent or incorrectly analyzed, may also sway practice the wrong way if they are incorporated into meta-analyses. For instance, there have been conventional aggregate data meta-analyses of aspirin use in in vitro fertilization (IVF) showing a benefit on pregnancy rate (5Ruopp M.D. Collins T.C. Whitcomb B.W. Schisterman E.F. Evidence of absence or absence of evidence? A reanalysis of the effects of low-dose aspirin in in vitro fertilization.Fertil Stertil. 2008; 90: 71-76Abstract Full Text Full Text PDF PubMed Scopus (40) Google Scholar). When an IPD meta-analysis of this topic was performed, only 6 of 10 eligible RCTs in the study could provide independent patient data (6Groeneveld E. Broeze K.A. Lambers M.J. Haapsamo M. Dirckx K. Schoot B.C. et al.Is aspirin effective in women undergoing in vitro fertilization (IVF)? Results from an individual patient data meta-analysis (IPD MA).Hum Reprod Update. 2011; 17: 501-509Crossref PubMed Scopus (30) Google Scholar). The conventional meta-analysis using the aggregate data from all 10 studies showed an effect favoring aspirin, whereas when only the aggregate data from the 6 studies providing IPD data were used the effect direction reversed against the use of aspirin in IVF. Such findings are sobering, and while full disclosure of clinical trial datasets has its own issues as noted by my fellow Inklings authors, the benefits of transparency of data both in primary and secondary analyses as well as in evidence based syntheses of clinical trials far outweigh the burdens of reporting. Following the stream, mandatory data sharing is likely in the near future and we should prepare for it. As Bob Dylan sang in 1964, “…the waters [a]round [us] have grown…” (and they are not retreating). He advised that if you wanted to survive, “…you better start swimming/or you'll sink like a stone,” for not only have the times changed, they are still a-changing'- for the better. The ICMJE have put forth an aggressive agenda to have data sharing become a routine practice for medical publications in the near future (1Taichman D. Sahni P. Pinborg A. Peiperl L. Laine C. James A. et al.Data sharing statements for clinical trials: a requirement of the International Committee of Medical Journal Editors.JAMA. 2017; 317: 2491-2492Crossref PubMed Scopus (55) Google Scholar). This action, in response to the World Health Organization and possibly other regulatory agencies and academic groups seeks to improve the quality of the medical information placed into the literature by increasing transparency around the data and the methodology and analyses which have been done. The potential advantages of such a precedent are appealing. The availability of data will assure greater integrity in collecting and evaluating data. Greater scrutiny might lead to more accurate analyses which do not over-reach the limits intrinsic to their experimental design or actual data. It would also be of great value when preforming meta-analyses. While all of these goals are noble, the reality is that they bring enormous complexity and challenges. This type of transparency will impact the way investigators, universities, funding agencies, and biotechnology programs interact with journals and the peer review process. What follows is an abbreviated list of potential consequences to complete transparency within medical studies. It will be more difficult to recruit patients to participate in research studies. This will be particularly true for those studies which involve genetic analyses/genotyping including exome and whole genome analyses. There is no effective way to de-identify genotyping data. In addition to the data derived from their own genome (health risks, etc.), the identification of the patient will be linked with all other data collected in the study including reproductive history. In our field, this would inevitably include parity including pregnancy terminations, sexually transmitted diseases, and even number of partners in the relatively recent past (oocyte donors). With increased use of genotyping for many indications including people studying their own genetic heritage, it will be impossible to reassure patients that their privacy can be protected. Based on our experience, this will be an enormous problem as patients are already heavily fixated on this question. It will result in reduced funding from biotechnology and Pharma. If there is a requirement to share all the data, then industry may hesitate to sponsor time consuming and expensive studies if the data will be readily available to competitors. Inevitably, there will be efforts to cross reference data for registration and validation purposes. Whoever owns the intellectual property rights will be hesitant to publish all their data as it may “show the way” to their competitors. This will not be limited to industry. Universities are amongst the most capitalistic organizations in the world and work hard to secure and protect their intellectual property. It is almost inevitable that they will want to be cautious about some publications if they could dilute the value of those rights. In reality, there is no requirement to publish validations studies, and data sharing could reduce the number of validation studies done for various technologies. Consider the example of a new diagnostic test. If the specimens are sent out to a third-party laboratory, then only those results would be reported as they were the only data in the study. The validation studies will not be published for novel things as it will give away the methodology and in the end, investigators can publish the clinical data anyway. Another thing to consider is how authorship will be attributed in publications. Will other investigators be allowed to use a group's data and then publish the results without recognizing the actual investigators at the author level? That seems ridiculous. On the other hand, what if the original group never consents to authorship? And will those using the data be required to pay their fair share of the costs generated to create those data? Who will have access to the data and what it can be used for is another important aspect to consider. Can anyone, independent of their expertise, evaluate large and complex data sets? This will be problematic. One problem which our society clearly has is the assumption that the original investigators are mad scientists greedy for academic fame, money, or some other nefarious cause. Anyone who reevaluates the data will be assumed by the lay press to be virtuous – no matter how ridiculous and poor quality their work. Will lawyers, law enforcement, or employers be able to mine these data? Can you imagine DNA from a crime scene from years ago potentially matching? We know their fingerprinting process is imperfect. Will we be putting our study subjects at risk to some fairly dramatic adverse consequences as a result of their participation? How the data will be stored and who will shoulder this responsibility and cost is another great unknown. What safeguards will be put into place to ensure the integrity of the process of attaining and reevaluating those data? The real question here is why we are assuming these risks now. Is there rampant corruption in the world of medical publications? It would seem that many of the problems are identified and eliminated during peer review. It is likely that some degree of data sharing will be an important part of the future. However, prior to implementation of the changes decreed by et al, a great deal of thoughtful work needs to be done to be certain that we do not dramatically undermine investigative and publication processes.