Data discrepancies and substandard reporting of interim data of Sputnik V phase 3 trial
2021; Elsevier BV; Volume: 397; Issue: 10288 Linguagem: Inglês
10.1016/s0140-6736(21)00899-0
ISSN1474-547X
AutoresEnrico Bucci, Johannes Berkhof, André Gillibert, Gowri Gopalakrishna, Raffaele Calogero, L.M. Bouter, Konstantin Andreev, Florian Naudet, Vasily Vlassov,
Tópico(s)Healthcare Technology and Patient Monitoring
ResumoRestricted access to data hampers trust in research. Access to data underpinning study findings is imperative to check and confirm the findings claimed. It is even more serious if there are apparent errors and numerical inconsistencies in the statistics and results presented. Regrettably, this seems to be what is happening in the case of the Sputnik V phase 3 trial.1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar Several experts3Vlassov V Rebrova O Aksenov V Commentary on the publication of preliminary results of the Sputnik-V vaccine phase 3 trial.http://osdm.org/english/2021/02/06/a-commentary-on-the-publication-of-preliminary-results-of-the-sputnik-v-vaccine-phase-3-trial/Date: Feb 5, 2021Date accessed: February 18, 2021Google Scholar, 4Bucci E Andreev K Björkman A et al.Safety and efficacy of the Russian COVID-19 vaccine: more information needed.Lancet. 2020; 396: e53Summary Full Text Full Text PDF PubMed Scopus (11) Google Scholar found problematic data in the published phase 1/2 results.2Logunov DY Dolzhikova IV Zubkova OV et al.Safety and immunogenicity of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine in two formulations: two open, non-randomized phase 1/2 studies from Russia.Lancet. 2020; 396: 887-897Summary Full Text Full Text PDF PubMed Scopus (296) Google Scholar We have made multiple independent requests for access to the raw dataset, but these were never answered. Despite publicly denying some problems, formal corrections were made to the Article,2Logunov DY Dolzhikova IV Zubkova OV et al.Safety and immunogenicity of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine in two formulations: two open, non-randomized phase 1/2 studies from Russia.Lancet. 2020; 396: 887-897Summary Full Text Full Text PDF PubMed Scopus (296) Google Scholar thus addressing some concerns.5Logunov DY Dolzhikova IV Tukhvatullin AI Shcheblyakov DV Safety and efficacy of the Russian COVID-19 vaccine: more information needed—Authors' reply.Lancet. 2020; 396: e54-e55Summary Full Text Full Text PDF PubMed Scopus (8) Google Scholar Notwithstanding the previous issues and lack of transparency, the interim results from the phase 3 trial of the Sputnik V vaccine1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar again raise serious concerns. We have a serious concern regarding the availability of the data from which the investigators draw their conclusions. The investigators state that data will not be shared before the trial is completed, and then only by approval of stakeholders, including a so-called security department. Data sharing is one of the cornerstones of research integrity; it should not be conditional and should follow the FAIR principles. The second concern pertains to the trial protocol, as already described in an open letter by the Russian Society for Evidence-Based Medicine.3Vlassov V Rebrova O Aksenov V Commentary on the publication of preliminary results of the Sputnik-V vaccine phase 3 trial.http://osdm.org/english/2021/02/06/a-commentary-on-the-publication-of-preliminary-results-of-the-sputnik-v-vaccine-phase-3-trial/Date: Feb 5, 2021Date accessed: February 18, 2021Google Scholar The Sputnik V investigators mention that three interim analyses were added to the study on Nov 5, 2020,1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar but this change was not recorded on ClinicalTrials.gov (NCT04530396). Unfortunately, the full study protocol has not been made publicly available, so the rationale behind this change or the type I error rate adjustment, if any, is not known. According to the ClinicalTrials.gov record NCT04530396, the primary outcome was changed on Sept 17, 2020. Initially, the primary outcome was to be assessed after the first dose, but the evaluation was postponed to after the second dose. The presented primary result (efficacy of 91·6%) is dependent on this change, but the reasons for the change have not been made public. Moreover, the latest ClinicalTrials.gov record (Jan 22, 2021) defines the primary outcome inconsistently: “Primary Outcome Measures: percentage of trial subjects…after the first dose…based on the percentage…after the second dose”. Besides these protocol amendments, the definition of the primary outcome is unclear in the Article,1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar where it says that when COVID-19 was suspected, participants were assessed with “COVID-19 diagnostic protocols, including PCR testing”. Here, we lack some crucial information, such as the clinical parameters determining suspected COVID-19, what diagnostic protocols were used, when the PCR testing was done, what specific method was used, or how many amplification cycles were used. The way cases of suspected COVID-19 were defined could have led to bias in PCR testing used to assess the number of confirmed COVID-19 cases, which is crucial for the efficacy determination. A final point of concern about the study protocol relates to the enrolment and randomisation of patients. According to the trial profile in figure 1 of the Article,1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar 35 963 individuals were screened and 21 977 individuals were randomised. The ClinicalTrials.gov record for NCT04530396 (Jan 20, 2021) mentions that 33 758 patients were enrolled. We would expect that this last figure should be equal to either the number of participants screened or randomised. Moreover, there is no information about what caused the exclusion of 13 986 participants, as per the trial profile. The third concern relates to the data reported and numerical results. We found the following data inconsistencies: (1) in figure 2 of the Article,1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar data for the vaccinated group on day 20 refer to more individuals than at day 10, as if there was either information missing for 100 participants at day 10, or participants were enrolled after day 10 (figure 2 was formally corrected on Feb 20, 2021, but the correction statement did not state the reasons leading to such correction); and (2) in table S1 of the appendix,1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar the number of participants reported for the different vaccinated age cohorts do not add up to the reported total (n=338 vs n=342). With such inconsistencies, we question the accuracy of the reported data. A very peculiar result of the major subgroup analysis of the primary outcome caught our attention. The vaccine efficacy was said to be high for all age groups. The reported percentages were 91·9% in the 18–30-year age group, 90·0% in the31–40-year age group, 91·3% in the 41–50-year age group, 92·7% in the 51–60-year age group, and 91·8% in participants older than 60 years. We checked the homogeneity of vaccine efficacy across age groups (interaction tests): the p value of the Tarone-adjusted Breslow-Day test was 0·9963, and the p value of a non-asymptotic test was 0·9956,6Sangnawakij P Böhning D Holling H On the exact null-distribution of a test for homogeneity of the risk ratio in meta-analysis of studies with rare events.J Stat Comput Simul. 2021; 91: 420-434Crossref Scopus (1) Google Scholar indicating a very low probability of observing a homogeneity this good if the actual homogeneity is perfect. By applying 18 other homogeneity tests (six in table 1, seven in table S6, six in table 2 of the Article1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar), we could not find other major abnormality in the overall distribution of p values (appendix). We also found some highly coincidental results reported in table S3 of the appendix. In particular, two upper confidence limit values for two different distributions (placebo group at baseline for unstimulated and antigen-stimulated measures) both equal 0·708. Of course, this is possible, but we call once more for access to the data from which the statistics originate for close scrutiny. In line with our earlier concerns with the phase 1/2 results4Bucci E Andreev K Björkman A et al.Safety and efficacy of the Russian COVID-19 vaccine: more information needed.Lancet. 2020; 396: e53Summary Full Text Full Text PDF PubMed Scopus (11) Google Scholar and the substandard reporting of the phase 3 interim results,1Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (312) Google Scholar we invite the investigators once more to make publicly available the data on which their analyses rely. Access to the protocol, its amendments, and the individual patient records is paramount, as much for clarification as for open discussion of all the issues. We also invite the Editors of The Lancet to clarify the consequences of further denying access to the data needed for assessing the results presented, should the authors still deny it. EMB is the owner of Resis Srl. All other authors declare no competing interests. Code to test the homogeneity of vaccine efficacy across age groups is available on the Open Science Framework, https://osf.io/sudxe/ Download .pdf (.07 MB) Help with pdf files Supplementary appendix Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in RussiaThis interim analysis of the phase 3 trial of Gam-COVID-Vac showed 91·6% efficacy against COVID-19 and was well tolerated in a large cohort. Full-Text PDF Data discrepancies and substandard reporting of interim data of Sputnik V phase 3 trial – Authors' replyClear and transparent regulatory standards exist for provision of clinical trial data, including data reported in clinical study reports that are considered sufficient for regulatory review and approvals. The reporting of the interim analysis1 in the phase 3 Sputnik V clinical trial fully complies with those standards. It is on this basis that Sputnik V has received registration in 51 countries, which confirms our full transparency and compliance with regulatory requirements. Full-Text PDF
Referência(s)