Portal de Periódicos da CAPES

Significant significance?

2021; Wiley; Volume: 232; Issue: 4 Linguagem: Italiano

10.1111/apha.13665

1748-1716

Tomas L. Bothe, Andreas Patzak,

Cardiac Health and Mental Health

Acta PhysiologicaVolume 232, Issue 4 e13665 EXACTAFree Access Significant significance? Tomas L Bothe, Corresponding Author Tomas L Bothe tomas-lucca.bothe@charite.de orcid.org/0000-0001-7569-4527 Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Vegetative Physiology, Berlin, Germany Correspondence Tomas L Bothe, Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Vegetative Physiology, Charitéplatz 1, D-10117 Berlin, Germany. Email: tomas-lucca.bothe@charite.deSearch for more papers by this authorAndreas Patzak, Andreas Patzak orcid.org/0000-0002-1088-6875 Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Vegetative Physiology, Berlin, GermanySearch for more papers by this author Tomas L Bothe, Corresponding Author Tomas L Bothe tomas-lucca.bothe@charite.de orcid.org/0000-0001-7569-4527 Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Vegetative Physiology, Berlin, Germany Correspondence Tomas L Bothe, Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Vegetative Physiology, Charitéplatz 1, D-10117 Berlin, Germany. Email: tomas-lucca.bothe@charite.deSearch for more papers by this authorAndreas Patzak, Andreas Patzak orcid.org/0000-0002-1088-6875 Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Vegetative Physiology, Berlin, GermanySearch for more papers by this author First published: 21 April 2021 https://doi.org/10.1111/apha.13665Citations: 1AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Modern medicine is a game of reason. The times of harmonizing bodily fluids and quackery seem long gone. We no longer believe in phlebotomy or burning women to free our communities from bewitched spirits, even though bloodletting had occasionally worked based on chance for some medical practitioners during the Middle Ages. Individual experience is still an invaluable skill for any clinician, albeit more in the realm of visual diagnosis than sensing imbalanced spirits. Ultimately, it was the 18th century's period of Enlightenment and the subsequent ~250 years of scientific, rational, objective, and most importantly evidence-based research that has paved the way for our current state of rapidly evolving and continuously improving medical comprehension. So much for the theory. As with most modern scientific theories, this is not a bad one. Hence, life expectancies have been growing rapidly since the advent of modern medicine and are now projected to outgrow barriers by 2030 which had commonly been deemed unconquerable at the beginning of the century.1, 2 These are splendid results, facilitated by a plethora of advances in individual medical fields: HIV treatment is in a state which not only allows for longer patient survival but may enable disease eradication plans.3 New kinds of cancer treatments have substantially changed lives for millions of patients around the world.4, 5 These and countless more phenomenal developments allowed evidence-based medicine to make headway in ridding our world from the scourges of humanity. What happens though when the desire for scientific advancement outruns healthy scepticism? It turns out, in modern medicine all that glitters is not gold. The infamous "Meta-Analysis of Multiple Primary Prevention Trials of Cardiovascular Events Using Aspirin" by Bartolucci et al impressively highlighted the pitfalls associated with the uncritical and unchallenged quest for medical evidence.6 The indisputably well-written study examined acetylsalicylic acid (ASS) capability of preventing myocardial infarction (MI) by comprising data of over 22.000 patients for a timespan of on average five years. The authors reported a P-value of P < .00001, which led to the recommendation of ASS for the general population. The reported P-value indicated a positive effect of ASS for MI risk reduction beyond any measure of reasonable doubt. Another victory of modern medicine, based on a highly significant statistical analysis: Sounds great, doesn't it? Not so much on second sight: There was a simple but very important insight missing while recommending ASS for basically everybody. While it was impressively shown that taking ASS had an effect on the rate of MIs, nobody cared for what this effect actually was. This is where statistics get tricky: It turned out to be barely anything. Even though a P-value smaller than a tenth of a percent of a percent was reported there was only a risk reduction of 0.77% and a correlation coefficient between ASS intake and reduced MI of r2 = .001. The actual effect of ASS on a single patient seemed not only clinically negligible but furthermore did most certainly not make up for ASS's adverse side effects. Subsequently, the correct question to ask is: How could this happen? It is worth the time to take a step back and think about what happened in this study. The researchers reported a P-value, indicating an effect, which in this case was the reduction of MI events in patients under ASS treatment. The critical misinterpretation is to link a low P-value with a relevant effect. P-values indicate the statistical certainty with which a hypothesis can be claimed to be true and are tested against an alpha-level. Less technical, a P-value tells us how likely it is that an observed effect is not truly there and is much rather observed by chance, just like there is a chance of rolling a six for five times straight on a fair die. A P-value would tell us how likely this event is to be observed by chance and thought backwards, how sure we can be that the observed effect is not a product of chance. The alpha-level is our decision boundary – P-values below the chosen alpha are considered significant, while P-values greater than alpha are generally interpreted as no shown difference. Notably, the value of alpha is arbitrary and commonly set to 0.05 due to nothing else but convention. Quite intuitively, after around 10 straight rolled sixes we all would heavily suspect the die to be loaded and most statistical tests would suggest a P-value well below commonly chosen alpha-levels. Measures of certainty play a critical role in medical science: Nobody would want to take a drug "proven" to work if there was a 40% chance of this "prove" being fallacious. Then again, there is something else you would want to know before taking a drug: How much does it help? Frustratingly, P-values do not tell us anything about effect size. Coming back to our die, rolling a six three of ten times is not suspicious at all but rolling 3.000 sixes out of 10.000 rolls would leave us convinced that the die is loaded. Markedly, in both cases, it could have been the same loaded die. When playing games most people do not care about being cheated a little – after all, there are no completely equal games (such as no die is manufactured with infinite precision). What people do rightfully care a lot about is whether they are cheated badly. Translated into scientific terms: Most people do not care about P-values. They care about effect sizes. What is to be learned from the ASS example is that there are more layers to statistical interpretation than just the calculation of P-values and that the sizes of discovered effects often add more to our medical understanding than P-values. Luckily, effect sizes are in most cases easily and intuitively calculated. The most straight-up options are 95% confidence intervals and simple regression coefficients. Gail Sullivan and Richard Feinn have authored a highly impactful editorial about the correct interpretation of P-values and effect sizes which can be recommended as a short handbook for scientific reporting.7 Equally important to reporting effect sizes is to interpret them correctly and comprehensibly. At this point researchers are asked to look at the effects they have shown and tell their readers what they think about them from their scientific perspective. This is not about intellectual paternalism but much rather about enhancing papers readability and correct interpretability for readers. This is increasingly important, given the fact that overall research output has skyrocketed while scientific advances have created more and more specialized fields of research.8 Of course, only reading papers from your very own field of research might limit the problem of readability but it is mostly reading and understanding things very different from one's everyday workflow that spark the creation of truly original ideas. Unsurprisingly, all of this is not limited to clinical sciences, but might be even more pronounced in basic research contexts. Fundamental medical research is based on a myriad of elaborate methods and spans across a vast field of medical domains. Scanning through recent physiological research presents the reader with a multitude of methods and statistical tests, all to be interpreted differently. Not to forget, the most fundamental limitation of biomedical sciences: Every experiment, every result, and every interpretation is manmade and therefore a flawed model of reality. P-values allow us to estimate the statistical probability that our model's results are due to chance. The choice of an appropriate model is therefore the most important feat of any scientific pursuit. Staggeringly low P-values promise nothing more than false-positive conclusions if the chosen model does not generalize adequately to real-world scenarios while, vice versa, unsound model selection can lead to large P-values even if there is a very significant and clinically important dependency to be found. No clear-cut way exists to determine which model is best to use as the field is too diverse, but a good rule of thumb would be to focus on a model's predictive scope. Biochemical models are best suited to describe biological dependencies on a cellular level.9-11 Deeper down, they even can be capable of characterizing specific intracellular pathways, broadening our understanding of the most basic biological levels.12-14 The examination of more complex systems depends on models focussing more on tissue and organ-related, functional models, both on a microscopical15, 16 and macroscopical17, 18 levels. Research about organ system interactions and multifocal dependencies is often too complex for a deep-down functional analysis and must rely on models based on systemic physiological and mechanistic understanding.19-21 Then, there are research questions which require investigation of whole body morphological or behavioural traits which can only be observed in living animals22 and lastly, there are the research questions which depend on human subjects of research and the heaps of models related to research in humans.23, 24 It is a hard task to choose the correct model and in many cases, only a combination of multiple models can lead to insightful results. Interpreting those models based on P-values only will lead to described false positive and negative interpretations. Physiologically comprehensible, well-described models and ideally a reported dose-response relationship can be crucial first measures of model integrity and therefore offer a more complete synopsis of research results than crude P-values. Even if all of this sounds very difficult, the situation is not too dire. Researchers should try to address the concern of declining interprofessional readability, but the tools to do so are right there and easy to administer. Explaining and interpreting effect sizes can go a long way and emphasizing those ideas in research papers helps to guide the reader's attention to the most pivotal results. Explaining to what degree muscles fibres were expected to grow and how much protein levels would have to rise to have a significant effect are key elements of excellent research papers, much more so than hunting P-values down to another decimal place. Our capability of understanding, interpreting and being able to trust evidence-based research works are the foundation of modern medicine's wonders. We as the research community have the tools available to make our results more accessible to readers and prevent them from flawed interpretations. Emphasizing effect sizes over plain P-values is another important contribution to distance our medical understanding from foregone maladministration. So, the next time writing a paper we are to make sure to take that step and not only present significant P-values but a truly significant significance. CONFLICT OF INTEREST There is no conflict of interest to declare. REFERENCES 1Mackenbach JP. Political conditions and life expectancy in Europe, 1900–2008. Soc Sci Med. 2013; 82: 134- 146. CrossrefPubMedWeb of Science®Google Scholar 2Kontis V, Bennett JE, Mathers CD, Li G, Foreman K, Ezzati M. Future life expectancy in 35 industrialised countries: projections with a Bayesian model ensemble. Lancet. 2017; 389(10076): 1323- 1335. CrossrefPubMedWeb of Science®Google Scholar 3Eisinger RW, Fauci AS. Ending the HIV/AIDS pandemic. Emerg Infect Dis. 2018; 24(3): 413- 416. CrossrefPubMedWeb of Science®Google Scholar 4Waks AG, Winer EP. Breast cancer treatment: a review. JAMA. 2019; 321(3): 288- 300. CrossrefCASPubMedWeb of Science®Google Scholar 5Howlader N, Forjaz G, Mooradian MJ, et al. The effect of advances in lung-cancer treatment on population mortality. N Engl J Med. 2020; 383(7): 640- 649. CrossrefCASPubMedWeb of Science®Google Scholar 6Bartolucci AA, Tendera M, Howard G. Meta-analysis of multiple primary prevention trials of cardiovascular events using aspirin. Am J Cardiol. Excerpta Medica, 2011; 107: 1796- 1801. CrossrefCASPubMedWeb of Science®Google Scholar 7Sullivan GM, Feinn R. Using effect size—or why the P value is not enough. J Grad Med Educ. 2012; 4(3): 279- 282. CrossrefPubMedGoogle Scholar 8Fontelo P, Liu F. A review of recent publication trends from top publishing countries. Syst Rev. 2018; 7(1): 147. https://doi.org/10.1186/s13643-018-0819-1CrossrefPubMedWeb of Science®Google Scholar 9Ansermet C, Centeno G, Lagarrigue S, et al. Renal tubular arginase-2 participates in the formation of the corticomedullary urea gradient and attenuates kidney damage in ischemia-reperfusion injury in mice. Acta Physiol. 2020; 229(3):e13457. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 10Zhu H, Toan S, Mui D, Zhou H. Mitochondrial quality surveillance as a therapeutic target in myocardial infarction. Acta Physiol. 2021; 231(3):e13590. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 11Li Z, Luo W, Fang S, et al. Prostacyclin facilitates vascular smooth muscle cell phenotypic transformation via activating TP receptors when IP receptors are deficient. Acta Physiol. 2021; 231(2):e13555. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 12Ying L, Benjanuwattra J, Chattipakorn SC, Chattipakorn N. The role of RIPK3-regulated cell death pathways and necroptosis in the pathogenesis of cardiac ischaemia-reperfusion injury. Acta Physiol. 2021; 231(2):e13541. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 13Gou D, Zhou J, Song Q, et al. Mog1 knockout causes cardiac hypertrophy and heart failure by downregulating tbx5-cryab-hspb2 signalling in zebrafish. Acta Physiol. 2021; 231: e13567. https://doi.org/10.1111/apha.13567Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 14Solagna F, Nogara L, Dyar KA, et al. Exercise-dependent increases in protein synthesis are accompanied by chromatin modifications and increased MRTF-SRF signalling. Acta Physiol. 2020; 230(1): 13496. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 15Xu N, Jiang S, Persson PB, Persson EAG, Lai EY, Patzak A. Reactive oxygen species in renal vascular function. Acta Physiol. 2020; 229(4):e13477. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 16Taguchi K, Kaneko N, Okudaira K, Matsumoto T, Kobayashi T. GLP-1 modulates insulin-induced relaxation response through β-arrestin2 regulation in diabetic mice aortas. Acta Physiol. 2021; 231(3):e13573. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 17Murtaza B, Hichami A, Khan AS, et al. Implication of TRPC3 channel in gustatory perception of dietary lipids. Acta Physiol. 2021; 231(2):e13554. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 18Lawrence MM, van Pelt DW, Confides AL, et al. Massage as a mechanotherapy promotes skeletal muscle protein and ribosomal turnover but does not mitigate muscle atrophy during disuse in adult rats. Acta Physiol. 2020; 229(3):e13460. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 19Franken GAC, Adella A, Bindels RJM, Baaij JHF. Mechanisms coupling sodium and magnesium reabsorption in the distal convoluted tubule of the kidney. Acta Physiol. 2021; 231(2):e13528. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 20Odening KE. Mechanism-based QT/APD-shortening therapy for potassium-channel-based long-QT syndromes? Acta Physiol. 2020; 229(4):e13481. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 21Berg P, Svendsen SL, Hoang TTL, Praetorius HA, Sorensen MV, Leipziger J. Impaired renal HCO3− secretion in CFTR deficient mice causes metabolic alkalosis during chronic base-loading. Acta Physiol. 2021; 231: e13591. https://doi.org/10.1111/apha.13591Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 22Moretti CH, Schiffer TA, Li X, Weitzberg E, Carlström M, Lundberg JO. Germ-free mice are not protected against diet-induced obesity and metabolic dysfunction. Acta Physiol. 2021; 231(3):e13581. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 23Monti E, Toniolo L, Marcucci L, et al. Are muscle fibres of body builders intrinsically weaker? A comparison with single fibres of aged-matched controls. Acta Physiol. 2021; 231(2):e13557. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar 24Pallubinsky H, Phielix E, Dautzenberg B, et al. Passive exposure to heat improves glucose metabolism in overweight humans. Acta Physiol. 2020; 229(4): 13488. Wiley Online LibraryCASPubMedWeb of Science®Google Scholar Citing Literature Volume232, Issue4August 2021e13665 ReferencesRelatedInformation