Portal de Periódicos da CAPES

A special issue on Bayesian inference: challenges, perspectives and prospects

2023; Royal Society; Volume: 381; Issue: 2247 Linguagem: Inglês

10.1098/rsta.2022.0155

1471-2962

Christian P. Robert, Judith Rousseau,

Statistical Methods and Bayesian Inference

Open AccessMoreSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InRedditEmail Cite this article Robert Christian P. and Rousseau Judith 2023A special issue on Bayesian inference: challenges, perspectives and prospectsPhil. Trans. R. Soc. A.3812022015520220155http://doi.org/10.1098/rsta.2022.0155SectionOpen AccessIntroductionA special issue on Bayesian inference: challenges, perspectives and prospects Christian P. Robert Christian P. Robert http://orcid.org/0000-0001-6635-3261 CEREMADE, University of Paris Dauphine, Paris, France Department of Statistics, University of Warwick, Coventry, UK [email protected] Contribution: Writing – original draft, Writing – review & editing Google Scholar Find this author on PubMed Search for more papers by this author and Judith Rousseau Judith Rousseau http://orcid.org/0000-0002-0998-6174 Department of Statistics, University of Oxford, Oxford, UK Contribution: Writing – original draft, Writing – review & editing Google Scholar Find this author on PubMed Search for more papers by this author Christian P. Robert Christian P. Robert http://orcid.org/0000-0001-6635-3261 CEREMADE, University of Paris Dauphine, Paris, France Department of Statistics, University of Warwick, Coventry, UK [email protected] Contribution: Writing – original draft, Writing – review & editing Google Scholar Find this author on PubMed and Judith Rousseau Judith Rousseau http://orcid.org/0000-0002-0998-6174 Department of Statistics, University of Oxford, Oxford, UK Contribution: Writing – original draft, Writing – review & editing Google Scholar Find this author on PubMed Published:27 March 2023https://doi.org/10.1098/rsta.2022.0155This special issue is dedicated to Sir Adrian Smith, whose contributions to Bayesian analysis have deeply impacted the field (or rather fields) of Bayesian inference, decision theory and statistical computing. When contemplating his past achievements, it is striking to align the emergence of massive advances in these fields with some papers or books of his. For instance, Lindley's & Smith's 'Bayes Estimates for the Linear Model' (1971) [1], a Read Paper at the Royal Statistical Society, is making the case for the Bayesian analysis of the most standard statistical model, as well as emphasizing the notion of exchangeability that is foundational in Bayesian statistics, and paving the way to the emergence of hierarchical Bayesian modelling. It thus makes a link between the early days of Bruno de Finetti, whose work Adrian Smith translated into English, and the current research in non-parametric and robust statistics. Bernardo's & Smith's masterpiece, Bayesian Theory (1994) [2], sets statistical inference within decision- and information-theoretic frameworks in a most elegant and universal manner that could be deemed a Bourbaki volume for Bayesian statistics if this classification endeavour had reached further than pure mathematics. It also emphasizes the central role of hierarchical modelling in the construction of priors, as exemplified in Carlin's et al. 'Hierarchical Bayesian analysis of changepoint problems' (1992) [3].The series of papers published in 1990 by Alan Gelfand & Adrian Smith, esp. 'Sampling-Based Approaches to Calculating Marginal Densities' (1990) [4], is overwhelmingly perceived as the birth date of modern Markov chain Monte Carlo (MCMC) methods, as it brought to the whole statistics community (and then quickly wider communities) the realization that MCMC simulation was the sesame to unlock complex modelling issues. The consequences on the adoption of Bayesian modelling by non-specialists are enormous and long-lasting.Similarly, Gordon's et al. 'Novel approach to nonlinear/non-Gaussian Bayesian state estimation' (1992) [5] is considered as the birthplace of sequential Monte Carlo, aka particle filtering, with considerable consequences in tracking, robotics, econometrics and many other fields.Titterington's, Smith's & Makov's reference book, 'Statistical Analysis of Finite Mixtures' (1984) [6] is a precursor in the formalization of heterogeneous data structures, paving the way for the incoming MCMC resolutions like Tanner & Wong (1987) [7], Gelman & King (1990) [8] and Diebolt & Robert (1990) [9]. Denison et al.'s book, 'Bayesian methods for nonlinear classification and regression' (2002) [10] is another testimony to the influence of Adrian Smith on the field, stressing the emergence of robust and general classification and nonlinear regression methods to analyse complex data, prefiguring in a way the later emergence of machine-learning methods, with the additional Bayesian assessment of uncertainty. It is also bringing forward the capacity of operating Bayesian non-parametric modelling that is now broadly accepted, following a series of papers by Denison et al. in the late 1990s [11] like CART and MARS.Peter Grünwald [12] proposes a decision-based construction of Bayesian inference and uncertainty quantification via e-posteriors that bypasses the precise construction of a prior distribution. It is advanced as a more robust form of inference, with guaranteed frequentist properties.Chris Holmes & Stephen Walker [13] connect exchangeability with Bayesian bootstrap. Exchangeability is at the core of Bayesian thinking, as introduced by de Finetti (1937) [14] and emphasized in Bernardo & Smith (1994) [2]. Their perspective reinforces this position by explaining exchangeability through learning, which is the main argument for pursuing the Bayesian approach to statistics, while connecting with a more classical approach by predicting an infinite population from the current dataset. By extending the notion of Bayesian bootstrap, they reformulate the Bayesian paradigm as purely predictive.Sonia Petrone [15] also focuses on the predictive aspect of Bayesian inference, making the crucial point that it is a natural way to connect with machine-learning imperatives. She makes the method appear as naturally embedded within non-parametric statistics, as a form of Polva urn sampling, and establishes limit coverage properties that grant frequentist properties to the ensuing Bayesian procedures.Veronika Rockova & Lizhen Nie [16] brings forward another perspective on the Bayesian bootstrap. The approach there is more likelihood-free in the sense that the sampling model is no longer defined through a likelihood but instead through a loss function as in Bissiri et al. (2016) [17]. A strong innovation in this approach is to rely on a machine-learning-based tool, a generative sampler that allows for a generalization of the Bayesian bootstrap of Newton & Raftery (1994) [18] and that brings both computational advantages and theoretical guarantees, including under model mis-specification, a perspective under-studied in the field.When seeking computational solutions to approximate an essential Bayesian entity, namely the Bayes factor (or the evidence), Arnaud Doucet et al. [19] similarly exploit machine-learning tools like variational auto-encoders and normalizing flows to build rich families of approximations in settings where the likelihood function is unavailable. They bring in an annealing perspective to improve further the computational efficiency of these importance sampling methods.Alicia Carriquiry & Kori Khan [20] build a safer environment for forensic calibration studies by resorting to hierarchical Bayes modelling, investigated by Adrian Smith in his early career, and borrowing from small area estimation, thus bringing a quantitative evaluation of the imprecision in using this tool for reporting error rates.Matthew Stephens [21] contributes a broader perspective to the volume by connecting Bayesian principles with non-Bayesian procedures, like confidence intervals and p-values, thus connecting with Peter Grünwald e-values. He also advocates considering alternative approaches for faster or more robust resolutions.Sylvia Früwirth-Schnatter [22] broadens the range of shrinkage priors in connection with spike-and-slab priors used in variable selection and factor analysis. She reveals new connections with Bayesian non-parametrics and exchangeability. Her computational aspects are also most relevant for the theme of this volume since they involve MCMC and data augmentation, where Adrian Smith's contributions have been fundamental.Sara Wade [23] advocates for a Bayesian approach to cluster analysis, of which Adrian Smith has been an early researcher when working on mixtures of distributions (Titterington et al. 1984 [6]). She brings another connection with the Gibbs priors of Bissiri et al. (2016) [17].Kerrie Mengersen & her co-authors [24] also offer a very broad perspective on the opportunities and challenges of applied Bayesian analysis, including design (with links to decision theory), data sources and citizen science, federated learning and data privacy, with MCMC consequences, implicit models and ABC, and model transfer. All these points are illustrated by diverse and exciting case studies.Richard Nickl & co-authors [25] study hindrances to MCMC convergence in high dimensions, reconnecting with the early days when MCMC and simulated annealing were created in parallel and phase transition sounded like an epiphenomenon. The notion of entropy wells is quite appealing in demonstrating that convergence issues can arise with MCMC even in the case of unimodal (if not log-concave) targets. This paper provides further arguments against local Monte Carlo methods and possibly in favour of new schemes like PDMPs although none is included here. For a specific Gaussian tensor model, the posterior concentrates exponentially fast (with the dimension) over a small region of the space, inducing an equally exponential time for 'small step' MCMC algorithms to converge. This includes both unadjusted and Metropolis-adjusted Langevin algorithms.Peter Müller et al. [26] open a new perspective on the Bayesian exploitation of complex data sources like medical records. They demonstrate how Bayesian non-parametrics can be used in that context, with consequence on drug trials. The approach relies on MCMC methods. The paper also relates to Adrian Smith's work on mixtures and Bayesian non-parametrics.Sumio Watanabe [27] presents his original perspective on Bayesian information criteria (BIC), within the context of cross-validation, including new results on the asymptotic behaviour of Bayes factors, new types of losses and BICs. The approach defended therein also allows for an assessment of prior distributions and a novel form of empirical Bayes analysis by optimizing hyperparameters.Beatrice Franzolini & co-authors [28] study an integer-supported zero-inflated model called the hurdle model, with the addition of a clustering mechanism, connecting with Sara Wade's article [24]. The work builds on earlier advances on finite mixtures with random number of components, to bring higher computational efficiency in the Bayesian analysis of the model than when using non-parametric alternatives.Data accessibilityThis article has no additional data.Authors' contributionsC.P.R.: writing—original draft, writing—review and editing; J.R.: writing—original draft, writing—review and editing.All authors gave final approval for publication and agreed to be held accountable for the work performed therein.Conflict of interest declarationWe declare we have no competing interest.FundingWe received no funding for this study.FootnotesOne contribution of 16 to a theme issue 'Bayesian inference: challenges, perspectives, and prospects'.© 2023 The AuthorsPublished by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.References1. Lindley DB , Smith AFM . 1972Bayes estimates for the linear model. J. R. Stat. Soc., Ser. B, Methodol. 34, 1-41. (doi:10.1111/j.2517-6161.1972.tb00885.x) Google Scholar2. Bernardo JM , Smith AFM . 1994Bayesian Theory. Hoboken, NY: John Wiley & Sons. Crossref, Google Scholar3. Carlin BP , Gelfand , Smith AFM . 1992Hierarchical Bayesian analysis of changepoint problems. J. R. Stat. Soc. Ser. C Appl. Stat. 41, 389-405. (doi:10.2307/2347570) Google Scholar4. Gelfand A, Smith A. 1990Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc 85, 398-409. (doi:10.1080/01621459.1990.10476213) Crossref, ISI, Google Scholar5. Gordon NJ , Salmond , Smith AFM . 1992Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F. 140, 107-113. (doi:10.1049/ip-f-2.1993.0015) Google Scholar6. Titterington , Smith , Makov . 1984Statistical analysis of finite mixtures. Chichester, NY: John Wiley & Sons. Google Scholar7. Tanner M Wong W. 1987The calculation of posterior distributions by data. J. Am. Stat. Assoc 82, 528-550. (doi:10.1080/01621459.1987.10478458) Crossref, Google Scholar8. Gelman A, King G. 1990Estimating the electoral consequences of legislative redistricting. J. Am. Stat. Assoc 85, 274-282. (doi:10.1080/01621459.1990.10476199) Crossref, Google Scholar9. Diebolt J, Robert C. 1990Estimation des paramètres d'un mélange par échantillonage bayésien. Notes aux Comptes Rendus de l'Académie des Sciences 311, 653-658. Google Scholar10. Denison DGT , Holmes CC , Mallick BK, Smith AFM . 2002Bayesian methods for nonlinear classification and regression. New York, NY: John Wiley & Sons. Google Scholar11. Dension DGT, Mallick BK, Smith AFM. 1990A Bayesian CART algorithm. Biometrika 85, 363-377. (doi:10.1093/biomet/85.2.363) Crossref, Google Scholar12. Grünwald PD. 2023The e-posterior. Phil. Trans. R. Soc. A 381, 20220146. (doi:10.1098/rsta.2022.0146) Link, Google Scholar13. Holmes CC, Walker SG. 2023Statistical inference with exchangeability and martingales. Phil. Trans. R. Soc. A 381, 20220143. (doi:10.1098/rsta.2022.0143) Link, Google Scholar14. de Finetti B. 1937La prévision : ses lois logiques, ses sources subjectives. Annales de l'institut Henri Poincaré 7, 1-68. Google Scholar15. Fortini S, Petrone S. 2023Prediction-based uncertainty quantification for exchangeable sequences. Phil. Trans. R. Soc. A 381, 20220142. (doi:10.1098/rsta.2022.0142) Link, Google Scholar16. Nie L, Ročková V. 2023Deep bootstrap for Bayesian inference. Phil. Trans. R. Soc. A 381, 20220154. (doi:10.1098/rsta.2022.0154) Link, Google Scholar17. Bissiri PG, Holmes CC, Walker SG. 2016A general framework for updating belief distributions. J. R. Stat. Soc. Series B Stat. Methodol. 78, 1103-1130. (doi:10.1111/rssb.12158) Crossref, PubMed, Google Scholar18. Newton MA, Raftery AE. 1994Approximate Bayesian inference by the weighted likelihood bootstrap (with discussion). J. R. Stat. Soc. Series B Stat. Methodol. 56, 1-48. (doi:10.1111/j.2517-6161.1994.tb01956.x) Google Scholar19. Doucet A, Moulines E, Thin A. 2023Differentiable samplers for deep latent variable models. Phil. Trans. R. Soc. A 381, 20220147. (doi:10.1098/rsta.2022.0147) Link, Google Scholar20. Khan K, Carriquiry A. 2023Hierarchical Bayesian non-response models for error rates in forensic black-box studies. Phil. Trans. R. Soc. A 381, 20220157. (doi:10.1098/rsta.2022.0157) Link, Google Scholar21. Stephens M. 2023The Bayesian lens and Bayesian blinkers. Phil. Trans. R. Soc. A 381, 20220144. (doi:10.1098/rsta.2022.0144) Link, Google Scholar22. Frühwirth-Schnatter S. 2023Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis. Phil. Trans. R. Soc. A 381, 20220148. (doi:10.1098/rsta.2022.0148) Link, Google Scholar23. Wade S. 2023Bayesian cluster analysis. Phil. Trans. R. Soc. A 381, 20220149. (doi:10.1098/rsta.2022.0149) Link, Google Scholar24. Bon JJet al.2023Being Bayesian in the 2020s: opportunities and challenges in the practice of modern applied Bayesian statistics. Phil. Trans. R. Soc. A 381, 20220156. (doi:10.1098/rsta.2022.0156) Link, Google Scholar25. Bandeira AS, Maillard A, Nickl R, Wang S. 2023On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributions. Phil. Trans. R. Soc. A 381, 20220150. (doi:10.1098/rsta.2022.0150) Link, Google Scholar26. Müller P, Chandra NK, Sarkar A. 2023Bayesian approaches to include real-world data in clinical studies. Phil. Trans. R. Soc. A 381, 20220158. (doi:10.1098/rsta.2022.0158) Link, Google Scholar27. Watanabe S. 2023Mathematical theory of Bayesian statistics for unknown information source. Phil. Trans. R. Soc. A 381, 20220151. (doi:10.1098/rsta.2022.0151) Link, Google Scholar28. Franzolini B, Cremaschi A, van den Boom W, De Iorio M. 2023Bayesian clustering of multiple zero-inflated outcomes. Phil. Trans. R. Soc. A 381, 20220145. (doi:10.1098/rsta.2022.0145) Link, Google Scholar Next Article VIEW FULL TEXT DOWNLOAD PDF FiguresRelatedReferencesDetails This Issue15 May 2023Volume 381Issue 2247Theme issue 'Bayesian inference: challenges, perspectives, and prospects' compiled and edited by Christian P. Robert, Judith Rousseau and Michael I. Jordan Article InformationDOI:https://doi.org/10.1098/rsta.2022.0155PubMed:36970829Published by:Royal SocietyPrint ISSN:1364-503XOnline ISSN:1471-2962History: Manuscript received22/02/2023Manuscript accepted22/02/2023Published online27/03/2023Published in print15/05/2023 License:© 2023 The AuthorsPublished by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited. Citations and impact KeywordsBayesian inferencehierarchical Bayes modelingcomputational statisticsstatistical decision theoryMarkov chain Monte Carlo methodsBayesian statistics Subjectsstatistics