Portal de Periódicos da CAPES

Guidance on Communication of Uncertainty in Scientific Assessments

2019; Wiley; Volume: 17; Issue: 1 Linguagem: Inglês

10.2903/j.efsa.2019.5520

1831-4732

Andrew Hart, Laura Maxim, Michael Siegrist, Natalie von Goetz, Cristina da Cruz, Caroline Merten, Olaf Mosbach‐Schulz, E.S. Lahaniatis, Anthony Smith, Amy Hardy,

Risk and Safety Analysis

EFSA JournalVolume 17, Issue 1 e05520 Guidance DocumentOpen Access Guidance on Communication of Uncertainty in Scientific Assessments European Food Safety Authority, European Food Safety AuthoritySearch for more papers by this authorAndrew Hart, Andrew HartSearch for more papers by this authorLaura Maxim, Laura MaximSearch for more papers by this authorMichael Siegrist, Michael SiegristSearch for more papers by this authorNatalie Von Goetz, Natalie Von GoetzSearch for more papers by this authorCristina da Cruz, Cristina da CruzSearch for more papers by this authorCaroline Merten, Caroline MertenSearch for more papers by this authorOlaf Mosbach-Schulz, Olaf Mosbach-SchulzSearch for more papers by this authorMajlinda Lahaniatis, Majlinda LahaniatisSearch for more papers by this authorAnthony Smith, Anthony SmithSearch for more papers by this authorAnthony Hardy, Anthony HardySearch for more papers by this author European Food Safety Authority, European Food Safety AuthoritySearch for more papers by this authorAndrew Hart, Andrew HartSearch for more papers by this authorLaura Maxim, Laura MaximSearch for more papers by this authorMichael Siegrist, Michael SiegristSearch for more papers by this authorNatalie Von Goetz, Natalie Von GoetzSearch for more papers by this authorCristina da Cruz, Cristina da CruzSearch for more papers by this authorCaroline Merten, Caroline MertenSearch for more papers by this authorOlaf Mosbach-Schulz, Olaf Mosbach-SchulzSearch for more papers by this authorMajlinda Lahaniatis, Majlinda LahaniatisSearch for more papers by this authorAnthony Smith, Anthony SmithSearch for more papers by this authorAnthony Hardy, Anthony HardySearch for more papers by this author First published: 16 January 2019 https://doi.org/10.2903/j.efsa.2019.5520Citations: 26 Correspondence: socialscience@efsa.europa.eu Requestor: EFSA Question number: EFSA-Q-2017-00466 Acknowledgements: EFSA wishes to thank the Scientific Committee members for the support provided on this scientific output: Vasileios Bampidis, Diane Benford, Jos Boesten, Claude Bragard, Thorhallur Ingi Halldorsson, Antonio F. Hernández-Jerez, Susanne Hougaard Bennekou, Konstantinos Panagiotis Koutsoumanis, Simon John More, Hanspeter Naegeli, Søren Saxmose Nielsen, Josef Rudolf Schlatter, Dieter Schrenk, Vittorio Silano, Dominique Turck, Maged Younes. This document benefited also from the review of Barbara Gallani and the support of Peter Craig. Adopted: 21 November 2018 This publication is linked to the following EFSA Supporting Publications article: http://onlinelibrary.wiley.com/doi/10.2903/sp.efsa.2019.EN-1540/full AboutSectionsPDF ToolsExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This document provides guidance for communicators on how to communicate the various expressions of uncertainty described in EFSA's document: 'Guidance on uncertainty analysis in scientific assessments'. It also contains specific guidance for assessors on how best to report the various expressions of uncertainty. The document provides a template for identifying expressions of uncertainty in scientific assessments and locating the specific guidance for each expression. The guidance is structured according to EFSA's three broadly defined categories of target audience: 'entry', 'informed' and 'technical' levels. Communicators should use the guidance for entry and informed audiences, while assessors should use the guidance for the technical level. The guidance was formulated using evidence from the scientific literature, grey literature and two EFSA research studies, or based on judgement and reasoning where evidence was incomplete or missing. The limitations of the evidence sources inform the recommendations for further research on uncertainty communication. Summary The European Food Safety Authority (EFSA) requested its Scientific Committee and Emerging Risk Unit (SCER) and its Communication Unit (COM) to establish a working group to develop practical guidance for EFSA communicators on how to communicate the various expressions of uncertainty resulting from the uncertainty analyses (e.g. qualitative, quantitative) described in EFSA's document: 'Guidance on uncertainty analysis in scientific assessments' (EFSA Scientific Committee, 2018a; henceforth, 'Uncertainty Analysis GD'). The European Union (EU) Food Law identifies the target audiences of risk communication on EU food and feed safety: 'risk assessors, risk managers, consumers, feed and food businesses, the academic community and other interested parties'. EFSA tailors its communication messages according to the expected scientific literacy of these audiences and their varying interests, splitting them into three broad categories – 'entry', 'informed' and 'technical' levels. EFSA delivers layered content to these audiences through a mixture of channels (meetings, media, website, social media, journal) and formats (scientific opinions, news stories, multimedia, posts) depending on their information needs and behaviours. The application of EFSA's Uncertainty Analysis GD can result in eight main types of expressions of uncertainty: (1) unqualified conclusions with no expression of uncertainty; (2) description of a source of uncertainty; (3) qualitative description of the direction and/or magnitude of uncertainty; (4) inconclusive assessment; (5) a precise probability; (6) an approximate probability; (7) a probability distribution; and (8) a two-dimensional probability distribution. In this communication guidance document (henceforth, 'Uncertainty Communication GD'), these eight expressions provide the framework for the discussion of evidence sources and the guidance on communication of uncertainty. This Uncertainty Communication GD is primarily aimed at communicators at EFSA, but could be applied by risk communicators in other institutions that provide scientific advice. It should be used as a supporting document alongside the EFSA handbook: When food is cooking up a storm – proven recipes for risk communications (EFSA, 2017; henceforth, EFSA Risk Communication Handbook), which is a practical guide for food safety risk communicators. This document contains general and specific guidance for communication to audiences within the three levels: guidance for the entry and informed levels is addressed to communicators producing supporting communications (e.g. news stories), while the guidance for the technical level is addressed to assessors producing scientific outputs (e.g. opinions, reports). The guidance consists of clear instructions, precautions, examples and choices to consider. The general guidance should be used for all uncertainty communication. Use of the specific guidance is case-by-case; an easy-to-use form is the main tool provided for identifying the specific guidance applicable to the eight different uncertainty expression types. The general and specific guidances were developed from an analysis of key evidence sources including selected academic literature, extracts from similar frameworks or guidance documents from other national or international advisory bodies, and the results of EFSA's own target audience research studies. Although the available evidence on the best ways to communicate the uncertainty expressions was limited overall, the expert analysis of selected academic literature provided a useful starting point. All but two of the eight expressions of uncertainty resulting from uncertainty analyses, i.e. 'unqualified conclusion with no expression of uncertainty' and 'inconclusive assessment', were extracted from the selected papers, either as formulated by the authors themselves or interpreted and drafted by the working group. The recommendations from this literature were applied and tested on real examples of EFSA scientific assessments, to draft messages and select visual aids for communicating the related uncertainties. Since the available evidence does not, however, address every aspect of communicating uncertainty, some guidance in the current document is based on judgement and reasoning. EFSA commissioned a focus group study in 2016 (Etienne et al., 2018) and carried out its own follow-up multilingual online survey in 2017 (EFSA, 2018) that represented the early development phase of the Communication GD. The studies provided indications of target audience perspectives of the usefulness of uncertainty information, the impact of language, culture and professional background, and on audiences' understanding of and/or preferences for messages describing four types of expressions of uncertainty. Both studies have limitations in their design and execution but, considered cautiously, they are useful sources of insights directly from EFSA's key target audiences. How uncertainties should be conveyed to enable non-scientists to make informed decisions is still an under-researched field. Experience gained during the implementation of this Uncertainty Communication GD itself will provide new insights into the best way to communicate different expressions of uncertainty in scientific assessments. EFSA therefore intends to review and, if needed, update the Uncertainty Communication GD over the next five years. Future research should address how various subjective probabilities could be communicated to laypeople so that they understand the information. Additional research should examine how well test subjects understand uncertainty communication and whether various communication formats result in different decisions. All such research should also involve decision-makers in the risk management domain, because these stakeholders may reach different conclusions depending on how uncertainty is communicated. 1 Introduction 1.1 Background The European Food Safety Authority's (EFSA) Guidance document on uncertainty analysis in scientific assessment (EFSA Scientific Committee, 2018a; henceforth, 'Uncertainty Analysis GD') defines 'uncertainty' as a general term referring to all types of limitations in available knowledge that affect the range and probability of possible answers to an assessment question. Available knowledge refers here to the knowledge (evidence, data, etc.) available to assessors at the time the assessment is conducted and within the time and resources agreed for the assessment. Sometimes 'uncertainty' is used to refer to a source of uncertainty (see separate definition), and sometimes to its impact on the conclusion of an assessment. In science and statistics, we are familiar with concepts such as measurement uncertainty and sampling uncertainty, and that weaknesses in methodological quality of studies used in assessments can be important sources of uncertainty. Uncertainties in how evidence is used and combined in assessment – e.g. model uncertainty, or uncertainty in weighing different and sometimes conflicting lines of evidence in a reasoned argument – are also important sources of uncertainty. General types of uncertainty that are common in EFSA assessments are outlined in Section 8 of its 'Scientific opinion on the principles and methods behind EFSA's guidance on uncertainty analysis in scientific assessment' (EFSA Scientific Committee, 2018b). There the Scientific Committee recommended closer interaction between assessors and decision-makers both during the assessment process and when communicating the conclusions. Understanding of the type and degree of uncertainties identified in the assessment helps to characterise the level of risk to the recipients and is therefore essential for informed decision-making (EFSA Scientific Committee, 2018b). Communication helps them to understand the range and likelihood of possible consequences. During the development of the Uncertainty Analysis GD, the Scientific Committee reviewed literature on the communication of uncertainty information as a basis for developing a common approach for EFSA's communications on uncertainty to different target audiences, including decision-makers and the general public. That initial review indicated that the literature is equivocal about the most effective strategies to communicate scientific uncertainties and that, on the whole, there is a lack of empirical data in the literature on which to base a working model. In EFSA's own publication on best practice in risk communication 'When food is cooking up a storm – proven recipes for risk communications' (EFSA, 2017), which was developed with national partners, the advice to communicators is only of a general nature. Therefore, the Scientific Committee recommended that EFSA should initiate research activities to explore best practices and develop further guidance in areas where this would benefit implementation of the Uncertainty Analysis GD, and the communication of uncertainties in scientific assessments targeted at different audiences. This would allow EFSA to identify how changes could be made to its current communications practices in relation to uncertainties and to tailor key messages to specific target audience needs. As EFSA completed its research activities on communication of uncertainties to different target audiences (EFSA, 2018; Etienne et al., 2018), the Scientific Committee proposed to develop a separate Guidance document on communication of uncertainty in scientific assessments (henceforth 'Uncertainty Communication GD'). The Scientific Committee considered that the significance of the research results and the different purpose, scope and target of the communication methodology warranted a stand-alone document for communication practitioners. The Uncertainty Communication GD is a companion document to the Uncertainty Analysis GD. 1.2 Terms of Reference as provided by the requestor The European Food Safety Authority (EFSA) asked its Scientific Committee and Emerging Risk Unit (SCER) and its Communication Unit (COM) to establish a working group to develop guidance on how to communicate uncertainty on the basis of EFSA's Uncertainty Analysis GD. The Scientific Committee advised that to carry out this work expertise in social sciences (e.g. psychology, risk communication, uncertainty communication and public perceptions) was needed to join its working group on uncertainty. The working group had the following three objectives: Develop practical guidance for EFSA communicators on how to communicate the various expressions of the uncertainty analyses (e.g. qualitative, quantitative) described in EFSA's Uncertainty Analysis GD, and in the 'Scientific opinion on the principles and methods behind EFSA's guidance on uncertainty analysis in scientific assessment' (EFSA Scientific Committee, 2018b) with the aim of bridging risk assessors and the different EFSA target audiences. Advise assessors on the ways in which uncertainties are reported in EFSA assessments in relation to the need to communicate. Advise EFSA on its current communication approach for dealing with uncertainty as described in the EFSA Risk Communication Handbook (EFSA, 2017). 1.3 Interpretation of the Terms of Reference The working group developed the following work plan to reach the three objectives of the Terms of Reference: Develop practical guidance for EFSA communicators on how to communicate the various expressions of the uncertainty analyses (e.g. qualitative, quantitative) described in EFSA's Uncertainty Analysis GD and in the 'Scientific opinion on the principles and methods behind EFSA's guidance on uncertainty analysis in scientific assessment' (EFSA Scientific Committee, 2018b). The following tasks were planned to reach objective 1: – Identify review papers for risk communication and relate them to uncertainty communication. – Perform a literature search on uncertainty communication and review the resulting literature. – Complement the literature with the insights gained from the EFSA research projects on communication of uncertainty from 2016 and 2017. – Perform an online search on approaches to communicating uncertainty by relevant national and international organisations. – Identify a representative set of outputs with case studies upon which EFSA might communicate, map the different sensitivities of topics dealt with by EFSA, target audiences, different types of assessment and expressions of uncertainty (e.g. probabilities, quantitative, qualitative), linking them to the methods described in the Uncertainty Analysis GD. – Consider whether different communications on uncertainty are needed for EFSA's defined target audiences and/or whether new categories are required. – Develop a practical communications approach and supporting tools for communications practitioners who are required to communicate scientific uncertainties to different target audiences. Use the literature review and the results of EFSA's target audience research activities conducted in 2016 and 2017 to inform the approach. – Draft this Guidance document on communication of uncertainty for consultation and follow-up on feedback from the consultation activities with a report and input for finalising the Guidance document. – Consult EFSA's risk communication partners in the European Union (EU) Member States, the EU institutions and other interested parties (e.g. other EU agencies, international organisations, non-EU countries) before finalising the Communication GD. Advise assessors on the ways in which uncertainties are reported in EFSA assessments in relation to the need to communicate. The following tasks were identified to reach objective 2: – For the results of objective 1, consider whether there are additional requirements and/or recommendations (e.g. terminology, data format, graphics) that Panels should be aware of when drafting their opinions and especially the conclusions of their assessments. Advise EFSA on its current communication approach for dealing with uncertainty as described in the EFSA Risk Communication Handbook (EFSA, 2017). The following tasks were identified to reach objective 3: – Review the relevant section of the EFSA Risk Communication Handbook. – Identify key examples of past communication challenges in which uncertainty was a decisive issue to determine the impact of a new approach to uncertainty communication. 1.4 Scope, audience and use The Terms of Reference require providing guidance for EFSA on how to communicate uncertainty on the basis of its Uncertainty Analysis GD. This Uncertainty Communication GD provides a practical framework for communicating uncertainties in scientific assessments. It does not provide a template for EFSA's risk communication activities as this takes place within a well-defined legal framework. However, a short description of EFSA's risk communication role, strategies and target audiences follows below (Section 1.5) as background and context. 1.4.1 Audience for this guidance This Uncertainty Communication GD is primarily aimed at risk communicators at EFSA, but it may also be helpful to risk communicators at other institutions that provide scientific advice. It does not replace the current EFSA Risk Communication Handbook (EFSA, 2017), which describes the overall framework and practical approaches for risk communication at EFSA, but supports and complements that publication. In addition to guidance for communicators, this document contains some guidance for assessors on how best to report the various expressions of uncertainty resulting from their uncertainty analyses so that they can support effective uncertainty communication. 1.4.2 Structure of the guidance document This Uncertainty Communication GD first explains EFSA's three broad communication target audiences (Section 1.5.2) and the eight different possible expressions of uncertainty as described in the Uncertainty Analysis GD (Section 2). As shown in Figure 1, Risk communicators looking for straightforward instructions on how best to communicate the various expression of uncertainty should go to Section 3. The guidance for the entry- and informed-level audiences is addressed to communicators, while the guidance for the technical level is addressed to assessors and advises them on the information to provide in their assessments that is needed for communication. Section 4 describes the evidence sources that contributed to development of the guidance. Additional reasoning is summarised in Appendix B. Section 5 provides recommendations for further research and Section 6 outlines EFSA's plans for implementing the approach in its working practices and subsequent evaluation. Figure 1Open in figure viewerPowerPoint Visual map of the structure and how to use this Uncertainty Communication GD 1.5 Risk communication at EFSA EFSA's science communications role within the EU food safety system is discussed in this section. For terminology, the Uncertainty Analysis GD refers to 'scientific assessment' rather than 'risk assessment' to recognise that the Uncertainty Analysis GD is applicable to all of EFSA's scientific advisory work, not solely its risk assessments. Notwithstanding this, most of the following section refers to EFSA's 'risk communication' role because this is the terminology used in the relevant legal texts establishing EFSA. However, this Uncertainty Communication GD applies to all of EFSA's 'science communication' activities and so is titled in full: 'Guidance on communication of uncertainty in scientific assessments'. 1.5.1 EFSA's risk communication role and strategies Under the EU Food Law, Regulation (EC) No 178/20021, by which EFSA was founded, EFSA is 'an independent scientific source of advice, information and risk communication to improve consumer confidence'. It defines risk communication as 'the interactive exchange of information and opinions throughout the risk analysis process as regards hazards and risks, risk-related factors and risk perceptions'. As stated by Codex (2018), risk communication should include a 'transparent explanation of the risk assessment policy and of the assessment of risk, including the uncertainty'. EFSA is charged with communicating the results of its work in the fields within its mission (food and feed safety, animal and plant health, nutrition) and with explaining its risk assessment findings. The European Commission is responsible for communicating its risk management decisions and the basis for them (i.e. scientific and/or other considerations). EU Member States are also charged with public communication on food and feed safety and risk. Given these overlapping roles, the Regulation also requires that EFSA collaborate closely with the Commission and the Member States to 'promote the necessary consistency in the risk communication processes'.2 Risk management measures, if needed, usually take further time to agree and implement after the completion of an assessment. Public perceptions of the related risks may be affected by such a timescale, requiring linked communication over the entire public information campaign. Critically, this also allows opportunities for feedback and dialogue with the representatives of consumers, producers and other interested parties, which is also required under the Food Law. With a clear mandate to communicate its scientific assessment results 'on its own initiative', EFSA follows a communications strategy based on guidelines in its EFSA Risk Communication Handbook (EFSA, 2017) that was developed together with the European Commission and members of EFSA's Communications Experts Network, which comprises communications representatives of EU national competent authorities on food and feed safety. The guidelines provide a framework to assist decision-making about appropriate communication approaches in a wide variety of situations that can occur when assessing and communicating on risks related to food safety in Europe. EFSA publishes over 500 scientific assessments and reports annually, but only about one-tenth of these are accompanied by supporting communications (e.g. press releases/interviews, FAQs, videos, briefings). EFSA's communicators weigh up several factors in selecting assessments for supporting communication and the mix of communication approaches and formats to employ for each. Broadly, these include: significance of the scientific results (e.g. routine vs new findings); nature of the risk (e.g. emerging, possible, identified or confirmed); potential public/animal health/environmental impact (e.g. there is a safety concern); public perception and anticipated reactions/sensitivity of subject area; legislative and market contexts (e.g. a request for authorisation); urgency of request (e.g. in an outbreak situation, or a long-term review); institutional risk (national, European, international contexts). The existence of one or more of the above factors in relation to an upcoming assessment or other activity is a potential trigger for such supporting communications. The communicator analyses the key results of the assessment or report, discusses key messages and supporting points with the assessors (scientific officers, EFSA Panel or working group members), agreeing a communications plan in the process. The plan identifies the rationale for communicating, the key messages and supporting points, and also defines the target audiences for the communication. 1.5.2 EFSA's target audiences The EU Food Law identifies the target audiences of risk communication on EU food and feed safety as 'risk assessors, risk managers, consumers, feed and food businesses, the academic community and other interested parties'. For EFSA's risk communication, the potential audience is therefore 500 million people residing in the European Union. Their interest, knowledge and concerns about food and food safety vary widely. Language, tradition, culture, age and education (e.g. scientific literacy) are among the variables that affect their understanding of messages about food safety-related assessments. Communicating directly to everyone is unrealistic given this diversity and complexity. Therefore, in the EU food safety system, many players share responsibility for communicating about food risks (and benefits) to consumers, food chain operators and other interested parties. EFSA cooperates with these other players: its partners in the EU Member States, EU institutions and 'stakeholder' groups (e.g. consumer organisations and public health professionals) to further disseminate the outcomes of its scientific assessments. To enable this, EFSA tailors its communication messages in layers to the expected scientific literacy and interests of these audiences and targets them through a mixture of channels (meetings, the media, website, social media, scientific journal) and formats (scientific opinions, news stories, multimedia, tweets). In devising its external communications, EFSA follows an approach for mapping and targeting these audiences (see Table 1) that was codified internally in 2014–2015 for the redesign of EFSA's corporate website. It was subsequently adapted to other communication channels and formats; more detailed segmentation of the audiences is possible for specific types of communication (e.g. to attract participants to events on specialist topics). The approach was based on the analysis of extensive user-centred research involving interviews, online surveys, analytics data (web metrics, media pick up) and external expertise, as well as the frameworks that guide EFSA's work: EU Food Law, EFSA's strategic documents and plans (see Annex A for further details). The table shows key target audience groups for EFSA that were identified through this research: decision-makers, assessors, industry, non-governmental organisations (NGOs)/specialised media, general media, and informed/concerned citizens. It also clusters them according to their scientific literacy and temporal relationship with EFSA's communications into three broad categories – 'entry', 'informed' and 'technical' levels. EFSA generally seeks to layer its communications content to improve accessibility for users in these broad audience categories. Certain audiences may generally prefer different communications products and channels, and the content can be tailored accordingly. The layering of information can also be within a single communication output. For example, a news story headline and introductory paragraphs are worded to be understandable to 'entry' level users while subsequent paragraphs may provide more details for 'informed' and 'technical' audiences. The mapping of target audiences and strategy for content development comes with important caveats. There is much diversity within the target audiences and considerable overlap between them in terms of the assumptions made about them (e.g. their scientific literacy) and the communication products they may use when informing themselves about EFSA. For example, industry representatives have widely varying degrees of scientific literacy and users in all categories have personal preferences that may guide them in selecting one format over another. Nevertheless this approach proved practical and was already in use at EFSA, making it a functioning framework for the purposes of the Communication GD. Therefore, these groups of target audiences will be used as a parameter in structuring the guidance in Section 3. For communicators not involved in developing or further disseminating EFSA's communic