Organising evidence for environmental management decisions: a ‘4S’ hierarchy
2014; Elsevier BV; Volume: 29; Issue: 11 Linguagem: Inglês
10.1016/j.tree.2014.09.004
ISSN1872-8383
AutoresLynn V. Dicks, Jessica C. Walsh, William J. Sutherland,
Tópico(s)Risk Perception and Management
Resumo•Available environmental science is often not used in policy and practice decisions.•We advocate a '4S' hierarchy to incorporate science into environmental management decisions.•It comprises studies, systematic reviews, summaries, and decision support systems.•Every level of the hierarchy is poorly developed for use in environmental management decisions.•The top level – synthesized science in decision support systems – is the least developed. Making decisions informed by the best-available science is an objective for many organisations managing the environment or natural resources. Yet, available science is still not widely used in environmental policy and practice. We describe a '4S' hierarchy for organising relevant science to inform decisions. This hierarchy has already revolutionised clinical practice. It is beginning to emerge for environmental management, although all four levels need substantial development before environmental decision-makers can reliably and efficiently find the evidence they need. We expose common bypass routes that currently lead to poor or biased representation of scientific knowledge. We argue that the least developed level of the hierarchy is that closest to decision-makers, placing synthesised scientific knowledge into environmental decision support systems. Making decisions informed by the best-available science is an objective for many organisations managing the environment or natural resources. Yet, available science is still not widely used in environmental policy and practice. We describe a '4S' hierarchy for organising relevant science to inform decisions. This hierarchy has already revolutionised clinical practice. It is beginning to emerge for environmental management, although all four levels need substantial development before environmental decision-makers can reliably and efficiently find the evidence they need. We expose common bypass routes that currently lead to poor or biased representation of scientific knowledge. We argue that the least developed level of the hierarchy is that closest to decision-makers, placing synthesised scientific knowledge into environmental decision support systems. The use of relevant scientific evidence to inform decisions in policy or practice is an aspiration widely shared by private and public institutions, and strongly advocated by scientists and science funders. In the environmental field, 'evidence-based conservation' or 'evidence-informed conservation' has been discussed in the literature for almost 15 years [1Adams W.M. Sandbrook C. Conservation, evidence and policy.Oryx. 2013; 47: 329-335Crossref Scopus (146) Google Scholar, 2Keene M. Pullin A.S. Realizing an effectiveness revolution in environmental management.J. Environ. Manage. 2011; 92: 2130-2135Crossref PubMed Scopus (65) Google Scholar, 3Pullin A.S. Knight T.M. Effectiveness in conservation practice: pointers from medicine and public health.Conserv. Biol. 2001; 15: 50-54Crossref Scopus (333) Google Scholar, 4Pullin A.S. Knight T.M. Support for decision making in conservation practice: an evidence-based approach.J. Nat. Conserv. 2003; 11: 83-90Crossref Scopus (174) Google Scholar, 5Pullin A.S. Knight T.M. Doing more good than harm – building an evidence-base for conservation and environmental management.Biol. Conserv. 2009; 142: 931-934Crossref Scopus (195) Google Scholar, 6Segan D.B. et al.Using conservation evidence to guide management.Conserv. Biol. 2011; 25: 200-202Crossref PubMed Scopus (65) Google Scholar, 7Sutherland W.J. The Conservation Management Handbook. Blackwell Science, 2000Crossref Scopus (25) Google Scholar, 8Sutherland W.J. et al.The need for evidence-based conservation.Trends Ecol. Evol. 2004; 19: 305-308Abstract Full Text Full Text PDF PubMed Scopus (1283) Google Scholar, 9Dicks L.V. et al.A transparent process for 'evidence-informed' policy making.Conserv. Lett. 2014; 7: 119-125Crossref Scopus (86) Google Scholar, 10Haddaway N. Pullin A.S. Evidence-based conservation and evidence-informed policy: a response to Adams & Sandbrook.Oryx. 2013; 47: 336-338Crossref Scopus (16) Google Scholar], and evidence-based policy is often claimed in other areas, such as energy and climate change [11Juntti M. et al.Evidence, politics and power in public policy for the environment.Environ. Sci. Policy. 2009; 12: 207-215Crossref Scopus (134) Google Scholar]. Yet, recent studies show that scientific information is still not widely used in environmental policy and practice [8Sutherland W.J. et al.The need for evidence-based conservation.Trends Ecol. Evol. 2004; 19: 305-308Abstract Full Text Full Text PDF PubMed Scopus (1283) Google Scholar, 11Juntti M. et al.Evidence, politics and power in public policy for the environment.Environ. Sci. Policy. 2009; 12: 207-215Crossref Scopus (134) Google Scholar, 12Bayliss H.R. et al.Does research information meet the needs of stakeholders? Exploring evidence selection in the global management of invasive species.Evid. Policy. 2012; 8: 37-56Crossref Scopus (33) Google Scholar, 13Bernhardt E.S. et al.Restoring rivers one reach at a time: results from a survey of US river restoration practitioners.Restor. Ecol. 2007; 15: 482-493Crossref Scopus (316) Google Scholar, 14Cook C.N. et al.Managers consider multiple lines of evidence important for biodiversity management decisions.J. Environ. Manage. 2012; 113: 341-346Crossref PubMed Scopus (85) Google Scholar, 15Cook C.N. et al.Conservation in the dark? The information used to support management decisions.Front. Ecol. Environ. 2010; 8: 181-186Crossref Scopus (233) Google Scholar, 16Matzek V. et al.Closing the knowing-doing gap in invasive plant management: accessibility and interdisciplinarity of scientific research.Conserv. Lett. 2014; 7: 208-215Crossref Scopus (89) Google Scholar, 17Pullin A.S. Knight T.M. Assessing conservation management's evidence base: a survey of management-plan compilers in the United Kingdom and Australia.Conserv. Biol. 2005; 19: 1989-1996Crossref Scopus (117) Google Scholar, 18Sharman A. Holmes J. Evidence-based policy or policy-based evidence gathering? Biofuels, the EU and the 10% target.Environ. Policy Gov. 2010; 20: 309-321Crossref Scopus (85) Google Scholar, 19Young K.D. Van Aarde R.J. Science and elephant management decisions in South Africa.Biol. Conserv. 2011; 144: 876-885Crossref Scopus (42) Google Scholar]. In this opinion article, we argue that this is because the infrastructure needed to incorporate scientific evidence into decisions is largely missing. Evidence-informed decision-making for environment and natural resource management is best achieved through a hierarchical framework that channels unbiased synthesis of research findings through systematic reviews (see Glossary) and summaries into decision support systems. The basic structure is emerging, but coverage of important environmental issues is poor at every level. The costs of developing this infrastructure could be met within existing investments in science and science–policy interactions. The limited use of scientific information in environmental decisions has been attributed partly to decision-makers' lack of access to relevant scientific literature [12Bayliss H.R. et al.Does research information meet the needs of stakeholders? Exploring evidence selection in the global management of invasive species.Evid. Policy. 2012; 8: 37-56Crossref Scopus (33) Google Scholar, 16Matzek V. et al.Closing the knowing-doing gap in invasive plant management: accessibility and interdisciplinarity of scientific research.Conserv. Lett. 2014; 7: 208-215Crossref Scopus (89) Google Scholar, 17Pullin A.S. Knight T.M. Assessing conservation management's evidence base: a survey of management-plan compilers in the United Kingdom and Australia.Conserv. Biol. 2005; 19: 1989-1996Crossref Scopus (117) Google Scholar, 19Young K.D. Van Aarde R.J. Science and elephant management decisions in South Africa.Biol. Conserv. 2011; 144: 876-885Crossref Scopus (42) Google Scholar, 20Bainbridge I. How can ecologists make conservation policy more evidence based? Ideas and examples from a devolved perspective.J. Appl. Ecol. 2014; https://doi.org/10.1111/1365-2664.12294Crossref Scopus (31) Google Scholar], and to a lack of effort to incorporate the growing evidence base into decision frameworks [6Segan D.B. et al.Using conservation evidence to guide management.Conserv. Biol. 2011; 25: 200-202Crossref PubMed Scopus (65) Google Scholar]. This contrasts with the situation in medicine, where evidence-based clinical practice is now routine [21Graham R. Clinical Practice Guidelines We Can Trust. National Academic Press, 2011Crossref Google Scholar]. The methods used to make scientific research usable and relevant to clinical decision-making have been conceptualised in what was initially called a '4S' hierarchy [22Haynes R.B. Of studies, summaries, synopses, and systems: the "4S" evolution of services for finding current best evidence.Evid. Based Med. 2001; 6: 36-38Crossref Scopus (36) Google Scholar], and subsequently developed into a '6S' hierarchy [23DiCenso A. et al.Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model.Evid. Based Nurs. 2009; 12: 99-101Crossref PubMed Scopus (160) Google Scholar, 24Haynes R.B. Of studies, syntheses, synopses, summaries, and systems: the "5S" evolution of information services for evidence-based healthcare decisions.Evid. Based Med. 2006; 11: 162-164Crossref PubMed Scopus (130) Google Scholar, 25Windish D. Searching for the right evidence: how to answer your clinical questions using the 6S hierarchy.Evid. Based Med. 2013; 18: 93-97Crossref PubMed Scopus (13) Google Scholar]. The 4S hierarchy shows the relation between different means of presenting science for use in environmental decisions (Figure 1, Table 1). Primary research is collated into systematic reviews, both studies and systematic reviews are presented in summaries, and decision support systems place the evidence into a decision-making context. The triangle shape illustrates the number of items at each level that could feed into a given decision. There might be hundreds of relevant research papers, several systematic reviews, one or a few summaries and one decision support system that accurately reflects the decision-making context. At each stage of ascent, the original scientific information is condensed, summarised, and becomes more accessible to decision-makers.Table 1A guide for environmental researchers or decision-makers on when and how to develop the different levels of evidence synthesis in the 4S hierarchyLevel of evidenceType of questionWhen to developNumber of specific questions addressedCostaCosts converted to US$ on the basis of exchange rates on 22 June 2014, rounded to two significant figures where necessary.Time requiredRenewal periodStudyA specific question about how an environmental system functions, how it is changing, or the differential effects of management. For example: how do carbon dioxide emissions change during wetland restoration on lowland peat?When existing research or data are scarce, or the effect varies with context and requires repeated experimental tests; the results will not be broadly applied.Few: often one main questionHighly context dependent; individual research grants from the UK Natural Environment Research Council range in value from US$3600 to US$21 millionbResults of a search for Natural Environment Research Council (NERC) on the UK Gateway to Research database (http://gtr.rcuk.ac.uk/).Anything from less than 1 year to decadesStudies should be replicated in different contexts to increase certaintySystematic reviewA specific question about how an environmental system functions, how it is changing, or the differential effects of management. For example: how are carbon stores and greenhouse gas fluxes affected by different land management on temperate and boreal lowland peatland ecosystems?When multiple studies have asked similar questions and their results can be compared; the issue has strong political or community interest, or is controversial; the results will be broadly applied.1–6 (mean 2.5)cNumber of primary and secondary objectives in the completed systematic reviews in the Collaboration for Environmental Evidence library (http://www.environmentalevidence.org/), calculated 22 June 2014.US$30 000–300 000 per reviewdInformation from [29].0.5–3 yearsdInformation from [29].Every 5 yearsdInformation from [29].SummaryA broad question relevant to managing the environment or natural resources, with many alternative options or aspects. For example: how can we reduce greenhouse gas emissions through land management?When multiple sources of relevant evidence exist, including studies and systematic reviews; the issue has strong political or community interest, and presents a current challenge for decision-makers; the results will be broadly applied.59–457eRange in the number of interventions covered by completed Conservation Evidence Synopses (http://www.conservationevidence.com).Initial cost of collating synopses of evidence: US$70 000–750 000 per subject. Update cost: approximately 20% of initial costfEstimates made by Conservation Evidence project (unpublished; http://www.conservationevidence.com).1–5 years for initial collation of evidencefEstimates made by Conservation Evidence project (unpublished; http://www.conservationevidence.com).Annual or biennial updatesDecision support systemA broad or specific question, applied in a specific institutional and environmental context. For example: what should the land management authority of a particular country or region do to reduce greenhouse gas emissions?When multiple sources of relevant evidence exist; if there are systematic reviews and synopses, they should be incorporated at the design stage; the issue presents a current challenge for decision-makers; the results will be applied in a specific context.Could address between one or several hundred questions, depending on the context and breadth of the overall questionWe found no literature data on comparative costs of environmental decision support systems. Based on recent examples, costs can range from US$540 000gThe cost of developing a decision support system for nitrogen fertiliser management on grasslands (Defra project NT1603; http://randd.defra.gov.uk/). to US$5.6 millionhThe cost of BRIDGE – a European Seventh Framework Programme project to build a decision support system for urban planning (http://cordis.europa.eu/projects/rcn/88630_en.html).; in medicine, a systematic review of cost-effectiveness provided costs ranging from US$3600 to US$45 000iCalculated from the number of patients and total cost per patient reported for five studies in [68].Usually several yearsModification as appropriatea Costs converted to US$ on the basis of exchange rates on 22 June 2014, rounded to two significant figures where necessary.b Results of a search for Natural Environment Research Council (NERC) on the UK Gateway to Research database (http://gtr.rcuk.ac.uk/).c Number of primary and secondary objectives in the completed systematic reviews in the Collaboration for Environmental Evidence library (http://www.environmentalevidence.org/), calculated 22 June 2014.d Information from 29Collaboration for Environmental EvidenceGuidelines for Systematic Review and Evidence Synthesis in Environmental Management. Version 4.2. Environmental Evidence, 2013http://www.environmentalevidence.org/Documents/Guidelines/Guidelines4.2.pdfGoogle Scholar.e Range in the number of interventions covered by completed Conservation Evidence Synopses (http://www.conservationevidence.com).f Estimates made by Conservation Evidence project (unpublished; http://www.conservationevidence.com).g The cost of developing a decision support system for nitrogen fertiliser management on grasslands (Defra project NT1603; http://randd.defra.gov.uk/).h The cost of BRIDGE – a European Seventh Framework Programme project to build a decision support system for urban planning (http://cordis.europa.eu/projects/rcn/88630_en.html).i Calculated from the number of patients and total cost per patient reported for five studies in 68Walsh T. et al.Undetermined impact of patient decision support interventions on healthcare costs and savings: systematic review.BMJ. 2014; 348https://doi.org/10.1136/bmj.g188Crossref PubMed Scopus (47) Google Scholar. Open table in a new tab The 4S hierarchy is particularly useful for diagnosing threats, selecting management actions, or deciding how to monitor environmental outcomes, areas where available scientific information is disparate, variable in relevance, quality, and extent, yet critical to success. It is less suitable for finding contextual information, such as species or ecosystem ecology, status, distribution, or local conditions, where only very specific information is relevant. We have not included two 'synopsis' levels from the 6S hierarchy [23DiCenso A. et al.Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model.Evid. Based Nurs. 2009; 12: 99-101Crossref PubMed Scopus (160) Google Scholar], which refer to individual descriptions of either studies or systematic reviews, not collated or assessed to extract messages or recommendations for decision-makers. These levels exist in environmental science (for example, policy briefs summarising systematic reviews: http://www.environmentalevidence.org/policy-briefs), but they are most useful when collated and regularly updated to form the basic units of summaries. We have modified the figure developed for evidence-based medicine into a framework that shows how decisions refer to compiled scientific evidence via experience and advice. Our 4S framework combines quality, rigour, and critical appraisal with easy routes for decision-makers to find the best-available scientific information. It also illustrates common bypass routes, through which environmental decisions can appear to be evidence-based without incorporating unbiased synthesis. Below, we explain each component of the framework. There are millions of scientific studies to be found as peer-reviewed papers in scientific journals, in published or unpublished reports, in books, or student research theses. For a given environmental management decision, a small subset of these will provide relevant information. For example, a summary of global evidence on bird conservation [26Williams D.R. et al.Bird Conservation: Evidence for the Effects of Interventions. Pelagic Publishing, 2013Google Scholar] included 1237 individual studies, each testing the effects of one or more specific actions to conserve birds. Published environmental studies are often not focussed on the needs of decision-makers, so they do not reliably supply information that is relevant and useful [27McNie E.C. Reconciling the supply of scientific information with user demands: an analysis of the problem and review of the literature.Environ. Sci. Policy. 2007; 10: 17-38Crossref Scopus (668) Google Scholar]. For example, a recent review showed that most published research on 'leopard conservation' was not relevant to leopard management, while practical leopard conservation projects tended not to publish their findings [28Balme G.A. et al.Failure of research to address the rangewide conservation needs of large carnivores: leopards in South Africa as a case study.Conserv. Lett. 2014; 7: 3-11Crossref Scopus (53) Google Scholar]. Systematic reviews analyse findings across all existing studies about a specific question, usually about the effectiveness of an intervention or the effect of exposure to a particular variable. They use transparent, unbiased search and appraisal methods, avoiding the vague methods or cherry picking of well-known studies that can make standard literature reviews unreliable or open to bias [29Collaboration for Environmental EvidenceGuidelines for Systematic Review and Evidence Synthesis in Environmental Management. Version 4.2. Environmental Evidence, 2013http://www.environmentalevidence.org/Documents/Guidelines/Guidelines4.2.pdfGoogle Scholar, 30Pullin A.S. Stewart G.B. Guidelines for systematic review in conservation and environmental management.Conserv. Biol. 2006; 20: 1647-1656Crossref PubMed Scopus (707) Google Scholar, 31Roberts P.D. et al.Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine.Biol. Conserv. 2006; 132: 409-423Crossref Scopus (102) Google Scholar]. The process generates better outcomes when there are multiple, well-designed relevant studies [32Mascia M.B. et al.Commonalities and complementarities among approaches to conservation monitoring and evaluation.Biol. Conserv. 2014; 169: 258-267Crossref Scopus (102) Google Scholar]. The Cochrane Collaboration, which publishes medical systematic reviews, has over 8300 reviews in its database (http://www.cochrane.org). The Campbell Collaboration oversees the production of systematic reviews in social policy areas such as crime and education (http://www.campbellcollaboration.org/) and has a library of over 250 reviews or review protocols. Following this model, several centres have been established to incorporate systematic review methods into environmental decision-making [5Pullin A.S. Knight T.M. Doing more good than harm – building an evidence-base for conservation and environmental management.Biol. Conserv. 2009; 142: 931-934Crossref Scopus (195) Google Scholar]. There are guidelines and training on how to conduct environmental systematic reviews [30Pullin A.S. Stewart G.B. Guidelines for systematic review in conservation and environmental management.Conserv. Biol. 2006; 20: 1647-1656Crossref PubMed Scopus (707) Google Scholar] and 62 completed reviews are currently published in the online library of the Collaboration for Environmental Evidence (www.environmentalevidence.org/completed-reviews). A study completed in 2012 assessed the contribution of systematic reviews included in online databases since 1945 to environmental management decisions [33Cook C.N. et al.Contribution of systematic reviews to management decisions.Conserv. Biol. 2013; 27: 902-915Crossref PubMed Google Scholar]. Of the 43 published at that time, 23 provided concrete conclusions relevant to conservation management. Many were too narrow in geographic scope or too broad in taxonomic scope to provide useful recommendations. The process of developing and framing appropriate questions for systematic reviews is key to ensuring they produce meaningful results [34Fazey I. et al.Can methods applied in medicine be used to summarize and disseminate conservation research?.Environ. Conserv. 2004; 31: 190-198Crossref Scopus (75) Google Scholar]. A similar, more exploratory technique called systematic mapping has recently been adopted for environmental questions (for examples, see [35Munroe R. et al.Review of the evidence base for ecosystem-based approaches for adaptation to climate change.Environ. Evid. 2012; 1: 1-11Crossref Scopus (46) Google Scholar, 36Randall N. et al.How effective are slurry storage, cover or catch crops, woodland creation, controlled trafficking or break-up of compacted layers, and buffer strips as on-farm mitigation measures for delivering an improved water environment?.Environ. Evid. 2012; 1: 12Crossref Scopus (4) Google Scholar, 37Randall N. James K. The effectiveness of integrated farm management, organic farming and agri-environment schemes for conserving biodiversity in temperate Europe – a systematic map.Environ. Evid. 2012; 1: 1-21Google Scholar]). Systematic maps use the rigorous search methods of systematic review to build a database or catalogue of evidence, which represents an unbiased picture of scientific information in a given area. They do not set out to answer a specific question, and do not always include analysis of results. If the full text of studies included in the map has been read, it might be accompanied by an appraisal of the quality of evidence and its results, based on author judgement rather than in-depth data extraction and analysis (demonstrated by [36Randall N. et al.How effective are slurry storage, cover or catch crops, woodland creation, controlled trafficking or break-up of compacted layers, and buffer strips as on-farm mitigation measures for delivering an improved water environment?.Environ. Evid. 2012; 1: 12Crossref Scopus (4) Google Scholar]). Systematic maps have an important role in the hierarchical synthesis of evidence, because they are able to cover the breadth of science often needed for policy-relevant questions. They can form the basic evidence-gathering phase for scoping a systematic review [38Stewart R. et al.The size and nature of the evidence-base for smallholder farming in Africa: a systematic map.J. Dev. Effect. 2014; 6: 58-68Crossref Scopus (7) Google Scholar], or provide the set of studies and reviews to be described in a summary (as in [39Dicks L.V. et al.Farmland Conservation: Evidence for the Effects of Interventions in Northern Europe. Pelagic Publishing, 2014Google Scholar], for example). Summaries are regularly updated descriptions of evidence across a range of possible solutions or approaches relevant to a particular type of decision. They integrate evidence-based information, preferably using an explicit review process to identify all relevant systematic reviews, and studies where systematic reviews are lacking. Summaries are written in simple, nontechnical language, suitable and sufficiently concise for a busy practitioner or policymaker to understand quickly. They can be organised collections of short synopses, each providing enough information about a single systematic review or study to use it in support of an environmental decision, without the decision-maker needing to read the whole article or review. Summaries often include recommendations about what practitioners should do, based on the evidence. In medicine, Clinical Practice Guidelines are an example [21Graham R. Clinical Practice Guidelines We Can Trust. National Academic Press, 2011Crossref Google Scholar], as is Clinical Evidence (http://www.clinicalevidence.com), an online database of systematic overviews that assesses benefits and harms of medical treatments. For environmental science, at least one set of resources has been developed that is moving towards offering the summary level in our 4S hierarchy, for management of biodiversity and ecosystem services (http://www.conservationevidence.com). These resources, called 'Conservation Evidence Synopses', include collations of global evidence from systematic reviews and studies on bird conservation [26Williams D.R. et al.Bird Conservation: Evidence for the Effects of Interventions. Pelagic Publishing, 2013Google Scholar] and amphibian conservation [40Smith R.K. Sutherland W.J. Amphibian Conservation: Global Evidence for the Effects of Interventions. Pelagic Publishing, 2014Google Scholar]. They have been applied in several contexts to inform policy or set research priorities [9Dicks L.V. et al.A transparent process for 'evidence-informed' policy making.Conserv. Lett. 2014; 7: 119-125Crossref Scopus (86) Google Scholar, 41Sutherland W.J. et al.Methods for collaboratively identifying research priorities and emerging issues in science and policy.Methods Ecol. Evol. 2011; 2: 238-247Crossref Scopus (251) Google Scholar, 42Sutherland W.J. et al.Quantifying the impact and relevance of scientific research.PLoS ONE. 2011; 6https://doi.org/10.1371/journal.pone.0027537Crossref Scopus (56) Google Scholar] and used directly in the development of UK policy on agriculture and pollinators (cited in [43Defra A Consultation on the National Pollinator Strategy: for Bees and other Pollinators in England. Department for Environment, Food, and Rural Affairs, 2014Google Scholar] for example). So far, none has incorporated practical recommendations based on the evidence, or been updated. A key element for both systematic reviews and summaries in environmental science is the involvement of practitioners and policymakers in framing the question and choosing the interventions or variables covered. This ensures that the scope remains relevant to environmental management or policy, regardless of what is covered in the scientific literature, and makes systematic reviews, maps, and summaries valuable tools for identifying gaps in knowledge [9Dicks L.V. et al.A transparent process for 'evidence-informed' policy making.Conserv. Lett. 2014; 7: 119-125Crossref Scopus (86) Google Scholar]. The final step in the 4S hierarchy links the full body of evidence, synthesised from the bottom up, into decision support systems operating at the point of decision. In medicine, these have been described as 'point-of-care evidence-based services' [44Moja L. Banzi R. Navigators for medicine: evolution of online point-of-care evidence-based services.Int. J. Clin. Pract. 2011; 65: 6-11Crossref PubMed Scopus (21) Google Scholar, 45Banzi R. et al.A review of online evidence-based practice point-of-care information summary providers.J. Med. Internet Res. 2010; 12: e39Crossref PubMed Google Scholar]. In environmental management, decision support systems, sometimes called 'decision support tools', are increasingly being used to help decision-making [46Laniak G.F. et al.Integrated environmental modeling: a vision and roadmap for the future.Environ. Model. Software. 2013; 39: 3-23Crossref Scopus (370) Google Scholar]. They are usually software based, and assist with decisions by illustrating possible outcomes visually or numerically or leading users through logical decision steps. Some are complex models, only reliably operated by their developers. Others have simple interfaces designed to be used by non-experts. While challenging, this range of system types is probabl
Referência(s)