Portal de Periódicos da CAPES

Lost in translation? Comparing the effectiveness of electronic-based and paper-based cognitive aids

2017; Elsevier BV; Volume: 119; Issue: 5 Linguagem: Inglês

10.1093/bja/aex263

1471-6771

Stuart Marshall,

Simulation-Based Education in Healthcare

It is now established that the use of cognitive aids, such as checklists and algorithms leads to improved technical1Marshall SD Use of cognitive aids during emergencies in anesthesia: a systematic review.Anesth Analg. 2013; 117: 1162-1171Crossref PubMed Scopus (152) Google Scholar and team performance2Marshall SD Sanderson P McIntosh C Kolawole H The effect of two cognitive aid designs on team functioning during intra-operative anaphylaxis emergencies: a multi-centre simulation study.Anaesthesia. 2016; 71: 389-404Crossref PubMed Scopus (52) Google Scholar during anaesthetic emergencies. The unpredictable nature of emergencies means that it is difficult to examine these outside of a simulation setting. Consequently, it is challenging to prove these observed behaviours translate to improved patient outcomes, although many anecdotal reports exist.3Sillen A Cognitive tool for dealing with unexpected difficult airway.Br J Anaesth. 2014; 112: 773-774Abstract Full Text Full Text PDF PubMed Scopus (13) Google Scholar 4Ranganathan P Phillips JH Attaallah AF Vallejo MC The use of cognitive aid checklist leading to successful treatment of malignant hyperthermia in an infant undergoing cranioplasty.Anesth Analg. 2014; 118: 1389Crossref PubMed Scopus (19) Google Scholar Research is now focused on the nature of these cognitive aids and how they can be integrated into the clinical setting, working with, rather than against the instincts of experienced practitioners.5Goldhaber-Fiebert SN Howard SK Implementing emergency manuals: can cognitive aids help translate best practices for patient care during acute events?.Anesth Analg. 2013; 117: 1149-1161Crossref PubMed Scopus (104) Google Scholar In cognitive engineering terms, cognitive aids are a type of 'artifact' – objects found in the work situation that aid in the completion of work.6Hutchins E Cognition in the Wild. MIT Press, Cambridge, MA1995Google Scholar More generally, artifacts encompass instructions, charts, displays or tools that coordinate team members and provide a shared understanding (or 'mental model') of the problem, the tasks to be completed and the team roles.7Xiao Y Henrickson-Parker S Manser T Teamwork and collaboration.Rev Hum Factors Ergon. 2013; 8: 55-102Crossref Scopus (32) Google Scholar Consequently, artifacts may not be fixed objects, and may convey a social function. They are often assigned additional meaning by modification by users over time, becoming more applicable to their context. A good example is the theatre suite or emergency department 'whiteboard' that displays the patients awaiting treatment.8Xiao Y Artifacts and collaborative work in healthcare: methodological, theoretical, and technological implications of the tangible.J Biomed Informat. 2005; 38: 26-33Crossref PubMed Scopus (122) Google Scholar Annotations are commonly handwritten, or symbols applied that add further meaning and communicate detail that was not initially captured by the designed artifact. In the context of paper-based cognitive aids these annotations may include information such as telephone numbers written in permanent marker on the laminated surface, or a sticker placed to omit or amend a relevant drug dosing. A design of a new cognitive aid requires attention to the function of potential modifications that add further function. With the ubiquity of handheld electronic devices such as smartphones and tablets, there is a strong temptation to translate paper-based cognitive aids into electronic ones. Digitisation of work artifacts means that they can be instantly updated when consensus guidelines change and disseminated rapidly. However, care needs to be taken to ensure that either the context-related modifications made by the users are retained, or at the very least examined and taken into account in the design. In the case of the whiteboard example, many researchers have advised against transferring to an electronic version as key aspects of information and functionality can be lost.9Pennathur PR Bisantz AM Fairbanks RJ Perry SJ Zwemer F Wears RL Assessing the impact of computerization on work practices: information technology in emergency departments.Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting. Baltimore. 2007; : 377-381Crossref Scopus (23) Google Scholar In this issue, Lelaidier and colleagues10Lelaidier R Balanca B Boet S et al.Use of a hand-held digital cognitive aid in simulated crises: the MAX randomised controlled trial.Br J Anaesth. 2017; 119: 1015-1021Abstract Full Text Full Text PDF PubMed Scopus (20) Google Scholar demonstrated that non-technical (teamwork) skills displayed by anaesthetists in a number of anaesthetic emergencies were improved by the provision of a cognitive aid on a mobile phone. The majority of research on cognitive aids in anaesthesia to date has been performed using paper-based cognitive aids. Although the use of personal digital assistants and computers have been shown in some instances to improve technical performance and knowledge recall during emergencies, they have not always fulfilled their promise. Few studies have looked at the effects of electronic cognitive aids on non-technical skills and this study represents the first to suggest an improvement in this area of performance. Computer-based prompts in anaesthesia have only recently been tested in the context of a simulated real emergency1Marshall SD Use of cognitive aids during emergencies in anesthesia: a systematic review.Anesth Analg. 2013; 117: 1162-1171Crossref PubMed Scopus (152) Google Scholar 11McEvoy MD Hand WR Stoll WD Furse CM Nietert PJ Adherence to guidelines for the management of local anesthetic systemic toxicity is improved by an electronic decision support tool and designated "Reader".Reg Anesth Pain Med. 2014; 39: 299-305Crossref PubMed Scopus (37) Google Scholar and in some instances have been found to hinder rather than help.12Coopmans VC Biddle C CRNA performance using a handheld, computerized, decision-making aid during critical events in a simulated environment: a methodologic inquiry.AANA J. 2008; 76: 29-35PubMed Google Scholar 13Watkins SC Anders S Clebone A et al.Paper or plastic? Simulation based evaluation of two versions of a cognitive aid for managing pediatric peri-operative critical events by anesthesia trainees: evaluation of the society for pediatric anesthesia emergency checklist.J Clin Monit. 2016; 30: 275-283Crossref Scopus (26) Google Scholar Clearly, good design in cognitive aids is more than merely translating a paper aid into an electronic format. It's therefore important to understand how and why Lelaidier and colleagues' electronic cognitive aid worked to improve technical and non-technical performance where others have failed. We can theorise those reasons based on previous successes and failures from both paper-based and electronic aids. The first reason electronic cognitive aids might fail is one that has already been touched on – that of context. The social, and physical context of a workplace may necessitate workarounds or modification. What may be appropriate in one context might not work in another. Hart and Owen14Hart EM Owen H Errors and omissions in anesthesia: a pilot study using a pilot's checklist.Anesth Analg. 2005; 101: 246-250Crossref PubMed Scopus (105) Google Scholar successfully introduced a 'pilots' checklist' to prevent omission of items during provision of general anaesthesia for caesarean section. The checklist consisted of both a display and audio prompt. The audio prompt was found to be useful, as it eliminated the need to read most items when visual attention to other tasks was precious, but despite their usefulness, it is easy to imagine the drawbacks of such audio prompts in causing anxiety in an awake patient, or an inability to hear them in a noisy environment. Consequently, the mode of information transmission should be tailored to the environment. In Lelaidier and colleagues' case a visual only prompt was deemed to be appropriate. Translating cognitive aids to electronic formats may render them less visible when an emergency develops. Rather than being available with other information in a pocket or laptop computer, a paper-based cognitive aid has the advantage of being placed where it might be seen and accessed. Its position, close to associated equipment, may then act as a reminder for it to be used.15Marshall SD Helping experts and expert teams perform under duress: an agenda for cognitive aid research.Anaesthesia. 2017; 73: 289-295Crossref Scopus (32) Google Scholar The inability to associate a difficult airway algorithm with the difficult airway trolley, or the local anaesthetic systemic toxicity cognitive aid with the lipid micelle antidote may mean the clinicians forget to access the aid and rely on memory alone. Alternatively, if the electronic cognitive aid can be associated and linked with the monitoring and information systems, it could theoretically improve uptake of its use. Simulation studies such as the one by Lelaidier and colleagues15Marshall SD Helping experts and expert teams perform under duress: an agenda for cognitive aid research.Anaesthesia. 2017; 73: 289-295Crossref Scopus (32) Google Scholar,16Goldhaber-Fiebert SN Pollock J Howard SK Bereknyei-Merrell S Emergency manual uses during actual critical events and changes in safety culture from the perspective of anesthesia residents: a pilot study.Anesth Analg. 2016; 123: 641-649Crossref PubMed Scopus (36) Google Scholar are not ideal to test such shortcomings. In the study the participants were given the cognitive aid and shown how to use it directly before the scenario rather than having to remember it was available. Indeed, one of the main drawbacks of cognitive aids is their infrequent use both in training and during the emergency. Some estimates suggest as few as 7% of users access a cognitive aid when available during an emergency17Neily J DeRosier JM Mills PD Bishop MJ Weeks WB Bagian JP Awareness and use of a cognitive aid for anesthesiology.Jt Comm J Qual Patient Saf. 2007; 33: 502-511Abstract Full Text Full Text PDF PubMed Scopus (52) Google Scholar 18Mills PD DeRosier JM Neily J McKnight SD Weeks WB Bagian JP A cognitive aid for cardiac arrest: You can't use it if you don't know about it.Jt Comm J Qual Patient Saf. 2004; 30: 488-496Scopus (40) Google Scholar even when 80% or more of participants in simulation studies report they would use them.16Goldhaber-Fiebert SN Pollock J Howard SK Bereknyei-Merrell S Emergency manual uses during actual critical events and changes in safety culture from the perspective of anesthesia residents: a pilot study.Anesth Analg. 2016; 123: 641-649Crossref PubMed Scopus (36) Google Scholar Similarly the workflow during the emergency needs careful consideration. Adding more physical tasks such as the requirement to 'click through' a checklist on a screen is impossible for a single operator when both hands are already being used for the clinical task. In this way, a cognitive aid that is easily remembered, rather than a complex, branched algorithm is preferable.2Marshall SD Sanderson P McIntosh C Kolawole H The effect of two cognitive aid designs on team functioning during intra-operative anaphylaxis emergencies: a multi-centre simulation study.Anaesthesia. 2016; 71: 389-404Crossref PubMed Scopus (52) Google Scholar 19Chrimes N The Vortex: a universal 'high-acuity implementation tool' for emergency airway management.Br J Anaesth. 2016; 117: i20-i27Abstract Full Text Full Text PDF PubMed Scopus (93) Google Scholar Where a cognitive aid needs to be read or interacted with, evidence suggests a dedicated team member should be assigned for this task.20Burden AR Carr ZJ Staman GW Littman JJ Torjman MC Does every code need a "reader?" Improvement of rare event management with a cognitive aid "reader" during a simulated emergency: a pilot study.Simul Healthc. 2012; 7: 1-9Crossref PubMed Scopus (82) Google Scholar In Lelaidier and colleagues' study a linear algorithm was used with a simple, clear display of prompts that was operated by the single participant in the study. The application of the device to this context seems appropriate, but the lack of a beneficial effect in the Ventricular Fibrillation scenario could equally indicate that the hands-on activities required in this scenario did not allow the participant to easily operate the device and perform the tasks. For teamwork to be enhanced by the cognitive aid it must also provide a common language for the team in the emergency and reinforce the most effective terminology used in terms of simplicity and clarity.21Chrimes N Cook TM Critical airways, critical language.Br J Anaesth. 2017; 118: 649-654Abstract Full Text Full Text PDF PubMed Scopus (22) Google Scholar Communication patterns and coordination strategies were not measured in Lelaidier and colleagues' study, but the reading of items aloud most likely improved the effectiveness of both during the emergencies. As in other studies, coordination and communication would almost certainly have been affected by the use of the cognitive aid.22Manser T Harrison TK Gaba DM Howard SK Coordination patterns related to high clinical performance in a simulated anesthetic crisis.Anesth Analg. 2009; 108: 1606-1615Crossref PubMed Scopus (86) Google Scholar Trust and familiarity with cognitive aids are important for their effective use. Fitzgerald and colleagues23Fitzgerald M Cameron P Mackenzie C et al.Trauma resuscitation errors and computer-assisted decision support.Arch Surg. 2011; 146: 218-225Crossref PubMed Scopus (83) Google Scholar introduced a displayed cognitive aid for the management of severely injured trauma patients in the emergency department. They identified that there were fewer omissions and additionally, when the team leader needed to undertake clinical tasks, team leadership was more easily transferred to a second person if the cognitive aid was present.24Jeffcott S Farrow N Marshall S Fitzgerald M Cameron P Optimising trauma team performance.Injury. 2010; 41: S38-S39Abstract Full Text Full Text PDF PubMed Google Scholar For a cognitive aid to support this fluid model of leadership, it is likely that the information displayed has to be trusted by the team members. Trust of the team in an electronic handheld device may be different to a clearly endorsed, printed-paper aid. The effects of different levels of trust on the functioning of the team using the different presentations are not yet known. The level of detail required in the cognitive aid's content is an important consideration. The aid, electronic or paper, should include items that are commonly forgotten or that need detailed information such as drug doses, but not be cluttered with items that are self-evident, easily remembered or detract from these important items.15Marshall SD Helping experts and expert teams perform under duress: an agenda for cognitive aid research.Anaesthesia. 2017; 73: 289-295Crossref Scopus (32) Google Scholar Critical decisions and actions that experts attend to should be identified through careful questioning and supported.25Schnittker R Marshall SD Horberry T Young K Lintern G Exploring decision pathways in challenging airway management episodes.J Cognit Eng Decis Making. 2017; (Advance Access published on June 21)doi: 10.1177/1555343417716843Crossref Scopus (7) Google Scholar These items could potentially strengthen behaviours that prevent further errors or mitigate their effects could in theory improve safety.26Schnittker R Marshall SD Safe anaesthetic care: further improvements require a focus on resilience.Br J Anaesth. 2015; 115: 643-645Abstract Full Text Full Text PDF PubMed Scopus (8) Google Scholar The ability to present complex information is more limited on a small screen than on paper. Translation of a paper-based aid onto an electronic screen, may therefore lead to unintended consequences. Careful design and testing in a simulation environment that closely reflects the clinical setting can help tailor the design to the context of use, as in the recent demonstration of improved compliance with a tablet based an intelligent dynamic clinical checklist for use in the intensive care unit.27De Bie AJR Nan S Vermeulen LRE et al.Intelligent dynamic clinical checklists improved checklist compliance in the intensive care unit.Br J Anaesth. 2017; 119: 231-238Abstract Full Text Full Text PDF PubMed Scopus (23) Google Scholar 28Webster CS Checklists, cognitive aids and the future of patient safety.Br J Anaesth. 2017; 119: 178-181Abstract Full Text Full Text PDF PubMed Scopus (16) Google Scholar Lelaidier and colleagues' study shows for the first time that the benefits of paper-based cognitive aids need not be 'lost in translation' when moving to an electronic version. S.M. is in receipt of an Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship grant (Grant number 1130929) investigating the nature of effective cognitive aids in medical emergencies. He has no other conflicts of interest to declare.