Portal de Periódicos da CAPES

Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS

2018; Frontiers Media; Volume: 12; Linguagem: Inglês

10.3389/conf.fnhum.2018.227.00039

1662-5161

Adrian Curtin, Hasan Ayaz,

EEG and Brain-Computer Interfaces

Event Abstract Back to Event Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS Adrian Curtin1, 2* and Hasan Ayaz1, 3, 4 1 School of Biomedical Engineering, Science and Health Systems, Drexel University, United States 2 Shanghai Jiao Tong University, School of Biomedical Engineering, China 3 University of Pennsylvania, Department of Family and Community Health, United States 4 Children's Hospital of Philadelphia, The Division of General Pediatrics, United States The way in which we effortlessly understand speech makes it relatively easy to overlook the complexity of the neural mechanisms which support its operation. This perceived ease of speech comprehension has inspired researchers and interface designers seek to incorporate additional auditory channels of information to consumer and industrial devices, especially in circumstances where the user of the device is otherwise engaged in manual or visual tasks such as handsfree communication while driving. However, previous research has demonstrated that there are unconscious cognitive costs associated with simple speech comprehension which can be exacerbated by task performance(Treffner & Barrett, 2004) and audio quality(Delogu, Conte, & Sementina, 1998; Francis & Nusbaum, 2009). In a previous exploratory work, we have observed performance deficits and neural activation differences due to low-quality synthetic speech during speech comprehension tasks(Curtin & Ayaz, 2017), and here we extend this exploration by examining how the quality of synthetic speech affects auditory working memory (AWM) performance and neural load using functional Near Infrared Spectroscopy (fNIRS), a continuous and noninvasive neuroimaging technique capable of investigating brain dynamics under naturalistic and applied settings (Ayaz et al., 2013). Working memory is an important executive function in which a limited storage system is updated and maintained during the execution of other processing tasks. As a limited resource, tasks which require significant working memory resources are especially sensitive to distortions and other factors which may competitively demand additional attention(Banich, 2009), hence the development of systems and user interfaces must carefully consider added cognitive burdens imposed as part of the design process. Beyond design considerations, the wide variety of situations which require auditory working memory (AWD) and the way in which degraded audio quality adversely can affect these has important implications in day-to-day life whether the degraded audio signal is due to environmental noise, hearing loss, distortion produced by hearing aid devices, or low-quality synthetic speech. Here we present a neuroergonomic evaluation on the effects of synthetic speech quality during an auditory working memory task while functional Near-Infrared Spectroscopy (fNIRS) was used to noninvasively monitor both the dorsolateral-Prefrontal Cortex (dlPFC) and the left-Temporal-Parietal Junction region (TPJ) which includes the primary auditory cortex, Broca’s area and Wernicke’s area. Sixteen right-handed healthy controls with no history of neurological or mental illness were recruited to participate in this study. Participants completed an auditory version of the N-Back task using vocalized letters as stimuli while the audio quality was manipulated by using low-quality and high-quality synthetic voices. Results show that reaction time was significantly increased by the presentation of low-quality synthetic speech under both the 0-Back and 2-Back conditions. Additionally, increased working memory load was associated with behavioral performance decreases in terms of accuracy (Figure 1, F(1,225.12)=8.34,p<0.005) and reaction time (Figure 2, F(1,22.46)=30.32,p<0.001) and neural differences as measured by changes in oxygenated hemoglobin [HbO] as measured by fNIRS in both the left-dlPFC (F(1_200.43)=33.2,p<0.001) and the left-TPJ (Figure 3, F(1,160)=8.32,p<0.005), near Wernicke’s Area. Our results suggest both that the interpretation of low-quality voices results in significant behavioral deficits in terms of reaction time (F(1,22.33)=29.91, p<0.001) and that increased AWM load is reflected by changes in activation in not only areas of executive function, but also associative auditory areas. These preliminary results highlight the potential significant impact of neuroimaging for usability and user interface research. Various characteristics of audio user interface can negatively impact cognitive workload in ways that should be accounted for during the design of complex systems or interfaces. Wearable and mobile brain function monitoring can be used to detect these otherwise unobserved characteristics. Figure 1: Auditory N-Back accuracy vs Workload and Synthetic Audio Quality. Figure 2: Mean reaction time during Auditory N-Back vs Workload and Synthetic Audio Quality. Figure 3: Mean [HbO] changes in Left-Temporal region vs Working Memory Load and Synthetic Audio Quality Figure 1 Figure 2 Figure 3 Acknowledgements The authors would like to thank Dr. Paul Crawford for his help with this study. This study is made possible by a research award from the Intel Corporation. The content of the information herein does not necessarily reflect the position or the policy of the sponsor and no official endorsement should be inferred. Keywords: auditory working memory, fNIRS, n-back, synthetic speech, Prefrontal Cortex, Auditory Cortex Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Poster Presentation Topic: Neuroergonomics Citation: Curtin A and Ayaz H (2019). Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00039 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 03 Apr 2018; Published Online: 27 Sep 2019. * Correspondence: Mr. Adrian Curtin, School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States, abc48@drexel.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Adrian Curtin Hasan Ayaz Google Adrian Curtin Hasan Ayaz Google Scholar Adrian Curtin Hasan Ayaz PubMed Adrian Curtin Hasan Ayaz Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.