Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface
2018; Cell Press; Volume: 8; Linguagem: Inglês
10.1016/j.isci.2018.09.016
ISSN2589-0042
AutoresCiaran Cooney, Raffaella Folli, Damien Coyle,
Tópico(s)Reading and Literacy Development
ResumoA direct-speech brain-computer interface (DS-BCI) acquires neural signals corresponding to imagined speech, then processes and decodes these signals to produce a linguistic output in the form of phonemes, words, or sentences. Recent research has shown the potential of neurolinguistics to enhance decoding approaches to imagined speech with the inclusion of semantics and phonology in experimental procedures. As neurolinguistics research findings are beginning to be incorporated within the scope of DS-BCI research, it is our view that a thorough understanding of imagined speech, and its relationship with overt speech, must be considered an integral feature of research in this field. With a focus on imagined speech, we provide a review of the most important neurolinguistics research informing the field of DS-BCI and suggest how this research may be utilized to improve current experimental protocols and decoding techniques. Our review of the literature supports a cross-disciplinary approach to DS-BCI research, in which neurolinguistics concepts and methods are utilized to aid development of a naturalistic mode of communication. A direct-speech brain-computer interface (DS-BCI) acquires neural signals corresponding to imagined speech, then processes and decodes these signals to produce a linguistic output in the form of phonemes, words, or sentences. Recent research has shown the potential of neurolinguistics to enhance decoding approaches to imagined speech with the inclusion of semantics and phonology in experimental procedures. As neurolinguistics research findings are beginning to be incorporated within the scope of DS-BCI research, it is our view that a thorough understanding of imagined speech, and its relationship with overt speech, must be considered an integral feature of research in this field. With a focus on imagined speech, we provide a review of the most important neurolinguistics research informing the field of DS-BCI and suggest how this research may be utilized to improve current experimental protocols and decoding techniques. Our review of the literature supports a cross-disciplinary approach to DS-BCI research, in which neurolinguistics concepts and methods are utilized to aid development of a naturalistic mode of communication. A direct-speech brain-computer interface (DS-BCI) is one that captures and decodes neural signals corresponding directly to speech production, enabling a naturalistic mode of communication (Iljina et al., 2017Iljina O. Derix J. Schirrmeister R.T. Schulze-Bonhage A. Auer P. Aertsen A. Ball T. Neurolinguistic and machine-learning perspectives on direct speech BCIs for restoration of naturalistic communication.Brain Computer Interfaces. 2017; 4: 186-199Crossref Google Scholar). Such a system has the potential to transform the lives of patients with severe motor dysfunction, including pathologies such as amyotrophic lateral sclerosis resulting in locked-in syndrome. Loss of verbal communication has a profound effect on those inflicted, with loss of social interaction and the potential for isolation. In parallel with this personal degeneration, a caregiver faces a more difficult challenge in ascertaining the needs of the patient. These factors have played a crucial role in driving the development of DS-BCIs (Brumberg et al., 2011Brumberg J.S. Wright E.J. Andreasen D.S. Guenther F.H. Kennedy P.R. Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex.Front. Neurosci. 2011; 5: 1-12PubMed Google Scholar, Oken et al., 2014Oken B.S. Orhan U. Roark B. Erdogmus D. Fowler A. Mooney A. Peters B. Miller M. Fried-Oken M.B. Brain–computer interface with language model–electroencephalography fusion for locked-in syndrome.Neurorehabil. Neural Repair. 2014; 28: 387-394Crossref PubMed Google Scholar). It is our view that development of a functional DS-BCI must be predicated on imagined speech (see section "Imagined Speech: A Special Case of Speech" for a detailed description) as the communicative modality. However, several other types of speech have been utilized in experiments referenced throughout this text, making it important to define their meanings. Table 1 is a categorization of the different types of speech typically used in DS-BCI experimentation. Three types of speech are presented, namely, overt (Blakely et al., 2008Blakely T. Miller K.J. Rao R.P.N. Holmes M.D. Ojemann J.G. Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids.Conf. IEEE Eng. Med. Biol. Soc. 2008; 2008: 4964-4967PubMed Google Scholar), intended (Guenther et al., 2009Guenther F.H. Brumberg J.S. Joseph Wright E. Nieto-Castanon A. Tourville J.A. Panko M. Law R. Siebert S.A. Bartels J.L. Andreasen D.S. et al.A wireless brain-machine interface for real-time speech synthesis.PLoS One. 2009; 4: e8218Crossref PubMed Scopus (0) Google Scholar), and imagined (D'Zmura et al., 2009D'Zmura M. Deng S. Lappas T. Thorpe S. Srinivasan R. Toward {EEG} sensing of imagined speech. Hum. Comput. Interact.Part I. 2009; 5610: 40-48Google Scholar), and these are subcategorized according to whether the speech is being produced or perceived by a subject. Overt speech production results in an audible output that can be heard by the person speaking and by others within range of the sounds produced. Intended speech is the name given to describe when a person tries to speak but does not have the capacity to produce an audible output. Imagined speech is the internal pronunciation of words without any audible output or associated movement. These are types of speech production and possible methods of communication with DS-BCI. However, several studies have used decoding approaches applied to the neural correlates of speech perception as evidence for the potential of decoding speech processes for communication (Di Liberto et al., 2015Di Liberto G.M. O'Sullivan J.A. Lalor E.C. Low-frequency cortical entrainment to speech reflects phoneme-level processing.Curr. Biol. 2015; 25: 2457-2465Abstract Full Text Full Text PDF PubMed Scopus (46) Google Scholar, Wang et al., 2018Wang Y.Y. Wang P. Yu Y. Decoding English alphabet letters using EEG phase information.Front. Neurosci. 2018; 12: 1-10Crossref PubMed Scopus (0) Google Scholar). We consider it to be extremely important to distinguish speech perception studies from speech production studies and to be aware that the "speech" in these studies refers to different phenomena. In perception studies, the speech being considered is the stimulus provided by the experimenter. The corresponding response of the subject, typically in the auditory cortex, is the neural activity being decoded. This differs greatly from the study of speech production in which the subject is actively producing phones, words, or sentences, whether prompted or unprompted, with neural correlates typically corresponding to brain regions associated with speech production. Although speech perception studies are important for DS-BCI research, this review is primarily concerned with speech production and, in particular, imagined speech production.Table 1Categorization of Types of Speech Typically Used in DS-BCI ExperimentsProductionPerceptionOvertFully articulated speech with audible outputActive or passive hearing of audible speech (one's own speech or from another source)IntendedIntention to produce overt speech but without the capacity to produce audible outputPerception of one's own intended speech productionImaginedInternal pronunciation of words, independent of movement and without any audible outputPerception of one's own imagined speech production Open table in a new tab A DS-BCI consists of several important stages (see Figure 1). The stages depicted in Figures 1B–1G have each been extensively covered in the literature (Blakely et al., 2008Blakely T. Miller K.J. Rao R.P.N. Holmes M.D. Ojemann J.G. Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids.Conf. IEEE Eng. Med. Biol. Soc. 2008; 2008: 4964-4967PubMed Google Scholar, Guenther et al., 2009Guenther F.H. Brumberg J.S. Joseph Wright E. Nieto-Castanon A. Tourville J.A. Panko M. Law R. Siebert S.A. Bartels J.L. Andreasen D.S. et al.A wireless brain-machine interface for real-time speech synthesis.PLoS One. 2009; 4: e8218Crossref PubMed Scopus (0) Google Scholar; reviewed in Bocquelet et al., 2017Bocquelet F. Hueber T. Girin L. Chabardès S. Yvert B. Key considerations in designing a speech brain-computer interface.J. Physiol. 2017; 110: 392-401Google Scholar). However, there is relatively little consideration of the difficulty in modeling the first of these stages (Figure 1A), namely, imagined speech production, during which a participant articulates words internally without any motor movement. Neurolinguistics research is providing insight into the cognitive function, phenomenology, and neurobiology of speech production in general (Hickok, 2014Hickok G. The architecture of speech production and the role of the phoneme in speech processing.Lang. Cogn. Neurosci. 2014; 29: 2-20Crossref Scopus (51) Google Scholar) and imagined speech in particular (Alderson-Day and Fernyhough, 2015Alderson-Day B. Fernyhough C. Inner speech: development, cognitive functions, phenomenology, and neurobiology.Psychol. Bull. 2015; 141: 931-965Crossref PubMed Scopus (0) Google Scholar, Perrone-Bertolotti et al., 2014Perrone-Bertolotti M. Rapin L. Lachaux J.-P. Lœvenbruck H. What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring.Behav. Brain Res. 2014; 261: 220-239Crossref PubMed Scopus (48) Google Scholar), and it is our view that these insights should be utilized within DS-BCI research. We concur with the arguments expressed by Iljina et al., 2017Iljina O. Derix J. Schirrmeister R.T. Schulze-Bonhage A. Auer P. Aertsen A. Ball T. Neurolinguistic and machine-learning perspectives on direct speech BCIs for restoration of naturalistic communication.Brain Computer Interfaces. 2017; 4: 186-199Crossref Google Scholar that, given the complexity of speech production processes, combining research from the fields of BCI and neurolinguistics must be seen as an important approach for those seeking to capture and decode the phenomena. Imagined speech is the internal pronunciation of words without any motor movement or acoustic output (Torres-García et al., 2016Torres-García A.A. Reyes-García C.A. Villaseñor-Pineda L. García-Aguilar G. Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification.Expert Syst. Appl. 2016; 59: 1-12Crossref Scopus (9) Google Scholar) (see section "Imagined Speech: A Special Case of Speech"). Related, and overlapping, terminology for imagined speech includes self-talk, sub-vocal/covert speech, internal dialogue/monologue, sub-vocalization, utterance, self-verbalization, and self-statement (Morin and Michaud, 2007Morin A. Michaud J. Self-awareness and the left inferior frontal gyrus: inner speech use during self-related processing.Brain Res. Bull. 2007; 74: 387-396Crossref PubMed Scopus (61) Google Scholar). However, for the purposes of performing controlled experiments in the field of DS-BCI, it is necessary to maintain a consistent terminology and description of the phenomena (see section "Imagined Speech: A Special Case of Speech"). Although imagined and overt speech are not identical, there is overlap between imagined and overt speech production, and imagined speech has become an alternative neuro-paradigm for communicative BCI (D'Zmura et al., 2009D'Zmura M. Deng S. Lappas T. Thorpe S. Srinivasan R. Toward {EEG} sensing of imagined speech. Hum. Comput. Interact.Part I. 2009; 5610: 40-48Google Scholar, DaSalla et al., 2009DaSalla C.S. Kambara H. Sato M. Koike Y. Single-trial classification of vowel speech imagery using common spatial patterns.Neural Netw. 2009; 22: 1334-1339Crossref PubMed Scopus (59) Google Scholar, Deng et al., 2010Deng S. Srinivasan R. Lappas T. D'Zmura M. EEG classification of imagined syllable rhythm using Hilbert spectrum methods.J. Neural Eng. 2010; 7: 046006Crossref PubMed Scopus (0) Google Scholar). Such a system differs from other types of communicative BCIs (Chaudhary et al., 2017Chaudhary U. Xia B. Silvoni S. Cohen L.G. Birbaumer N. Brain–computer interface–based communication in the completely locked-in state.PLoS Biol. 2017; 15: 1-25Crossref Scopus (32) Google Scholar, Pandarinath et al., 2017Pandarinath C. Nuyujukian P. Blabe C.H. Sorice B.L. Saab J. Willett F.R. Hochberg L.R. Shenoy K.V. Henderson J.M. High performance communication by people with paralysis using an intracortical brain-computer interface.Elife. 2017; 6: 1-27Crossref Scopus (22) Google Scholar) in that it relies on tapping directly into a person's speech production processes, rather than using some unrelated neural activity as the method of communication. Several DS-BCI studies have used neurolinguistics approaches within their experimental procedures (González-Castañeda et al., 2017González-Castañeda E.F. Torres-García A.A. Reyes-García C.A. Villaseñor-Pineda L. Sonification and textification: proposing methods for classifying unspoken words from EEG signals.Biomed. Signal Process. Control. 2017; 37: 82-91Crossref Scopus (4) Google Scholar, Kim et al., 2013Kim, T., Lee, J., Choi, H., Lee, H., Kim, I.Y., Jang, D.P., 2013. Meaning based covert speech classification for brain-computer interface based on electroencephalography. Int. IEEE/EMBS Conf. Neural Eng. NER. 53–56.Google Scholar, Wang et al., 2011Wang W. Degenhart A.D. Sudre G.P. Pomerleau D.A. Tyler-Kabara E.C. Decoding semantic information from human electrocorticographic (ECoG) signals.Conf. Proc. IEEE Eng. Med. Biol. Soc. 2011; 2011: 6294-6298PubMed Google Scholar, Zhao and Rudzicz, 2015Zhao, S., Rudzicz, F., 2015. Classifying phonological categories in imagined and articulated speech. ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc. 2015, 992–996.Google Scholar). In general, the approaches used have been to design a constrained dictionary of words categorized according to their relative semantic or phonological relationships. The basic principle underpinning this approach is that the categorical features of a word may aid decoding accuracy in imagined speech. There is some evidence that this is a valid approach to take, particularly in relation to semantic categorization, which has received greater attention in the literature. Studies examining the feasibility of decoding semantic information from neural signals have shown that semantic category can be predicted from brain activity (Kim et al., 2013Kim, T., Lee, J., Choi, H., Lee, H., Kim, I.Y., Jang, D.P., 2013. Meaning based covert speech classification for brain-computer interface based on electroencephalography. Int. IEEE/EMBS Conf. Neural Eng. NER. 53–56.Google Scholar, Wang et al., 2011Wang W. Degenhart A.D. Sudre G.P. Pomerleau D.A. Tyler-Kabara E.C. Decoding semantic information from human electrocorticographic (ECoG) signals.Conf. Proc. IEEE Eng. Med. Biol. Soc. 2011; 2011: 6294-6298PubMed Google Scholar). However, further research is required to determine the true potential of neurolinguistics research in relation to the neurobiology of imagined speech and the structured processes underlying speech production, to inform DS-BCI research. Here, we review trends in DS-BCI research, and the current understanding of speech production processes, with an emphasis on imagined speech. We consider the potential implications of attempting to harness neurolinguistics concepts and the limitations of working directly with imagined speech. An argument is presented that effective research in the field of DS-BCI should incorporate neurolinguistics research and a thorough understanding of imagined speech where possible to aid the development of a naturalistic mode of communication. The development of a "silent" interface has long been an active area of research to enable users to communicate without audible articulation of their speech. Several modalities have been developed to facilitate such communication through movement-independent BCI, including BCI-spellers (e.g., D'albis et al., 2012D'albis T. Blatt R. Tedesco R. Sbattella L. Matteucci M. A predictive speller controlled by a brain-computer interface based on motor imagery.ACM Trans. Comput. Interact. 2012; 19: 1-25Crossref Scopus (18) Google Scholar), BCIs based on steady-state visually evoked potential (e.g., Bin et al., 2009Bin G. Gao X. Yan Z. Hong B. Gao S. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method.J. Neural Eng. 2009; 6: 046002Crossref PubMed Scopus (0) Google Scholar), and BCIs based on motor imagery (e.g., Tabar and Halici, 2017aTabar Y.R. Halici U. A novel deep learning approach for classification of EEG motor imagery signals.J. Neural Eng. 2017; 14: 016003Crossref PubMed Scopus (37) Google Scholar) (see AlSaleh et al., 2016AlSaleh, M.M., Arvaneh, M., Christensen, H., Moore, R.K., 2016. Brain-computer interface technology for speech recognition: a review. Asia-Pacific Signal Inf. Process. Assoc. Annu. Summit Conf. (APSIPA ASC 2016).Google Scholar, Tabar and Halici, 2017bTabar Y.R. Halici U. Brain computer interfaces for silent speech.Eur. Rev. 2017; 25: 208-230Crossref Scopus (0) Google Scholar for reviews). There are numerous forms that these silent interfaces have taken to provide a more naturalistic, language-based mode of communication, including ultrasound imaging of lip profiles (Denby et al., 2006Denby, B., Oussar, Y., Dreyfus, G., Stone, M., 2006. Prospects for a silent speech interface using ultrasound imaging. Proc. 2006 IEEE Int. Conf. Acoust. Speed Signal Process. Proc. 1, I-365-I-368.Google Scholar) and word recognition using magnetic implants and sensors (Gilbert et al., 2010Gilbert J.M. Rybchenko S.I. Hofe R. Ell S.R. Fagan M.J. Moore R.K. Green P. Isolated word recognition of silent speech using magnetic implants and sensors.Med. Eng. Phys. 2010; 32: 1189-1197Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar). However, approaches such as these require active motor skills that can be readily utilized as the communicative modality and are therefore not movement-independent BCIs. The utility of BCI as a mode for language-based communication has been noted by researchers for many years (Denby et al., 2006Denby, B., Oussar, Y., Dreyfus, G., Stone, M., 2006. Prospects for a silent speech interface using ultrasound imaging. Proc. 2006 IEEE Int. Conf. Acoust. Speed Signal Process. Proc. 1, I-365-I-368.Google Scholar, Donchin et al., 2000Donchin E. Spencer K.M. Wijesinghe R. The mental prosthesis: assessing the speed of a P300-based brain- computer interface.IEEE Trans. Rehabil. Eng. 2000; 8: 174-179Crossref PubMed Scopus (0) Google Scholar), with the concept for a DS-BCI being a movement-independent BCI based on neural activity corresponding directly to imagined speech production processes. However, the possibility of developing a BCI predicated purely on imagined speech has only recently begun to gather momentum (Ikeda et al., 2014Ikeda S. Shibata T. Nakano N. Okada R. Tsuyuguchi N. Ikeda K. Kato A. Neural decoding of single vowels during covert articulation using electrocorticography.Front. Hum. Neurosci. 2014; 8: 1-8Crossref PubMed Scopus (7) Google Scholar, Yoshimura et al., 2016Yoshimura N. Nishimoto A. Belkacem A.N. Shin D. Kambara H. Hanakawa T. Koike Y. Decoding of covert vowel articulation using electroencephalography cortical currents.Front. Neurosci. 2016; 10: 1-15Crossref PubMed Scopus (5) Google Scholar, Nguyen et al., 2017Nguyen C.H. Karavas G. Artemiadis P. Inferring imagined speech using EEG signals: a new approach using Riemannian Manifold features.J. Neural Eng. 2017; 15: 016002Crossref Scopus (1) Google Scholar) as researchers have revealed promising results in attempts to classify units of imagined speech (González-Castañeda et al., 2017González-Castañeda E.F. Torres-García A.A. Reyes-García C.A. Villaseñor-Pineda L. Sonification and textification: proposing methods for classifying unspoken words from EEG signals.Biomed. Signal Process. Control. 2017; 37: 82-91Crossref Scopus (4) Google Scholar, Martin et al., 2014Martin S. Brunner P. Holdgraf C. Heinze H.-J. Crone N.E. Rieger J. Schalk G. Knight R.T. Pasley B.N. Decoding spectrotemporal features of overt and covert speech from the human cortex.Front. Neuroeng. 2014; 7: 14Crossref PubMed Scopus (41) Google Scholar, Pei et al., 2011aPei X. Barbour D.L. Leuthardt E.C. Schalk G. Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans.J. Neural Eng. 2011; 8: 046028Crossref PubMed Scopus (0) Google Scholar, Yoshimura et al., 2016Yoshimura N. Nishimoto A. Belkacem A.N. Shin D. Kambara H. Hanakawa T. Koike Y. Decoding of covert vowel articulation using electroencephalography cortical currents.Front. Neurosci. 2016; 10: 1-15Crossref PubMed Scopus (5) Google Scholar, Zhao and Rudzicz, 2015Zhao, S., Rudzicz, F., 2015. Classifying phonological categories in imagined and articulated speech. ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc. 2015, 992–996.Google Scholar). There have been several incarnations of DS-BCIs, including a wireless BCI for real-time speech synthesis (Guenther et al., 2009Guenther F.H. Brumberg J.S. Joseph Wright E. Nieto-Castanon A. Tourville J.A. Panko M. Law R. Siebert S.A. Bartels J.L. Andreasen D.S. et al.A wireless brain-machine interface for real-time speech synthesis.PLoS One. 2009; 4: e8218Crossref PubMed Scopus (0) Google Scholar) and a concept for continuous speech recognition (Herff et al., 2017Herff C. de Pesters A. Heger D. Brunner P. Schalk G. Schultz T. Towards continuous speech recognition for {BCI}.in: Guger C. Allison B. Ushiba J. Brain-Computer Interface Research. A State-of-the-Art Summary 5. Springer International Publishing, 2017: 21-29Crossref Google Scholar). The current stream of DS-BCI research indicates a trend toward improved classification of imagined speech units for decoded brain activity (González-Castañeda et al., 2017González-Castañeda E.F. Torres-García A.A. Reyes-García C.A. Villaseñor-Pineda L. Sonification and textification: proposing methods for classifying unspoken words from EEG signals.Biomed. Signal Process. Control. 2017; 37: 82-91Crossref Scopus (4) Google Scholar, Martin et al., 2014Martin S. Brunner P. Holdgraf C. Heinze H.-J. Crone N.E. Rieger J. Schalk G. Knight R.T. Pasley B.N. Decoding spectrotemporal features of overt and covert speech from the human cortex.Front. Neuroeng. 2014; 7: 14Crossref PubMed Scopus (41) Google Scholar) and the development of methodologies for continuous decoding of imagined speech (Brumberg et al., 2016Brumberg J.S. Krusienski D.J. Chakrabarti S. Gunduz A. Brunner P. Ritaccio A.L. Schalk G. Spatio-temporal progression of cortical activity related to continuous overt and covert speech production in a reading task.PLoS One. 2016; 11: 1-21Crossref Scopus (6) Google Scholar). There have also been recent developments in the classification of the neural correlates of speech perception (Di Liberto et al., 2015Di Liberto G.M. O'Sullivan J.A. Lalor E.C. Low-frequency cortical entrainment to speech reflects phoneme-level processing.Curr. Biol. 2015; 25: 2457-2465Abstract Full Text Full Text PDF PubMed Scopus (46) Google Scholar, Wang et al., 2018Wang Y.Y. Wang P. Yu Y. Decoding English alphabet letters using EEG phase information.Front. Neurosci. 2018; 12: 1-10Crossref PubMed Scopus (0) Google Scholar), one of which demonstrates real-time classification of auditory sentences from neural activity (Moses et al., 2018Moses D.A. Leonard M.K. Chang E.F. Real-time classification of auditory sentences using evoked cortical activity in humans.J. Neural Eng. 2018; 15: 036005Crossref PubMed Scopus (1) Google Scholar). Although this research is vital for the implementation of a closed-loop DS-BCI, it is important that results from speech perception studies are assessed independently of speech production studies, as the neural activity corresponding to each cannot be assumed to have similar properties. There have been notable successes in attempts to improve the decoding of language content directly from neural activity. The neural correlates of vowels and consonants (Idrees and Farooq, 2016Idrees, B.M., Farooq, O., 2016. EEG based vowel classification during speech imagery. 2016 3rd Int. Conf. Comput. Sustain. Glob. Dev. 1130–1134.Google Scholar, Pei et al., 2011bPei X. Leuthardt E.C. Gaona C.M. Brunner P. Wolpaw J.R. Schalk G. Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition.Neuroimage. 2011; 54: 2960-2972Crossref PubMed Scopus (86) Google Scholar, Yoshimura et al., 2016Yoshimura N. Nishimoto A. Belkacem A.N. Shin D. Kambara H. Hanakawa T. Koike Y. Decoding of covert vowel articulation using electroencephalography cortical currents.Front. Neurosci. 2016; 10: 1-15Crossref PubMed Scopus (5) Google Scholar), phonemes (Brumberg et al., 2011Brumberg J.S. Wright E.J. Andreasen D.S. Guenther F.H. Kennedy P.R. Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex.Front. Neurosci. 2011; 5: 1-12PubMed Google Scholar, Leuthardt et al., 2011Leuthardt E.C. Gaona C. Sharma M. Szrama N. Roland J. Freudenberg Z. Solis J. Breshears J. Schalk G. Using the electrocorticographic speech network to control a brain–computer interface in humans.J. Neural Eng. 2011; 8: 036004Crossref PubMed Scopus (0) Google Scholar), syllables (Deng et al., 2010Deng S. Srinivasan R. Lappas T. D'Zmura M. EEG classification of imagined syllable rhythm using Hilbert spectrum methods.J. Neural Eng. 2010; 7: 046006Crossref PubMed Scopus (0) Google Scholar), whole words (González-Castañeda et al., 2017González-Castañeda E.F. Torres-García A.A. Reyes-García C.A. Villaseñor-Pineda L. Sonification and textification: proposing methods for classifying unspoken words from EEG signals.Biomed. Signal Process. Control. 2017; 37: 82-91Crossref Scopus (4) Google Scholar, Martin et al., 2016Martin S. Brunner P. Iturrate I. Millán J.d R. Schalk G. Knight R.T. Pasley B.N. Word pair classification during imagined speech using direct brain recordings.Sci. Rep. 2016; 6: 25803Crossref PubMed Scopus (15) Google Scholar), and even sentences (Herff et al., 2015Herff C. Heger D. de Pesters A. Telaar D. Brunner P. Schalk G. Schultz T. Brain-to-text: decoding spoken phrases from phone representations in the brain.Front. Neurosci. 2015; 9: 1-11Crossref PubMed Scopus (29) Google Scholar) have all been evaluated using advanced decoding algorithms. Decoding of discrete units of speech, single vowels, for example, has been a popular experimental paradigm in DS-BCI to date (Ikeda et al., 2014Ikeda S. Shibata T. Nakano N. Okada R. Tsuyuguchi N. Ikeda K. Kato A. Neural decoding of single vowels during covert articulation using electrocorticography.Front. Hum. Neurosci. 2014; 8: 1-8Crossref PubMed Scopus (7) Google Scholar, Sereshkeh et al, Rezazadeh Sereshkeh et al., 2017aRezazadeh Sereshkeh, A., Trott, R., Bricout, A., Chau, T., 2017a. EEG classification of covert speech using regularized neural networks. IEEE/ACM Trans. Audio Speech Lang. Process. 25, 2292–2300.Google Scholar) presented evidence suggesting that it is possible to classify units of imagined speech from electroencephalogram (EEG), presenting 63.2% ± 6.4 accuracy for pairwise classification tasks. Other studies have shown that decoding accuracies of vowels and consonants were similar for both overt and imagined speech (Pei et al., 2011aPei X. Barbour D.L. Leuthardt E.C. Schalk G. Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans.J. Neural Eng. 2011; 8: 046028Crossref PubMed Scopus (0) Google Scholar). Elsewhere, linguistic content has been harnessed to aid discrimination of both overt and imagined speech, with phonology (Zhao and Rudzicz, 2015Zhao, S., Rudzicz, F., 2015. Classifying phonological categories in imagined and articulated speech. ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc. 2015, 992–996.Google Scholar), semantics (Kim et al., 2013Kim, T., Lee, J., Choi, H., Lee, H., Kim, I.Y., Jang, D.P., 2013. Meaning based covert speech classification for brain-computer interface based on electroencephalography. Int. IEEE/EMBS Conf. Neural Eng. NER. 53–56.Google Scholar), and syntax (Herff et al., 2015Herff C. Heger D. de Pesters A. Telaar D. Brunner P. Schalk G. Schultz T. Brain-to-text: decoding spoken phrases from phone representations in the brain.Front. Neurosci. 2015; 9: 1-11Crossref PubMed Scopus (29) Google Scholar) each showing some potential to aid classification in DS-BCI. Figure 2, and the corresponding data in Table 2, categorizes DS-BCI studies according to recording technique and the type of speech being investigated. The time period for this analysis begins with the study of Blakely et al., 2008Blakely T. Miller K.J. Rao R.P.N. Holmes M.D. Ojemann J.G. Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids.Conf. IEEE Eng. Med. Biol. Soc. 2008; 2008: 4964-4967PubMed Google Scholar, because this is the first study based on the BCI paradigm depicted, and runs through to 2018. Criteria for inclusion in this analysis are those studies using typical recording techniques (EEG, electrocorticogram [ECoG], micro-arrays, functional magnetic resonance imagining [fMRI], and functional near-infrared spectroscopy [fNIRS]) to decode speech production (overt, imagined, intended), but not speech perception, directly from neu
Referência(s)