Portal de Periódicos da CAPES

Functional Materials and Devices for XR (VR/AR/MR) Applications

2021; Wiley; Volume: 31; Issue: 39 Linguagem: Inglês

10.1002/adfm.202106546

1616-3028

Seung Hwan Ko, John A. Rogers,

Tactile and Sensory Interactions

The “Metaverse” (meaning “beyond universe”) is the next big subject in the online world and it is essentially a shared virtual space in which virtual life and reality interact, co-exist, evolve together, and various cultural, social, and economic activities take place within them to create value. It combines an enhanced physical reality with a virtual space to encompass them with special technology; XR (eXtended Reality) which includes VR (Virtual Reality), AR (Augmented Reality) and MR (Mixed Reality). Steven Spielberg's recent science fiction action-adventure film, “Ready Player One” (2018) well illustrated a possible future metaverse society. Beyond this science fiction vision of the future, we are already entering the era of the metaverse, where the current COVID-19 pandemic situation and the associated demands for remote forms of social interactions are accelerating factors. The technical issues in XR (VR+AR+MR) lie in development of the software and the hardware system as well. We are currently witnessing a remarkable development in XR software engineering, while XR devices and materials development remain in the very infancy stage. XR hardware systems are expected to artificially reconstruct various human feelings and transmit the information to the human sensory systems to make them believe they are really feeling the virtual world. Oculus (Facebook), Vive (HTC), Index (Valve) are the popular commercial VR devices, but they are limited to specific feelings originating mainly from visual and auditory cues and realized only in rigid forms of materials and devices. Compared with visual and auditory senses, less advancement has been made in tactile senses (haptics) including temperature, texture, pressure, and touch. At the same time, rigid material and devices deteriorate the immersion in haptic feedback systems because intimate contact between haptic device and skin is very important to replicate the faithful haptic feedback. By introducing new soft skin-like materials and devices to the XR research field, various next-generation haptic feedback systems are under development. Among various tactile senses, vibro-haptic (mechanical feeling by vibration) and thermo-haptic (thermal feeling by heating or cooling) are being studied. Vibro-haptic relies on the electrical/mechanical components to stimulate the skin to deliver sensations of physical touch in the virtual world. Rogers and colleagues (article number 2008805) review the full range of research activities in skin integrated vibro-haptic interfaces. Shea and colleagues (article number 2006639) demonstrate feel-through haptics that are ultra-thin and soft for more faithful feeling delivery directly to the skin with sufficient force to make virtual objects feel tangible, or to change the perceived texture of a physical object. In addition to the mechanical sensation, thermal sensation is very important to generate a more realistic artificial feeling in XR since heat carries fluent information on the surrounding environment. Ko and colleagues (article number 2007376) and Lee and colleagues (article number 2007952) review the fundamental mechanism, design strategies, and the rational guidelines for the adoption of thermo-haptic technology in XR application. Multifunctional and high-resolution sensors are another integral part of natural and continuous interactions between human and XR devices. Especially, skin-compatible soft sensor development is important for implementing the user comfort and immersive feeling of feedback. Yeo and colleagues discuss packaging strategies, specific device designs and physiological sensors for XR application (article number 2005692). Javey and colleagues (article number 2008087) and Takei and colleagues (article number 2007436) review the important factors for materials and structures of wearable sensors, and their power sources and data communication. Skin electronics are one of the most promising platforms for sensing and actuation in future VR/AR devices. The detailed engineering aspects of e-skin devices such as input/output device, energy source and integrated systems are discussed by Someya and colleagues (article number 2009602). In addition to elastomer material, fiber and fiber assembly-based sensors are discussed for interactive textile electronics for XR application by Tao and colleagues (article number 2007254). Two new types of e-skin type sensors for XR application are demonstrated by Makarov and colleagues (article number 2007788) for a magnetosensitive E-Skin type sensor that can track motion and orientation in 3D and by Tee and colleagues (article number 2008650) for a predictive materials design system consisting of a capacitive micropyramidal structure sensor and its augmented reality application in surgical training. The haptic feedback and sensor technology can be integrated to make more complex XR devices. Bae and colleagues (article number 2007772) develop a new multimodal sensor with a printed liquid metal circuit and two modes of haptic feedback (vibro-haptic + thermo-haptic) device is demonstrated to deliver temperature and tactile feeling in virtual reality. Majidi and colleagues (article number 2007428) discuss soft materials based on-body devices for wearable sensors and haptic feedback, as well as their current challenges and future perspective. Robotic research is very closely related to XR technology such as robotic surgery or soft robotic teleoperation. Shepherd and colleagues (article number 2009364) review the recent advances in soft actuators and sensors for application in haptic feedback and sensing for soft robotics in XR application. The interface between machine and user is discussed by Chen and colleagues (article number 2008807) for the fusion of emerging stretchable electronics and machine learning technology and by Cheng and Colleagues (article number 2008347) for nanowire-based soft wearable HMI sensors and haptic feedback systems. Soft actuators for soft robotics are reviewed by Yi and colleagues (article number 2009835) for liquid crystal soft actuators for mixed reality applications and by Liu and colleagues (article number 2007437) for soft multifunctional tensile and torsional actuators. Optical components are also important for head mound display. New characteristics of image sensors and displays for developing future XR systems such as transparent or deformable display are discussed (article number 2009281). Lee and colleagues (article number 2104105) present a study on azopolymeric optical Fourier surfaces for AR. Lastly, three extensive reviews on the material for haptics technology are presented for polymeric material design and chemistry by Lipomi and colleagues (article number 2008375), the current trends in and perspectives for active materials in haptic technology by Kim and colleagues (article number 2008831), and haptic perception, mechanics, and material technologies for VR by Visell and colleagues (article number 2008186). As with radical advances in any new technology, ethical issues must be considered. Although such topics lie outside of the scope of a journal like Advanced Functional Materials, in this editorial, we provide some comments. As James Spiegel[1] discussed, XR technologies have the following potential hazards: “First, XR poses potential mental health risks, including depersonalization/derealization disorder. Second, XR technology raises serious concerns related to personal neglect of users’ own actual bodies and real physical environments. Third, XR technologies may be used to record personal data which could be deployed in ways that threaten personal privacy and present a danger related to manipulation of users’ beliefs, emotions, and behaviors. Finally, there are other moral and social risks associated with the way XR blurs the distinction between the real and illusory.” Beyond the excitement around technological advancements in functional materials and devices for XR applications, these issues should be kept in mind in the process of XR development. We believe that this collection of research articles and reviews provides a thorough outline of the recent progress, challenges, and future opportunities in functional materials and devices for XR (VR/AR/MR) and, furthermore, that this issue of the journal can serve as a guideline and inspiration to current and next generation researchers in this rapidly expanding field. Seung Hwan Ko is a professor in Applied Nano & Thermal Science (ANTS) Lab, Mechanical Engineering department., Seoul National University, Korea. Before joining Seoul National University, he had been a faculty at KAIST, Korea since 2009. He received his Ph.D. degree in mechanical engineering from UC Berkeley in 2006. He worked as a postdoc at UC Berkeley until 2009. His current research interests are stretchable/flexible electronics, transparent electronics, soft robotics, wearable electronics, laser-assisted nano/micro fabrication, and crack assisted nanomanufacturing. John A. Rogers is the Louis Simpson and Kimberly Querrey Professor of Materials Science and Engineering, Biomedical Engineering and Medicine at Northwestern University, with affiliate appointments in Mechanical Engineering, Electrical and Computer Engineering and Chemistry, where he is also Director of the recently endowed Querrey Simpson Institute for Bioelectronics. He received his Ph.D. degree in physical chemistry from MIT in 1995. He was a Junior Fellow in the Harvard University Society of Fellows from 1995–1997. From 1997–2002 he was on the staff at Bell Laboratories, and from 2003–2016 he was on the faculty at the University of Illinois at Urbana/Champaign. His current research interests are in advanced materials for bio-inspired and bio-integrated systems.