Portal de Periódicos da CAPES

A Whole-Body Integrated AVATAR System: Implementation of Telepresence With Intuitive Control and Immersive Feedback

2023; Institute of Electrical and Electronics Engineers; Linguagem: Inglês

10.1109/mra.2023.3328512

1558-223X

Sungman Park, Junsoo Kim, H. P. Lee, Minwoong Jo, Dohoon Gong, D.Y. Ju, Dami Won, Sihyeon Kim, Jinhyeok Oh, Hun Jang, Joonbum Bae,

Tactile and Sensory Interactions

This article proposes an intuitive and immersive whole-body teleoperation system with motion-based control and multimodal feedback. The system consists of an anthropomorphic teleoperated robot and a haptic interface platform. The teleoperated robot has dual arms with dexterous hands, a head with a neck, and waist, giving it a human-like appearance and wide range of motion as well as an omnidirectional mobile platform for improved mobility. The haptic interface platform enables a human operator to control the robot intuitively by measuring the operator’s motion with a motion-capture system, providing haptic feedback to the user’s arms, fingers, and feet, and providing 3D image feedback. Additionally, facial animation further enhances immersion by synchronizing the facial expression of the robot with the user’s voice. The proposed teleoperation system offers a promising solution for the human-oriented robotic avatar system, which was verified through a global competition: the $10 M All Nippon Airways (ANA) Avatar XPRIZE. The system was successfully evaluated with 45 min of training time for users who were new to our system. And the lessons learned from the competition and future improvements are discussed.