Gesticulating with NAO: real-time context-aware co-speech gesture generation for human-robot interaction

Nguyen, Tan Viet Tuyen, Schmuck, Viktor and Celiktutan, Oya (2023) Gesticulating with NAO: real-time context-aware co-speech gesture generation for human-robot interaction. André, Elisabeth, Chetouani, Mohamed, Vaufreydaz, Dominique, Lucas, Gale, Schultz, Tanja, Morency, Louis-Philippe and Vinciarelli, Alessandro (eds.) In ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. Association for Computing Machinery. 3 pp . (doi:10.1145/3610661.3620664).

Abstract

Humans naturally produce nonverbal behaviours via facial, body, and vocal expressions to signal their messages, intentions, and feelings to their interacting partners during interactions. Considering robots are progressively moving out from research laboratories into human environments, there is an increasing need for them to be able to develop similar social intelligence skills. Equipping robots with human nonverbal communication skills, therefore, has been an active research area, where data-driven, end-to-end learning approaches have become predominant, offering scalability and generalisability. However, most recent works only consider a single character for modelling intrapersonal dynamics without paying attention to the interacting partner’s behaviours. Our research aims to address the gap in the literature by introducing a generative framework allowing social robots to produce co-speech gestures to convey their speech in a scenario of real-time human-robot interaction. Notably, the system also considers non-verbal signals observed from the interacting partner as a conditional input for producing robots’ communicative gestures.

This record has no associated files available for download.

Contributors

Author: Tan Viet Tuyen Nguyen

Download statistics