Leveraging error correction in voice-based text entry by Talk-and-Gaze

We present the design and evaluation of Talk-and-Gaze (TaG), a method for selecting and correcting errors with voice and gaze. TaG uses eye gaze to overcome the inability of voiceonly systems to provide spatial information. The user’s point of gaze is used to select an erroneous word either by dwelling on the word for 800 ms (D-TaG) or by uttering a “select” voice command (V-TaG). A user study with 12 participants compared D-TaG, V-TaG, and a voice-only method for selecting and correcting words. Corrections were performed more than 20% faster with D-TaG compared to the V-TaG or voice-only methods. As well, D-TaG was observed to require 24% less selection effort than V-TaG and 11% less selection effort than voice-only error correction. D-TaG was well received in a subjective assessment with 66% of users choosing it as their preferred choice for error correction in voice-based text entry.

eye tracking, interaction design, multimodal, text entry, usability, voice

10.1145/3313831.3376579

1-11

Sengupta, Korok

72599c31-2af0-4bc4-961e-de785c13a69b

Bhattarai, Sabin

d35c8f9d-aba5-4804-ad17-28fca90e8c40

Sarcar, Sayan

7db408dd-2b88-47e5-86e8-de98c735fc03

MacKenzie, Scott

6cf71d46-4a6a-443e-bb37-d939c8a2098a

Staab, Steffen

bf48d51b-bd11-4d58-8e1c-4e6e03b30c49

21 April 2020

Sengupta, Korok

72599c31-2af0-4bc4-961e-de785c13a69b

Bhattarai, Sabin

d35c8f9d-aba5-4804-ad17-28fca90e8c40

Sarcar, Sayan

7db408dd-2b88-47e5-86e8-de98c735fc03

MacKenzie, Scott

6cf71d46-4a6a-443e-bb37-d939c8a2098a

Staab, Steffen

bf48d51b-bd11-4d58-8e1c-4e6e03b30c49

Sengupta, Korok, Bhattarai, Sabin, Sarcar, Sayan, MacKenzie, Scott and Staab, Steffen (2020) Leveraging error correction in voice-based text entry by Talk-and-Gaze. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems,, , Honululu, United States. 25 - 30 Apr 2020. pp. 1-11 . (doi:10.1145/3313831.3376579).

Record type: Conference or Workshop Item (Paper)

Abstract

Text

CHI2020_TAG____VoiceGazeTextEdit-camera-ready-copy - Author's Original

Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (5MB)

More information

Accepted/In Press date: 11 January 2020

e-pub ahead of print date: April 2020

Published date: 21 April 2020

Additional Information: Funding Information: This work has been performed as part of the MAMEM9 project with funding from the European Union’s Horizon 2020 research and innovation program under grant agreement number 644780. We would like to thank Pooya Oladazimi and Tara Morovatdar (University of Koblenz-Landau) for their assistance during the evaluation phase and all the participants who took part in the evaluation studies and provided us with relevant feedback. Publisher Copyright: © 2020 ACM.

Venue - Dates: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems,, , Honululu, United States, 2020-04-25 - 2020-04-30

Keywords: eye tracking, interaction design, multimodal, text entry, usability, voice

Learn more about Web and Internet Science research