The University of Southampton
University of Southampton Institutional Repository

The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface

The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface
The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface
This paper conducts a comparison of the performance and cognitive workload between three UAV control interfaces on an nCA (navigation control autonomy) Tier 1-III flight navigation task. The first interface is the standard RC Joystick (RCJ) controller, the second interface is the multimodal speech and visual gesture (mSVG) interface, and the third interface is the modified version of the RCJ interface with altitude, attitude, and position (AAP) assist. The modified RCJ interface was achieved with the aid of the Keyboard (KBD). A model of the mSVG interface previously designed and tested was used in this comparison. An experiment study was designed to measure the completion time and navigation accuracy of participants using each of the three interfaces, on a developed path_v02 test flight path. Thirty-seven (37) participants volunteered. The NASA task load index (TLX) survey questionnaire was administered at the end of each interface experiment to access the participants experience and to estimate the interface cognitive workload. A commercial software, the RealFlight Drone Simulator (RFDS) was used to estimate the RCJ skill level of the participants. From the results of the experiment, it was shown that the flying hours, the number of months flying, and the RFDS Level 4 challenge performance was a good estimator for participants RCJ flying skill level. A two-way result was obtained in the comparison of the RCJ and mSVG interfaces. It was concluded that, although the mSVG was better than the standard RCJ interface, the AAP-assisted RCJ was found to be as effective as (in some cases better than) the mSVG interface. It was also shown, from the speech gesture ratio result, that the participants had a preference for gesture over speech when using the mSVG interface. Some further works such as an outdoor field test and a performance comparison at higher nCA levels were suggested.
Aerobot, RFDS (RealFlight Drone Simulator), Speech, Visual gesture, mSVG (multimodal speech and visual gesture), nCA (navigation control autonomy)
0921-8890
Abioye, Ayodeji
3ec89a0b-8e78-4ef6-a1d7-342d8f6da021
Prior, Stephen
9c753e49-092a-4dc5-b4cd-6d5ff77e9ced
Saddington, Peter
5f3b9162-2a5d-4cae-8cfc-a5116d3edef6
Ramchurn, Sarvapali
1d62ae2a-a498-444e-912d-a6082d3aaea3
Abioye, Ayodeji
3ec89a0b-8e78-4ef6-a1d7-342d8f6da021
Prior, Stephen
9c753e49-092a-4dc5-b4cd-6d5ff77e9ced
Saddington, Peter
5f3b9162-2a5d-4cae-8cfc-a5116d3edef6
Ramchurn, Sarvapali
1d62ae2a-a498-444e-912d-a6082d3aaea3

Abioye, Ayodeji, Prior, Stephen, Saddington, Peter and Ramchurn, Sarvapali (2022) The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface. Robotics and Autonomous Systems, 147, [103915]. (doi:10.1016/j.robot.2021.103915).

Record type: Article

Abstract

This paper conducts a comparison of the performance and cognitive workload between three UAV control interfaces on an nCA (navigation control autonomy) Tier 1-III flight navigation task. The first interface is the standard RC Joystick (RCJ) controller, the second interface is the multimodal speech and visual gesture (mSVG) interface, and the third interface is the modified version of the RCJ interface with altitude, attitude, and position (AAP) assist. The modified RCJ interface was achieved with the aid of the Keyboard (KBD). A model of the mSVG interface previously designed and tested was used in this comparison. An experiment study was designed to measure the completion time and navigation accuracy of participants using each of the three interfaces, on a developed path_v02 test flight path. Thirty-seven (37) participants volunteered. The NASA task load index (TLX) survey questionnaire was administered at the end of each interface experiment to access the participants experience and to estimate the interface cognitive workload. A commercial software, the RealFlight Drone Simulator (RFDS) was used to estimate the RCJ skill level of the participants. From the results of the experiment, it was shown that the flying hours, the number of months flying, and the RFDS Level 4 challenge performance was a good estimator for participants RCJ flying skill level. A two-way result was obtained in the comparison of the RCJ and mSVG interfaces. It was concluded that, although the mSVG was better than the standard RCJ interface, the AAP-assisted RCJ was found to be as effective as (in some cases better than) the mSVG interface. It was also shown, from the speech gesture ratio result, that the participants had a preference for gesture over speech when using the mSVG interface. Some further works such as an outdoor field test and a performance comparison at higher nCA levels were suggested.

Text
abioye_elsevier_ras3_revised - Accepted Manuscript
Restricted to Registered users only
Download (2MB)
Request a copy

More information

Accepted/In Press date: 4 October 2021
e-pub ahead of print date: 13 October 2021
Published date: January 2022
Keywords: Aerobot, RFDS (RealFlight Drone Simulator), Speech, Visual gesture, mSVG (multimodal speech and visual gesture), nCA (navigation control autonomy)

Identifiers

Local EPrints ID: 453153
URI: http://eprints.soton.ac.uk/id/eprint/453153
ISSN: 0921-8890
PURE UUID: 971af48c-1bfe-4caa-a9a9-0c4e06dcd07a
ORCID for Ayodeji Abioye: ORCID iD orcid.org/0000-0003-4637-3278
ORCID for Stephen Prior: ORCID iD orcid.org/0000-0002-4993-4942
ORCID for Sarvapali Ramchurn: ORCID iD orcid.org/0000-0001-9686-4302

Catalogue record

Date deposited: 10 Jan 2022 17:47
Last modified: 28 Mar 2024 02:47

Export record

Altmetrics

Contributors

Author: Ayodeji Abioye ORCID iD
Author: Stephen Prior ORCID iD
Author: Peter Saddington
Author: Sarvapali Ramchurn ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×