The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface
The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface
This paper conducts a comparison of the performance and cognitive workload between three UAV control interfaces on an nCA (navigation control autonomy) Tier 1-III flight navigation task. The first interface is the standard RC Joystick (RCJ) controller, the second interface is the multimodal speech and visual gesture (mSVG) interface, and the third interface is the modified version of the RCJ interface with altitude, attitude, and position (AAP) assist. The modified RCJ interface was achieved with the aid of the Keyboard (KBD). A model of the mSVG interface previously designed and tested was used in this comparison. An experiment study was designed to measure the completion time and navigation accuracy of participants using each of the three interfaces, on a developed path_v02 test flight path. Thirty-seven (37) participants volunteered. The NASA task load index (TLX) survey questionnaire was administered at the end of each interface experiment to access the participants experience and to estimate the interface cognitive workload. A commercial software, the RealFlight Drone Simulator (RFDS) was used to estimate the RCJ skill level of the participants. From the results of the experiment, it was shown that the flying hours, the number of months flying, and the RFDS Level 4 challenge performance was a good estimator for participants RCJ flying skill level. A two-way result was obtained in the comparison of the RCJ and mSVG interfaces. It was concluded that, although the mSVG was better than the standard RCJ interface, the AAP-assisted RCJ was found to be as effective as (in some cases better than) the mSVG interface. It was also shown, from the speech gesture ratio result, that the participants had a preference for gesture over speech when using the mSVG interface. Some further works such as an outdoor field test and a performance comparison at higher nCA levels were suggested.
Aerobot, RFDS (RealFlight Drone Simulator), Speech, Visual gesture, mSVG (multimodal speech and visual gesture), nCA (navigation control autonomy)
Abioye, Ayodeji
3ec89a0b-8e78-4ef6-a1d7-342d8f6da021
Prior, Stephen
9c753e49-092a-4dc5-b4cd-6d5ff77e9ced
Saddington, Peter
5f3b9162-2a5d-4cae-8cfc-a5116d3edef6
Ramchurn, Sarvapali
1d62ae2a-a498-444e-912d-a6082d3aaea3
January 2022
Abioye, Ayodeji
3ec89a0b-8e78-4ef6-a1d7-342d8f6da021
Prior, Stephen
9c753e49-092a-4dc5-b4cd-6d5ff77e9ced
Saddington, Peter
5f3b9162-2a5d-4cae-8cfc-a5116d3edef6
Ramchurn, Sarvapali
1d62ae2a-a498-444e-912d-a6082d3aaea3
Abioye, Ayodeji, Prior, Stephen, Saddington, Peter and Ramchurn, Sarvapali
(2022)
The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface.
Robotics and Autonomous Systems, 147, [103915].
(doi:10.1016/j.robot.2021.103915).
Abstract
This paper conducts a comparison of the performance and cognitive workload between three UAV control interfaces on an nCA (navigation control autonomy) Tier 1-III flight navigation task. The first interface is the standard RC Joystick (RCJ) controller, the second interface is the multimodal speech and visual gesture (mSVG) interface, and the third interface is the modified version of the RCJ interface with altitude, attitude, and position (AAP) assist. The modified RCJ interface was achieved with the aid of the Keyboard (KBD). A model of the mSVG interface previously designed and tested was used in this comparison. An experiment study was designed to measure the completion time and navigation accuracy of participants using each of the three interfaces, on a developed path_v02 test flight path. Thirty-seven (37) participants volunteered. The NASA task load index (TLX) survey questionnaire was administered at the end of each interface experiment to access the participants experience and to estimate the interface cognitive workload. A commercial software, the RealFlight Drone Simulator (RFDS) was used to estimate the RCJ skill level of the participants. From the results of the experiment, it was shown that the flying hours, the number of months flying, and the RFDS Level 4 challenge performance was a good estimator for participants RCJ flying skill level. A two-way result was obtained in the comparison of the RCJ and mSVG interfaces. It was concluded that, although the mSVG was better than the standard RCJ interface, the AAP-assisted RCJ was found to be as effective as (in some cases better than) the mSVG interface. It was also shown, from the speech gesture ratio result, that the participants had a preference for gesture over speech when using the mSVG interface. Some further works such as an outdoor field test and a performance comparison at higher nCA levels were suggested.
Text
abioye_elsevier_ras3_revised
- Accepted Manuscript
Restricted to Registered users only
Request a copy
More information
Accepted/In Press date: 4 October 2021
e-pub ahead of print date: 13 October 2021
Published date: January 2022
Keywords:
Aerobot, RFDS (RealFlight Drone Simulator), Speech, Visual gesture, mSVG (multimodal speech and visual gesture), nCA (navigation control autonomy)
Identifiers
Local EPrints ID: 453153
URI: http://eprints.soton.ac.uk/id/eprint/453153
ISSN: 0921-8890
PURE UUID: 971af48c-1bfe-4caa-a9a9-0c4e06dcd07a
Catalogue record
Date deposited: 10 Jan 2022 17:47
Last modified: 12 Nov 2024 02:52
Export record
Altmetrics
Contributors
Author:
Ayodeji Abioye
Author:
Peter Saddington
Author:
Sarvapali Ramchurn
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics