AI supporting AAC pictographic symbol adaptations
AI supporting AAC pictographic symbol adaptations
The phenomenal increase in technological capabilities that allow the design and training of systems to cope with the complexities of natural language and visual representation in order to develop other formats is remarkable. It has made it possible to make use of image to image and text to image technologies to support those with disabilities in ways not previously explored. It has opened the world of adaptations from one picture to another in a design style of a user's choosing. Automated text simplification alongside graphical symbol representations to enhance understanding of complex content is already being used to support those with cognitive impairments and learning difficulties. Symbol sets have become embedded within applications as dictionaries and look up systems, but the need for flexibility and personalization remains a challenge. Most pictographic symbols are created over time within the bounds of a certain style and schema for particular groups such as those who use augmentative and alternative forms of communication (AAC). By using generative artificial intelligence, it is proposed that symbols could be produced based on the style of those already used by an individual or adapted to suit different requirements within local contexts, cultures and communities. This paper explores these ideas at the start of a small six-month pilot study to adapt a number of open licensed symbols based on the symbol set's original style. Once a collection has been automatically developed from image to image and text descriptions, potential stakeholders will evaluate the outcomes using an online voting system. Successful symbols will be made available and could potentially be added to the original symbol set offering a flexible personalized approach to AAC symbol generation hitherto not experienced by users.
artificial intelligence, augmentative and alternative communication, cognitive impairment, pictographic symbols, symbol adaptations
215-221
Draffan, E.A.
021d4f4e-d269-4379-ba5a-7e2ffb73d2bf
Wald, Mike
90577cfd-35ae-4e4a-9422-5acffecd89d5
Ding, Chao
22f1f673-adf7-48db-9ff0-211280213153
Yin, Yuanyuan
cdb7e6d5-a9d9-4ecc-bbaa-a10ea4350f39
23 August 2023
Draffan, E.A.
021d4f4e-d269-4379-ba5a-7e2ffb73d2bf
Wald, Mike
90577cfd-35ae-4e4a-9422-5acffecd89d5
Ding, Chao
22f1f673-adf7-48db-9ff0-211280213153
Yin, Yuanyuan
cdb7e6d5-a9d9-4ecc-bbaa-a10ea4350f39
Draffan, E.A., Wald, Mike, Ding, Chao and Yin, Yuanyuan
(2023)
AI supporting AAC pictographic symbol adaptations.
Studies in Health Technology and Informatics, 306, .
(doi:10.3233/SHTI230622).
Abstract
The phenomenal increase in technological capabilities that allow the design and training of systems to cope with the complexities of natural language and visual representation in order to develop other formats is remarkable. It has made it possible to make use of image to image and text to image technologies to support those with disabilities in ways not previously explored. It has opened the world of adaptations from one picture to another in a design style of a user's choosing. Automated text simplification alongside graphical symbol representations to enhance understanding of complex content is already being used to support those with cognitive impairments and learning difficulties. Symbol sets have become embedded within applications as dictionaries and look up systems, but the need for flexibility and personalization remains a challenge. Most pictographic symbols are created over time within the bounds of a certain style and schema for particular groups such as those who use augmentative and alternative forms of communication (AAC). By using generative artificial intelligence, it is proposed that symbols could be produced based on the style of those already used by an individual or adapted to suit different requirements within local contexts, cultures and communities. This paper explores these ideas at the start of a small six-month pilot study to adapt a number of open licensed symbols based on the symbol set's original style. Once a collection has been automatically developed from image to image and text descriptions, potential stakeholders will evaluate the outcomes using an online voting system. Successful symbols will be made available and could potentially be added to the original symbol set offering a flexible personalized approach to AAC symbol generation hitherto not experienced by users.
Text
AI supporting AAC Pictographic Symbol Adaptations
- Accepted Manuscript
Available under License Other.
More information
e-pub ahead of print date: 1 August 2023
Published date: 23 August 2023
Additional Information:
Publisher Copyright:
© 2023 IOS Press. All rights reserved.
Keywords:
artificial intelligence, augmentative and alternative communication, cognitive impairment, pictographic symbols, symbol adaptations
Identifiers
Local EPrints ID: 483681
URI: http://eprints.soton.ac.uk/id/eprint/483681
ISSN: 0926-9630
PURE UUID: b180e6f6-ccb3-4d2e-a7d6-629e604ded3f
Catalogue record
Date deposited: 03 Nov 2023 17:51
Last modified: 18 Mar 2024 03:15
Export record
Altmetrics
Contributors
Author:
E.A. Draffan
Author:
Mike Wald
Author:
Chao Ding
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics