Expert Evaluation of ChatGPT Performance for Risk Management Process Based on ISO 31000 Standard
Expert Evaluation of ChatGPT Performance for Risk Management Process Based on ISO 31000 Standard
ChatGPT is widely known for its ability to facilitate knowledge exchange, support research endeavours, and enhance problem-solving across various scientific disciplines. However, to date, no empirical research has been undertaken to evaluate ChatGPT's performance against established standards or professional guidelines. Consequently, the present study aims to evaluate the performance of ChatGPT for the risk management (RM) process based on ISO 31000 standard using expert evaluation. The authors (1) identified the key indicators for measuring the performance of ChatGPT in managing construction risks based on ISO 31000 and determined the key assessment criteria for evaluating the identified indicators using a focus group session with Iraqi experts; and (2) quantitatively analysed the level of performance of ChatGPT under a fuzzy environment. The findings indicated that ChatGPT's overall performance was high. Specifically, its ability to provide relevant risk mitigation strategies was identified as its strongest aspect. However, the research also revealed that ChatGPT's consistency in risk assessment and prioritization was the least effective aspect. This research serves as a foundation for future studies and developments in the field of AI-driven risk management, advancing our theoretical understanding of the application of AI models like ChatGPT in real-world risk scenarios.
ChatGPT, ChatGPT Performance, AI, Risk, Risk management, ISO 31000
1-6
Al-Mhdawi, M.K.S.
e23cdd27-fe4c-4aec-81b3-be2b2616bf6c
Qazi, Abroon
9ef4fdc7-0a1a-4805-a46c-5d210b523517
Alzarrad, Ammar
1fa41d65-399c-423f-9102-0201ea8d258d
Dacre, Nicholas
90ea8d3e-d0b1-4a5a-bead-f95ab32afbd1
Rahimian, Farzad
73b2c53f-c29e-440c-bbc0-64e64e1b5d62
Buniya, Mohanad K.
70fb4b40-9052-4bce-aa57-afeda01f9fcc
Zhang, Hanqin
50d1aead-830d-47b8-b327-659341cb404a
17 July 2023
Al-Mhdawi, M.K.S.
e23cdd27-fe4c-4aec-81b3-be2b2616bf6c
Qazi, Abroon
9ef4fdc7-0a1a-4805-a46c-5d210b523517
Alzarrad, Ammar
1fa41d65-399c-423f-9102-0201ea8d258d
Dacre, Nicholas
90ea8d3e-d0b1-4a5a-bead-f95ab32afbd1
Rahimian, Farzad
73b2c53f-c29e-440c-bbc0-64e64e1b5d62
Buniya, Mohanad K.
70fb4b40-9052-4bce-aa57-afeda01f9fcc
Zhang, Hanqin
50d1aead-830d-47b8-b327-659341cb404a
Al-Mhdawi, M.K.S., Qazi, Abroon, Alzarrad, Ammar, Dacre, Nicholas, Rahimian, Farzad, Buniya, Mohanad K. and Zhang, Hanqin
(2023)
Expert Evaluation of ChatGPT Performance for Risk Management Process Based on ISO 31000 Standard.
SSRN Electronic Journal, .
(doi:10.2139/ssrn.4504409).
Abstract
ChatGPT is widely known for its ability to facilitate knowledge exchange, support research endeavours, and enhance problem-solving across various scientific disciplines. However, to date, no empirical research has been undertaken to evaluate ChatGPT's performance against established standards or professional guidelines. Consequently, the present study aims to evaluate the performance of ChatGPT for the risk management (RM) process based on ISO 31000 standard using expert evaluation. The authors (1) identified the key indicators for measuring the performance of ChatGPT in managing construction risks based on ISO 31000 and determined the key assessment criteria for evaluating the identified indicators using a focus group session with Iraqi experts; and (2) quantitatively analysed the level of performance of ChatGPT under a fuzzy environment. The findings indicated that ChatGPT's overall performance was high. Specifically, its ability to provide relevant risk mitigation strategies was identified as its strongest aspect. However, the research also revealed that ChatGPT's consistency in risk assessment and prioritization was the least effective aspect. This research serves as a foundation for future studies and developments in the field of AI-driven risk management, advancing our theoretical understanding of the application of AI models like ChatGPT in real-world risk scenarios.
Text
ssrn-4504409
- Accepted Manuscript
More information
Submitted date: 8 July 2023
Published date: 17 July 2023
Additional Information:
This paper is particularly relevant to both researchers and project professionals as it evaluates the performance of ChatGPT in managing risks within the framework of ISO 31000 standards. For researchers, the study investigates the practical application of AI, specifically ChatGPT, in risk management processes. Employing expert evaluation and a fuzzy-based assessment model, the research introduces a novel approach to quantify AI performance in a standard-driven environment, setting a foundation for further studies in AI-driven risk management. For project professionals, especially those in the construction industry, this research provides empirical insights into how AI can enhance the risk management process. The findings offer a detailed evaluation of ChatGPT’s capabilities in identifying risks, suggesting mitigation strategies, and prioritising actions, which are crucial for maintaining compliance with established risk management standards like ISO 31000.
Keywords:
ChatGPT, ChatGPT Performance, AI, Risk, Risk management, ISO 31000
Identifiers
Local EPrints ID: 492591
URI: http://eprints.soton.ac.uk/id/eprint/492591
PURE UUID: c0a109b1-0471-4227-9fd6-baa6c1d67739
Catalogue record
Date deposited: 07 Aug 2024 16:41
Last modified: 07 Aug 2024 16:45
Export record
Altmetrics
Contributors
Author:
M.K.S. Al-Mhdawi
Author:
Abroon Qazi
Author:
Ammar Alzarrad
Author:
Farzad Rahimian
Author:
Mohanad K. Buniya
Author:
Hanqin Zhang
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics