Evaluating Large Language Models (LLMs)' understanding of ‘Black Crusts’ predictive models for built heritage preservation

Abstract

This study evaluates the understanding of LLMs in predicting limestone sulphation, described in common language as ‘black crusts’, which is an environmental decay damaging for historic buildings, implying gypsum formation on the surface of carbonatic materials. The research question is: ‘To what extent can built heritage managers use LLMs for preservation advice?’. GPT-4, Claude, and OpenArt were used for evaluation, and prompts were designed to test different aspects of limestone sulphation predictions. Gypsum thickness calculated at various time intervals from published studies was prompted to the LLMs to generate new predictions. These results were then cross-referenced with predictive models to assess accuracy. The findings indicate that LLMs produced varying results each time, with significant discrepancies compared to published models. Numerical predictions, data fitting, and image forecasting based on LLMs were explored, underscoring the limitations of LLMs in predictive modelling. Further testing is required to leverage LLMs capabilities in heritage preservation.

More information

Identifiers

ORCID for Stephanie Gauthier: orcid.org/0000-0002-1720-1736

Catalogue record

Export record

Altmetrics