Visualisation and analysis of large distributed temperature sensing (DTS) datasets from high voltage cables

Abstract

High-voltage cables are a critical part of a countries power infrastructure, whether transferring power from large-scale, onshore and offshore renewable projects, or expanding network capacity for more conventionally generated energy sources. Ensuring the integrity of these assets requires continuous thermal monitoring; distributed temperature sensing (DTS) is the standard tool but produces archives of extreme size at metre-scale and minute-scale sampling over ~100 KM and years, ≥100 GB. Analysis of such huge archives has up until now been impossible. The challenges are twofold: interactive out-of-core computation over full resolution and display-aware visualization that preserves local transients under hard pixel limits. This work delivers a preliminary approach: an out-of-core stack ordered tiling, zstd compression, memory-mapped streaming, and adaptive, pixel-aware aggregation; a layered analysis pipeline from coarse heatmaps to contours and fixed quantiles, perceptual composites, fixed-scale wavelet energy, and UMAP–HDBSCAN segmentation. Finally, a Bayesian-optimized weighted baseline correction combining TOPHAT, JBCD, MORMOL, and IARPLS is also presented. Results demonstrate efficient and robust visualisation of 100 GB/3.4 billion-points of data on 32 GB RAM; contours and quantiles make seasonal structure and thresholds explicit; composites separate temporal from spatial temperature spikes and highlight sparse sampling; and baseline removal suppresses drift to reveal load-correlated vertical banding in the residuals.

More information

Identifiers

ORCID for Sunny Chaudhary: orcid.org/0000-0003-2664-7083

ORCID for Justin Dix: orcid.org/0000-0003-2905-5403

ORCID for Paul Lewin: orcid.org/0000-0002-3299-2556

Catalogue record

Export record