Evaluating the performance of automated detection systems for long-term monitoring of delphinids in diverse marine soundscapes

Abstract

There is an increasing reliance on passive acoustic monitoring (PAM) as a cost-effective method for monitoring cetaceans, necessitating robust and efficient automated tools for extracting species presence. This work compares two methods, one based on the ‘off-line’ analysis of raw PAM data, using Convolutional Neural Networks (CNNs), and the second based on in-situ detections, implemented within the C-POD. The C-POD is a rapid, low-cost choice for monitoring of odontocetes, while CNNs, requiring large efforts to train, are gaining traction within bioacoustics as they offer performance benefits above standard detection and classification tools. This work represents the first empirical comparison of a C-POD with a system using a CNN on recorded raw acoustic data for monitoring delphinids. The comparison is based on 3000 hours of PAM data, collected off the west coast of Scotland, using a collocated C-POD and SoundTrap acoustic recorder. Results show that the system using a CNN achieves an overall accuracy of 0.82, and an effectiveness (F1-Score) of 0.78 as a click detector, whilst the C-POD achieves scores of 0.71 and 0.62, respectively. The method employing a CNN provides a lower missed detection rate, with the C-POD failing to detect > 90% delphinid positive hours at one focal site. However, the C-POD offered a lower false-positive rate across all analysis sites. This work highlights the importance of incorporating the right automated tools for long-term species monitoring, as the C-POD offers high precision rates for click detections, while the CNN based system provides a robust approach to identifying seasonal and diurnal trends in long-term dolphin occurrence.

More information

Identifiers

ORCID for Ellen L. White: orcid.org/0000-0002-3787-8699

ORCID for Paul R. White: orcid.org/0000-0002-4787-8713

ORCID for Jonathan M. Bull: orcid.org/0000-0003-3373-5807

Catalogue record

Export record