Proactive multi-USV maritime search and rescue in stochastic wave environments: a hierarchical, non-causal reinforcement learning framework

Abstract

Maritime search and rescue (SAR) in stochastic wave environments presents a critical challengefor multi-Unmanned Surface Vehicle (USV) systems and demands a fine balance betweensearch efficiency and operational safety. This paper proposes a novel hierarchical reinforcementlearning framework, termed Non-Causal Reward Multi-Agent Proximal Policy Optimization(NCR-MAPPO), to address this challenge. Our framework decouples the mission by employinga strategic guidance system based on International Maritime Organization (IMO) standards forsystematic coverage while a tactical motion controller built upon the Multi-Agent ProximalPolicy Optimization (MAPPO) algorithm learns cooperative execution. The core innovation isa Non-Causal Reward (NCR) mechanism that incorporates short-term wave field prediction intothe decision process, enabling a shift from reactive collision avoidance to proactive seakeepingcontrol. Through comprehensive simulations, we demonstrate the superiority of our framework.Compared to the standard MAPPO baseline, NCR-MAPPO significantly enhances survivabilityby reducing wave impact incidents by 27% and exposure to hazardous sea states by 25% whilemaintaining high mission efficiency. Thiswork provides a robust solution for autonomous marinesystems by bridging the gap between regulatory compliance and predictive safety control.

More information

Identifiers

ORCID for Tahsin Tezdogan: orcid.org/0000-0002-7032-3038

Catalogue record

Export record