PhySwin: an efficient and physically-informed foundation model for multispectral earth observation

Abstract

Recent progress on Remote Sensing Foundation Models (RSFMs) aims toward universal representations for Earth observation imagery. However, current effort soften scale up in size significantly without addressing efficiency constraints critical for real-world applications (e.g., onboard processing, rapid disaster response) or treat multispectral (MS) data as generic imagery, overlooking valuable physical priors. We introduce PhySwin, a foundation model for MS data that integrates physical priors with computational efficiency. PhySwin combines three innovations:(i) physics-informed pretraining objectives leveraging radiometric constraints to enhance feature learning; (ii) an efficient Mix MAE formulation tailored to SwinV2for low-FLOP, scalable pretraining; and (iii) token-efficient spectral embedding to retain spectral detail without increasing token counts. Pretrained on over 1MSentinel-2 tiles, PhySwin achieves SOTA results (+1.32% mIoU segmentation,+0.80% F1 change detection) while reducing inference latency by up to 14.4× and computational complexity by up to 43.6× compared to ViT-based RSFMs.

More information

Identifiers

ORCID for Alex S. Weddell: orcid.org/0000-0002-6763-5460

Catalogue record

Export record