Cross-domain analysis of 3D object detection with point clouds

Abstract

Deep learning models such as convolutional neural networks and transformers have
been widely applied to solve 3D object detection problems in the domain of autonomous driving. However, the robustness of models related to cross-domain tasks is usually ignored. To adapt models to other domains, including different cities, countries and weathers, re-training with the target domain data is usually necessary for existing 3D object detection methods, which hinders the wide application of autonomous driving. To better understand the challenges in cross-domain 3D object detection tasks, we first deeply analyse the cross-domain performance of the state-of-the-art models. We observe that most models will overfit the training domains and it is challenging to directly adapt them to other domains. Meanwhile, existing domain adaptation solutions are shifting the knowledge domain of the models instead of improving their generalisation ability. To encounter the overfitting problem, we propose a novel sub-task that pays more attention to the detection quality of the bounding box’s surfaces and corners that are closer to the LiDAR sensor. Two additional evaluation metrics and a novel refinement head named EdgeHead are proposed for this new task. By guiding models to focus more on the learnable closer surfaces, EdgeHead greatly improves the cross-domain performance of existing models. Furthermore, we propose a more robust cross-domain 3D object detector, CornerPoint3D, which focuses more on the visible point cloud data and the nearest corners of objects towards the LiDAR sensor, realising a balanced trade-off between the detection quality of entire bounding boxes and the closer surfaces to the LiDAR sensor, providing a more practical and robust cross-domain 3D object detection solution. In addition, we also explore the 3D referring expression comprehension task. We introduce Talk2Radar, the first dataset including both the LiDAR and radar data for this task, and an efficient LiDAR/radar-based 3D REC model, T-RadarNet, which achieves SOTA performance on the Talk2Radar dataset. These works present deep investigations and more robust solutions for cross-domain 3D object detection, as well as advances in the interactive perception capabilities of LiDAR and 4D radar for environmental understanding in autonomous driving.

More information

Identifiers

ORCID for Xiaohao Cai: orcid.org/0000-0003-0924-2834

Catalogue record

Export record