Visual navigation of forest environments for small-scale rovers

Abstract

Off-road navigation in forest environments is a challenging problem in field robotics. Rovers are required to infer the traversability of a priori unknown and dynamically changing forest terrain using noisy onboard navigation sensors. The problem is compounded for small-sized rovers, such as those in a swarm. While such rovers may be useful for collaboratively monitoring large tracts of forests as a swarm, with low environmental impact, their size-proportional low-viewpoint affords them a restricted view for navigation, which may be partially occluded by forest vegetation. In addition, environmental factors (varying weather and lighting conditions), rover motion factors (blurred image and changes in camera pitch angle) and low-cost requirements for deploying large swarms pose more challenges. To address these challenges, we first developed a forest environment dataset for smallscale rovers to navigate forest terrain. Around calibrated, synchronized, aligned and timestamped 200,000 RGB-Depth image pairs annotated with GPS, IMU, and wheel rotary encoder data were recorded with a custom rig from the perspective of a small ground rover moving through a forest. We provide this dataset to the community to fill a need identified in our own research and hope it will accelerate progress in robots navigating the challenging forest environment. This thesis describes our custom hardware and methodology to collect the data, subsequent processing and quality of the data, and how to access it. Benefiting from the dataset, we have designed a low-cost low viewpoint navigation system tailored for small-sized forest rovers. First, we have developed a geometry-based approach because of (i) the high cost of LiDAR and depth sensors used for obtaining geometric information, (ii) incomplete depth information resulting from environmental factors such as high contrast and high dynamic range lighting conditions, (iii) unreliability of depth information in unstructured environments due to depth sensor limitation, and (iv) difficulty capturing geometric features of compliant vegetation using depth cameras. A lightweight convolution neural network is used to predict depth images from RGB input images from a low-viewpoint monocular camera. Subsequently, iv a simple coarse-grained navigation algorithm aggregates the predicted depth information to steer our mobile platform towards open traversable areas in the forest while avoiding obstacles. Our navigation algorithm has been extensively tested in high fidelity forest simulations and in field trials. Using no more than a 16 × 16 pixel depth prediction image from a 32 × 32 pixel RGB image, our algorithm was able to successfully navigate a total of over 750 m of real-world forest terrain. Due to properties of poor discriminative power, and computationally expensive in hand-crafted features that may often be brittle in varying environmental conditions, we thus have also developed an appearance-based approach using self-learned features. Four different state-of-the-art DenseNet-121, MobileNet-V1, MobileNet-V2 and NASNetMobile have been investigated for multiclass classification of steering actions. From the four models, the MobileNet-V1 and MobileNet-V2 are selected for field experiments due to their high accuracy and runtime performance. The MobileNet-V1 and MobileNet-V2 models, trained following an end-to-end learning approach, are deployed to steer a mobile platform towards traversable paths while avoiding obstacles. Receiving a 128 × 96 pixels RGB image as input, the algorithm was able to successfully navigate a total of over 3 km of real-world forest terrain. The field experiments were conducted in a forest environment comprising shrubs, dense bushes, tall grass, fallen branches, fallen tree trunks, small ditches and mounds, and standing trees, under five different weather conditions and at four different times of day. Furthermore, our algorithms exhibit robustness to motion blur, low lighting at dusk, and high-contrast lighting conditions. In addition, the geometry-based approach is robust to robustness to changes in the mobile platform’s camera pitch angle, In this study, the steering commands output from our navigation algorithms direct an operator pushing the mobile platform. The sensor hardware for off-road navigation is a Logitech C270 camera and Raspberry Pi 4 embedded computer. The thesis demonstrates that it is promising to employ hand-crafted features and selflearned features for forest navigation. Our findings suggest that the use of low-cost hardware and low-resolution imagery allows for a reconceptualization of sparse rover swarms. This PhD research is challenging the traditional notion and encouraging a paradigm shift towards considering low-cost approaches that prioritize feasibility and practicality, rather than blindly relying on costly sensors and complex solutions. This study has a direct impact on improving the capabilities of small-scale rovers in navigating through forests, opening up new possibilities for autonomous exploration and research in challenging terrains. It also provides valuable insights and inspiration for the initial phases of low-cost, low-viewpoint forest navigation, which serves as a crucial stepping stone toward achieving navigation capabilities on an autonomous small-scale rover.

More information

Identifiers

ORCID for Chaoyue Niu: orcid.org/0000-0001-7626-0317

ORCID for Danesh Tarapore: orcid.org/0000-0002-3226-6861

Catalogue record

Export record