Heng, Yuwen, Wu, Yihong, Dasmahapatra, Srinandan and Kim, Hansung (2023) Dense material segmentation with context-aware network. In, Augusto de Sousa, A., Debattista, Kurt, Paljic, Alexis, Ziat, Mounia, Hurter, Christophe, Purchase, Helen, Farinella, Giovanni Maria, Radeva, Petia and Bouatouch, Kadi (eds.) Computer Vision, Imaging and Computer Graphics Theory and Applications: 17th International Joint Conference, VISIGRAPP 2022, Virtual Event, February 6–8, 2022, Revised Selected Papers. (Communications in Computer and Information Science, 1815) 1 ed. Springer Cham, pp. 66-88. (doi:10.1007/978-3-031-45725-8_4).
Abstract
The dense material segmentation task aims at recognising the material for every pixel in daily images. It is beneficial to applications such as robot manipulation and spatial audio synthesis. Modern deep-learning methods combine material features with contextual features. Material features can generalise to unseen images regardless of appearance properties such as material shape and colour. Contextual features can reduce the segmentation uncertainty by providing extra global or semi-global information about the image. Recent studies proposed to crop the images into patches, which forces the network to learn material features from local visual clues. Typical contextual information includes extracted feature maps from networks targeting object and place related tasks. However, due to the lack of contextual labels, existing methods use pre-trained networks to provide contextual features. As a consequence, the trained networks do not give a promising performance. Their accuracy is below 70%, and the predicted segments have coarse boundaries. Considering this problem, this chapter introduces the Context-Aware Material Segmentation Network (CAM-SegNet). The CAM-SegNet is a hybrid network architecture to simultaneously learn from contextual and material features jointly with labelled materials. The effectiveness of the CAM-SegNet is demonstrated by training the network to learn boundary-related contextual features. Since the existing material datasets are sparsely labelled, a self-training approach is adopted to fill in the unlabelled pixels. Experiments show that CAM-SegNet can identify materials correctly, even with similar appearances. The network improves the pixel accuracy by 3–20% and raises the Mean IoU by 6–28%.
This record has no associated files available for download.
More information
Identifiers
Catalogue record
Export record
Altmetrics
Contributors
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.