The University of Southampton
University of Southampton Institutional Repository

Joint deep learning for land cover and land use classification

Joint deep learning for land cover and land use classification
Joint deep learning for land cover and land use classification
Land cover (LC) and land use (LU) have commonly been classified separately from remotely sensed imagery, without considering the intrinsically hierarchical and nested relationships between them. In this paper, for the first time, a highly novel Joint Deep Learning framework is proposed and demonstrated for LC and LU classification. The proposed Joint Deep Learning (JDL) model incorporates a multilayer perceptron (MLP) and convolutional neural network (CNN), and is implemented via a Markov process involving iterative updating. In the JDL, LU classification conducted by the CNN is made conditional upon the LC probabilities predicted by the MLP. In turn, those LU probabilities together with the original imagery are re-used as inputs to the MLP to strengthen the spatial and spectral feature representations. This process of updating the MLP and CNN forms a joint distribution, where both LC and LU are classified simultaneously through iteration. The proposed JDL method provides a general framework within which the pixel-based MLP and the patch-based CNN provide mutually complementary information to each other, such that both are refined in the classification process through iteration. Given the well-known complexities associated with the classification of very fine spatial resolution (VFSR) imagery, the effectiveness of the proposed JDL was tested on aerial photography of two large urban and suburban areas in Great Britain (Southampton and Manchester). The JDL consistently demonstrated greatly increased accuracies with increasing iteration, not only for the LU classification, but for both the LC and LU classifications, achieving by far the greatest accuracies for each at around 10 iterations. The average overall classification accuracies were 90.18% for LC and 87.92% for LU for the two study sites, far higher than the initial accuracies and consistently outperforming benchmark comparators (three each for LC and LU classification). This research, thus, represents the first attempt to unify the remote sensing classification of LC (state; what is there?) and LU (function; what is going on there?), where previously each had been considered separately only. It, thus, has the potential to transform the way that LC and LU classification is undertaken in future. Moreover, it paves the way to address effectively the complex tasks of classifying LC and LU from VFSR remotely sensed imagery via joint reinforcement, and in an automatic manner.
0034-4257
173-187
Zhang, Ce
72e137e7-06c5-483e-bdc7-21629e03bb5b
Sargent, Isabel
c0ae2d59-039b-45f2-a906-069fe46c6633
Pan, Xin
387a1d0d-63a4-432a-a443-0654cfcc9321
Li, Huapeng
9e72ecd5-9964-4038-ab71-9bc7a2fb0510
Gardiner, Andy
0c87983a-019a-4206-842b-ad12603b99e4
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Atkinson, Peter M.
985bc8d3-e826-4e02-8060-8388183eb697
Zhang, Ce
72e137e7-06c5-483e-bdc7-21629e03bb5b
Sargent, Isabel
c0ae2d59-039b-45f2-a906-069fe46c6633
Pan, Xin
387a1d0d-63a4-432a-a443-0654cfcc9321
Li, Huapeng
9e72ecd5-9964-4038-ab71-9bc7a2fb0510
Gardiner, Andy
0c87983a-019a-4206-842b-ad12603b99e4
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Atkinson, Peter M.
985bc8d3-e826-4e02-8060-8388183eb697

Zhang, Ce, Sargent, Isabel, Pan, Xin, Li, Huapeng, Gardiner, Andy, Hare, Jonathon and Atkinson, Peter M. (2019) Joint deep learning for land cover and land use classification. Remote Sensing of Environment, 221, 173-187. (doi:10.1016/j.rse.2018.11.014).

Record type: Article

Abstract

Land cover (LC) and land use (LU) have commonly been classified separately from remotely sensed imagery, without considering the intrinsically hierarchical and nested relationships between them. In this paper, for the first time, a highly novel Joint Deep Learning framework is proposed and demonstrated for LC and LU classification. The proposed Joint Deep Learning (JDL) model incorporates a multilayer perceptron (MLP) and convolutional neural network (CNN), and is implemented via a Markov process involving iterative updating. In the JDL, LU classification conducted by the CNN is made conditional upon the LC probabilities predicted by the MLP. In turn, those LU probabilities together with the original imagery are re-used as inputs to the MLP to strengthen the spatial and spectral feature representations. This process of updating the MLP and CNN forms a joint distribution, where both LC and LU are classified simultaneously through iteration. The proposed JDL method provides a general framework within which the pixel-based MLP and the patch-based CNN provide mutually complementary information to each other, such that both are refined in the classification process through iteration. Given the well-known complexities associated with the classification of very fine spatial resolution (VFSR) imagery, the effectiveness of the proposed JDL was tested on aerial photography of two large urban and suburban areas in Great Britain (Southampton and Manchester). The JDL consistently demonstrated greatly increased accuracies with increasing iteration, not only for the LU classification, but for both the LC and LU classifications, achieving by far the greatest accuracies for each at around 10 iterations. The average overall classification accuracies were 90.18% for LC and 87.92% for LU for the two study sites, far higher than the initial accuracies and consistently outperforming benchmark comparators (three each for LC and LU classification). This research, thus, represents the first attempt to unify the remote sensing classification of LC (state; what is there?) and LU (function; what is going on there?), where previously each had been considered separately only. It, thus, has the potential to transform the way that LC and LU classification is undertaken in future. Moreover, it paves the way to address effectively the complex tasks of classifying LC and LU from VFSR remotely sensed imagery via joint reinforcement, and in an automatic manner.

Text
JDL_Manuscript_accepted - Accepted Manuscript
Download (3MB)

More information

Accepted/In Press date: 10 November 2018
e-pub ahead of print date: 21 November 2018
Published date: February 2019

Identifiers

Local EPrints ID: 427873
URI: http://eprints.soton.ac.uk/id/eprint/427873
ISSN: 0034-4257
PURE UUID: ae13fbc3-0853-4124-9388-af0bc4059db8
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283

Catalogue record

Date deposited: 30 Jan 2019 17:30
Last modified: 07 Oct 2020 06:09

Export record

Altmetrics

Contributors

Author: Ce Zhang
Author: Isabel Sargent
Author: Xin Pan
Author: Huapeng Li
Author: Andy Gardiner
Author: Jonathon Hare ORCID iD
Author: Peter M. Atkinson

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×