Semantics of texture

Abstract

In this thesis we investigate means by which the semantic and visual spaces of texture may be tied together, and argue for the importance of explicit semantic modelling inhuman-centred texture analysis tasks such as retrieval, annotation, synthesis, and zeroshot learning.

We take a new approach to semantic texture labelling by adopting a pairwise comparison framework robust to human biases, and within a semantic space consisting of attributes,low-level visual features acting as building blocks for more expressive semantic ontologies. We crowdsource a dataset of approximately 140;000 pairwise comparisons across 319 classes of texture and 98 attributes | to our knowledge the largest of its kind.

To aid in learning from sparsely labelled pairwise comparison datasets such as this we derive a new Bayesian probabilistic approach, providing a natural framework in which to incorporate prior knowledge and to measure uncertainty, and outperforming the often used Ranking SVM on incomplete and unreliable data. We demonstrate how the error variance present in our pairwise comparison data may be precisely quantified, allowing us to identify and discard rogue responses in a principled way.

Existing texture descriptors are then assessed in terms of their correspondence to the attributes comprising the semantic space. Textures with strong presence of attributes connoting randomness and complexity are shown to be poorly modelled by existing descriptors. These effects are likely due to disparities between human perception of what texture entails, and definitions adopted prior to (or, sometimes, after) the design of computational texture analysis systems.

Despite the decifiencies of the visual descriptors they are based upon, we demonstrate the benefit of semantically enriched descriptors in a retrieval experiment. Semantic modelling of texture is shown to provide considerable value in both feature selection and in analysis tasks.

More information

Identifiers

ORCID for Mark Nixon: orcid.org/0000-0002-9174-5934

Catalogue record

Export record