The University of Southampton
University of Southampton Institutional Repository

GANmut: learning interpretable conditional space for a gamut of emotions

GANmut: learning interpretable conditional space for a gamut of emotions
GANmut: learning interpretable conditional space for a gamut of emotions
Humans can communicate emotions through a plethora of facial expressions, each with its own intensity, nuances and ambiguities. The generation of such variety by means of conditional GANs is limited to the expressions encoded in the used label system. These limitations are caused either due to burdensome labelling demand or the confounded label space. On the other hand, learning from inexpensive and intuitive basic categorical emotion labels leads to limited emotion variability. In this paper, we propose a novel GAN-based framework that learns an expressive and interpretable conditional space (usable as a label space) of emotions, instead of conditioning on handcrafted labels. Our framework only uses the categorical labels of basic emotions to learn jointly the conditional space as well as emotion manipulation. Such learning can benefit from the image variability within discrete labels, especially when the intrinsic labels reside beyond the discrete space of the defined. Our experiments demonstrate the effectiveness of the proposed framework, by allowing us to control and generate a gamut of complex and compound emotions while using only the basic categorical emotion labels during training. Our source code is available at https://github.com/stefanodapolito/GANmut.
568-577
D'Apolito, Stefano
d2e8d5cd-f79d-4f29-beca-cf680e2bb906
Paudel, Danda Pani
92cefdf8-92e7-43ff-b952-6290a9844be0
Huang, Zhiwu
84f477cd-9097-44dd-a33e-ff71f253d36b
Vergara, Andres Romero
06431161-b4d2-4ca1-95ec-8229e77f2480
Van Gool, Luc
4d5f6fcf-2814-45d4-9028-86378b4b76b7
D'Apolito, Stefano
d2e8d5cd-f79d-4f29-beca-cf680e2bb906
Paudel, Danda Pani
92cefdf8-92e7-43ff-b952-6290a9844be0
Huang, Zhiwu
84f477cd-9097-44dd-a33e-ff71f253d36b
Vergara, Andres Romero
06431161-b4d2-4ca1-95ec-8229e77f2480
Van Gool, Luc
4d5f6fcf-2814-45d4-9028-86378b4b76b7

D'Apolito, Stefano, Paudel, Danda Pani, Huang, Zhiwu, Vergara, Andres Romero and Van Gool, Luc (2021) GANmut: learning interpretable conditional space for a gamut of emotions. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 568-577 . (doi:10.1109/CVPR46437.2021.00063).

Record type: Conference or Workshop Item (Paper)

Abstract

Humans can communicate emotions through a plethora of facial expressions, each with its own intensity, nuances and ambiguities. The generation of such variety by means of conditional GANs is limited to the expressions encoded in the used label system. These limitations are caused either due to burdensome labelling demand or the confounded label space. On the other hand, learning from inexpensive and intuitive basic categorical emotion labels leads to limited emotion variability. In this paper, we propose a novel GAN-based framework that learns an expressive and interpretable conditional space (usable as a label space) of emotions, instead of conditioning on handcrafted labels. Our framework only uses the categorical labels of basic emotions to learn jointly the conditional space as well as emotion manipulation. Such learning can benefit from the image variability within discrete labels, especially when the intrinsic labels reside beyond the discrete space of the defined. Our experiments demonstrate the effectiveness of the proposed framework, by allowing us to control and generate a gamut of complex and compound emotions while using only the basic categorical emotion labels during training. Our source code is available at https://github.com/stefanodapolito/GANmut.

This record has no associated files available for download.

More information

Published date: 19 June 2021

Identifiers

Local EPrints ID: 501679
URI: http://eprints.soton.ac.uk/id/eprint/501679
PURE UUID: 8dd9f7d4-c920-4d8f-9fc3-0c62ac16fc3b
ORCID for Zhiwu Huang: ORCID iD orcid.org/0000-0002-7385-079X

Catalogue record

Date deposited: 05 Jun 2025 16:57
Last modified: 06 Jun 2025 02:06

Export record

Altmetrics

Contributors

Author: Stefano D'Apolito
Author: Danda Pani Paudel
Author: Zhiwu Huang ORCID iD
Author: Andres Romero Vergara
Author: Luc Van Gool

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×