FSPool: Learning set representations with featurewise sort pooling
FSPool: Learning set representations with featurewise sort pooling
Traditional set prediction models can struggle with simple datasets due to an issue we call the responsibility problem. We introduce a pooling method for sets of feature vectors based on sorting features across elements of the set. This can be used to construct a permutation-equivariant auto-encoder that avoids this responsibility problem. On a toy dataset of polygons and a set version of MNIST, we show that such an auto-encoder produces considerably better reconstructions and representations. Replacing the pooling function in existing set encoders with FSPool improves accuracy and convergence speed on a variety of datasets.
Zhang, Yan
0edf84ab-1e32-4239-bef6-7fe80d6bc7a7
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Prugel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
26 April 2020
Zhang, Yan
0edf84ab-1e32-4239-bef6-7fe80d6bc7a7
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Prugel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
Zhang, Yan, Hare, Jonathon and Prugel-Bennett, Adam
(2020)
FSPool: Learning set representations with featurewise sort pooling.
International Conference on Learning Representations, Millennium Hall, Addis Ababa, Ethiopia.
26 - 30 Apr 2020.
Record type:
Conference or Workshop Item
(Paper)
Abstract
Traditional set prediction models can struggle with simple datasets due to an issue we call the responsibility problem. We introduce a pooling method for sets of feature vectors based on sorting features across elements of the set. This can be used to construct a permutation-equivariant auto-encoder that avoids this responsibility problem. On a toy dataset of polygons and a set version of MNIST, we show that such an auto-encoder produces considerably better reconstructions and representations. Replacing the pooling function in existing set encoders with FSPool improves accuracy and convergence speed on a variety of datasets.
This record has no associated files available for download.
More information
Accepted/In Press date: 19 December 2019
Published date: 26 April 2020
Venue - Dates:
International Conference on Learning Representations, Millennium Hall, Addis Ababa, Ethiopia, 2020-04-26 - 2020-04-30
Identifiers
Local EPrints ID: 436921
URI: http://eprints.soton.ac.uk/id/eprint/436921
PURE UUID: 83ea56fc-2baf-407a-ba9a-4749d6b36c8d
Catalogue record
Date deposited: 14 Jan 2020 17:30
Last modified: 17 Mar 2024 03:05
Export record
Contributors
Author:
Yan Zhang
Author:
Jonathon Hare
Author:
Adam Prugel-Bennett
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics