RotLSTM: rotating memories in recurrent neural networks
RotLSTM: rotating memories in recurrent neural networks
Long Short-Term Memory (LSTM) units have the ability to memorize and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrized by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.
cs.LG
Velici, Vlad
9c9e1a57-8667-4239-a31d-234da5ce9f4b
Prügel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
1 May 2021
Velici, Vlad
9c9e1a57-8667-4239-a31d-234da5ce9f4b
Prügel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
Velici, Vlad and Prügel-Bennett, Adam
(2021)
RotLSTM: rotating memories in recurrent neural networks.
arXiv.
Abstract
Long Short-Term Memory (LSTM) units have the ability to memorize and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrized by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.
More information
Published date: 1 May 2021
Keywords:
cs.LG
Identifiers
Local EPrints ID: 449306
URI: http://eprints.soton.ac.uk/id/eprint/449306
ISSN: 2331-8422
PURE UUID: 23623fac-95eb-418a-9176-859cdde43a72
Catalogue record
Date deposited: 24 May 2021 16:30
Last modified: 16 Mar 2024 12:20
Export record
Contributors
Author:
Vlad Velici
Author:
Adam Prügel-Bennett
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics