Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitner, Grey Nearing, Sepp Hochreiter, and Günter Klambauer

MC-LSTM

MC-LSTM architecture

Many real-world systems are governed by conservation of mass, energy, number of particles, or other properties. For modeling such systems, specialized mechanisms should be used to conserve and redistribute certain inputs across the system storage. We will refer to any type of conserved quantity as “mass”.

In this work, we introduce Mass-Conserving Long Short-Term Memory (MC-LSTM) network that enforces the conservation of mass. The original LSTM model incorporated memory cells into recurrent neural networks (RNNs), which alleviated the vanishing gradient problem. Here, the memory cells are used as mass accumulators, or mass storage. The sum of the memory cells in the network represents the current mass of the system and is conserved over time. The MC-LSTM gates operate as control units on mass flux. The inputs are divided into mass inputs which are conserved and auxiliary inputs which are used to control the gates. We show that MC-LSTM provides a powerful neural arithmetic unit. We apply MC-LSTM to traffic forecasting, modeling a pendulum with friction, and modeling hydrological processes, and demonstrate that MC-LSTM excels at task requiring conservation of mass.

arXiv:2101.05186, 2021-01-13.

Download
View paper
IARAI Authors
Dr Sepp Hochreiter
Research
A Theory of AI
Keywords
LSTMs

©2021 IARAI - INSTITUTE OF ADVANCED RESEARCH IN ARTIFICIAL INTELLIGENCE

Imprint | Privacy Policy

Stay in the know with developments at IARAI

We can let you know if there’s any

updates from the Institute.
You can later also tailor your news feed to specific research areas or keywords (Privacy)
Loading

Log in with your credentials

Forgot your details?

Create Account