A Remark on a Paper of Krotov and Hopfield

Fei Tang & Michael Kopp

In their recent paper titled “Large Associative Memory Problem in Neurobiology and Machine Learning” the authors gave a biologically plausible microscopic theory from which one can recover many dense associative memory models discussed in the literature. We show that the layers of the recent “MLP-mixer” as well as the essentially equivalent model in “Feed-Forward Layers” are amongst them.

arXiv:2105.15034, 2021-06-03