Performers and Memory – fireside chat between Sepp Hochreiter, Krzysztof Choromanski and Johannes Brandstetter

The paper “Rethinking Attention with Performers” introduced the Performer, a new model that approximates Transformer architectures and significantly improves their space and time complexity. 

Recently, Sepp Hochreiter and his team showed connections between transformer self-attention and Modern Hopfield networks which can also be extended towards new Performer networks. 

We are delighted to host a fireside session with the authors of these papers, Sepp Hochreiter (IARAI), Krzysztof Choromanski (Google Research) and Johannes Brandstetter (JKU Linz). The panelists will discuss the new linear attention mechanism in Performers, which significantly improves computational efficiency for large inputs, and its connection to classical Hopfield networks.