The Library / Intelligence & systems
Worth keeping
- 1
The Transformer processes sequences using attention without relying on recurrence or convolution.
- 2
Attention lets the model weight relationships among different positions in a sequence.
- 3
The architecture enabled more parallel training and demonstrated strong results on machine translation.
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
In Not Boring4 essays that cite it
welcome to not boring world
Jan 2026
Weekly Dose of Optimism #37
Apr 2023
Attention is All You Need
Mar 2023
Atoms Are Local
Nov 2022