Hugging Face Reads, Feb. 2021 - Long-range Transformers
This is a blog post summarizing four papers on long-range transformers: Longformer, Compressive Transformer, Linformer, and Performer. It discusses their approaches to reducing the quadratic cost of transformer self-attention with respect to sequence length.