BigBird's block sparse attention mechanism, its implementation, and advantages over BERT's full attention, including that a BigBird RoBERTa-like model is now available in 🤗Transformers.

31 Mar

← EarlierWeek 12 Later →Week 14

← Week 12Week 13, 2021Week 14 →

AI lab announcements, 29 March – 4 April 2021 — forck

← Week 12

Week 13, 2021 · AI lab announcements, 29 March – 4 April 2021

Week 14 →

Wednesday 31 March1 post

Hugging Face Story of the week

Understanding BigBird's Block Sparse Attention

BigBird's block sparse attention mechanism, its implementation, and advantages over BERT's full attention, including that a BigBird RoBERTa-like model is now available in 🤗Transformers.

31 Mar

← EarlierWeek 12 Later →Week 14