# Hugging Face — Introducing RWKV - An RNN with the advantages of a transformer

- Company: Hugging Face (huggingface.co)
- Announced: 2023-05-15
- Category: new-model
- Coverage: not counted
- Announcement: yes
- Group: models
- Source: https://huggingface.co/blog/rwkv
- Record: https://forck.live/items/2063-introducing-rwkv-an-rnn-with-the-advantages-of-a-transformer
- Subject: Platform
- Models affected: RWKV, RWKV-4
- Context window: ctx8192

Hugging Face announces the integration of the RWKV architecture into its transformers library. RWKV combines the strengths of RNNs and transformers, offering efficient long-context processing. The blog details the architecture and mentions RWKV-4 models with a context length of 8192 tokens.

## Evidence

Verbatim from https://huggingface.co/blog/rwkv:

> Through this blogpost, we will introduce the integration of a new architecture, RWKV, that combines the advantages of both RNNs and transformers, and that has been recently integrated into the Hugging Face transformers library.

---

Record: https://forck.live/items/2063-introducing-rwkv-an-rnn-with-the-advantages-of-a-transformer
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md