# Hugging Face — StarCoder: A State-of-the-Art LLM for Code

- Company: Hugging Face (huggingface.co)
- Announced: 2023-05-04
- Category: new-model
- Coverage: not counted
- Announcement: yes
- Group: models
- Source: https://huggingface.co/blog/starcoder
- Record: https://forck.live/items/2067-starcoder-a-state-of-the-art-llm-for-code
- Subject: Platform
- Models affected: StarCoder, StarCoderBase, StarEncoder, StarPii
- License: BigCode OpenRAIL-M v1
- Context window: over 8,000 tokens

Hugging Face and ServiceNow's BigCode collaboration introduces StarCoder and StarCoderBase, 15B parameter open Code LLMs trained on permissively licensed GitHub data with an 8,000+ token context window. StarCoder is a Python-fine-tuned version that outperforms existing open models and matches closed models like code-cushman-001. Released under the BigCode OpenRAIL-M v1 license with safety measures.

## Evidence

Verbatim from https://huggingface.co/blog/starcoder:

> StarCoder and StarCoderBase are Large Language Models for Code (Code LLMs) trained on permissively licensed data from GitHub, including from 80+ programming languages, Git commits, GitHub issues, and Jupyter notebooks. Similar to LLaMA, we trained a ~15B parameter model for 1 trillion tokens. We fine-tuned StarCoderBase model for 35B Python tokens, resulting in a new model that we call StarCoder.

---

Record: https://forck.live/items/2067-starcoder-a-state-of-the-art-llm-for-code
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md