# Replit — Productizing Large Language Models

- Company: Replit (replit.com)
- Announced: 2022-09-21
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://replit.com/blog/llms
- Record: https://forck.live/items/18103-productizing-large-language-models
- Subject: Replit Agent
- Models affected: GhostWriter

Replit describes its experience deploying transformer-based language models ranging from ~100 million to over 100 billion parameters, including its GhostWriter code autocomplete product. The post covers techniques for reducing nonsense output, maintaining quality via benchmarks and A/B testing, and improving inference speed through FasterTransformer, knowledge distillation, and quantization.

## Evidence

Verbatim from https://replit.com/blog/llms:

> At Replit we have deployed transformer-based language models of all sizes: ~100m parameter models for search and spam, 1-10B models for a code autocomplete product we call GhostWriter, and 100B+ models for features that require a higher reasoning ability.

---

Record: https://forck.live/items/18103-productizing-large-language-models
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md