# Hugging Face — Introducing the Hugging Face LLM Inference Container for Amazon SageMaker

- Company: Hugging Face (huggingface.co)
- Announced: 2023-05-31
- Category: developer-tool-release
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://huggingface.co/blog/sagemaker-huggingface-llm
- Record: https://forck.live/items/2051-introducing-the-hugging-face-llm-inference-container-for-amazon-sagemaker
- Subject: Platform
- Models affected: BLOOM, Pythia Open Assistant Model, StarCoder, GPT-NeoX, Llama, T5, BLOOMZ, MT0-XXL, Galactica, SantaCoder, GPT-Neox 20B, FLAN-T5-XXL, Falcon 7B, Falcon 40B

Hugging Face releases a new LLM Inference Container for Amazon SageMaker, a purpose-built inference container to deploy open-source LLMs in a managed environment, powered by Text Generation Inference (TGI).

## Evidence

Verbatim from https://huggingface.co/blog/sagemaker-huggingface-llm:

> Hugging Face LLM DLC is a new purpose-built Inference Container to easily deploy LLMs in a secure and managed environment.

---

Record: https://forck.live/items/2051-introducing-the-hugging-face-llm-inference-container-for-amazon-sagemaker
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md