# Amazon — Building a self-service LLM gateway with Amazon Bedrock and LiteLLM

- Company: Amazon (amazon.com)
- Announced: 2026-11-12T18:41:12+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/building-a-self-service-llm-gateway-with-amazon-bedrock-and-litellm/
- Record: https://forck.live/items/35472-building-a-self-service-llm-gateway-with-amazon-bedrock-and-litellm
- Subject: Bedrock / Nova
- Models affected: Claude Sonnet, Claude Haiku

Amazon's guide shows platform teams how to build a self-service LLM gateway that puts Amazon Bedrock behind an OpenAI-compatible API using LiteLLM on AWS Fargate. The pattern answers three pressures: platform teams need latency and error monitoring, finance teams need inference costs attributed to teams and projects, and security teams need consistent access controls. Developers get a single base URL and a per-user API key, so tools such as Claude Code, Continue and Cursor can swap models, add fallbacks or change providers without touching developer configurations. Each model, including Claude Sonnet and Claude Haiku, gets its own Amazon Bedrock Application Inference Profile for per-model quota isolation, while Amazon RDS (PostgreSQL) holds key metadata and spend state and Amazon ElastiCache for Valkey handles caching and rate limiting. LiteLLM's AI Gateway is open source and free for up to five users, but the self-service key issuance described requires the licensed Enterprise version, which adds single sign-on, audit logs and delegated admin roles. The post notes that Amazon Bedrock Guardrails are not covered and that responsible AI controls would normally be added before production.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/building-a-self-service-llm-gateway-with-amazon-bedrock-and-litellm/:

> Developers want AI coding assistants with minimal setup. With a large language model (LLM) gateway and LiteLLM, they can quickly access foundation models (FMs) available on Amazon Bedrock while you gain visibility and usage control. Platform teams need to monitor latency and errors, finance teams need to attribute inference costs to teams and projects, and security teams need consistent access controls. An LLM gateway can help address these needs by providing a centrally managed access point to the models that power coding assistants.
> 
> In this post, we show how to build a self-service LLM gateway using Amazon Bedrock Application Inference Profiles with LiteLLM running on AWS Fargate to abstract Amazon Bedrock behind an OpenAI-compatible API. With this pattern, developers can access foundation models through a single API with a dedicated key while you gain per-user cost controls and model performance observability. By the end, you will understand the pattern well enough to build your own version from the illustrative snippets and AWS Cloud Development Kit (AWS CDK) constructs. This post uses Claude Sonnet and Claude Haiku models for example purposes.

---

Record: https://forck.live/items/35472-building-a-self-service-llm-gateway-with-amazon-bedrock-and-litellm
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
