From the source
Amazon's guide shows platform teams how to build a self-service LLM gateway that puts Amazon Bedrock behind an
OpenAI-compatible API using LiteLLM on AWS Fargate.
- The pattern answers three pressures: platform teams need latency and error monitoring, finance teams need inference costs attributed to teams and projects, and security teams need consistent access controls.
- Developers get a single base URL and a per-user API key, so tools such as Claude Code, Continue and
Cursor can swap models, add fallbacks or change providers without touching developer configurations.
- Each model, including Claude Sonnet and Claude Haiku, gets its own Amazon Bedrock Application Inference Profile for per-model quota isolation, while Amazon RDS (PostgreSQL) holds key metadata and spend state and Amazon ElastiCache for Valkey handles caching and rate limiting.
- LiteLLM's AI Gateway is open source and free for up to five users, but the self-service key issuance described requires the licensed Enterprise version, which adds single sign-on, audit logs and delegated admin roles.
- The post notes that Amazon Bedrock Guardrails are not covered and that responsible AI controls would normally be added before production.






