From the source
Amazon is adding usage-based metering to SageMaker AI bidirectional streaming so AWS Marketplace model providers can bill for what customers actually consume during a live session.
- Deepgram, the real-time speech AI company, is the launch partner, bringing usage-based Speech-to-Text and Text-to-Speech billing to AWS Marketplace.
- The change answers a billing gap: request-response metering reports a unit count in an HTTP response header, but a streaming session can last 30 seconds or 30 minutes and its final total is not known until it ends.
- Model servers opt in by returning the header X-Amzn-SageMaker-Metadata-Stream-Supported: true, after which SageMaker AI opens a companion WebSocket at /invocations-bidirectional-stream-metadata for JSON metering records.
- The metadata stream allows one message per second per connection with a maximum message size of 512 bytes, and Amazon recommends sending records incrementally as usage accrues so less goes unreported if a session ends unexpectedly.
- SageMaker AI endpoint infrastructure charges still apply while the endpoint is provisioned, and Amazon notes that transport delivery alone does not establish that usage has been billed.






