From the source
Amazon published a guide to building a centralized observability data lake for Amazon Bedrock invocations, aimed at platform and ML engineers running Bedrock workloads across many accounts.
- The problem it addresses is that Bedrock's model invocation logging delivers a full record of requests and responses, but the data stays locked inside individual accounts, so teams see an aggregate bill without knowing what happened inside each call.
- The design pulls raw invocation logs from linked accounts into CloudWatch, forwards them through subscription filters to a central account, processes them with Amazon Data Firehose, and exposes them through Athena as one queryable table.
- Each flattened record carries the prompt, the model response, input and output token counts, the model used, the stop reason, the account and timestamp, and the IAM role that sent the prompt.
- A Lambda function unwraps CloudWatch's batched, compressed events, discards housekeeping messages, and lifts the nested caller identity to the top level.
- Firehose converts the records from JSON to Parquet and files each object under a date path, and an AWS Glue table lets Athena read the files with ordinary SQL.
- Everything deploys as infrastructure as code through two CloudFormation templates, with templates, Lambda code and SQL queries in the AWS Sample Bedrock Invocation Data Lake
GitHub repository.






