From the source
Amazon's guide introduces the Agent Development Lifecycle (Agent DLC), a lifecycle with two loops and one explicit standard for judging every change to an AI agent before release.
- The post argues that conventional unit tests and CI/CD are necessary but not sufficient for agents, because agents are non-deterministic, because prompt, tool, retrieval or memory changes alter behavior in ways text review cannot reveal, and because model endpoints, knowledge bases and external APIs can change independently of the repository.
- It presents a measured reference implementation, an internal HR policy assistant answering questions over a document knowledge base, and states the results are not a customer outcome or a benchmark result.
- In the first evaluation round, chunk-level source precision was 1.00 for all three golden questions, while fact-level precision was 0.59, 0.53 and 0.09, so a release gate using chunk-level precision alone would have accepted all three answers.
- After a prompt-only change that added three constraints, the first question improved substantially and the third got worse, which the evaluation traced to a knowledge-coverage problem: the corpus lacked a dedicated sick-leave policy document.
- The accompanying AWS Samples repository includes infrastructure deployment, the HR agent, custom evaluators, and scripts for baseline evaluation and prompt-change re-evaluation, and the post notes scores can vary across runs and repository versions.






