Cutting our inference bill by 68% by being obsessive about optimization

Summary by forck

Cloudflare cut the average cost of its AI code review from $2.43 to $0.77, a 68% reduction, after spending rose from $1.19 in April to about $2.46 by late August.

• The company says most of the increase came from model changes rather than architecture: moving every reviewer onto Anthropic models in early August raised the cost per job from $1.38 to $2.03, and its internal standards reviewer, Codex, went from about $0.25 to $0.88 per review on Claude Opus 4.7.
• About 40% of the savings came from code changes to the reviewer, with the rest from experimenting with which models it uses and how much effort they put in.
• Cloudflare found that output tokens were only about 23% of spend, while cache writes cost 43% and cache reads 34%, because each session sent the whole conversation as context.
• The reviewer now runs on OpenCode v2, a ground-up rewrite that agreed with v1 on the approval decision in 98% of tested merge requests and finishes full-tier reviews about 13% faster.