From the source
Summary by forck
Asana used GPT-6 Astra in Codex to optimize its StackAI browser agent on GPT-6.1 Sol, cutting estimated model costs 76x and run times 5x in a 144-run study.
- The agent had cached fixed instructions and tool definitions but resent its growing history of page text and screenshots at full price on every request, and it dropped older screenshots and trimmed text at nearly every step, so caching the history alone would not have helped.
- Asana's StackAI CTO Frank Hidalgo directed GPT-6 Astra to map the codebase, propose fixes and refactor the code so one frontend and backend could support many workflows in parallel, work he estimates would have taken one to two months by hand and took about a week.
- The study tested history budgets of 120,000 and 480,000 characters and six caching and screenshot policies, each three times on four models; the best policy let screenshots accumulate to 20 before cutting back to the most recent one.
- The optimized workflow on GPT-6.1 Sol averaged $0.47 in estimated model costs and about four minutes per run, against at least $36.21 and at least 22.5 minutes on the original production setup on Model B, and every optimized run completed the task with the correct answer.
- On GPT-6.1 Sol alone the new caching and screenshot policy cut cost 4x, from $1.97 to $0.47 per run, with each call about 3x cheaper because 89% of the input came from cache at 5% of the uncached price.
