# Hugging Face — ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

- Company: Hugging Face (huggingface.co)
- Announced: 2026-10-20T15:19:48+00:00
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://huggingface.co/blog/ibm-research/altk-evolve-bc
- Record: https://forck.live/items/25265-altk-evolve-how-learned-memory-tops-the-bench-leaderboard-on-customer-service
- Subject: Platform
- Models affected: GPT-4.1, gpt-4.1, gpt-5.5

IBM Research's ALTK-Evolve learns a playbook of behavioral rules that lifts a basic GPT-4.1 customer-service agent to 67.5% Pass^4 on τ²-bench retail, above every retail entry in its October 12, 2026 leaderboard snapshot. The method analyzes the agent's attempts on training tasks and distills successes and failures into short rules, such as how to authenticate a customer or handle a return, which are appended to the system prompt. The same model writes the guidelines and runs the agent, so no weight updates or human annotation are needed. The team tested the approach on τ²-bench, a Sierra benchmark where an LLM-simulated customer, domain tools and a written policy constrain each conversation, and a task counts as solved only when the final database state matches the expected outcome. On held-out tasks, mean accuracy rose 4.4 points in retail and 7.5 points in airline, while the stricter Pass^4 rose 10 points in each domain. The team compared no guidelines against a high-confidence core and a lossless set, and reports that the gains come from better-informed calls rather than longer conversations.

## Evidence

Verbatim from https://huggingface.co/blog/ibm-research/altk-evolve-bc:

> Using GPT-4.1, an older, non-reasoning model, a basic customer-service agent reaches 67.5% Pass^4 on τ²-bench retail — meaning it solves the task on all four independent attempts for 67.5% of tasks. It scores above every retail entry in our October 12, 2026 leaderboard snapshot.

---

Record: https://forck.live/items/25265-altk-evolve-how-learned-memory-tops-the-bench-leaderboard-on-customer-service
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md