# Together AI — Making open models work for long-horizon agentic workloads

- Company: Together AI (together.ai)
- Announced: 2026-11-16
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://www.together.ai/blog/making-open-models-work-for-long-horizon-agentic-workloads
- Record: https://forck.live/items/36909-making-open-models-work-for-long-horizon-agentic-workloads
- Subject: Inference platform
- Models affected: DeepSeek-V4.1-Flash, GLM-5.3

Together AI and Ollama have written a joint guide on what the harness and the inference engine each must get right so long agent sessions stay fast. Coding agents resend the whole conversation every turn, and prefix caching spares the engine from redoing that work, but only if the harness leaves the start of the prompt alone and the engine sends each turn back to the GPU that already holds it cached. The two companies work on opposite sides of that handoff: Ollama decides what goes into each request, Together AI decides what happens when the request arrives. The post walks through the prefill and decode phases, noting that an agent turn is mostly prefill because the input is the whole session and the output is usually a short tool call. It gives an idealized arithmetic example: ten turns without caching process 55 units of prompt against 10 with caching, and a hundred turns process 5,050 against 100. A prefix cache matches from the beginning of the prompt up to the first differing token and stops there, so changing anything earlier means re-processing everything after it. Ollama says it shipped the ability to connect harnesses such as Claude Code, Codex and Pi to open models like DeepSeek-V4.1-Flash and GLM-5.3 through the ollama launch command.

## Evidence

Verbatim from https://www.together.ai/blog/making-open-models-work-for-long-horizon-agentic-workloads:

> What the harness and the inference engine each have to get right so a long agent session stays fast for long horizon tasks.
> 
> Authors
> 
> Zain Hasan, Parth Sareen (Ollama)
> 
> Table of contents
> 
> 40+ Models Chosen for Production...40+ Models Chosen for Production...40+ Models Chosen for Production...
> 
> Summary
> 
> Coding agents send the whole conversation again on every turn. Prefix caching keeps the inference engine from redoing that work, but only if the harness leaves the start of the prompt alone and the engine sends each turn back to the GPU that already has it cached. Together AI and Ollama work on opposite sides of that handoff, so we wrote down what each, the client side and the inference engine side, needs to do to successfully build agentic apps.
> 
> Coding agents have an odd traffic pattern. Every time the agent takes a step, the harness sends the model the entire conversation so far: the system prompt, every tool definition, your request, every file the agent has read, and every command it has run. What comes back is usually a tool call a few dozen tokens long. Then it happens again, with everything from last time plus a little more.

---

Record: https://forck.live/items/36909-making-open-models-work-for-long-horizon-agentic-workloads
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
