From the source
Together AI and Ollama have written a joint guide on what the harness and the inference engine each must get right so long agent sessions stay fast.
- Coding agents resend the whole conversation every turn, and prefix caching spares the engine from redoing that work, but only if the harness leaves the start of the prompt alone and the engine sends each turn back to the GPU that already holds it cached.
- The two companies work on opposite sides of that handoff: Ollama decides what goes into each request, Together AI decides what happens when the request arrives.
- The post walks through the prefill and decode phases, noting that an agent turn is mostly prefill because the input is the whole session and the output is usually a short tool call.
- It gives an idealized arithmetic example: ten turns without caching process 55 units of prompt against 10 with caching, and a hundred turns process 5,050 against 100.
- A prefix cache matches from the beginning of the prompt up to the first differing token and stops there, so changing anything earlier means re-processing everything after it.
- Ollama says it shipped the ability to connect harnesses such as Claude Code, Codex and Pi to open models like
DeepSeek-V4.1-Flash and GLM-5.3 through the ollama launch command.






