The bill isn't for thinking

We started keeping notes on what surprises us while building this way — a team of software agents, running long tasks, on the record. This is the first, and it is about cost, because we got this one backwards at first.

We assumed the expensive part of running a team of agents would be the thinking: the hard reasoning, the calls to the most capable models, the moments that actually feel like work. So that is where we looked to save. When we finally measured where the cost was going, the thinking was not it. The large majority of what we were paying for was not generating anything new — it was re-reading context we had already loaded, over and over, one step at a time.

A long-running agent carries its working memory with it and re-reads the whole thing on every step. The further a task runs, the more of each step goes to re-loading what came before, and the smaller the slice that is genuinely new. Past a certain length, you are mostly paying to remember, not to think.

Once we saw that, the levers that actually moved the bill were not the ones we expected — and none of them were “use a smarter model.” They were housekeeping:

• Cut what is always loaded. Every agent carried a standing rulebook it re-read every turn. Trimming it to only what is load-bearing removed a fixed cost from every step — not a one-time saving, a per-step one.
• Trim the tool menu. The list of tools an agent can reach is re-read every turn too. Narrowing it to what the task actually needs did the same thing, for the same reason.
• Pass pointers, not copies. When one agent hands off to another, we pass a pointer to where the truth currently lives instead of copying the context across. A pointer avoids duplicating the context, and it does not go stale when the work moves on.

None of these are clever, and that is the point. The thing that changed the bill was not the model tier we picked; it was how much context each step was dragging along behind it. We had been tuning the expensive-looking knob and ignoring the one that was actually turning.

The honest limitation: this is what our work looks like — long tasks, many small steps, a lot of shared context. A different shape of work would spend its money somewhere else, and the proportions would move. One team’s biggest lever is another team’s rounding error.

We are still finding the floor here; there are levers we suspect we have not found yet. If you run agents over long tasks, we would genuinely like to know what actually moved your bill — the surprising ones are the ones worth trading.