The Token Tax of Multi-Agent Systems

A single agent that re-reads its whole history every turn is wasteful. A fleet of them is the same mistake, multiplied— and most of the bill never had to exist.

When a multi-agent system gets expensive, the reflex is to reach for a smarter model or a bigger context window. But in a fleet, the tokens that dominate the bill aren’t spent on reasoning — they’re spent on repetition. Every agent re-sends its growing transcript. Orchestrators shuttle full histories between steps. The same facts sit duplicated in every agent’s window. And worst of all, agents re-derive what a sibling already figured out an hour ago, paying full price to rediscover it. A bigger window doesn’t remove that cost. It just raises the ceiling on a bill that shouldn’t be there.

The bill scales with the wrong things

It helps to be precise about what you’re paying for. In a naive multi-agent loop, input tokens scale roughly with agents × history length × redundancy — three multipliers, none of which is capability:

• 1Per-agent history, re-sent every turn. Each agent carries its own transcript so it “remembers,” and pays for the early turns again at every later one.
• 2Full context passed between agents. Hand-offs and orchestrator steps that forward the entire conversation rather than the relevant slice.
• 3The same knowledge, duplicated N ways. Shared conventions, specs, and decisions copied into every agent’s window instead of stored once.
• 4Re-derivation across agents. One agent solves a problem; a sibling hits the same wall later and solves it again from scratch — the multiplier unique to fleets.

Why a bigger window doesn’t help

Enlarging the context window treats the symptom. Cost still rises with every token you place in it; attention is roughly quadratic, so latency climbs too; and models reliably lose the thread in the middle of very long contexts, so you pay more to get noisier answers. None of that touches the redundancy problem — five agents with huge windows is just five times the duplicated spend. The real lever is architectural: stop keeping knowledge in the prompt at all.

What proper memory infrastructure does

“Memory infrastructure” here means a store that agents read from and write to, sitting outside the prompt — whatever you build it on. A few principles make it pay off, and they compound in a fleet:

• 1Externalize state. Knowledge lives in the store, not baked into the prompt. The prompt becomes a working set, not an archive.
• 2Retrieve, don't stuff. Each agent pulls back only the few memories its current task needs — ranked by hybrid retrieval (vector + keyword + graph) and tunable per agent (top_k, min_similarity, graph_max_hops) — so prompt size stops tracking history length.
• 3Share once, recall many. A substrate every agent reads means one agent’s discovery — written at scope_team or scope_org— is recalled by the rest instead of re-derived and re-sent. In a fleet, this is the multiplier that matters: redundant work collapses to a single write.
• 4Keep the store lean. A crystallizer merges near-duplicates into atomic facts, and an eight-status lifecycle retires stale ones (active → … → superseded) — so retrieval stays small and precise. You don’t pay to carry five copies or last week’s wrong answer.
• 5Scope what's returned. Visibility scopes (scope_agent / scope_team / scope_org) and per-agent trust tiers decide what each agent receives. Precision is also economy: a tighter, permissioned result is simply fewer tokens.

Principle 2 is where the obvious savings come from, and it’s worth seeing directly. Re-sending the transcript makes the per-turn prompt grow with the conversation; retrieving only what’s relevant keeps it flat, no matter how deep the history runs.

What it looks like on the bill

Take a long-running session whose full context would reach roughly 50,000 tokens. Re-sent every turn, that’s an input bill that climbs the whole way up Fig 1. Retrieve instead, and each turn pulls back only what’s pertinent — on the order of a thousand tokens — regardless of how deep the history goes.

That’s roughly a 97% cut for a single agent — and tokens are priced linearly, so it’s close to a 97% cut in input cost, with a faster prompt as a bonus. Now multiply by the fleet: the same reduction repeats for every agent, and share once, recall many removes the re-derivation across them entirely. The savings in a multi-agent system are larger than in a single one, for the same reason the waste was.

The loop, in MemClaw’s tools

Concretely, each agent runs a small recall–act–write loop against the shared store. With MemClaw these are MCP tool calls — an MCP-compatible client (Claude Code, Cursor, Windsurf) issues them, and the tenant is resolved from the API key:

# once per session: load mandatory policy — overrides conflicting instructions
memclaw_keystones()

# recall only what this task needs — hybrid vector + keyword + graph, scope-filtered
hits = memclaw_recall(query=task, top_k=8)   # include_brief=true adds an LLM brief

# ... the agent acts on `hits` + recent turns ...

# write the outcome back, stamped with a scope the fleet can recall
memclaw_write(content="<lesson learned>", visibility="scope_team")

# report the result against the recalled memories — the Karpathy Loop tunes retrieval
memclaw_evolve(outcome="success")

Recall returns only what the calling agent’s visibility scope and trust tierpermit, and every read is audited; writes are auto-enriched (type, summary, tags, entities, PII scan) on the way in. Wiring it in is one config block — paste it into any MCP client and the tools appear:

{
  "mcpServers": {
    "memclaw": {
      "url": "https://memclaw.net/mcp",
      "headers": { "X-API-Key": "mc_your_key" }
    }
  }
}

The interface is small; the discipline it enforces — externalize, retrieve, share, prune — is what does the work.

You don’t lose the signal

The reasonable objection: if agents stop sending everything, won’t they miss something? In practice what gets dropped is the irrelevant bulk, not the pertinent facts — retrieval keeps answer quality roughly intact while collapsing token count. (In our own benchmarking we measured token reductions in the 96–98% range versus full context with accuracy holding; methodology here, and your mileage will depend on workload and how you tune retrieval.)

Name the costs honestly: recall adds a small call — an embedding plus a query — and a top_k set too low can miss context while one set too high gives the savings back. memclaw_tune lets each agent learn its own profile (top_k, min_similarity, graph_max_hops) from memclaw_evolvefeedback. A handful of tasks genuinely want a whole document in view; for those, put it in the prompt. For everything else, which is most of it, retrieval wins clearly on cost, latency, and headroom — and in a multi-agent system it wins by the size of the fleet.

The takeaway is almost a slogan: in a multi-agent system, the cheapest token is the one you don’t re-send — and the second cheapest is the one only one agent ever had to learn. That’s an infrastructure decision, not a context-window setting.