A user asks the agent for a release report: filter a project’s issues to the open blockers, group them by assignee, and produce a PDF. The issue export is 2.1 MB of JSON. In a naive harness that value enters the model’s context twice: once when the query tool returns it (~500k tokens of input — if it fits at all), and again when the model passes the rows to the PDF tool, because the only way to hand a value to the next tool is to write it out token by token. That second copy is the expensive kind — output tokens — and the model is now a lossy, non-deterministic wire: it will truncate, “fix,” or hallucinate rows in transit.

Part 1 kept the domain model out of standing context; part 2 kept the tool catalog out. Both were fixed-size wins. This one is unbounded: bulk data never has to enter context in full. The model gets a bounded working view plus references, and the harness resolves the exact values server-side at the moment a tool actually needs them.

The vocabulary

A reference is not a URI or an opaque handle. It’s a small typed JSON object the model can read, and — in two of four cases — write:

{ "$ref": { "type": "Message", "seq": 42, "part": 2, "field": "result.value" } } the envelope — replaces any top-level property, or the whole input the address space: Message · Blob · (derived, server-only) which message in the transcript — omit for the current turn which part of it dot-path within Message seq · part · field Blob key · or source JsonPointer source · path JsonQuery source · path · filter the model may author these server-only recipes — the model forwards, never writes
The reference the model sees, and the four-member union behind it. Message and Blob references address things the model has legitimately seen named; JsonPointer and JsonQuery are derivation recipes that only the server may construct.

Any top-level property of any tool call can be replaced by { "$ref": … }, or the entire input can be one. A Message reference addresses a part of the transcript itself — message 42, part 2, field result.value — which means the transcript doubles as an addressable store: anything a tool ever returned is a coordinate, not a copy. A Blob reference addresses stored bytes by key or through another reference. The two derived forms — JsonPointer (one node of a JSON document) and JsonQuery (an array filtered by field equality or substring) — are recipes: the model can forward one it received, but the input parser rejects any it tries to author. That asymmetry closes an injection surface — the model cannot fabricate a pointer into data it was never shown — and it keeps recipes an implementation detail the harness is free to change.

How values leave context

Two complementary mechanisms enforce the same invariant: context gets a working view; durable state keeps the exact value.

The first is type-based offloading. Any Blob in a tool result is intercepted on the way out. Its bytes stream to object storage, chunk by chunk, and the persisted result keeps a typed marker. When a provider projects that marker for the model, the storage key disappears and the reference sits directly on the visible object:

{
  "opaque": true,
  "size": 2214731,
  "format": { "mediaType": "application/json" },
  "$ref": {
    "type": "Blob",
    "source": {
      "type": "Message",
      "part": 2,
      "field": "result.value"
    }
  }
}

Metadata and an address: never bytes, never even the storage key. File size no longer determines context size.

The second mechanism is size-based rendering, applied to every successful tool result after reference projection. Results that fit pass through unchanged. Oversized results become structure-aware views: objects keep their keys and shallow values before deep detail; arrays keep a prefix of complete items; strings keep a verbatim head. Each omission becomes a {"$truncated": …} placeholder that says what is missing and points to the exact stored subtree.

ONE RESULT, THREE REPRESENTATIONS Persisted message part result: { total: 148, rows: [all exact rows] } Bounded renderer replace opaque values with refs then allocate the view stored value untouched Model context total: 148 rows: [complete prefix, { "$truncated": { items, omitted, $ref } } THE BUDGET PRESERVES STRUCTURE, NOT JUST CHARACTERS STRING verbatim head preview · omitted · $ref ARRAY whole-item prefix items omitted · whole-array $ref OBJECT keys + shallow values first deep subtree → its own $ref BATCH ENVELOPE small results take only what they need; the remainder is fairly redistributed Even a crowded parallel batch leaves every result with a recoverable structural skeleton. $ref resolves the exact stored subtree
One cross-cutting boundary for every successful tool result. The model gets a faithful structural view and an address; the persisted result is never modified, so passing or reading the reference recovers the exact omitted value.

There are budgets for one result and for all results arriving together. The batch budget is fair-shared: small results keep only what they need, and the surplus flows to larger ones. That matters when the model parallelizes calls — one response cannot flood the next turn simply by returning several individually reasonable payloads at once.

This boundary is a safety net, not a replacement for good tool design. A generic prefix is less useful than a page chosen with domain knowledge, so read_json and query_json still return a deliberate preview + reference pair. Their derived references — JsonPointer for one node, JsonQuery for a filtered array — stay hidden behind a Message reference. The model sees enough to decide, while the exact value remains available if a later tool resolves it.

The flow, end to end

MODEL CONTEXT — EVERY CHARACTER HERE COSTS TOKENS, EVERY TURN query_json({ source, filter }) the call — a few dozen chars preview: 10 of 148 items value: { $ref } ≤6,000 chars + a ~60-char ref generate_pdf({ rows: { $ref } }) forwards the ref, not the rows ~60 chars the context boundary — whatever crosses it, costs read + filter, server-side bounded preview + derived $ref resolver re-runs the recipe issues.json — 2.1 MB in the object store agent/<agentId>/<key> filtered where status = "open" → 148 rows; the recipe {path, filter} is kept, not the rows ≈500k tokens if inlined — never crosses resolve → execute full 148-row array handed to generate_pdf; result pdf stored as a new blob → { opaque, size, $ref } bytes stay below HARNESS + OBJECT STORE — BYTES ARE FREE HERE Round trip: 2.1 MB brokered by the model for the cost of one preview and two ~60-char references.
The release-report scenario. The 2.1 MB issue export is read, filtered and re-read entirely below the context boundary; the model orchestrates it with one bounded preview and two short references. The exact rows never enter the model's view.

The step that still surprises me is the third arrow: when generate_pdf receives rows: {$ref}, the resolver walks Message → result field → recovers the stored JsonQuery recipe → re-reads the blob → re-applies the filter — and hands the tool the full 148-row array. The recipe is lazy: it was never evaluated when query_json returned, only now, at the moment of use. The model brokered a value it never held. Resolution is recursive with two independent caps — one at validation, a tighter one at resolution — so Blob(source: JsonQuery(source: Message)) chains are legal but bounded.

The same route now covers code execution. Exact stdout streams into a Blob while context gets only a compact head-and-tail preview; files enter and leave the sandbox by reference. A parser can consume an attachment and hand its output to another tool without either file passing through the model.

Cheaper is also more correct

The token arithmetic gets the headlines, but the correctness argument is what makes references non-negotiable. When values travel tool-to-tool through the model, the model is part of the data path: every row is re-generated token by token by a sampler. It abbreviates long fields, “fixes” values that look inconsistent, rounds numbers, drops a row and keeps going — none of which is a bug in the model. That is simply what generative transport is.

By reference, the model is only the control path. It routes addresses; the data path is deterministic server code. The 148 rows that reach generate_pdf are byte-for-byte the filter’s output, not the model’s recollection of it. And the failure modes improve along with the fidelity:

  • A bounded result never passes a plausible-looking partial value as the whole. Visible content is verbatim; every absence is marked, counted and paired with an address for the exact omitted subtree.
  • A stale pointer throws — “Derived JSON path … no longer exists.” — loudly and immediately, instead of silently shipping plausible-looking rows.
  • The model cannot author a derivation recipe, so it cannot point a tool at data it was never shown; a fabricated reference fails validation instead of fabricating data.
  • The preview and the reference travel together, describing the same exact value — there is no gap between what the model inspected and what the next tool receives.

Token savings you can buy with a bigger context window. This property you can’t: the only way the model corrupts a value in transit is if the value transits the model. The bounded view also protects attention: large results stop competing with the instructions and evidence the current decision actually needs, without making their full values unreachable.

The plumbing that makes it declarable

For the model to pass a reference anywhere, every tool’s schema must say so. The harness rewrites each tool’s input schema so that every top-level property accepts anyOf: [original, $ref]. Two details echo part 2’s discipline:

  • The alternative is attached per property, never at the schema root — the Anthropic API rejects oneOf/anyOf at the top level of input_schema, so the envelope design bends around a provider constraint.
  • The rewritten schema is cached in a WeakMap keyed by the original parameters object — so the declaration bytes are stable across turns, and the prefix cache keeps hitting.

Tools that want laziness inside their implementation opt out of eager resolution: read_json’s ref parameter receives the reference object itself rather than resolved bytes, so the tool can stream from the store on its own terms. Everything else gets its $ref arguments resolved before execute runs — tool authors mostly never know references exist.

The envelope is the taught, canonical form, but the boundary is deliberately forgiving where the intended value is unambiguous. A reference accidentally encoded as a JSON string is parsed back before validation. If the model drops the {$ref: …} wrapper around a reference in an ordinary value field, the harness considers repairing it only after validation fails, only at the fields that failed, and accepts the repair only if the resolved call then passes the original schema. Valid literal inputs are never reinterpreted.

That closes the loop this part opened: the model routes data by address, the harness moves bytes the model never sees, and the token bill and the error rate drop together. But the standing context still has rows we haven’t examined — and they’re different in kind. The ontology, the catalog and the data are all knowledge we shipped. The agent also picks things up along the way: the user’s preferences, a project’s conventions, corrections it shouldn’t need twice. That accumulating knowledge needs the index-and-payload treatment more than anything else — it grows without bound by design — and it’s the subject of part 4. What users deliberately teach the agent is a different mechanism again, and it closes the series in part 5.