Our agent has 169 tools. In the issue-tracker example that surface includes creating issues, configuring workflows, assigning people, linking blockers, planning releases, reading attachments and exporting reports — plus less familiar operations such as executing Python in a sandbox. Measured across the real catalog, their JSON schemas alone total 84,407 characters — about 21,000 tokens before you count a single description. Declare all of that natively and you pay it on every turn, and you pay twice: once in tokens, once in quality, because a model choosing between 169 options mis-picks more than a model choosing between a dozen.

Part 1 compacted the domain model into an index with drill-down. Tools get the same treatment, with one extra twist at the end of the pipeline — a search lens that decides which characters of a tool’s description are worth showing, based on how rare your query words are.

A tool is in one of three states

Undiscovered — the other ~137 tools ≈0 tokens per tool <tool_capabilities> attachment, code, export, file, issue, label, … … project, search, user, view, workflow </tool_capabilities> one shared line — 34 namespace roots · ≈330 chars for the whole catalog search_tools({ query, capabilities, domains }) Discovered — a menu row, not a declaration capped excerpt · shared page budget { name: "read_json", description: "Read a node of a JSON document… bounded preview plus a reference…", capabilities: ["file:read"], domains: [] } returned in the tool result — declares nothing, cannot evict the working set activate_tool({ names: ["read_json"] }) Activated — full schema, natively declared median 351 chars · worst 3,182 read_json — description() + parameters JSON schema, callable this turn one of a bounded set of LRU slots — the least-recently-used tool is evicted when it fills; a direct call keeps a tool warm, so the set follows what the model actually uses
The three representations of a tool, measured across our 169-tool catalog. Search and activation are the explicit route; observed usage can also seed likely tools directly into the activated set when an agent starts.

The standing cost is the top band. Undiscovered tools — the overwhelming majority — appear in the prompt only as their capability namespaces. Every tool declares capabilities like issue:create or file:read, and the prompt carries just the deduplicated roots, one comma-separated line inside <tool_capabilities>: 34 words standing in for the entire catalog. Alongside it, a fixed scaffolding of ~12 tools (search, activation, ontology, skills, background jobs and memory) is always declared, and a short instruction tells the model the rule of the game: your declared tools are a bounded working set, not the full catalog — and never claim a capability is unavailable without searching for it first.

Cold start (no usage signal) capability line + scaffolding schemas ≈9.6k chars ≈ 2.4k tok Working set full + a full LRU working set of schemas ≈17k chars ≈ 4.2k tok Everything declared all 169 schemas, before descriptions ≈84k chars ≈ 21k tok
Standing tool surface per turn vs. declaring the full catalog. The typical case sits between the first two bars: scaffolding plus however many LRU slots are occupied.

Note what the middle state does not do: a search_tools result declares nothing. It’s a menu, rendered into the tool result, and the model must explicitly activate_tool the entries it wants. The code comments on why: decoupling find from load means a broad search can never flood or evict your working set. Browsing is free; holding costs a slot.

Finding: the ontology pays its first dividend

search_tools takes the exact coordinate system from part 1. You can constrain by capabilities (prefix-matched, so file covers file:read and file:write), by domains, entities or relationships — the same ids the ontology prompt taught the model — or by free-text query, and the axes intersect. A tool tagged with the relationship issue.assigned-to.user is findable from either endpoint entity, so “tools touching user” surfaces the assignment tool even though it lives in the issues domain.

The text layer is classic BM25 (via Orama, English stemming, snake_case split into words), over a four-field document with descending boosts: name ×3, description ×2, capabilities ×2, glossary ×1. That last field is the ontology again: each tool’s document embeds the glossary text of the concepts it’s tagged with. The label entity’s description mentions “colored tag” — so a model searching “colored tag” finds add_label_to_issue and define_label, whose own names and descriptions never contain those words. A curated synonym layer, no embeddings involved.

Ranking: rarity beats repetition

BM25 produces scores, but the harness doesn’t trust them as the primary ordering. Query tokens are deduplicated up front, then every candidate is re-ranked by three keys:

hits.sort((left, right) => {
    const coverage =
        right.matchedTerms.length - left.matchedTerms.length;
    if (coverage !== 0) return coverage;
    const leftWeight = left.matchedTerms.reduce(
        (total, term) => total + (weightByTerm.get(term) ?? 0),
        0
    );
    const rightWeight = right.matchedTerms.reduce(
        (total, term) => total + (weightByTerm.get(term) ?? 0),
        0
    );
    return rightWeight - leftWeight || right.score - left.score;
});

Distinct terms covered first, summed term weight second, raw BM25 last. The weight is a smoothed inverse document frequency:

w(t)=1+ln⁡N+1df(t)+1w(t) = 1 + \ln\frac{N + 1}{\mathit{df}(t) + 1}

with one crucial property: NN is not the corpus size. It’s the size of the already-filtered candidate set — rarity is computed within the scope of this search, per query. “File” might be a rare, informative word among issue tools and a worthless one among file tools. The smoothing keeps a single-document scope finite and gives ubiquitous terms a baseline weight of one instead of zero.

SCOPE — 3 CANDIDATE TOOLS AFTER FILTERS label_issue “Add a label to an issue” create_issue “Create an issue” search_issues “Search for an issue” query: “label issue” TERM WEIGHTS — RARITY WITHIN THE SCOPE label df 1/3 1 + ln(4/2) 1.69 issue df 3/3 1 + ln(4/4) 1.00 RANKING — COVERAGE, THEN Σ WEIGHT, THEN BM25 1. label_issue 2 distinct terms · Σw 2.69 2. create_issue 1 term · Σw 1.00 · BM25 breaks the tie 3. search_issues 1 term · Σw 1.00
An issue-tracker rendering of the harness's worked example. Coverage puts label_issue first regardless of BM25; among the one-term matches, weight can't separate them, so BM25 finally gets a vote. Repeating a query word buys nothing — tokens are deduplicated before scoring.

The excerpt lens: spending characters where information lives

Here’s my favorite mechanism in the whole harness. A search result page gets a fixed budget of description text, and each entry a capped excerpt. Most tool descriptions are longer than their cap. Which characters survive?

The naive answer — the leading ones — shows you boilerplate. The harness instead builds each excerpt from the query outward, and it spends its budget by the same IDF weights that ranked the results:

1 · excerptMatches — locate query-term hits in the 426-char description issue ×3 label 2 · clusterExcerptMatches — hits ≤36 chars apart share a window (span ≤120) 3 hits → 1 window 1 hit → 1 window 3 · allocateExcerptContext — leftover budget distributed ∝ term weight issue — w 1.00 → 105 chars kept label — w 1.69 → 160 chars kept 4 · render — windows joined with “ … ”, capped at 300 271 / 300 chars … Create an issue from a template and apply issue defaults issue-by-issue. Preserves assignee and priority, … validates the chosen id before applying the label and returns the updated issue without reloading the project. Retur… The rare term gets the wider window: 160 chars around one “label” vs 105 around three “issue” hits.
A 426-char issue-tool description, query “label issue”, run through the real allocation pipeline. Every number in this figure is computed; only the domain vocabulary is illustrative.

Matches near each other merge into one window (its span capped), a greedy set-cover picks the fewest windows explaining the most distinct terms, and then the leftover budget — after a small floor per window — is distributed proportional to each window’s term weight. In the figure, three occurrences of the common word “issue” earn 105 characters of context; a single “label” earns 160. The excerpt literally allocates explanation to the words most likely to be the reason you searched.

Two smaller touches complete it. If a hit came from a field other than the description — name, capability, glossary — the excerpt prepends it (Tool name: archive issue), so a name-only match doesn’t render an unrelated description opening. And the page budget is water-filled: every shown entry gets an equal share, and entries with short descriptions donate their slack to the long ones. A lone result gets nearly the whole page; a terse “Widget lookup.” hands its unused budget to its neighbours.

Prefetching: a learned cold start

Search remains the general escape hatch, but making the same discovery call in every familiar context is wasted work. When an agent starts, its scope already identifies the relevant ontology domains. The harness uses observed tool usage for those domains to predeclare a few likely tools before the model asks for them. An agent opened on an issue can therefore begin with common issue tools ready to call, while the rest of the catalog stays deferred.

The preference order moves from specific to general: this user’s habits in the current workspace first, then workspace-wide usage, then the global aggregate. Each broader level fills only what the narrower evidence did not. A new user can inherit useful workspace defaults; a new workspace still has a sensible global starting point.

SPAWN CONTEXT Agent opened on ISSUE-142 scope → domains: ["issues", "projects"] MOST SPECIFIC AVAILABLE USAGE FIRST User in this workspace personal, local habits Workspace shared local habits Global aggregate broad fallback broader levels fill only the gaps DECLARATION BUDGET — PRIORITY ORDER 1 · Current iteration what the model just touched 2 · Conversation history recent activations and calls 3 · Usage prefetch fills remaining slots only bounded native tool declarations
Scope chooses the domains; observed usage ranks likely tools within them. The most specific available habits win, while broader aggregates fill gaps. Current-turn and conversation activity always outrank these prefetched seeds.

This is deliberately a seed, not a new authority. Current-turn tool activity gets first claim on the declaration budget, conversation history comes next, and prefetched tools fill only unused slots. Usage history cannot evict a tool the current conversation has demonstrated it needs, and explicit search remains available whenever the learned guess is wrong.

Holding: a working set that follows the model around

Activation is bounded by a small, fixed working-set budget. What I find elegant is where the conversation’s contribution lives: in the transcript, not a mutable session of “loaded tools.” Each turn, the provider walks the transcript newest-to-oldest and reconstructs the most recently touched tools up to the budget: an activate_tool call adds its names, a direct tool call keeps a tool warm even if its activation has aged out, and everything past the budget falls off. Prefetched tools then fill any remaining slots. activate_tool.execute itself only validates names.

The live portion is still a pure projection of the conversation — replay the transcript, recover the same priorities — while the learned seed improves the cold start. The budget earns its keep twice: a small working set is cheaper than the catalog, and it is a cleaner decision surface than everything the model could ever call.

activated this turn held kept warm by a direct call − evicted (least recently used) t1 activate read_pdf read_pdf t2 activate query_json, read_json read_pdf query_json read_json t3 call read_pdf directly read_pdf query_json read_json t4 activate run_view,search_text, merge_pdfs read_pdf read_json run_view search_text merge_pdfs − query_json t5 activate generate_pdf read_pdf run_view search_text merge_pdfs generate_pdf − read_json 5 slots shown — the production budget is larger
The conversation side of the working set is re-derived each turn: a direct call (t3) refreshes read_pdf's recency and the least recently touched tool is evicted when the budget fills. Usage-based seeds can fill any slots left open by this history.

The quiet discipline: byte stability

Deferred loading changes the prompt from turn to turn, which threatens something valuable: automatic prefix caching. The harness’s countermeasures are small and everywhere. Declared tools are name-sorted so the declaration block is byte-identical whenever the set is unchanged. And the one line that must vary — “You currently have N of M activated tools loaded” — is deliberately appended last, after every static section, so the varying suffix invalidates nothing before it.

the system instruction, re-assembled every iteration system prompt 8 sections protocol capabilities 34 roots discovery tool schemas name-sorted byte-identical across turns → served from the provider’s prefix cache “You currently have N of M activated tools loaded.” the only varying text — deliberately appended last
Cache-aware prompt assembly: everything static first, byte-identical across turns; the single varying line last. No explicit cache_control anywhere — the harness simply makes the automatic prefix cache's job easy.

Descriptions as live data

One tool takes discovery a step further. run_python executes in a sandbox with preinstalled packages — and its description includes the live package inventory, refreshed from the sandbox every 60 seconds. Because description is a method, not a field, the text can change under the harness’s feet; when it does, the tool re-indexes itself in place. Search “openpyxl” and you find run_python, not because anyone hardcoded a package list into a description, but because the description is a projection of deployed reality. The model also stops burning a turn asking which packages exist — and stops hallucinating ones that don’t.

What it costs, honestly

Discovery is not free. The worst case adds two full model iterations — search, activate, then finally call — with the whole context re-read each time (cache-discounted, but not free), and genuinely novel work still depends on the model knowing when to search. The harness spends real effort compensating: scaffolding tools skip discovery, scope selects domain-specific usage seeds, and the anti-deferral instruction discourages a model with a bounded toolset from claiming things are impossible. Prefetching adds a small speculative cost when its guesses are irrelevant, which is why it is capped and subordinate to conversation activity. The budget is also a bet on locality; a task that needs more tools than the working set holds will thrash. Ours don’t, today. That’s an empirical fact about our product, not a law.

We’ve now kept the domain model and the tool catalog out of standing context. What remains is the largest object in the room: the data itself. A single query_json over a project export can return more characters than every schema we just avoided declaring — and worse, the model would have to re-type those characters to pass them to the next tool. Part 3 is about never letting that happen.