Find, don’t load.

Our agent has 169 tools. Creating work items, configuring workflows, running dashboard queries, reading PDFs, executing Python in a sandbox — the whole surface of the product, wrapped for an LLM. Measured across the catalog, their JSON schemas alone total 84,407 characters — about 21,000 tokens before you count a single description. Declare all of that natively and you pay it on every turn, and you pay twice: once in tokens, once in quality, because a model choosing between 169 options mis-picks more than a model choosing between a dozen.

Part 1 compacted the domain model into an index with drill-down. Tools get the same treatment, with one extra twist at the end of the pipeline — a search lens that decides which characters of a tool’s description are worth showing, based on how rare your query words are.

A tool is in one of three states

Undiscovered — the other ~137 tools ≈0 tokens per tool <tool_capabilities> asset, code, concept, dashboard, data-point, document, … … view, work-item, work-item-definition, work-type </tool_capabilities> one shared line — 34 namespace roots · ≈330 chars for the whole catalog search_tools({ query, capabilities, domains }) Discovered — a menu row, not a declaration capped excerpt · shared page budget { name: "read_json", description: "Read a node of a JSON document… bounded preview plus a reference…", capabilities: ["file:read"], domains: [] } returned in the tool result — declares nothing, cannot evict the working set activate_tool({ names: ["read_json"] }) Activated — full schema, natively declared median 351 chars · worst 3,182 read_json — description() + parameters JSON schema, callable this turn one of a bounded set of LRU slots — the least-recently-used tool is evicted when it fills; a direct call keeps a tool warm, so the set follows what the model actually uses
The three representations of a tool, measured across our 169-tool catalog. Between states sit two always-declared scaffolding tools: search_tools and activate_tool.

The standing cost is the top band. Undiscovered tools — the overwhelming majority — appear in the prompt only as their capability namespaces. Every tool declares capabilities like work-item:create or file:read, and the prompt carries just the deduplicated roots, one comma-separated line inside <tool_capabilities>: 34 words standing in for the entire catalog. Alongside it, a fixed scaffolding of ~12 tools (search, activate, ontology, skills, tasks, memory) is always declared, and a short instruction tells the model the rule of the game: your declared tools are a bounded working set, not the full catalog — and never claim a capability is unavailable without searching for it first.

Turn 1 (nothing active) capability line + scaffolding schemas ≈9.6k chars ≈ 2.4k tok Working set full + a full LRU working set of schemas ≈17k chars ≈ 4.2k tok Everything declared all 169 schemas, before descriptions ≈84k chars ≈ 21k tok
Standing tool surface per turn vs. declaring the full catalog. The typical case sits between the first two bars: scaffolding plus however many LRU slots are occupied.

Note what the middle state does not do: a search_tools result declares nothing. It’s a menu, rendered into the tool result, and the model must explicitly activate_tool the entries it wants. The code comments on why: decoupling find from load means a broad search can never flood or evict your working set. Browsing is free; holding costs a slot.

Finding: the ontology pays its first dividend

search_tools takes the exact coordinate system from part 1. You can constrain by capabilities (prefix-matched, so file covers file:read and file:write), by domains, entities or relationships — the same ids the ontology prompt taught the model — or by free-text query, and the axes intersect. A tool tagged with the relationship work-item.assigned-to.member is findable from either endpoint entity, so “tools touching member” surfaces the assignment tool even though it lives in the work-items domain.

The text layer is classic BM25 (via Orama, English stemming, snake_case split into words), over a four-field document with descending boosts: name ×3, description ×2, capabilities ×2, glossary ×1. That last field is the ontology again: each tool’s document embeds the glossary text of the concepts it’s tagged with. The label entity’s description mentions “colored tag” — so a model searching “colored tag” finds add_label_to_work_item and define_label, whose own names and descriptions never contain those words. A curated synonym layer, no embeddings involved.

Ranking: rarity beats repetition

BM25 produces scores, but the harness doesn’t trust them as the primary ordering. Query tokens are deduplicated up front, then every candidate is re-ranked by three keys:

hits.sort((left, right) => {
    const coverage =
        right.matchedTerms.length - left.matchedTerms.length;
    if (coverage !== 0) return coverage;
    const leftWeight = left.matchedTerms.reduce(
        (total, term) => total + (weightByTerm.get(term) ?? 0),
        0
    );
    const rightWeight = right.matchedTerms.reduce(
        (total, term) => total + (weightByTerm.get(term) ?? 0),
        0
    );
    return rightWeight - leftWeight || right.score - left.score;
});

Distinct terms covered first, summed term weight second, raw BM25 last. The weight is a smoothed inverse document frequency:

w(t)=1+ln⁡N+1df(t)+1w(t) = 1 + \ln\frac{N + 1}{\mathit{df}(t) + 1}

with one crucial property: NN is not the corpus size. It’s the size of the already-filtered candidate set — rarity is computed within the scope of this search, per query. “File” might be a rare, informative word among work-item tools and a worthless one among file tools. The smoothing keeps a single-document scope finite and gives ubiquitous terms a baseline weight of one instead of zero.

SCOPE — 3 CANDIDATE TOOLS AFTER FILTERS excel_reader “Read an Excel file” text_reader “Read a text file” pdf_reader “Read a PDF file” query: “Excel file” TERM WEIGHTS — RARITY WITHIN THE SCOPE excel df 1/3 1 + ln(4/2) 1.69 file df 3/3 1 + ln(4/4) 1.00 RANKING — COVERAGE, THEN Σ WEIGHT, THEN BM25 1. excel_reader 2 distinct terms · Σw 2.69 2. text_reader 1 term · Σw 1.00 · BM25 breaks the tie 3. pdf_reader 1 term · Σw 1.00
The worked example from the harness's own tests. Coverage puts excel_reader first regardless of BM25; among the one-term matches, weight can't separate them, so BM25 finally gets a vote. Repeating a query word buys nothing — tokens are deduplicated before scoring.

The excerpt lens: spending characters where information lives

Here’s my favorite mechanism in the whole harness. A search result page gets a fixed budget of description text, and each entry a capped excerpt. Most tool descriptions are longer than their cap. Which characters survive?

The naive answer — the leading ones — shows you boilerplate. The harness instead builds each excerpt from the query outward, and it spends its budget by the same IDF weights that ranked the results:

1 · excerptMatches — locate query-term hits in the 426-char description file ×3 excel 2 · clusterExcerptMatches — hits ≤36 chars apart share a window (span ≤120) 3 hits → 1 window 1 hit → 1 window 3 · allocateExcerptContext — leftover budget distributed ∝ term weight file — w 1.00 → 105 chars kept excel — w 1.69 → 160 chars kept 4 · render — windows joined with “ … ”, capped at 300 271 / 300 chars … Read a spreadsheet file and turn each sheet file-by-file into structured records. Handles column headers, … s are collected instead of aborting. Requires the openpyxl engine for native Excel workbooks; legacy binary formats are converted first. Retur… The rare term ends up with the wider window: 160 chars of context around one “Excel” vs 105 around three “file” hits.
A 426-char description, query “excel file”, run through the real pipeline and its constants. Every number in this figure is computed, not illustrative.

Matches near each other merge into one window (its span capped), a greedy set-cover picks the fewest windows explaining the most distinct terms, and then the leftover budget — after a small floor per window — is distributed proportional to each window’s term weight. In the figure, three occurrences of the common word “file” earn 105 characters of context; a single “Excel” earns 160. The excerpt literally allocates explanation to the words most likely to be the reason you searched.

Two smaller touches complete it. If a hit came from a field other than the description — name, capability, glossary — the excerpt prepends it (Tool name: archive work item), so a name-only match doesn’t render an unrelated description opening. And the page budget is water-filled: every shown entry gets an equal share, and entries with short descriptions donate their slack to the long ones. A lone result gets nearly the whole page; a terse “Widget lookup.” hands its unused budget to its neighbours.

Holding: a working set that follows the model around

Activation is bounded by a small, fixed working-set budget. What I find elegant is where that state lives — nowhere. There is no server-side session of “loaded tools.” Each turn, the provider walks the transcript newest-to-oldest and reconstructs the most recently touched tools up to the budget: an activate_tool call adds its names, a direct tool call keeps a tool warm even if its activation has aged out, and everything past the budget falls off. activate_tool.execute itself only validates names. The declared set is a pure projection of the conversation — replay the transcript, get the same working set, no migration or cleanup code anywhere. And the budget earns its keep twice: a small working set is cheaper than the catalog, and it’s a cleaner decision surface — the model chooses among tools its own recent behavior says are relevant, instead of scanning everything it could ever call.

activated this turn held kept warm by a direct call − evicted (least recently used) t1 activate read_pdf read_pdf t2 activate query_json, read_json read_pdf query_json read_json t3 call read_pdf directly read_pdf query_json read_json t4 activate run_view,search_text, merge_pdfs read_pdf read_json run_view search_text merge_pdfs − query_json t5 activate generate_pdf read_pdf run_view search_text merge_pdfs generate_pdf − read_json 5 slots shown — the production budget is larger
The working set as a transcript projection. Nothing is stored: each turn re-derives the set, so a direct call (t3) refreshes read_pdf's recency and the least recently touched tool is evicted when the budget fills.

The quiet discipline: byte stability

Deferred loading changes the prompt from turn to turn, which threatens something valuable: automatic prefix caching. The harness’s countermeasures are small and everywhere. Declared tools are name-sorted so the declaration block is byte-identical whenever the set is unchanged. And the one line that must vary — “You currently have N of M activated tools loaded” — is deliberately appended last, after every static section, so the varying suffix invalidates nothing before it.

the system instruction, re-assembled every iteration system prompt 8 sections protocol capabilities 34 roots discovery tool schemas name-sorted byte-identical across turns → served from the provider’s prefix cache “You currently have N of M activated tools loaded.” the only varying text — deliberately appended last
Cache-aware prompt assembly: everything static first, byte-identical across turns; the single varying line last. No explicit cache_control anywhere — the harness simply makes the automatic prefix cache's job easy.

Descriptions as live data

One tool takes discovery a step further. run_python executes in a sandbox with preinstalled packages — and its description includes the live package inventory, refreshed from the sandbox every 60 seconds. Because description is a method, not a field, the text can change under the harness’s feet; when it does, the tool re-indexes itself in place. Search “openpyxl” and you find run_python, not because anyone hardcoded a package list into a description, but because the description is a projection of deployed reality. The model also stops burning a turn asking which packages exist — and stops hallucinating ones that don’t.

What it costs, honestly

Discovery is not free. The worst case adds two full model iterations — search, activate, then finally call — with the whole context re-read each time (cache-discounted, but not free), and a cold model can’t know what it doesn’t know to search for. The harness spends real effort compensating: scaffolding tools skip discovery entirely, scope references pre-expand the relevant domains, and the anti-deferral instruction exists precisely because a model with a bounded toolset will otherwise claim things are impossible. The budget is also a bet that conversations have locality; a task that genuinely needs more tools than the working set holds will thrash. Ours don’t, today. That’s an empirical fact about our product, not a law.

We’ve now kept the domain model and the tool catalog out of standing context. What remains is the largest object in the room: the data itself. A single query_json over a project export can return more characters than every schema we just avoided declaring — and worse, the model would have to re-type those characters to pass them to the next tool. Part 3 is about never letting that happen.