docs/design/toolsearch-preload-threshold.md
Deferred tools (shouldDefer=true) are unconditionally hidden behind
ToolSearch: every MCP tool (hardcoded in DiscoveredMCPTool) plus a set of
bundled built-ins (web_search, web_fetch, cron, monitor, worktree, …).
Deferral saves prompt tokens when the deferred set is large, but it is not
free: every mid-session reveal rewrites the function-declaration list, which
sits at the front of the tools→system→messages prefix, so a single ToolSearch
load invalidates the entire prompt KV cache. For a small deferred set the
deferral saves little and the cache damage plus the extra ToolSearch
round-trip make it a net loss.
Claude Code models this tradeoff with ENABLE_TOOL_SEARCH=auto / auto:N:
"tools load upfront if they fit within 10% of the context window, deferred
otherwise" (code.claude.com/docs/en/agent-sdk/tool-search). This change adds
the equivalent gate.
New setting tools.toolSearch.threshold (number, percent, default 10).
At session start (GeminiClient.startChat, before the deferred-tools reminder
is resolved), when ToolSearch is registered and the threshold is > 0:
JSON.stringify(tool.schema).length / CHARS_PER_TOKEN).threshold% of the context window
(contentGeneratorConfig.contextWindowSize, falling back to
tokenLimit(model)), reveal them all via the existing
revealDeferredTool mechanism. All-or-nothing — a partial reveal would
leave an arbitrary subset behind ToolSearch, and any tool left deferred
can still bust the cache on first use.threshold: 0
restores the old behavior unconditionally.Preloaded tools therefore land in the initial declaration list, are filtered out of the startup deferred-tools reminder, and the declaration list stays stable for the whole session.
setTools(). Revealing a tool the startup
reminder already announced would make queueAddedMcpToolsReminder flag it
as "removed", and a mid-session declaration change busts the very cache the
preload exists to protect. Tools from servers that connect later stay
deferred (announced via the added-tools reminder, reachable through
ToolSearch) until the next session start. /clear clears the revealed set
and re-runs the decision.tool_reference block — "The prefix is
untouched, so prompt caching is preserved"
(platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool).
Here every reveal — bundled or MCP — goes through setTools() and rewrites
the declaration list. Excluding the ~14 bundled deferred tools (web_search,
web_fetch, …) would leave the prefix one common tool-load away from a full
cache bust, forfeiting exactly the stability the preload buys. When the
union exceeds the budget everything stays deferred, which matches the
pre-threshold baseline for bundled tools.auto opt-in — affordable there because a deferred tool's first use costs
no cache invalidation. Here it costs a full prefix rebuild, so the
auto-style gate is on by default; threshold: 0 reproduces Claude Code's
always-defer default.startChat) cannot ratchet the
revealed set past the budget as servers come and go.resolveDeferredToolsForReminder already exposes everything.