plans/agent-manager-git-poller-optimization.md
Agent Manager periodically computes exact diff and ahead/behind statistics for the local checkout and every visible managed worktree. The timer itself is not the problem. The expensive part is that every poll reconstructs the same state through several independent Git processes and filesystem passes, even when a worktree has not changed.
For each worktree, the current hot path does the following:
git merge-base HEAD <base>git diff --name-status --no-renames <merge-base>git diff --numstat --no-renames <merge-base>git ls-files --others --exclude-standardgit rev-list --left-right --count <base>...HEADlstat, binary detection, and line counting for every untracked fileThe poll also runs git worktree list --porcelain once for presence and branch
information. GitStatsPoller suppresses an unchanged webview message only after
all of the work above has finished, so the existing result hash does not reduce
Git CPU, process creation, or disk reads.
The cost scales linearly with the number of worktrees and expanded projects. The shared semaphore limits concurrent child processes, but it does not reduce the total work. On endpoint-protected machines, every extra Git process and file scan also creates security-agent work.
snapshot(true) refresh behavior through
the command-consolidation and cache phases. Permit bounded timer-based
sharding only if measurement proves the mandatory status scans remain above
the CPU/disk or endpoint-security goals.remoteRef(wt). Do not fetch from remotes.core.fsmonitor, untracked cache, split index, or maintenance as a
side effect of opening Agent Manager.Semaphore and abort
controller.kilo serve because the local path intentionally avoids Bun child-process
memory growth on Windows.Make each periodic poll proportional to the amount of changed state rather than repeating every exact diff calculation for every worktree.
For an unchanged worktree, the steady-state poll should perform one read-only Git status probe and no base-relative diff or history walk. When a worktree does change, it should compute the exact UI statistics with fewer Git processes and fewer repeated index/worktree traversals than today.
Use a two-level timer-driven poll:
The cache is an optimization, not a source of truth. The first poll, a forced snapshot, a failed probe, a changed base, or an uncertain fingerprint always falls back to exact calculation.
The status, merge-base numstat, ref snapshot, and rev-list commands used here
are available in current Git and have stable machine-readable output. Every
optimization also has a failure fallback.
This design does not make polling free. Without filesystem watchers or Git fsmonitor, there is no repository-level hash that reveals arbitrary unstaged or untracked file changes. Exact periodic detection must scan each worktree. Git cannot status several independent worktree/index pairs in one invocation, so one status process and one working-tree scan per active worktree per interval is the practical lower bound under these constraints.
The expected gain comes from removing duplicate scans, exact line-count diffs, and history walks after that mandatory status scan. Process-count savings will therefore be larger than CPU and disk savings. CrowdStrike CPU normalization is a measured acceptance gate, not an assumed consequence of reducing process launches.
Add a polling-specific helper that runs:
git --no-optional-locks status \
--porcelain=v2 --branch -z \
--no-ahead-behind --untracked-files=all --no-renames
Parse its stable machine format into:
HEAD OID and branch,The status payload alone is not a safe cache key. Editing a file that is
already reported as modified can leave the porcelain text unchanged. Complete
the fingerprint in Node with bigint lstat metadata for every changed
non-deleted path: size, nanosecond mtime/ctime when supported, inode, mode, and
file type. Include these values in a deterministic hash together with:
HEAD OID,This does not hash file contents. It detects normal editor writes, atomic renames, staging, commits, branch switches, untracked-file changes, and local tracking-ref changes while avoiding a full exact diff on an unchanged tree.
If any path cannot be statted, contains unsupported status data, or changes during probing, mark the fingerprint uncertain and run the exact path. Never reuse cached stats on uncertainty.
The fingerprint cache is in memory and scoped to one GitStatsPoller. Store per
worktree:
type CachedStats = {
base: string
fingerprint: string
stats: WorktreeStats
}
Do not retain every WorktreeDiffEntry in the poller cache. Large worktrees can
have thousands of changed files, while the poller only needs aggregate stats.
Keep review and file-detail metadata outside this cache. Bound aggregate cache
entries by active worktree IDs and clear them on stop() so project switches
and disposal cannot leak stale state.
When a fingerprint changes, reuse the probe's untracked paths instead of
running git ls-files --others --exclude-standard again.
Replace the separate merge-base process with Git's built-in merge-base diff form:
git -c core.quotepath=false diff \
--merge-base --numstat -z --no-renames <base>
The poller only renders aggregate file/addition/deletion counts, so it does not
need tracked-file status records or WorktreeDiffEntry objects. Numstat alone
provides the tracked file count, exact text counts, and binary markers currently
obtained from merge-base, diff --name-status, and diff --numstat. -z
makes paths safe for tabs, newlines, Unicode, and unusual filenames.
The polling helper should therefore compute a changed worktree with:
Do not change diffFile's on-demand detail path in this work. It is not periodic
and has different materialization requirements.
Run one project-level ref snapshot per poll before processing individual worktrees:
git for-each-ref \
--format=%(refname)%00%(objectname)%00%(upstream)%00%(symref)%00 \
refs/heads refs/remotes
Use it to map every local branch and remote base ref to an OID, each local
branch to its configured upstream, and each remote HEAD to its symbolic
default branch. Combine this with git worktree list --porcelain -z, whose
records already contain each worktree's HEAD OID and branch. Extend the
existing worktree parser and listWorktreePaths result instead of launching
rev-parse, symbolic-ref, or config queries in each worktree.
For the local checkout, preserve the current tracking-resolution fallback when the ref snapshot is insufficient (for example, unusual configuration with a branch remote but no upstream merge ref). Cache that resolution as today. Do not trade a small process reduction for a change in which base branch is used.
The project snapshot supplies three things:
HEAD OID for each worktree,Cache ahead/behind by <headOID>\0<baseOID>. If both OIDs are unchanged, reuse
the previous counts even when only working-tree files changed. This removes
history walks from normal editing polls.
The implementation uses the ref snapshot to cache base OIDs and reuses
ahead/behind counts when the worktree HEAD and base OID are unchanged. A
changed worktree uses the existing rev-list --left-right --count <base>...HEAD
path. Batched ahead/behind and merge-base fast paths were tested but removed
because they did not improve the dominant steady-state workload enough to
justify their compatibility and maintenance cost.
snapshot(true) bypasses fingerprint reuse and recomputes exact results for
skipped and unskipped worktrees, matching current behavior.HEAD OID change invalidates both caches.syncSkips() must not discard cached data. Collapsing and re-expanding a
section should reuse the last exact result after a confirming fingerprint.Create packages/kilo-vscode/src/agent-manager/git-stats-snapshot.ts as a
VS Code-free module. Keep parsing, hashing, cache decisions, and aggregation out
of GitStatsPoller.ts so the existing file does not grow into another mixed
responsibility controller.
The module should own:
-z parsing,-z parsing,Use explicit parse-result errors rather than silently treating malformed output as a clean worktree.
GitOpsUse the existing GitOps buffered execution methods through the narrow
GitStatsSnapshot source boundary. Do not expose the generic private raw()
method or bypass the existing semaphore. Use execGitBuffer() for NUL-delimited
output.
Retain the current untracked-file safeguards in the aggregate implementation:
content reads capped at 1 MB and content-based binary detection. Do not call
git status without --no-ahead-behind; otherwise every supposedly cheap probe
can perform the history walk this design is trying to cache.
GitStatsPollerReplace the independent localDiff + aheadBehind calls with a project poll
coordinator:
HEADs, and ref OIDs once,lastStats and emit only when the aggregate
UI hash changed.Replace the injected localDiff callback with a narrow exact aggregate callback
or snapshot service. Do not construct full WorktreeDiffEntry[] merely to reduce
them to three numbers.
The local checkout should use the same snapshot/cache path instead of separately
resolving branch, tracking branch, diff, and ahead/behind every tick. Preserve
the existing no-base fallback to workingTreeStats.
Keep the current trailing-delay scheduler. Do not change polling intervals in this optimization.
Leave local-diff.ts, createLocalDiff, diffSummary, and diffFile behavior
unchanged in this optimization. Review and file-detail requests need per-file
status, stamps, merge-base identity, and materialization data that the poller
does not need. Sharing those objects would retain potentially thousands of
entries per worktree and broaden the regression surface for an infrequent path.
The measured implementation combines the useful parts of the original proposal:
HEAD and base OIDs,snapshot(true) refreshes, andThe clean dormant sharding was added only after direct workload and extension-host profiles showed that status probes remained the dominant steady-state cost. It is timer-based polling, not filesystem event invalidation. Product approval is still required for the bounded 30-second freshness tradeoff.
Batched ahead/behind, merge-base shortcuts, persistent Git workers, and shipped benchmark or profiling hooks were explicitly discarded after measurement.
Add focused tests for the new snapshot module using the real temporary-repository
fixtures and helpers in tests/unit/local-diff.test.ts:
For every fixture, compare the optimized aggregate output to the sum of the
current diffSummary entries and to the current aheadBehind result before
switching the poller to the new implementation.
Extend tests/unit/git-stats-poller.test.ts with command recording and assert:
second unchanged poll runs a status probe but no exact diff or rev-list,
editing an already-modified file changes the metadata fingerprint and updates line counts,
staging without changing file contents invalidates the fingerprint,
commit and branch changes invalidate diff and ahead/behind,
remote-tracking ref changes invalidate base-dependent values,
working-tree-only edits reuse ahead/behind,
snapshot(true) bypasses cache reuse,
skipped worktrees remain cached but are not emitted,
missing worktrees evict cached state,
malformed/failed probes fall back to exact computation,
exact failures retain last-known stats,
stop() clears caches and stale generations cannot publish results.
hot worktrees are polled on every visible tick,
clean dormant worktrees rotate without starvation and are sampled within 30 seconds,
a dirty result promotes a worktree to hot immediately,
two consecutive clean polls return it to the dormant queue,
forced refreshes bypass the shard budget,
deleted sessions cannot leave stale busy IDs indefinitely, and
every Git operation remains behind the shared semaphore.
Use disposable fixtures and isolated VS Code instances for before/after measurements. Do not add benchmark scripts, Git hooks, or profiling observers to the product diff.
The final matched extension-host profile used 40 rendered worktrees for 30 seconds. It recorded 1,994 baseline GitOps commands versus 466 optimized commands and 42.16 seconds versus 13.62 seconds of cumulative Git command time. The direct workload profile also showed a 78.5% reduction in combined Git CPU.
Acceptance criteria now are:
CrowdStrike CPU remains an external managed-endpoint measurement. It must be recorded directly before claiming an endpoint-security reduction; Git command or process reductions are not a substitute for that measurement.
From packages/kilo-vscode/:
bun run test:unit -- --grep "GitStatsPoller|diffSummary|GitOps|parseWorktreeList"bun run typecheckbun run lintbun run knipbun run check-kilocode-changeManually verify that Agent Manager stats update within one visible poll after editing, staging, committing, switching branches, and updating a local tracking ref. Verify that clean dormant worktrees rotate within 30 seconds and that forced refreshes bypass the shard budget.
--no-renames, -z, and --no-optional-locks.merge-base and rev-list commands;
do not require an upgrade merely to show stats./usr/bin/git launcher to avoid double execution.Implementation retained only the optimizations that reduced the real workload:
Removed after measurement:
Measurements on the same repository on 2026-08-05:
The final comparison is a local process-level measurement, not a direct CrowdStrike process measurement because this session has no sudo access. Falcon CPU must still be checked on the managed endpoint after deployment; no security policy exception is justified by this implementation result alone.