Terminating a worker thread while it is still loading its modules can crash
the whole process on Windows (exit 3221225477 / 0xC0000005, no error, no
dump). QueryPool.destroy() terminated every worker at once, and the daemon
destroys its pool whenever it stops - a session that ends within a few
seconds of starting had its eager worker in exactly that state.
MCPEngine.stop() didn't even wait for the pool before the daemon called
process.exit().
- QueryPool tracks each worker's start (settled on 'ready', error or exit);
destroy() answers outstanding calls first, then terminates each worker once
it has started, capped at 15 s - the parse pool's #2301 mechanism.
- MCPEngine.stop() awaits the pool.
- #2301's explanation is corrected: on the Windows VM, terminating workers
mid grammar compile never crashed (0 in 2,161), but terminating them while
they load CodeGraph's extraction modules did. The parse pool's wait already
covers that window, so only its comment and changelog wording change.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Workers whose project open failed were enrolled as idle and returned errors indefinitely.
Route failed readiness through the existing idempotent retirement and replacement path.
Ignore retired-worker messages and duplicate readiness handshakes while preserving the crash budget and startup limits.
Cover mixed pools, repeated failures, lifecycle duplicates, and real SQLite recovery.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(mcp): keep an explicit projectPath project in sync (#1835)
A project opened through a tool call's `projectPath` (a repository other
than the server's default — e.g. an indexed child of an un-indexed
workspace) was opened read-only: no catch-up sync on open and no file
watcher, so its answers went stale until someone ran `codegraph sync`.
The engine now owns the lifecycle of every project the ToolHandler opens
for an explicit path, mirroring the default project:
- the project's writer lock is acquired (#1740 single-writer rule); if
another live process holds it, that process keeps syncing and we only
read;
- a catch-up `sync()` runs on open and the first call against that
project awaits it, time-boxed like the default gate (#905);
- a file watcher runs for as long as the project stays cached.
Bounds: the cache is keyed by the canonical (realpath) root, so a
symlinked spelling shares one connection and one watcher; it is LRU with
`MAX_CACHED_PROJECTS = 8` — opening a ninth closes the least recently
used (watcher stopped, writer lock released, DB closed); `stop()` closes
them all. Un-indexed paths still get the success-shaped guidance.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): keep explicit projects synchronized (#1835)
Explicit projectPath connections bypassed catch-up and synchronization ownership.
Share canonical project leases and catch-up gates, retain daemon sessions, and retry ownership after a writer exits.
Advertise direct-writer readiness and drain active work before eviction or shutdown.
Add exact SQLite regressions for concurrent sessions, aliases, no-watch behavior, and teardown.
Co-authored-by: danusha2345 <danusha2345@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(mcp): retry catch-up quietly on sync lock contention
Handle LockUnavailableError without marking catch-up complete or logging an alarming failure, and verify timer-driven recovery against a real indexing lock.
Make the eviction regression deterministic by applying its short timeout only to the held catch-up. Other projects must finish reconciling before the test can assert that the oldest eligible graph was closed.
---------
Co-authored-by: danusha2345 <danusha2345@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): enable query workers in direct mode (#1465)
Direct and proxy-fallback engines never enabled the query pool, serializing concurrent reads despite CODEGRAPH_QUERY_POOL_SIZE.
Enable bounded workers in both paths and support sessions without a default index, including indexes discovered later.
Preserve main-thread catch-up, staleness notices, and session accounting, with regression coverage for concurrency, parity, configuration, and teardown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(mcp): enable query workers in direct mode (#1465)
Direct and proxy-fallback engines never enabled the query pool, serializing concurrent reads despite CODEGRAPH_QUERY_POOL_SIZE.
Enable bounded workers in both paths and support sessions without a default index, including indexes discovered later.
Preserve main-thread catch-up, staleness notices, and session accounting, with regression coverage for concurrency, parity, configuration, and teardown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(mcp): enable query workers in direct mode (#1465)
Direct and proxy-fallback engines never enabled the query pool, serializing concurrent reads despite CODEGRAPH_QUERY_POOL_SIZE.
Enable bounded workers in both paths and support sessions without a default index, including indexes discovered later.
Preserve main-thread catch-up, staleness notices, and session accounting, with regression coverage for concurrency, parity, configuration, and teardown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Large-codebase indexing died at the end of "Resolving refs" two ways:
watchdog kills of healthy work (24k-file Java on Windows, #1212 — third
iteration of the #1091/#1122 class) and hard OOMs (Linux kernel scale,
where v1.3.0 could not complete at any watchdog setting). Root causes:
~31 of 37 dynamic-edge synthesis passes ran start-to-finish with no
yield points, several materialized whole-graph snapshots (kotlin
expect/actual opened with getAllNodes() — 2M nodes in one array; the
C fn-pointer pass retained every C file's contents twice plus every
function node), and the post-index WAL checkpoint ran minutes of
synchronous IO on the main thread, killing even a successful index at
the finish line.
The pipeline tail now follows the same discipline as the rest: never
hold O(graph) in the heap, yield everywhere.
- All synthesis passes stream node-kind scans (cursors, not arrays) and
yield on time-budgeted checkpoints; language gates skip passes whose
filters a project's file languages provably can't satisfy.
- kotlin expect/actual filters SQL-side; c-fnptr caches are LRU-bounded,
units stream one file at a time, and the all-functions array +
write-only id map are gone; spring reads each .java once, not twice.
- runMaintenance moved to a worker thread (own SQLite connection);
per-file store commits chunk with yields behind a serialized flush
chain (preserving #1015 file-order determinism); resolver warm-up
streams the DISTINCT name set; resolution batch-tail and merged-edge
inserts run in bounded sub-transactions.
- Daemon: fixed a socket-handoff race that could leave a fresh MCP
session permanently silent (client-hello tail unshifted into a
flowing stream with zero listeners — the long-standing #662 test
flake was this real bug); first tool call no longer queues behind
the query pool's cold start (pool.ready gate).
Validation: Linux kernel (70,129 files, 2.05M nodes, 6.4M edges) fully
indexes in 27m8s on a 2-core/6GB container at default heap + default
watchdog; llvm-project (180k files) completes under 1GB RSS including
kill-and-sync recovery; synthesized-edge and full-graph parity are
byte-identical vs baseline on elasticsearch/redis/vim; the ex-flaky
daemon test passed 25/25 under load. Env-gated diagnostics kept:
CODEGRAPH_SYNTH_TIMINGS pass/phase timings, CODEGRAPH_MCP_DEBUG hop
tracing. Design record: docs/design/main-thread-stall-followup.md.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The shared daemon served every session on one event loop with synchronous
node:sqlite. codegraph_explore is CPU-bound work stitched together by microtask
awaits, so N concurrent explores keep the microtask queue continuously full and
starve the macrotask phases — timers AND socket I/O. The transport freezes: no
response can flush until the whole batch drains, so with ~10 subagents on a large
repo clients routinely time out (reported via X by @symbolic2020).
Move the heavy read-tool dispatch onto a worker-thread pool. Each worker holds
its own WAL read connection (verified: a worker reader sees the main writer's
committed catch-up/watcher writes); the single watcher/writer, the catch-up gate,
codegraph_status, and the staleness/worktree notices stay on the main thread.
Concurrent reads now run in true parallel up to core count and the main loop
stays free for the MCP transport, so responses flush incrementally instead of
all-at-once after the batch drains. Enabled for the shared daemon only; direct
(single-stdio-client) mode is unchanged.
- crash recovery: respawn + retry-once, with a circuit breaker that falls back
to in-process dispatch if workers can't run on this platform
- graceful backstop: an overloaded pool returns success-shaped "busy, retry"
guidance, never isError (so it can't teach the agent to abandon codegraph)
- pending-aware growth + capped concurrent cold-starts avoid a startup
thundering herd (N simultaneous module-loads + DB opens could stall the loop)
- config: CODEGRAPH_QUERY_POOL_SIZE (default clamp(cores-1, 1, 16); 0 disables
→ in-process), CODEGRAPH_QUERY_BUSY_TIMEOUT_MS (default 45s)
10 concurrent explores on vscode (10.5k files): 31s → ~9s, staggered flush,
0 timeouts, byte-identical output; scales with cores (≈3.3× on 8, 1.8× on 2).
Full suite passes plus 10 new query-pool tests (fake-worker injection so the
scheduling logic is covered without spawning threads).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Colby Mchenry