getDominantFile() backs context ranking's core-directory boost. Its answer
depends only on the graph, but the whole-graph aggregation behind it
(edges joined to both endpoints' nodes, grouped by file) ran on every
generic codegraph_explore - seconds per call on a large index.
Memoize it in QueryBuilder against a database change stamp:
total_changes() (rows this connection wrote) plus PRAGMA data_version
(commits by any other connection or process). Both are O(1), so no write
path has to remember to invalidate, and a sync made by another process is
seen on the next call. Any write forces a recompute; the heuristic's
result is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`codegraph index` recreates the database, then takes the write lock inside
indexAll. With the watcher now following a replaced file, a sync landing in
that gap reopened the empty file and started a full reconcile, holding the
lock the rebuild was about to request — `codegraph index` could then fail
with "another process may be indexing". Review flagged the gap; this closes
it. After following a replaced file, a sync that finds no index_state yet
(the rebuild has not started indexing) and a file written within the last
two minutes returns the lock-busy shape so the watcher retries; the first
sync after that reconciles the whole tree once, then syncs go back to scoped.
The time bound keeps a rebuild that died before indexing from parking the
watcher forever.
New test: fails without the guard, passes with it; 136 related tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`codegraph index` rebuilds through `CodeGraph.recreate()`, which unlinks
`.codegraph/codegraph.db` and creates a new file at the same path. A running
MCP daemon kept its handle on the unlinked inode and its watcher kept
"auto-syncing" into it, so no edit reached the on-disk index. The #925
self-heal only ran on the tool-call path and ran no catch-up after reopening.
- `sync()` now checks, once it holds the index mutex and the write lock,
whether the database was replaced on disk (one stat, the same inode check
`reopenIfReplaced` uses). If so it reopens the live file in place and widens
a scoped sync to a full reconcile. A failed reopen (rebuild mid-way) returns
the lock-busy shape so the watcher keeps its pending files and retries.
- `reopenIfReplaced()` refuses while an index/sync holds the mutex, so the
tool-call path never closes the handle an in-flight sync writes through.
- When a tool call's `freshen()` does reopen, the engine runs its existing
catch-up sync (serialized on the same mutex), but only for its own watched
instance, so an engine that is not the project's writer never writes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`readSourceOrStamp` took a `stats` argument no caller passed and returned
stats no caller read; it now returns the text or stamp change detection
hashes. The batch reader's two comments are back in the order its code runs:
the size gate, then the MPEG-TS head check. No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The size gate ran after `readFile`, so a committed video or blob fixture was
decoded in full — 3.4 GB of RSS for a 400 MB file, `Invalid string length`
past ~512 MB — only to be stored as skipped. It was then read again by every
pass that scans files by name: the framework detectors' `readFile` and the
resolver's cached reader, three full passes on the reporter's fixtures.
Stat first, everywhere a source file is read by path: the batch reader and
`extractFile` store an oversize file with a size stamp in place of its
content, change detection hashes the same stamp (so a same-size rewrite of a
file nothing is indexed from is not a change, while crossing the limit in
either direction is), and both by-name readers return null above the limit.
The limit moves to `src/file-limits.ts` so the three sites share one number.
strace on a sparse 400 MB `.ts`: 153 603 reads before, 0 after; indexing it
next to a real file costs no measurable RSS. Stacked on #1914, which shares
the batch-reader hunk.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review found two places that decide "is this a source file" without the
MPEG-TS check the scan applies: the git fast path's candidate loop in
getChangedFiles and the scoped-sync path filter (watcher events). An
untracked clip in a git repo was reported as `added`, skipped by sync
without a record, and reported as `added` again on every status; a tracked
.ts that became a clip stayed `modified` with its stale nodes. Both now
treat the clip as non-source: removed when tracked, ignored otherwise.
Regression tests fail without this change and pass with it, on both arms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`.ts` mapped to TypeScript by extension alone, so MPEG transport-stream
fixtures (`testdata/*.ts`) were parsed by tree-sitter: ~28 s of CPU for a
900 KB clip, minutes for a directory of them, for zero symbols.
Recognise the stream from the head of the file — the 0x47 sync byte at
offsets 0, 188, 376 and 564 plus a NUL byte, which every stream carries in
its first packets and UTF-8 source never does — and drop it at discovery:
not indexed, not parsed, not counted, not tallied as an unsupported
language. The check reads under 1 KB and only for `.ts` files; the batch
reader sniffs the bytes it already read, and the single-file path (sync,
watcher) does the same head read, so a clip handed in by name is skipped
too. The skip happens before parse dispatch, so no kernel mirror is needed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review found that a `.codegraph/codegraph.db` holding bytes that are not
SQLite at all counted as "schema-less", so `init` printed "rebuilding it"
and then failed with SQLite's "file is not a database". Only an empty file
or a SQLite database without the codegraph tables can be rebuilt in place;
`hasSchemalessDb` now means exactly that, and the new `hasForeignDbFile`
names the other case. `init` stops on it with exit 1, says which file it
is and to move or delete it, and never touches it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The schema probe failed CLOSED: a real WAL codegraph.db in a directory the
process cannot write (a read-only checkout, a mount, another user's tree)
made the read-only node:sqlite open throw — SQLite must create `-shm` to
open a WAL database — and isInitialized() reported the project as not
initialized, where main's file-exists check had been right.
The probe now fails OPEN. Once the size gate and the `SQLite format 3\0`
header have passed, the file is a SQLite database; only a SUCCESSFUL
sqlite_master query proving the `nodes` table absent may return false. Any
open/prepare error (locked, busy, cantopen, readonly, disk I/O) returns
true, the pre-existing behaviour for a database we cannot inspect. No error
strings are matched. The per path + mtime + size memo is unchanged, and
schema-less/empty files still return false, so `codegraph init` still
repairs them and never touches a real db.
Test: a WAL db copied into a `.codegraph/` made read-only with chmod stays
initialized — skipped on win32 and when running as root (root ignores mode
bits).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
isInitialized() accepted any existing .codegraph/codegraph.db, so an empty or
table-less file in an ancestor directory (an interrupted init, a stray touch,
a never-populated ~/.codegraph/) captured the upward walk of
findNearestCodeGraphRoot / resolveProjectPath / resolveServerRoot for every
project beneath it: the real index of a sub-project became unreachable, the
sub-project down-scan never ran, and commands either operated on the broken
file or failed opening it instead of showing the not-initialized guidance.
An initialized project is now one whose db carries the schema: the probe is
gated by file size (>= the 100-byte SQLite header), then the header magic,
then a read-only node:sqlite open with a single sqlite_master lookup for the
`nodes` table, memoized per path + mtime + size so hot callers (the prompt
hook, MCP root resolution on every call) never reopen an unchanged db. The
handle is closed immediately so the probe never holds the db on Windows; a
locked/busy db counts as live.
createDirectory() uses the same predicate, so `codegraph init` in the
directory with the broken file repairs it in place (the schema is applied
with CREATE ... IF NOT EXISTS) instead of refusing with "Already
initialized"; the CLI says what it found. Nothing is deleted automatically.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Fixes#1840. Reuses and extends #1855 with lexical/import ownership and cache invalidation controls.
Co-authored-by: Max Hsu <maxmilian@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes#1861. Reuses L4XB's receiver retention and owner resolution from #1866, with duplicate-owner and edit/sync guards.
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Preserve the indexed-commit approach and regression cases from #1848, with dirty-path retention, commit-race guards, NUL Git paths, and failed-I/O recovery.
Fixes#1829.
Co-authored-by: Eric Minish <eric.minish@gmail.com>
* fix(ui): a screen's steps read as clusters, and a link too far to follow is said in words
The mobile app's /capture came back as a web: 100 boxes in a 1,227x5,588
ribbon, 113 lines drawn at rest crossing each other 652 times, each one
running over about five other boxes' names. Measured, not guessed — three
separate causes, very unequal.
The region grouping had nothing to divide there (98 of 100 boxes take their
region from one memoized component), so the picture fell back to a single
719px column. But the region dimension was not the lever. The lever was that
`packRegions` packed every step of one distance onto shared rows and wrapped
those rows at a fixed 720px, so a box and the thing it fires landed seven
lines apart: 70 of the 113 lines joined boxes ONE step apart. That is what
the crossings were made of.
So a region is now packed as CLUSTERS — a step, then the steps it sets in
motion on the line under it, stepped in — while the starting points that fire
nothing still share a line, because a screen's handlers are siblings and
giving each its own line turned a flat region into a column. A region's line
width is earned rather than fixed (sqrt(total * pitch), clamped 720..2600),
so a big screen comes out about as wide as it is tall.
Clustering makes most links local but not all: a step reached from two places
is drawn under whichever reached it first, so the other way in still crosses
the picture. Those are now said in WORDS at both ends — `-> resumeInference`
under the box that leads there, `<- CaptureView` under the box it arrives at,
capped at three with `+N more` — rather than drawn. This is not a hiding: the
link is stated, which says more than a line vanishing off the edge of the
screen does, and selecting the box draws every one of its real lines exactly
as before. It is the one at-rest cut that does not produce the "box that leads
somewhere and draws nothing" every earlier cut produced.
Also fixed while here, and predicted by the earlier region work: the in-region
row relaxation had no cycle guard, so a region holding one loop pushed 65 of
its boxes to rows 294-301 while the rest sat at 0-2. `forwardLinks` sets
cycle-closing links aside first, as the order reading's `withoutBackEdges`
already did.
/capture: 1,227x5,588 -> 2,279x4,356, at-rest crossings 652 -> 1,
lines-over-boxes 553 -> 26, with half the links still drawn as real lines and
every quiet box still one the screen itself fires directly. The order reading
is untouched (it keeps every line).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NTUFNN5bH3aPw2gi2LqbYD
* fix(ui): a stub names its box without the mark the box wears for its kind
`← ⇠ onCaptureProgress +2` reads as two arrows arguing: the stub already
leads with a direction, and the box's own kind mark was competing with it.
Verified in the live canvas. The box keeps its mark, where nothing competes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NTUFNN5bH3aPw2gi2LqbYD
* fix(ui): the screen's line into each of its parts stops sweeping the picture
Audited all 51 screens of the mobile app on this branch. 96 lines — the
screen's own stand-in line into each region — were 17% of everything drawn
and caused 79% of every crossing left. A screen with ten regions tiles them
into bands, so the line into a region two bands down travelled the height of
the whole picture.
Two causes, both fixed. The entry the line lands on was the walk's first
member of the region; clustering moves a step that fires something BELOW the
ones that fire nothing, so that box could sit lines down inside the region
and the line had to reach past everything above it. It now lands on the box
nearest the region's top-left that the screen actually leads to. And the
stand-in line is no longer exempt from the stub rule — when the region is
still too far to follow, the link is said in words like any other. The rest
of the anchor's fan stays quiet as before: it is already stood in for.
Across the 51 screens: crossings 47 -> 10, no screen above 10 (worst was 19,
now 2); lines-over-boxes 236 -> 182; boxes with neither a line nor a word
150 -> 135.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NTUFNN5bH3aPw2gi2LqbYD
* fix(ui): a screen's parts fill the canvas instead of squaring off into rows
Audited the app's 28 regioned screens: the median canvas was 55% region and
45% nothing, and /home was 44% — 4,860px tall to hold about 2,160px of
picture. The cause is that regions were tiled a row at a time with each row
as tall as its tallest member, so one short region beside a tall one left the
rest of that row blank, and a reader scrolls through the blank.
Each region now goes as high as it can and then as far left as it can, over a
skyline of what is already placed. Reading order is untouched: regions are
still walked in the screen's own source order, so an earlier one is never
pushed below a later one — a short one just tucks under another short one
rather than waiting for the tall one beside it. Layering now comes from the
finished geometry rather than a band counter, since once regions drop
independently what a reader sees as one row IS one row.
/home 4,860px -> 3,584px, aspect 0.59 -> 0.94. Tallest screen in the app
4,860 -> 4,356. Crossings 10 -> 13 across all 51 screens, still none above 10.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NTUFNN5bH3aPw2gi2LqbYD
* fix(ui): the width a picture wraps at is tried, not estimated
A region's line width came from sqrt(total * pitch) — the width at which
total/width lines come out square. That estimate is wrong for how these
pictures are drawn: a cluster spends lines on its own structure (a hub gets a
line to itself, and what it fires starts another), so it undercounts a
region's lines badly and wrapped /capture's 98 boxes into a 4,356px column.
Laying a picture out is cheap and exact, so the widths are tried instead:
layoutAt runs the whole pack at each of eight widths and the best finished
canvas wins (~2ms for the model, all eight included). It has to be scored on
the CANVAS, not per region — squaring each region off individually leaves
fewer of them side by side, which took /home from 3,584px to 5,624px while
every region looked better on its own.
Also measured and rejected while here: dropping a region's CLUSTERS side by
side the way the regions drop onto the canvas. Total height 42,084 -> 39,756px
(-6%), but lines-over-boxes 120 -> 134 and crossings 5 -> 8, because two
clusters side by side put each one's lines through the other. Height is cheap
to scroll; a crossed line is what made this picture unreadable. The reasoning
is recorded in the code so it is not re-tried blindly. Regions differ — they
sit far enough apart that few lines run between them.
Across the app's 51 screens: tallest picture 4,356 -> 3,796px, total height
45,744 -> 42,084px, lines-over-boxes 184 -> 120, crossings 13 -> 5, and no
screen is a tall ribbon any more.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NTUFNN5bH3aPw2gi2LqbYD
* stuff
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reuse unsafeIndexRootReason before scanning indexed subprojects so stray manifests at home or broader roots cannot inject unrelated context. Preserve workspace adoption for #964.
Validation: four new regressions fail before the guard and pass after it; 48 relevant tests and npm run build pass. Confirmed the real os.homedir() leak before and after the fix with fixture cleanup.
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
* fix(db): graceful FTS5 fallback when Node.js build lacks FTS5 support (#1532)
Official Node.js binaries do not compile FTS5 by default, causing codegraph
init to fail with 'no such module: fts5'. Added runtime FTS5 detection:
- Split schema execution to try FTS5 separately, skip on failure with warning
- Added fts5Available flag to DatabaseConnection and QueryBuilder
- Bulk-load and search paths skip FTS5 operations when unavailable
- Search falls back to LIKE + fuzzy matching when FTS5 is missing
(cherry picked from commit ed708b7f60)
* fix(db): preserve core schema during FTS5 fallback (#1532)
Keep required tables and indexes after the FTS triggers outside the
optional schema block in the upstream #1625 fix. Without this boundary,
simulated-missing-FTS5 indexing still fails on name_segment_vocab.
Add seven regressions using real SQLite with FTS5 creation intercepted,
covering initialization/open, LIKE and fuzzy search, non-FTS schema parity,
bulk no-ops, and real FTS5 search and bulk-load recovery. Credit
@aniruddhaadak80 under Unreleased fixes.
Validation on Linux x64 with Node v22.19.0:
- npm run build passed, including viewer and grammar asset checks.
- 34 tests passed across fts5-fallback, node-sqlite-backend,
sqlite-backend, and db-perf.
- Rebuilt CodeGraph initialization, indexing, reopening, search, and
cross-file callers passed with simulated missing FTS5 and real FTS5.
Fixes#1532.
Supersedes #1625.
---------
Co-authored-by: Aniruddha Adak <aniruddhaadak80@users.noreply.github.com>
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
Adapt @uvmplus's PR #1481 (3ecf7479) to the shared symbol lookup and
named-symbol flow resolver on current main. Missing names return not found
with suggestions, and exact matches with no callers stay empty.
Preserve #1512 definition grouping and --file narrowing, #173 qualified
misses, and codegraph_node's intentional fuzzy file lookup. Port the
upstream regression suite and cover the moved shared lookup paths.
Validation on Linux with Node 22: project build, 74 requested tests,
47 related flow tests, and 16 same-fixture CLI/MCP checks pass. Baseline
captured 12 failing tests and 13 failing fixture checks.
Fixes#1473.
Supersedes #1481.
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
Adapt upstream PR #1485 by @valkyriweb (cc791cfc51)
onto current main. Keep the upstream alias-binding module and six-test suite
verbatim, preserve target-kind gating and default-export bindings, and add
one credited Unreleased changelog entry.
Linux verification (x86_64, Node 22.19.0): TypeScript build and asset copy
pass; the fresh ./impl.js repro changes callers/impact of realImpl from
missing consumerFn to including it. All 6 upstream tests and 231 related
resolver regression tests pass.
Fixes#1482
The Forge PR will supersede upstream PR #1485.
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
Land upstream PR #1687 by danusha2345 (fix commit 6e9bbb26), using the
PR tip implementation with only a Rust doc-comment placement cleanup.
Read parameter lists and return types positionally in the wasm extractor
and native kernel in lockstep, preserving verbatim signature text.
Verified on Linux x64 with Node 22.19.0: reproduced three undefined
signatures in both backends before the fix, then confirmed all three
expected signatures and exact wasm/kernel parity after rebuilding
TypeScript and the linux-x64 kernel. All 31 focused tests pass: 15 Kotlin
extraction, 6 Kotlin parity, and 10 kernel scaffold checks, with
CODEGRAPH_KERNEL_EXPECT=1 for the native suites.
Add the upstream #1495 changelog bullet while preserving all other
Unreleased entries. Keep EXTRACTION_VERSION unchanged for this bug fix.
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
A project CodeGraph has no grammar for was indistinguishable from an empty one: unsupported extensions are filtered out at discovery, so filesDiscovered was 0, the reconciliation in index.ts found no shortfall and recorded index_state as complete, and the CLI printed the same 'No files found to index' it prints for an empty repo.
That silence is what makes it costly over MCP: an empty result reads identically to 'no match', and the agent has been told to trust the graph rather than grep.
The scan already visits every file, so the tally of what it declined to index costs no extra I/O and no second pass. index_state itself is left alone: changing its values would change the status --json contract, which is a call for the maintainer to make.
(cherry picked from commit 2c20892789)
Co-authored-by: Max Hsu <maxmilian@gmail.com>
Co-authored-by: netbrah <netbrah@users.noreply.github.com>
Apply upstream commit 1572d90d71 from
PR #1570 by @rongbc.
Clarify that suggested explore call counts are advisory and extra calls
remain available. Sync the MCP initialization guidance and eval probe,
retain the upstream regression tests, close the footer's bold formatting,
and credit the fix under Unreleased in the changelog.
Validation: npm run build; focused Vitest (3 files, 53 tests); source and
dist wording greps; git diff --check.
Fixes#1504
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
Forge