Commit Graph

128 Commits

Author SHA1 Message Date
Forge 384daf329f feat: bound hosted publication windows and add HTTPS acceptance [skip ci] 2026-09-22 21:15:03 +00:00
Forge 7575dcc71e Bulk rebuild candidate indexes and retain concurrent config edits 2026-09-22 15:00:18 +00:00
Forge 018b840a3e Replace candidate text search atomically in bulk 2026-09-22 14:44:46 +00:00
Forge 5c27e4ffd2 Reuse pure core parses across atomic semantic updates 2026-09-22 14:34:14 +00:00
Forge 8936a3dc43 fix: reconcile global extension links atomically after source changes 2026-09-22 13:37:42 +00:00
Forge 2bc4029bf4 test: record Drupal profile and convergence methodology 2026-09-22 13:20:29 +00:00
Forge e72c064546 test: expand Drupal corpus proofs and native package regression 2026-09-22 13:15:58 +00:00
Forge 7e674aa0fa feat: pin reviewed Drupal 0.1.1 while retaining immutable 0.1.0 2026-09-22 13:08:13 +00:00
Forge b27bf875c9 fix: tighten Drupal wiring evidence and pending release metadata 2026-09-22 13:06:43 +00:00
Forge 1209454f3b test: add Drupal accuracy negatives and measured corpus samples 2026-09-22 12:59:47 +00:00
Forge 5d40696905 feat: add reviewed official Drupal registry operator 2026-09-22 12:29:38 +00:00
Forge 44aaa721b2 feat(marketplace): checkpoint Worker D1 and R2 adapter 2026-09-22 06:53:02 +00:00
Forge ebf32f8fb0 fix(marketplace): preserve author input when catalog loads late 2026-09-22 06:15:21 +00:00
Forge 63de525482 fix(marketplace): flush registry files with Windows-compatible handles 2026-09-22 06:08:03 +00:00
Forge 35d7d8c100 feat(marketplace): add persistent registry operations and verified backups 2026-09-22 05:39:24 +00:00
Forge 37d4dd8371 test(extensions): import directory helper from its defining module 2026-09-22 05:00:44 +00:00
Forge 70d24ab276 fix(extensions): honor configured graph directory during recovery 2026-09-22 04:58:58 +00:00
Forge 258cd005fe fix(extensions): preserve trust across native path aliases and publisher input 2026-09-22 04:47:58 +00:00
Forge bae8e16dde Add native test-only workflow and durable full regression shards 2026-09-22 04:35:43 +00:00
Forge 9ab6959d26 Keep legacy graph owners protected and root reads lightweight 2026-09-22 03:49:35 +00:00
Forge 4c13016561 Validate recovered trust and config; refresh readers after graph commits 2026-09-22 03:47:21 +00:00
Forge e7f89a43e6 Add journaled extension lifecycle recovery and process-kill harness 2026-09-22 03:44:33 +00:00
Forge 0730549f2a Retain semver build metadata in immutable release identities 2026-09-22 03:15:32 +00:00
Forge 79f2d9f1ca Preserve extension version pins through lazy CLI parsing 2026-09-22 03:10:06 +00:00
Forge eab93c3114 Resolve compatible stable extension releases across browser and CLI 2026-09-22 03:07:46 +00:00
Forge 8b30afb328 Add public extension author SDK, starter and graph test harness 2026-09-22 02:45:03 +00:00
Forge c33354c930 Recover project extensions and validate worker and marketplace lifecycle 2026-09-22 01:29:58 +00:00
Colby Mchenry 4297b8e2ed fix: restore call graph relationships and Steps effects (#1862)
* docs: record regression audit and validation

* test: cover audited call and extraction regressions

* test: cover audited calls and selector guards

* test: cover audited calls and selector guards

* test: cover audited calls and selector guards

* test: cover audited calls and selector guards

* test: cover audited calls and selector guards

* fix: retain qualified call sites and restore language-specific getters

* fix: resolve typed calls and bound store actions

* fix: retain qualified call sites in the native kernel

* docs: explain restored call and Steps coverage

* test: isolate watchdog and cover imported store aliases

* refactor: expose import lookup through resolver context

* refactor: remove import resolver cycle

* test: canonicalize Claude config temp paths

* docs: record review fixes and large TypeScript timings

* test: cover store cache edits and document verification

* perf: cache per-file store eligibility

* fix: observe large benchmarks without a wall timeout

* docs: explain completed indexing benchmarks

docs: explain completed indexing benchmarks
2026-09-15 05:30:09 +00:00
Colby Mchenry e79d5df448 fix(mcp): make explore guidance advisory (#1504) (#1805)
Apply upstream commit 1572d90d71 from
PR #1570 by @rongbc.

Clarify that suggested explore call counts are advisory and extra calls
remain available. Sync the MCP initialization guidance and eval probe,
retain the upstream regression tests, close the footer's bold formatting,
and credit the fix under Unreleased in the changelog.

Validation: npm run build; focused Vitest (3 files, 53 tests); source and
dist wording greps; git diff --check.

Fixes #1504

Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
2026-09-08 18:28:36 -05:00
Colby McHenry 6f4887db80 feat(steps): draw all arms of conditional navigations as separate edges
Adds multi-arm navigation support: when a destination is produced by a conditional, every arm is now drawn as its own edge. Introduces helpers (hrefArms, destinationsForHref) and updates framework resolvers and edge creation to emit multiple navigates edges (via alsoTargets) instead of a single one. Also introduces per-app rooted route tables to avoid cross-app crossings, and updates various resolvers (React Router, TanStack Router, Vue Router, SvelteKit, Vue, and SvelteKit’s linker) and the UI to reflect multiple possible destinations. Tests and docs updated to reflect the new behavior, ensuring the Screens tab shows all possible navigation paths from conditional destinations. This makes navigation visualization more accurate for forked destinations.
2026-08-31 11:34:26 -05:00
Colby McHenry dff4e50e98 chore(scripts): try-repo.sh — clone, index with the current build, and open the viewer on a framework preset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 10:26:13 -05:00
Colby McHenry 47576b392e feat(ui): saved trails — a walk you named, kept, and still true after a re-index (CG-60)
Save trail on the trail bar writes the walk to .codegraph/ui/trails/ as one
JSON file, listed on the empty screen and on Entry points above the derived
suggestions, reopened at the symbol you left with the whole path restored.

A hop is stored by qualified name, kind and file — never by node id, which
contains a start line and so changes the first time anybody edits above the
symbol. Every hop is re-resolved against the current index on the way out and
each row says what became of it: still here, moved to another file, now
ambiguous, or gone. A hole is never stitched over: the row opens the longest
run of CONSECUTIVE resolved hops and says which ones those are, because the
trail is a path and a skipped hop would draw a call that does not exist.

This is the first write the viewer makes, and the boundary moved with it:
POST/DELETE answer under /api/ only, must carry X-CodeGraph-UI and
application/json (neither of which a cross-origin form can produce without a
preflight this server answers none of), and --read-only refuses both while
still listing what is there. The blanket "read-only" claim is retired from the
banner, the README, the CLI help and the docs site in favour of the narrower
true one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 08:31:48 -05:00
Colby McHenry 56dfdb0655 feat(ui): dead code and islands — what nothing reaches, and everything that could still reach it (CG-59)
A Dead code screen and a mark on the Map, both drawn from one derivation in
src/graph/dead-code.ts so a second surface can never disagree with the first.

The SQL half is four lines — no incoming edge but `contains`. It returns ~2 500
candidates on this repository and the shipped list is 20; everything in between
is the feature. A candidate is dropped the moment there is any reason to believe
something outside the graph reaches it: exported symbols and header
declarations, test and generated files, abstract and interface members, anything
carrying a `decorates` edge, overrides of an ancestor's member, names the
language calls by itself, vendored directories, files nothing in the index
reaches (those are islands, and the Map says so instead), names the resolver
failed to resolve somewhere, and names shared with a symbol that IS referenced —
the mis-resolution that leaves a used method with a self-edge and its twin with
nothing. The last rule is the only one that is not a graph query: before a claim
is made, the declaring file and every file that reaches it are read and the
identifier counted, which is what catches the references the extractor never
recorded (`this.handleMessage.bind(this)`, a call inside an object literal, a
shorthand property).

Every subtraction is counted and printed under the list with the scale it came
from, and the caveat line above it never collapses: the claim is "no static
reference in the index", not "unused".

On the Map a module nothing depends on keeps its stroke and says so in its count
line, and tool-generated files and modules recede to ink-4 there, in the map's
file list, in search results and on the file screen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 07:57:00 -05:00
Colby McHenry 2a0c6dc58f feat(ui): the type hierarchy — what a type is built on, and what dispatches through it (CG-58)
A vertical tree above the members outline for classes, interfaces, structs,
traits, protocols, enums, unions and type aliases: ancestors above (the whole
chain, not just the direct parent), the focus in accent, subtypes below indented
per level. `extends` draws solid, `implements` dashed; a synthesized edge — Go's
implicit interface satisfaction — draws dashed wider and carries the site it was
wired at, so a relation the resolver inferred never reads like one the source
wrote down. For an interface the fan below IS the set of runtime targets a call
can land on, and a type with eight or more implementers leads with that in a
sentence. Members that redeclare an ancestor's are marked in the outline.

The walk lives in `src/graph/type-hierarchy.ts`, following CG-50/CG-51: shared
computation in `src/graph/`, presentation in the caller. Its `countImplementers`
is now also what `ToolHandler.buildPolymorphicBoundaries` counts with, so "N
types implement X" is the same N whether an agent reads it or a person does.
`/api/node` carries the block as `hierarchy` rather than a second endpoint —
it is part of the Symbol view's first paint, and gated to types, so a function
costs one kind test.

Layout is arithmetic (24px rows, 22px indent, orthogonal connectors computed
from the two): no ResizeObserver, same payload → same picture. The header's
`extends X` / `implemented by …` chips are suppressed while the tree is on
screen — two renderings of one relation in one column is how a reader ends up
trusting neither.

`TypeHierarchy` is exported from `@colbymchenry/codegraph-ui` and takes its data
as a prop, so a host holding a `WireSymbolPayload` renders it without a second
read.
2026-08-27 07:14:55 -05:00
Colby McHenry c15413f200 feat(ui): the viewer's screens as @colbymchenry/codegraph-ui, behind one adapter (CG-61)
`ui/src` now builds two ways from one tree: the static app `codegraph ui`
serves, and — via `svelte-package` — a Svelte library the Pro app imports.
A forked component would be a second answer to the same question about the
same graph, so there is no fork.

Everything a screen knows arrives through a `GraphAdapter`: eleven methods
answering the wire shapes verbatim, with `createHttpAdapter()` (the loopback
JSON API) as the default and a host's in-process engine reads as the point.
`lib/api.ts` became a one-line-per-call facade over it, which is why no call
site in the views changed. The payload types moved to `lib/wire.ts` — no
imports, no runtime — so a host can depend on the vocabulary alone.

Two more seams and one guard:

- `lib/navigation.ts` holds the href builders behind a `NavigationDriver`, so
  a host addresses its own URL space. The app's half — the hash parser and the
  live route, which attach window listeners at module scope — stays in
  `router.svelte.ts` and is pruned out of the package: rendering a Symbol view
  must not install a hash router in somebody else's application.
- `lib/theme.css` carries the design tokens and maps Svelte Flow's `--xy-*`
  variables onto them, so a host never sees library defaults. Dark now also
  answers to a bare `[data-theme]`, which is how `<CodegraphUi theme>` themes
  a container rather than the document.
- `scripts/check-ui-package.mjs` prunes the app's shell, resolves the
  extensionless specifiers svelte-package leaves behind, and asserts that
  nothing but `lib/adapter.js` reaches the network.

The search box, its keyboard and its panel are one component now
(`SearchPalette`), because splitting them is what breaks a palette.

`__tests__/ui-package.test.ts` mounts the three screens from the package entry
against a mock adapter in jsdom; it runs as a second vitest project so the
`browser` resolve condition it needs cannot reach the engine's suites.

Versioned with the engine. Prepared, not published: `private: true` is the
guard and `pack-npm.sh` only packs a tarball under CODEGRAPH_PACK_UI=1.
2026-08-27 06:52:57 -05:00
Colby McHenry ad91c8fdd8 feat(ui): classify code from the engine's own tree-sitter parse, retiring Shiki (CG-57)
The viewer ran a second highlighter over source the engine had already parsed
with a real grammar: Shiki, plus 56 pruned TextMate grammars shipped in
dist/textmate/. The classification now comes off that tree instead, so a file is
read by exactly the grammar that decided what its symbols are.

The swap is complete rather than flagged: @shikijs/core, @shikijs/engine-javascript
and @shikijs/langs are off the dependency list, scripts/prune-grammars.mjs and
`npm run build:textmate` are deleted, and check-ui-build.mjs asserts the
tree-sitter grammars in dist/extraction/wasm instead of dist/textmate.

The wire contract is unchanged — `[classId, text]` pairs with the class names
alongside — so the viewer's decoder and code blocks did not have to be rewritten.
Two classes are added to the six: `type` (a named type reference, painted at
plain ink) and `def` (the name a definition declares, weight 600), the latter
taken from the extractors' own definition tables so it cannot drift from what
indexing calls a definition.

Three differences are not cosmetic:

* Interpolations (`${…}`, `#{…}`, `$"{…}"`, f-strings) are classified as code,
  not as string. The call-site overlay refuses to claim a token classed string,
  so calls written inside interpolated strings now link.
* Built-in type words are emitted whole and classed `type` in every language.
  The grammars disagree about whether `string` is a type_identifier or an
  anonymous token inside a predefined_type, and TextMate scoped them
  inconsistently too.
* 3 000 lines of TypeScript cost 24-41 ms instead of ~700 ms.

Given up deliberately: Liquid, Razor, YAML, Twig, XML and .properties render
plain. .svelte/.vue/.astro are classified through their <script> blocks, the same
delegation the SFC extractors do. Pulling html/css/vue out of tree-sitter-wasms
would cover them, but those ABI-13 builds are the known cause of shared-WASM-heap
corruption for every other language in the same process.

Measured parity, per-language before/after screenshots and the reproduction
recipe: docs/design/cg57-highlighting-parity.md.
2026-08-27 06:22:11 -05:00
Colby McHenry 2ad836d935 feat(ui): highlight source server-side with a near-monochrome Shiki theme (CG-43)
The viewer's code block stops lexing with a hand-rolled dialect table and
reads real TextMate grammars instead, run once in `/api/source`.

Three things make that safe to depend on:

* Highlighting never fails a request. A missing grammar, an oversized
  slice, an ESM import that did not resolve — every one of them answers
  `engine: 'plain'` with a reason and the source still goes out.
* Identifiers survive whatever token boundaries a grammar chose. Every
  code token is split into identifier runs before it goes on the wire, so
  the graph's call-site overlay claims a token the highlighter produced
  rather than re-cutting the line. `assignRefs` now matches on a token's
  text rather than on the class a grammar gave it, so a language that
  scopes type names as `storage.type` still links.
* The theme classifies rather than colours: its foregrounds are sentinels
  the server maps back to class names, and the viewer paints them from
  CSS custom properties — one token stream serves light and dark with no
  refetch, and the ramp lives only in app.css.

Comments move from --ink-3 to a new --code-comment. --ink-3 measures
3.46:1 on paper and 3.00:1 on the hot-line tint, both under AA for 12.5px
text; --code-comment is the smallest step along the same ramp that clears
4.5:1 on every background a code line can have, and stays quieter than
the strings and numbers above it.

Shipping: @shikijs/core and @shikijs/engine-javascript are runtime
dependencies (no wasm, no native module); @shikijs/langs stays a
devDependency and `npm run build:textmate` writes only the closure the
engine's 40-odd languages reach — 56 grammars, 2.6 MB, against 11 MB for
all 722. check-ui-build.mjs asserts the tree after every build and inside
every release archive.
2026-08-27 01:23:10 -05:00
Colby McHenry a72f22a6d3 feat(ui): scaffold the codegraph ui viewer as a Svelte 5 + Vite workspace (CG-40)
Adds `ui/` as an npm workspace (Svelte 5.56 + Vite 7, devDependencies only —
the engine's runtime dependencies are untouched) and chains its build into
`npm run build`, so the browser viewer ships inside `dist/` with everything
else: `build-bundle.sh` already copies `dist` wholesale and `pack-npm.sh`
packs that bundle.

Output is `dist/viewer/`, NOT `dist/ui/`: `src/ui/` is the engine's terminal
ui (shimmer progress + its worker) and tsc compiles it to `dist/ui/`, so
emitting there both deletes those modules — the CLI then dies at startup with
`Cannot find module '../ui/shimmer-progress'` — and would leave the static
server handing out compiled engine internals. The design spec is corrected to
match.

`scripts/check-ui-build.mjs` is the release guard: index.html must exist, be
non-trivial, and every local asset it references must be on disk, and the
compiled engine next door must still be intact. It runs after every UI build,
again in `build-bundle.sh` once the bundle stage has copied `dist`, and again
in `pack-npm.sh` once each archive is unpacked — so a broken viewer fails the
release instead of shipping a CLI that serves a 404.

`vite build` does not override an ambient NODE_ENV, so a shell or runner with
NODE_ENV=development silently shipped dev-mode Svelte (~13 kB of dev-only
runtime checks, warning in the user's console). The config now pins production
for `command === 'build'`; macOS and Windows ARM64 then emit byte-identical
bundle hashes.

The shell itself follows docs/design/codegraph-ui-design-spec.md §2–§3.1:
design tokens as CSS custom properties (light on bare `:root`, dark under both
`prefers-color-scheme` and `[data-theme="dark"]`), square corners, hairline
rules, one oxblood accent; top bar 48px / trail bar 34px / main; a hash router
over `#/s/<id>`, `#/file/<path>`, with `#/map` and `#/flow` reserved for phase
2. Fonts are vendored through @fontsource rather than fetched, so a local
reader works offline and never announces the project to a CDN.

Verified: clean `npm run build` from an empty dist on macOS and on the Windows
ARM64 VM (forward-slash asset URLs, CLI still starts, both assertion failure
modes exit 1); `dist/viewer` present in a real darwin-arm64 bundle and in the
packed npm platform package; shell geometry, tokens, all seven routes, both
themes and font loading checked in headless Chromium with no console errors;
`npm test` unaffected.
2026-08-26 15:55:10 -05:00
Colby Mchenry 99f2ebf0d1 Merge pull request #1527 from colbymchenry/bugfix/CG-38
CG-38: guarantee an agent-named symbol renders, wherever it sits
2026-08-07 13:19:25 -05:00
Colby McHenry 2c708caf7c Merge branch 'main' into feature/CG-35 2026-08-06 21:17:56 -05:00
Colby McHenry 89c53ddf24 fix(explore): guarantee an agent-named symbol renders, wherever it sits (CG-38)
`codegraph_explore` never returned `queueMessage` (L1087) or
`flushQueuedMessages` (L1102) from a 1,414-line file, on a symbol bag or a
prose question, even with that file at rank #1 holding 67% of the envelope —
the agent got a same-stem `QueuedMessage` interface at L70 and had to Read the
file for the functions it had named. Pre-existing at every build including
pre-epic (controlled bisect, index held fixed).

Two independent causes:

1. `buildFlowFromNamedSymbols` returns the Flow prose AND the set of node ids
   the agent named — and the latter is the whole guarantee, since it injects a
   named def into its file's cluster ranges at importance 9. Its bail-outs
   returned EMPTY, zeroing the identity whenever there was nothing to PRINT.
   Two sibling closures that never call each other produce no chain, no synth
   hop and no boundary, so both defs lost importance 9 and the file rendered
   from its head. `identityOnly()` now separates the two, gated on
   shape-precise tokens so a prose word that exact-matches a callable cannot
   promote itself.

2. The ceiling trim filled in SOURCE order, so an over-ceiling render always
   dropped the END of a large file first. The shrink HAD kept both symbols
   (1022-1121); the trim cut back to 839. `windowToCeiling` now takes the
   spine call site plus every importance>=9 member as focus lines, tries the
   full ceiling first, and splits the held-back reserve evenly with
   carry-forward — greedy-in-source-order reproduced the bug one level down.

The shrink's loose size estimate is left alone deliberately, and the comment
now says why: making it exact was built and measured WORSE (it stops at the
last member that fits whole and the released bytes carry forward to
lower-ranked files, costing payroll-go's `s.store.Upsert`). `bound()` clamps to
the ceiling anyway, so the slack costs no bytes; it just must not pick the
survivors, which is what the trim now handles.

The measurement gap this closes: every existing probe is aggregate — envelope
share, per-file spend, source totals, file counts — and all are green on a
response that returns 25K from the right file and omits the named function.
`probe-named-symbol.mjs` checks the definition LINE against the response's
rendered lines, per symbol.

Suite envelope byte-identical to main on all six repos; probe-allocation 4/4,
no starvation flags; 180 files / 2,997 tests green. Fixture: 7/7 fail on main,
7/7 pass here, deterministic over 4 runs per arm.
2026-08-06 21:11:51 -05:00
Colby McHenry eed16447c3 fix(explore): shrink a later cluster into the remainder instead of dropping it (CG-36)
A file's ranked clusters were all-or-nothing past the first one: the top-ranked
cluster was taken (shrunk to fit when it had to be) and every cluster below it
was rendered whole, then either fit the remainder or was dropped entirely. On a
file whose top-ranked cluster is TRIVIAL that discards the answer — django's
`db/models/sql/query.py` kept a 22-line glue cluster and dropped the 624-line
`Query` body, spending 1,923 of a 7,947 reservation; okhttp's
`RealInterceptorChain.kt` did the same behind its import header.

The response stayed full, which is why this was invisible: the unspent
reservation carried forward exactly as designed and a file scoring a fifth as
much took the bytes.

Two sites, the same rule — hold the remainder while it is still worth a section
(CG-26's between-FILES lesson, applied between CLUSTERS):

- selection now shrinks a later cluster into what is left of the file's budget,
  by the same whole-member rule the first cluster already used;
- the ceiling trim re-renders the weakest cluster into the room that remains
  before dropping it. On excalidraw's `typeChecks.ts` the section-cost estimate
  missed by 13 chars and a 1,512-char cluster — the file's highest-SCORING one —
  was thrown away to pay for it.

Cluster RANKING is untouched: measured, both real cases lost on `maxImportance`,
not on the density tiebreak the issue suspected, and density-first is what keeps
Alamofire's `Session.swift` from burying its methods under the property list.

Suite (6 repos, clean-rebuilt indexes): all 8 starvation flags cleared,
+1,012 source chars net. django's `sql/query.py` 1,923 -> 10,082 of 7,947,
okhttp's `RealInterceptorChain.kt` 1,474 -> 6,038 of 6,058, gin's
`routergroup.go` 3,273 -> 5,632. okhttp trades its rank-6 file (score 21) for
+7,196 chars in the two files that answer the question.

Ships two fixtures pulling in opposite directions (`starved-cluster-ts` and
`dense-header-ts`), a `spendShareAtLeast` gate in probe-allocation, and
probe-file-spend.mjs — a standing per-file reservation-vs-delivered sweep.
2026-08-06 15:10:46 -05:00
Colby McHenry 9efae0f8f2 fix(explore): damp ambient declaration files on flow queries (CG-28)
A file that declares nothing but types and that nothing in the index depends
on — a hand-written ambient `.d.ts` of global shims, vendored typings, module
augmentation — cannot answer a flow question: no bodies, no call edges, no
behaviour, nothing typed by it. But the identifiers it declares are exactly the
generic ones a prose question uses (`Body`, `Message`, `ImageMetadata`,
`ReadableStream`), so on term overlap it out-scored the implementation. Measured
on the new fixture: rank #1 and 51% of delivered source, with the flow's own
entry file pushed out of the response entirely.

Measured first, per the issue: the Wrangler `worker-configuration.d.ts` that
opened this is already handled by CG-25's banner detection, worth 15-46 points
of envelope share across four flow queries. CG-25 credited; only the un-bannered
case needed anything.

`rankPenalty` now multiplies score and graph mass by 0.5 for such files, taken
as the STRONGER of it and the generated penalty rather than multiplied — one
property two signals see must not be charged twice. Detection is structural, not
by extension, and four conditions deep. Two of them were forced by measurement:
requiring every symbol to be type-level takes the corpus flag rate from 1-18%
(which swept in Kotlin sealed classes, Rust mod.rs re-exports and django's
locale tables) down to 0-4%; requiring that nothing depends on the file
separates an ambient shim from a working types module, and without it the rule
demoted displacement-ts's pipeline `types.ts` and broke the CG-31 gate.

A query that NAMES a declared type is exempt, so a question about a type still
reaches its declaration at full weight. Precise tokens only, so "…the file
body…" cannot exempt a `Body` interface it never meant to name; this needs its
own set because `namedSeedIds` is callable-only and a type never becomes one.

Regression evidence in docs/benchmarks/explore-declaration-only-cg28.md:
6-repo envelope sweep byte-identical against a clean baseline build, zero
ambient files reach the candidate set on VS Code across five queries, corpus
flag rate 0-0.74%, both allocation fixtures PASS, full suite 2,978 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:35:56 -05:00
Colby McHenry 91cb5b4317 measure(explore): the factory-closure envelope premise does not hold (CG-27)
CG-27 asked whether the >50%-of-file envelope drop should cover `function` /
`method`, so a `createFoo()` factory returning an object of closures stops
merging every closure inside it into one cluster. Measured on a hermetic
fixture, it should not, and the issue is closed as obsolete with CG-30 credited.

Two mechanisms already absorb the shape. shrinkCluster orders members by
(importance desc, size ASC) and refuses any member that overruns the cap once
something is kept, so a file-spanning member is only selected when it is the
sole member of the top importance tier — eight of nine query shapes never
selected it at all. When it IS selected, CG-30 windows it on whole lines, so
the file still delivers bounded, readable source (6 of 9 closure definitions
in that configuration).

Dropping the range instead SPLITS the file, and only the first-chosen cluster
may be shrunk: a trivial 7-line cluster won the density tiebreak and the
answer-bearing cluster was dropped whole — rank-#1 file 7,539 chars and 7 of 11
closures to 397 and none. Reaching the same intent more carefully (defer the
envelope MEMBER inside shrinkCluster, leaving clustering untouched) is noise:
69 vs 68 closure definitions across nine query shapes. Nothing shipped.

Adds the fixture, the probe, a standing gate on the outcome, and the record —
including a real defect the measurement exposed on the epic tip: django's
query.py leaves 8,212 of 10,135 unspent and drops a score-290 cluster to keep a
score-14 one. Filed separately.

No behaviour change, so no CHANGELOG entry.
2026-08-06 14:07:22 -05:00
Colby McHenry d49265043c test(explore): add the factory-closure fixture and its selection probe (CG-27)
A file whose top-level symbol spans almost all of it — createFoo() returning
an object of closures — is how Svelte 5 rune stores, React custom-hook modules,
IIFE module-pattern JS and Zustand's create((set,get)=>({…})) are all written.
probe-factory-closure.mjs measures what such a file DELIVERS from within: which
inner symbols' definitions reach the agent, not how many bytes did.
2026-08-06 13:58:36 -05:00
Colby McHenry 7cbde95ce2 fix(explore): pay every admitted file on every render path (CG-26)
The invariant this closes: every admitted file receives at least its
reservation before any file draws on carry-forward slack. CG-30 bounded an
oversize cluster member and CG-31 gave the cluster path a displacement guard;
three holes were left, and each one starved a file that had been admitted,
reserved and — in the worst case — rendered.

1. The whole-file arms had no displacement guard. BUY's fit test read
   `renderCeiling - totalChars` (everyone's room) while its source-space
   sibling refused the same trade, and GRACE was not fit-tested at all.
   okhttp's CallServerInterceptor.kt shipped 8,499 chars on a 5,964 funded
   ceiling and the rank-6 file below it delivered nothing. Both arms now test
   the render they actually produce against `fundedHeadroom`, and a whole
   render that does not fit falls through to clustering instead of skipping
   the file.

2. Every section was charged a flat 200 chars while a real header runs
   300-500. The loop believed it had room it did not have — okhttp allocated
   26,601 against a 24,400 ceiling — so the final truncation threw a
   fully-rendered section away. Sections are charged their real cost now, the
   owed-below arithmetic uses a per-file overhead estimated from the file's own
   symbols, and a marginal overrun trims the weakest cluster (or windows the
   last one into the room that is left) rather than skipping the file over a
   rounding difference.

3. `owedPayableBelow` held all-or-nothing. When the last admitted file's FULL
   reservation no longer fit, nothing was held for it: on the precise-query
   fixture the rank-5 file took 4,134 chars against a 2,948 reservation while
   rank 6 — admitted, reserved 2,539 — was left 4 chars and skipped. It now
   holds the remainder while that remainder is still worth a section
   (MIN_CHARS).

And the epilogue is budgeted instead of discarded. The flat 600-char margin was
neither the epilogue's size (1,064 gin, 1,788 django, 2,231 excalidraw) nor a
bound on it, so four of six suite repos shipped with no pointer list and no
reminders at all. The loop now reserves the epilogue's FLOOR — the one line
that says an uncovered area exists, plus a pointer for every file whose bytes
were deliberately withheld (CG-12) — and the rest is fitted to the room that
actually remains, in priority order, entry by entry. Sized from the real
strings; no constant was swept against the suite.

Deterministic, same clean-rebuilt indexes, baseline = CG-31 tip:

  repo         base source   new source   files      ceiling
  django            20,791       20,878   6 -> 6     was discarding its epilogue
  tokio             21,521       21,607   5 -> 5     was discarding its epilogue
  okhttp            19,034       18,870   5 -> 6     +1 file delivered
  excalidraw        20,204       19,652   8 -> 8     keeps its pointer list
  gin               10,776       10,776   4 -> 4     byte-identical
  alamofire         11,662       11,662   2 -> 2     byte-identical

No repo truncates any more and none loses a file. okhttp and excalidraw trade
164 and 552 source chars on their LAST-ranked file for the pointer list naming
what the response could not cover — bytes the CG-31 tip only had because it
over-filled a ceiling it mis-measured and then discarded the epilogue whole.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 03:45:28 -05:00
Colby McHenry 089dcc276f fix(explore): hold back what is still owed below a clustered render (CG-31)
Carry-forward slack let a file spend what the files ABOVE it left on the
table. Nothing held back what was promised BELOW it. The whole-file BUY arm
has always refused that trade (`owedBelow`); the cluster path read `headroom`
— what is left before the hard ceiling — instead of what is still owed, so
`fileBudget` and `SPINE_CEILING` could pay a 1.5x overshoot out of another
file's reservation.

`fundedHeadroom` is the same inequality in the units the cluster path spends
in: source PLUS the per-section overhead each unreached file will charge.
Floored at the file's own reservation — a kept promise is not a displacement —
and it is <= `headroom` by construction, so it is the only bound the three
render sites need. The skeleton path's `bodyCap` takes it too.

Measured on `__tests__/fixtures/displacement-ts` (a 4-stage pipeline padded
past 500 files, where the 24K envelope genuinely saturates the 24.4K render
ceiling):

  before  ingest.ts emitted 9,301 on a 6,289 spendable, then lost the whole
          section to the final ceiling — 0 delivered. types.ts and sink.ts
          skipped `budget-whole-file`. 3 of 6 admitted files delivered.
  after   ingest.ts bounded to the 4,913 actually free. 6 of 6 delivered,
          envelope 14,908 -> 22,066.

The self-query allocation fixture flips back to PASS with it, on a clean full
rebuild of this repo's index (CG-33). Its `afterCG30` verdict blamed an
over-RESERVED incidental file; the reservation was identical in both arms —
the file was over-SPENDING. Recorded honestly in `afterCG31`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 02:44:59 -05:00
Colby McHenry d652c148f6 docs(cg-30): changelog entry + record the self-query probe flip honestly
The self-query allocation probe fixture's delivered-share gates now fail. The
cause is not the new bound: allocation is unchanged between arms (parse-run.mjs
32.3% vs 33.9% on main) and tools.ts delivers the same 8,282 chars in both. What
changed is that the incidental file now DELIVERS — on main its whole section was
cut by the hard-ceiling truncation, so the fixture passed on truncation luck.
Every file on this repo obeys the new bound (max 1.40x of spendable).

Recorded as `afterCG30` with that reasoning rather than tuning the bound to
restore the pass. The over-reservation it exposes is epic CG-24's subject.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 01:47:02 -05:00
Colby McHenry 2cf63fd114 CG-33: record index-drift measurement and add a drift diff tool
A live, auto-sync-maintained index does not converge to a clean full
rebuild of the identical tree. On codegraph's own repo, 4.3% of distinct
edges are wrong in both directions (751 missing, 476 stale), dominated by
`calls` — the edges flow queries traverse and that feed the RWR mass
explore ranks files by.

Raw edge rows differ by only +0.7%, because the divergence is
bidirectional and nets out; any drift check must compare edge SETS.
Rebuild-vs-rebuild is 0, so the indexer is deterministic and this is not
noise. Node sets are identical and every integrity check is 0 on both
indexes, so this is stale cross-file resolution, not accumulated residue.

`diff-index-drift.mjs` is read-only and takes two index paths — rebuilding
is the caller's job, so the tool can never clobber the artifact it is
measuring. It also refuses a missing path, since node:sqlite creates an
empty database rather than failing and an empty schema reads exactly like
a stale pre-migration index.

Diagnostic captures from the originating incident are deliberately NOT
committed: they contain verbatim source from a private repo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 01:24:59 -05:00
Colby McHenry 04c0f8eab5 test(agent-eval): sum tokens per turn — result.usage stopped being cumulative
"Tokens processed" was read off result.usage. That was correct when the README
figures were measured; in current Claude Code the field reports the LAST turn
only. Nothing here changed — the host did, silently — and the harness kept
reporting the smaller number.

The error is one-sided, which makes it worse than noise: it under-counts
whichever arm takes more turns, and that is always the WITHOUT arm. On the
2026-08-05 campaign it turned a real 62% token saving into 19% and invented a
token REGRESSION on tokio (-41%) and alamofire (-25%) that does not exist. Those
numbers were one push away from the README.

Now summed per assistant request and deduped by message.id, the same rule the
occupancy timeline already used — Claude Code emits one event per content block
carrying identical usage, so summing per event double-counts (~1.7x measured).

CLAUDE.md already warned about this field. The code did not follow; it does now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:27:02 -05:00