The telemetry database hit D1's 10 GB cap on 2026-08-11 and refused nearly
every write for seven weeks: the dashboard kept showing Aug 9 as the latest
day, and its activation query (a cohort join over raw events, ~55 s per
week) stalled D1's single query lane until every panel failed.
Root cause was the client: usage counters were aggregated per process and
appended on exit, so every `serve` launch, CLI command and prompt hook
uploaded its own `count: 1` line - 30.3M of the 30.8M event rows.
- Client: merge count lines per (day, kind, name, client) on append,
stale-claim recovery (which also bypassed the size cap) and send.
- Ingest: usage counters ADD into usage_daily, one row per machine x day x
tool (migration 0003), so storage no longer depends on upload frequency.
- Rollup: reads usage from usage_daily, folds legacy usage rows out of
events in 50k-row transactions, maintains machine_first_seen.first_index_day
(0002), catches up on days a failed run missed, purges usage_daily with
retention. 0004 drops the now-unused events_machine_day index.
- Dashboard: ranges end today; rolled-up series return null past the
rollup's last day instead of a phantom zero; no panel reads raw events
(activation reads first_index_day); min/max lookups stay index-only;
one retry on transient 5xx; banners when ingest stalls or the rollup
falls behind.
- scripts/backfill-rollup.sh re-runs the rollup over a day range.
Already applied in production: DB 10 GB -> 467 MB, all usage counts kept,
rollups current through 2026-10-02.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* perf(resolution): resolve imports to definitions, not sibling import nodes (#915)
"Resolving refs" crawled (tens of minutes) on large projects — most painfully
ones mixing a big front-end and back-end. An external package or module imported
across hundreds/thousands of files (react, a shared UI package, Python
logging/typing) is re-declared as an `import` node in every importing file, so
its unresolved import ref fell through to the exact-name matcher, which scored
all K same-named import nodes via findBestMatch — K refs x K candidates = O(K^2)
per package, producing only meaningless import->import edges.
Fix: exclude `import`-kind nodes as name-match targets (they're statements, not
definitions; real import->definition resolution is the import resolver's job).
Plus two safe constant-factor wins in findBestMatch: hoist the per-candidate
ref.filePath split, and skip cross-language candidates when a same-language one
exists (provably the same winner — same-language scores >=50, cross-language
maxes at 35).
Measured: superset (Py+TS) candidates scored 7.5M -> 833K (9x), non-import edges
preserved (+1618 now resolve to real defs), ~22K useless import->import edges
removed; kubernetes (Go) computePathProximity 37.2s -> 5.0s; synthetic 8k-file
mixed repo (K=4000) resolution 16.0s -> 1.7s. Full suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: correct stale better-sqlite3/wasm references to node:sqlite
The SQLite backend has been Node's built-in node:sqlite (real SQLite, WAL + FTS5,
from the bundled runtime) for a while — there is no native build step and no
node-sqlite3-wasm fallback. README and the docs site were already updated; this
catches the stragglers:
- CLAUDE.md: the src/db/ backend description and the sqlite-backend test note.
- src/db/index.ts, src/mcp/tools.ts: two code comments that still blamed "the
wasm backend" for non-WAL behavior (reworded to "when WAL isn't in effect").
Leaves tree-sitter grammar wasm (web-tree-sitter / --liftoff-only) untouched —
that's a different, still-current use of wasm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(telemetry): drop the dead sqlite_backend field (schema v2)
node:sqlite is now the only backend, so the `index` event's `sqlite_backend`
field was a constant ("native") carrying no signal — and the `install` event
never actually sent it. Remove the field and the backendKind() helper, bump the
telemetry SCHEMA_VERSION 1 -> 2, and update TELEMETRY.md + docs/design/telemetry.md.
The ingest worker is deliberately left tolerant: `index` doesn't require the
field and schema_version validates as nonNegInt(99), so v2 events ingest fine and
old clients still sending v1 + sqlite_backend keep validating too. Added a legacy
comment there explaining it's safe to drop once old-client share is negligible.
telemetry.test.ts: the assertion pinning schema_version and a stale-claim fixture
line updated 1 -> 2. All telemetry tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds anonymous usage statistics (commands/tools used, languages indexed,
connecting agents) with a strict, auditable allowlist. Never code, paths,
file/symbol names, queries, or IPs.
- src/telemetry/: zero-dep client — consent resolution (DO_NOT_TRACK >
CODEGRAPH_TELEMETRY > stored choice > default-on), random machine UUID,
in-memory counters → capped JSONL buffer → completed-day rollups; sync
exit-append (survives process.exit) + opportunistic bounded sends; the
first-run notice gates the first SEND, never local buffering, so the
installer's consent toggle always precedes it. Off is off: no recording,
no socket, buffered data deleted.
- codegraph telemetry status|on|off; per-command counting via preAction hook.
- MCP: tool counting after the reply is on the wire (session + proxy
in-process fallback), agent attribution from initialize clientInfo,
unref'd daemon flush interval. Zero hot-path cost, zero stdout.
- Installer: visible default-on consent toggle (asked once, never re-asked),
install/index/uninstall lifecycle events.
- telemetry-worker/: public Cloudflare Worker behind telemetry.getcodegraph.com
— allowlist validation, IP stripping, per-machine rate limit, forwards to
PostHog as anonymous events. Ships nowhere with the npm package.
- TELEMETRY.md (field-by-field contract) + README section + design doc.
- 20 unit tests; suite-wide CODEGRAPH_TELEMETRY=0 guard so tests never
pollute real telemetry. Full suite: 1448 passing.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Colby Mchenry