mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-18 13:53:01 +08:00
Colby Mchenry
217190680d
The telemetry database hit D1's 10 GB cap on 2026-08-11 and refused nearly every write for seven weeks: the dashboard kept showing Aug 9 as the latest day, and its activation query (a cohort join over raw events, ~55 s per week) stalled D1's single query lane until every panel failed. Root cause was the client: usage counters were aggregated per process and appended on exit, so every `serve` launch, CLI command and prompt hook uploaded its own `count: 1` line - 30.3M of the 30.8M event rows. - Client: merge count lines per (day, kind, name, client) on append, stale-claim recovery (which also bypassed the size cap) and send. - Ingest: usage counters ADD into usage_daily, one row per machine x day x tool (migration 0003), so storage no longer depends on upload frequency. - Rollup: reads usage from usage_daily, folds legacy usage rows out of events in 50k-row transactions, maintains machine_first_seen.first_index_day (0002), catches up on days a failed run missed, purges usage_daily with retention. 0004 drops the now-unused events_machine_day index. - Dashboard: ranges end today; rolled-up series return null past the rollup's last day instead of a phantom zero; no panel reads raw events (activation reads first_index_day); min/max lookups stay index-only; one retry on transient 5xx; banners when ingest stalls or the rollup falls behind. - scripts/backfill-rollup.sh re-runs the rollup over a day range. Already applied in production: DB 10 GB -> 467 MB, all usage counts kept, rollups current through 2026-10-02. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
59 lines
2.2 KiB
HTML
59 lines
2.2 KiB
HTML
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
<title>codegraph telemetry</title>
|
|
<link rel="stylesheet" href="/styles.css" />
|
|
</head>
|
|
<body>
|
|
<header class="masthead">
|
|
<div>
|
|
<h1>codegraph telemetry</h1>
|
|
<p class="subtitle">Anonymous usage from the public engine, straight out of D1.</p>
|
|
</div>
|
|
<form method="post" action="/logout">
|
|
<button type="submit" class="secondary">Sign out</button>
|
|
</form>
|
|
</header>
|
|
|
|
<!-- One filter row for the whole page: every panel below is drawn against the
|
|
range chosen here, and no panel carries a time control of its own. The
|
|
panels themselves are built from the registry in public/panels.js. -->
|
|
<section class="filters" id="filters" aria-label="Time range">
|
|
<div class="filter-group" data-role="presets"></div>
|
|
|
|
<div class="filter-group custom-range">
|
|
<label for="custom-from">From</label>
|
|
<input type="date" id="custom-from" data-role="custom-from" />
|
|
<label for="custom-to">To</label>
|
|
<input type="date" id="custom-to" data-role="custom-to" />
|
|
<button type="button" class="secondary" data-role="custom-apply">Apply</button>
|
|
</div>
|
|
|
|
<div class="filter-group filter-end">
|
|
<button type="button" class="secondary" data-role="refresh">Refresh</button>
|
|
</div>
|
|
|
|
<p class="filter-status">
|
|
<span id="range-summary">Loading…</span>
|
|
<span class="dot" aria-hidden="true">·</span>
|
|
<span id="data-through"></span>
|
|
<span class="dot" aria-hidden="true">·</span>
|
|
<span id="refreshed-at"></span>
|
|
</p>
|
|
</section>
|
|
|
|
<!-- Filled from /api/meta when a writer has stopped: the ingest worker (no new
|
|
events) or the nightly rollup (numbers stuck on an old day). Both have
|
|
failed silently before, so the page says so instead of drawing old data
|
|
as if it were current. -->
|
|
<section class="data-warning" id="data-warning" role="status" hidden></section>
|
|
|
|
<main class="grid" id="grid"></main>
|
|
|
|
<script src="/vendor/chart.umd.js"></script>
|
|
<script type="module" src="/app.js"></script>
|
|
</body>
|
|
</html>
|