mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-25 20:12:11 +08:00
Colby MchenryandClaude Opus 5.5
a192df40f9
Three bugs in the self-hosted telemetry services (telemetry-worker/ and
telemetry-dashboard/), reported against 6560052a and all still on main.
1. An index run that uploads late never counted toward activation. The
dashboard's funnel reads machine_first_seen.first_index_day, which only the
nightly rollup wrote, and the rollup re-rolls just the last three days plus
days with no daily_machines row. Ingest accepts timestamps up to 30 days old
and the client keeps an event's original timestamp when it re-queues it, so
an index event 4+ days late landed on a day nothing revisited and
first_index_day stayed NULL. The ingest upsert of machine_first_seen now
lowers first_index_day the way it lowers first_day:
coalesce(min(old, new), old, new), because SQLite's min() is NULL if either
side is. It is the same statement and row, with no new index and no
migration (the column exists since 0002). The rollup still re-derives it
from raw events, which covers events stored before this change.
2. ingest_stalled read only max(day) FROM events, but usage counters have
lived in usage_daily since 0003, so a day with usage and no lifecycle events
read as "The ingest worker ... is not storing anything". /api/meta now also
reads max(day) FROM usage_daily (a primary-key lookup) and returns
latest_usage_day and latest_ingest_day. Ingest counts as stalled only when
there is no lifecycle event from yesterday or later and no usage counter
from the day before yesterday or later, because clients upload a day's
counters only after that day ends. The banner names latest_ingest_day.
3. runNightly rolled up the 3 regular days, then up to 31 missed days, and ran
purgeOldEvents last. Each day first folds its legacy usage_rollup rows,
which takes minutes on a heavy day, so a backlog could run past Cloudflare's
15-minute Cron Trigger limit and skip the purge and the summary line night
after night. The purge now runs first: it is bounded and touches only days
past the window, which no rollup reads. After that, rollups stop starting
new days or fold chunks 10 minutes in (NIGHTLY_BUDGET_MS). A day stopped
mid-fold gets no rollup at all, so it stays a missed day and the next night
continues from its last committed chunk. The summary line counts those days
as `deferred`, and `caught_up` now counts missed days actually rolled up.
Verification: __tests__/telemetry-services.test.ts (new; the root vitest
config picks it up) runs the worker's fetch/cron code and the dashboard API
against the checked-in migrations in in-memory node:sqlite through a small
D1-shaped adapter. With the source fix reversed, all 5 tests fail (activation
0, first_index_day null, ingest_stalled true, and "nightly rollup incomplete:
4 day(s) failed, purge failed" once statements pass the 15-minute mark). With
the fix, all 5 pass. `npm run check` is clean in both packages. Against
wrangler dev, run locally only: smoke:rollup 71/71, smoke:cutover 62/62,
smoke:api 122/122 and render-check 87/87. smoke (ingest) is 46/47; the one
failure is the 70 KB oversized-body curl argument hitting the Windows
command-line limit, and main fails it the same way. The new smoke assertions
fail on main.
Takes effect only once both workers are redeployed; no migration.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
211 lines
13 KiB
Markdown
211 lines
13 KiB
Markdown
# codegraph telemetry dashboard
|
||
|
||
The private admin view behind `stats.getcodegraph.com`. Its sibling
|
||
[`telemetry-worker/`](../telemetry-worker/) writes anonymous usage events into a D1 database;
|
||
this worker reads them back and draws the charts. Two people use it, so the auth is
|
||
deliberately the simplest thing that is actually safe: one shared password in a secret, and
|
||
a long-lived signed cookie.
|
||
|
||
This directory is in the public repo for the same reason the ingest worker is — the code
|
||
that touches telemetry should be readable by the people it collects from. Nothing secret
|
||
lives here: the password and the cookie-signing key are deployment secrets, and the D1
|
||
database ID is an identifier, not a credential.
|
||
|
||
## What is gated
|
||
|
||
Everything except the login page and `robots.txt`. `assets.run_worker_first` is `true` in
|
||
`wrangler.jsonc`, so Cloudflare hands *every* request to `src/index.ts` before the static
|
||
asset server sees it — the dashboard HTML, its JS, its CSS and the chart library are all
|
||
behind the session check, and a request without a valid cookie gets a redirect (pages) or a
|
||
`401` (`/api/*`). The login page is rendered inline by the worker rather than served from
|
||
`public/`, so the asset directory needs no "is this file public?" judgement calls.
|
||
|
||
| Route | Auth | Notes |
|
||
|---|---|---|
|
||
| `GET /login` | public | Password form. Redirects to `/` if already signed in. |
|
||
| `POST /login` | public | Rate-limited per IP; sets the session cookie on success. |
|
||
| `POST /logout` | public | Clears the cookie. |
|
||
| `GET /robots.txt` | public | `Disallow: /`. |
|
||
| `GET /api/*` | required | JSON. `401` without a session. See the API below. |
|
||
| everything else | required | Static assets from `public/`. `302 /login` without a session. |
|
||
|
||
## The API
|
||
|
||
Every endpoint is `GET`, session-gated, and scoped by `?from=YYYY-MM-DD&to=YYYY-MM-DD`
|
||
(inclusive, UTC days). Ranges wider than 366 days are clamped and say so in
|
||
`range.clamped`. Responses come back Chart.js-shaped — `labels[] + datasets[]` — plus a
|
||
`rows[]` in the data's natural shape, which is what each panel's "Show numbers" table
|
||
renders. Bad input is a `400` with a message, never a guess. Chart data carries
|
||
`Cache-Control: private, max-age=300`.
|
||
|
||
| Endpoint | Answers |
|
||
|---|---|
|
||
| `/api/meta` | How current the data is: `today`, the rollup's last day (`latest_rollup_day`), the last day ingest stored anything (`latest_ingest_day` — a lifecycle event or a usage counter), machines active yesterday, and two flags — `ingest_stalled` and `rollup_behind` — that put a warning above the panels when either writer stops. Cached 60 s. |
|
||
| `/api/summary` | Big numbers: production users, active machines, new machines, installs, uninstalls, indexing runs, tool calls. |
|
||
| `/api/timeseries?metric=` | `installs_uninstalls`, `new_installs`, `production_users`, `indexing_activity`, `tool_calls`, `duration_buckets`. One dense point per day — a day with nothing is a zero, not a gap. Days after the rollup's last day are `null` ("not counted yet") and `covered_through` says where that is; `new_installs` is written live and runs through today. |
|
||
| `/api/breakdown?dim=` | `os`, `arch`, `codegraph_version`, `node_major`, `language`, `file_count_bucket`, `duration_bucket`, `target`, `scope`, `kind`, `name`, `client_name`, `name_error`. Optional `&event=`, `&metric=count\|machines`, `&limit=`. |
|
||
| `/api/activation?window=7` | Install → first index funnel, plus the daily rate. |
|
||
| `/api/retention` | Day 0–14 cohort curve for machines first seen in the range. |
|
||
| `/api/health` | Liveness plus the latest event/rollup day. Uncached. |
|
||
|
||
Everything reads the `daily_*` rollups, `machine_days` and `machine_first_seen`, which are
|
||
kept forever, so a chart stays correct for days whose raw events have been purged. **No
|
||
panel reads raw `events`.** D1 runs one query at a time per database: the activation funnel
|
||
used to join every cohort machine against `events`, which took ~55 s for one week of
|
||
cohorts in production and failed every panel queued behind it. It now reads
|
||
`machine_first_seen.first_index_day`, which the ingest worker lowers as each index event is
|
||
stored and the nightly rollup re-derives from raw events.
|
||
|
||
The presets end on **today** (UTC — every event and rollup is keyed on the UTC day). Live
|
||
numbers — production users, new machines, retention — run through today; rolled-up ones
|
||
stop at the rollup's last day, normally yesterday, and the filter row says which day that
|
||
is.
|
||
|
||
### Two numbers that are easy to misread
|
||
|
||
Both are labelled honestly in the UI rather than rounded off into something friendlier:
|
||
|
||
- **Machine-days, not users.** `daily_dim_counts.machines` is per day, so summing it over
|
||
a range counts a machine once per day it was active. A range-wide distinct count per
|
||
dimension value is not recoverable from the rollups at all, so the panels that use it
|
||
say "machine-days" and are share-of-total panels where the distinction does not move the
|
||
shape. Where a dimension rides several event types, the per-day figure is the largest
|
||
single-event count rather than their sum, so one machine's install + index + usage on
|
||
one day is not counted three times.
|
||
- **Recent cohorts have not finished converting.** A machine that installed yesterday has
|
||
not had seven days to run an index, so the tail of the activation curve is a floor, not
|
||
a result. The API marks those days (`complete: false`, `incomplete_from`) and the panel
|
||
says so instead of drawing a cliff and calling it a drop in conversion. Retention does
|
||
the same thing with a per-day denominator: day *k* is measured only over the machines
|
||
that have actually had *k* days to come back.
|
||
|
||
## How the session works
|
||
|
||
- The password is compared in constant time, over SHA-256 digests so the operands are always
|
||
the same length and nothing about the secret leaks through timing.
|
||
- The cookie is a signed assertion — `base64url(payload).base64url(HMAC-SHA256)` — not a
|
||
lookup key. There is no session store; a tampered payload fails the signature check.
|
||
- `HttpOnly; Secure; SameSite=Lax; Path=/`, `Max-Age` one year. You sign in once per browser
|
||
and it survives restarts.
|
||
- The payload carries a fingerprint of the password it was minted against, so
|
||
**rotating `ADMIN_PASSWORD` signs everyone out** — that is the revocation story.
|
||
- Login attempts are capped at 5/min per IP. Unlike the ingest worker, which never reads the
|
||
client IP at all, this one does — solely as a rate-limit key, never stored or logged.
|
||
|
||
## Deploy
|
||
|
||
Prereqs: the `getcodegraph.com` zone on the deploying Cloudflare account (the custom domain
|
||
auto-provisions DNS + cert), and the D1 database from `telemetry-worker/` already created.
|
||
|
||
```bash
|
||
cd telemetry-dashboard
|
||
npm install
|
||
npx wrangler login # once
|
||
|
||
npx wrangler secret put ADMIN_PASSWORD # the shared password
|
||
npx wrangler secret put SESSION_SECRET # cookie-signing key, e.g. `openssl rand -base64 48`
|
||
|
||
npm run deploy
|
||
```
|
||
|
||
Both secrets are required — the worker refuses every request if either is missing, so a
|
||
half-configured deployment fails closed rather than becoming an open dashboard.
|
||
|
||
The activation panels read `machine_first_seen.first_index_day`, which arrives with
|
||
`telemetry-worker/migrations/0002_first_index_day.sql`. Apply the ingest worker's
|
||
migrations and deploy it **before** this worker, or those two panels fail with a query
|
||
error (the rest of the page is unaffected) — see "Backfilling first_index_day" in
|
||
`telemetry-worker/README.md`.
|
||
|
||
Rotating either one is a `wrangler secret put` away. Rotating `SESSION_SECRET` invalidates
|
||
outstanding cookies too, and is the right move if you think one leaked.
|
||
|
||
Migrations belong to the writer, not to this worker: apply schema changes from
|
||
`telemetry-worker/` (`npm run db:migrate`). D1 is read-only here.
|
||
|
||
## Local dev & checks
|
||
|
||
```bash
|
||
cp .dev.vars.example .dev.vars # placeholder secrets; also feeds `wrangler types`
|
||
npm run check # vendor + wrangler types + tsc --noEmit + deploy --dry-run
|
||
npm run seed # load scripts/fixture.sql into the LOCAL D1
|
||
npm run dev # http://localhost:8787
|
||
|
||
npm run smoke:auth # the auth gate (54 assertions)
|
||
npm run smoke:api # the SQL and its numbers (98 assertions)
|
||
npm run smoke:render # the panels, in a browser (79 assertions)
|
||
```
|
||
|
||
Each suite starts its own throwaway `wrangler dev` on its own port and cleans up after
|
||
itself, so they can be run in any order (`DASH_PORT` overrides the port).
|
||
|
||
The repo's own test suite also runs `src/api.ts`, with the ingest worker feeding it, against
|
||
the writer's migrations in an in-memory SQLite database — no wrangler needed. From the repo
|
||
root (it is part of `npm test` there): `npx vitest run __tests__/telemetry-services.test.ts`.
|
||
|
||
**`smoke-auth.sh`** is the regression net for the gate: unauthenticated requests reach
|
||
nothing (pages, API *and* static assets), the cookie is persistent and correctly flagged,
|
||
flipped/truncated/forged cookies are all rejected, brute force is capped, and rotating the
|
||
password invalidates existing sessions. Run it after touching `src/auth.ts` or the route
|
||
table in `src/index.ts`.
|
||
|
||
**`smoke-api.sh`** checks every endpoint against `scripts/fixture.sql` — twelve machines
|
||
over ten days, listed machine by machine in that file's header, small enough that every
|
||
expected number was worked out by hand rather than recorded from a passing run. It also
|
||
covers the boring half: bad dims, malformed dates, backwards ranges and over-wide ranges.
|
||
|
||
**`render-check.mjs`** loads the real page in whatever Chromium is already on the machine
|
||
(over the DevTools protocol — no new dependency; it *skips* if there is no browser) and
|
||
reads the live Chart.js instance behind each canvas, comparing what every panel plotted
|
||
against the same endpoint fetched from Node. That is what catches a panel wired to the
|
||
wrong dimension, which neither of the other two suites can see. It also drives the range
|
||
picker and asserts a clean console, so a CSP regression fails the build.
|
||
`RENDER_SHOT=/tmp/dash.png npm run smoke:render` writes a full-page screenshot — the only
|
||
way to check the things assertions cannot, like label collisions.
|
||
|
||
## Frontend
|
||
|
||
Plain static files in `public/` — one HTML page, ES modules, no framework, no build step.
|
||
|
||
| File | Holds |
|
||
|---|---|
|
||
| `index.html` | The shell: masthead, the one filter row, an empty grid. |
|
||
| `panels.js` | The panel registry — data in, chart config out, no DOM. Adding a panel is one entry. |
|
||
| `theme.js` | Palette, formatters, and the Chart.js defaults every panel inherits. |
|
||
| `app.js` | The page: range picker, one fetch per panel, loading/empty/error states. |
|
||
|
||
The split is what lets `render-check.mjs` import the *same* registry the browser just
|
||
rendered from, so its expectations cannot drift from the panels under test.
|
||
|
||
Panels fail alone: each fetches, draws and reports independently, so a failed query leaves
|
||
the other eighteen on screen. There is no client-side cache — the only reuse is
|
||
deduplicating identical URLs within a single render (four stat tiles share one
|
||
`/api/summary`), and that map is discarded afterwards, so refresh really does re-ask.
|
||
A refetch dims the previous render rather than tearing it down, so nothing jumps. Every
|
||
chart has a "Show numbers" table twin, which is what keeps a value from being reachable
|
||
only by hovering.
|
||
|
||
### Colours
|
||
|
||
Two scales, both run through the data-viz validator against this dashboard's actual chart
|
||
surface (`#ffffff`, the panel fill) rather than picked by eye — the exact results are
|
||
recorded at the top of `theme.js`:
|
||
|
||
- **Categorical** `#a8342a #2a6f9e #17916a #c98500` — identity (which series). Slot 1 is
|
||
the brand oxblood stepped up into the legible lightness band. Clears every gate
|
||
including all-pairs colour-vision separation, with no contrast relief needed.
|
||
- **Ordinal** `#d99a90 #c26a5c #a3423a #7a201a` — one hue, light to dark, for scales whose
|
||
order *is* their meaning (run length, codebase size), so the ordering is visible in the
|
||
colour instead of needing the legend.
|
||
|
||
Nominal bars all take slot 1: colouring them by value would spend the identity channel
|
||
re-encoding what bar length already shows. If you change a hex, re-run the validator — the
|
||
red/green pair that "looks fine" is the one that collapses under deuteranopia.
|
||
Workers Static Assets serves them verbatim, so third-party libraries are copied out of
|
||
`node_modules` into `public/vendor/` by `npm run vendor` (wired into `dev` and `deploy`).
|
||
That keeps the version pinned by the lockfile, avoids a third-party origin at runtime, and
|
||
lets the CSP stay `script-src 'self'`. `public/vendor/` is gitignored — it is build output.
|
||
|
||
Visual conventions follow the rest of codegraph: flat and editorial, square corners, hairline
|
||
rules, sentence-case headings, one oxblood accent, no tiny all-caps tracked labels.
|