feat(mcp): steer agents to explore-first; fix Kotlin/Swift test detection (#191)

* feat(mcp): steer agents to explore-first; fix Kotlin/Swift test detection Two changes from diagnosing why Claude Code's Explore agent wasn't using codegraph_explore on a benchmark run (37 calls / ~90k tokens via search+Read+grep, vs a general-purpose agent that led with explore: 13 calls / ~55k tokens for the same question). 1. Tool guidance reframed across server-instructions.ts, instructions-template.ts, and .cursor/rules/codegraph.mdc (+ the explore/search tool descriptions): codegraph_explore is the workhorse for understanding/architecture/"how does X work" questions. Seed it with the key symbol names (a quick search/context first if the question names nothing concrete), read its output, and fill gaps with node/Read — instead of searching then Reading each file. The old "search first to find names, then explore" wording was short-circuiting: agents searched, got file:line locations, and Read them, never reaching explore. 2. isTestFile now recognizes Kotlin (*Test.kt, jvmTest/commonTest/ androidTest source sets), Swift (*Tests.swift), and other camelCase test conventions, so test code is deprioritized in explore/context ranking. Previously only Java/JS/Python were known, letting tests dominate Kotlin/Swift exploration (OkHttp "trace a request" went from 8/9 test files to surfacing Call.kt/OkHttpClient.kt/Request.kt/Response.kt). Capital-led matching keeps latest.kt/manifest.kt unflagged. An IDF common-term down-weighting was prototyped for the cold-query case but dropped — it was a measured no-op (the "common" terms weren't actually common in the test indexes); the test-detection gap was the real cause. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(agent-eval): add agent-behavior eval harness for codegraph MCP usage Tooling to measure how a Claude Code agent actually uses the codegraph MCP tools on a real repo — does it lead with codegraph_explore, how many Read/Grep follow-ups, token cost — for validating tool-guidance changes (server-instructions, tool descriptions) against real agent behavior. - itrun.sh drives the real interactive TUI via tmux (the faithful Explore path). Hardened for unattended runs: type-and-verify prompt delivery (the ❯ glyph is drawn ~6s before the input accepts keys), auto-accepts the "trust this folder" dialog, busy-detection keys on the universal "(Ns · …)" spinner so the pre-stream thinking phase counts as busy, and fails loudly instead of capturing an empty pane. - parse-session.mjs reports the tool breakdown + token accounting (gen / fresh-in / cached-in / billable) from the session and subagent logs, consistent across main-thread and subagent runs; counts main-thread Bash in the grep verdict. - run-agent.sh / parse-run.mjs are the headless stream-json complement (exact per-tool tokens/cost via claude -p). - run-interactive-test.md documents how to run it and how completion is detected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-10-18 13:53:01 +08:00 · 2026-05-19 22:10:31 -05:00
parent 8125798e68
commit 37cf566dcf
12 changed files with 533 additions and 43 deletions
@@ -13,22 +13,22 @@ Use codegraph for **structural** questions — what calls what, what would break

 | Question | Tool |
 |---|---|
+| "How does X work? / trace X / explain a system / architecture" | `codegraph_explore` (seed with symbol names) |
 | "Where is X defined?" / "Find symbol named X" | `codegraph_search` |
 | "What calls function Y?" | `codegraph_callers` |
 | "What does Y call?" | `codegraph_callees` |
 | "What would break if I changed Z?" | `codegraph_impact` |
 | "Show me Y's signature / source / docstring" | `codegraph_node` |
 | "Give me focused context for a task/area" | `codegraph_context` |
-| "Survey an unfamiliar module/topic" | `codegraph_explore` |
 | "What files exist under path/" | `codegraph_files` |
 | "Is the index healthy?" | `codegraph_status` |

 ### Rules of thumb

+- **`codegraph_explore` is the workhorse for understanding questions** ("how does X work", "trace…", "explain the Y system"). Feed it the key symbol/file names and read its output (line-numbered source from many files in one call). If the question names nothing concrete, do one quick `codegraph_search`/`codegraph_context` to surface the names, then explore with them. Fill gaps with `codegraph_node`/Read — don't grep-and-read your way through; that's the loop explore replaces.
+- **Delegating exploration to a subagent?** Tell it to call `codegraph_explore` first and trust the result. A generic "explore"-style agent defaults to grep+Read and treats codegraph as just a search index, throwing away the token savings.
 - **Trust codegraph results.** They come from a full AST parse. Do NOT re-verify them with grep — that's slower, less accurate, and wastes context.
 - **Don't grep first** when looking up a symbol by name. `codegraph_search` is faster and returns kind + location + signature in one call.
- **Don't chain `codegraph_search` + `codegraph_node`** when you just want context — `codegraph_context` is one call.
- **`codegraph_explore` is the heavy hitter** for unfamiliar areas — it returns full source from all relevant files in one call, but is token-heavy. If your harness supports parallel subagents (e.g., Claude Code's Task tool), spawn one for explore-class questions to keep main session context clean.
 - **Index lag**: the file watcher debounces ~500ms behind writes; don't re-query immediately after editing a file in the same turn.

 ### If `.codegraph/` doesn't exist
@@ -44,6 +44,17 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  VS Code ~12%. Agent-trust floor still holds — the Relationships section,
  scored cluster selection, and structured-source output are all retained.
  Thanks to [@essopsp](https://github.com/essopsp) for the repro.
+- **MCP / tool guidance**: the tool descriptions and installed instructions
+  now steer agents to treat `codegraph_explore` as the workhorse for
+  understanding/architecture/"how does X work" questions — seed it with the
+  key symbol names (a quick `codegraph_search`/`codegraph_context` first if
+  the question names nothing concrete) and read its output, rather than
+  searching and then Reading each file. Diagnosed from a benchmark run where
+  Claude Code's Explore agent used `codegraph_search` + Read + grep (37 tool
+  calls, ~90k tokens) and never called `codegraph_explore`, vs a
+  general-purpose agent that led with explore (13 calls, ~55k tokens) for the
+  same VS Code question. Updated in lockstep across `server-instructions.ts`,
+  `instructions-template.ts`, and `.cursor/rules/codegraph.mdc`.

 ### Fixed
 - **MCP**: source-omission markers in `codegraph_explore` and
@@ -51,6 +62,15 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  `... (trimmed) ...`, `... (truncated) ...`) instead of C-style `//`
  comments, which were misleading inside Python, Ruby, and other non-C
  fenced source blocks.
+- **Search/explore ranking**: test-file detection now recognizes Kotlin
+  (`*Test.kt`, `jvmTest/`/`commonTest/`/`androidTest/` source sets), Swift
+  (`*Tests.swift`), and other camelCase test conventions, so test code is
+  properly deprioritized in `codegraph_explore` / `codegraph_context`
+  results. Previously only Java/JS/Python conventions were known, which let
+  test files dominate exploration of Kotlin/Swift codebases (e.g. an OkHttp
+  "trace a request" query returned 8/9 test files; now it surfaces
+  `Call.kt`, `OkHttpClient.kt`, `Request.kt`, `Response.kt`). Capital-led
+  matching keeps production files like `latest.kt` / `manifest.kt` unflagged.

 ## [0.7.10] - 2026-05-19

@@ -0,0 +1,53 @@
+/**
+ * isTestFile heuristic — test-file detection used to deprioritize test code in
+ * search/explore ranking.
+ *
+ * Regression coverage for the cold-query fix: the heuristic previously only
+ * knew Java/JS/Python conventions, so Kotlin (`*Test.kt`, `jvmTest/`), Swift
+ * (`*Tests.swift`), and camelCase test source-set dirs slipped through — which
+ * let OkHttp's tests flood `codegraph_explore` results on a plain-language
+ * query. The false-positive guards matter just as much: `latest.kt` /
+ * `manifest.kt` / a `RealCall.kt` production file must NOT be flagged.
+ */
+import { describe, it, expect } from 'vitest';
+import { isTestFile } from '../src/search/query-utils';
+
+describe('isTestFile', () => {
+  it('flags Kotlin test files and source sets', () => {
+    expect(isTestFile('okhttp/src/jvmTest/kotlin/okhttp3/CallTest.kt')).toBe(true);
+    expect(isTestFile('okhttp/src/commonTest/kotlin/okhttp3/CompressionInterceptorTest.kt')).toBe(true);
+    expect(isTestFile('app/src/androidTest/java/com/example/FooTest.kt')).toBe(true);
+    expect(isTestFile('module/src/integrationTest/kotlin/BarSpec.kt')).toBe(true);
+  });
+
+  it('flags Swift test files', () => {
+    expect(isTestFile('Tests/SessionTests.swift')).toBe(true);
+    expect(isTestFile('Sources/FooTest.swift')).toBe(true);
+  });
+
+  it('still flags the previously-supported conventions', () => {
+    expect(isTestFile('foo/test_bar.py')).toBe(true);
+    expect(isTestFile('pkg/bar_test.go')).toBe(true);
+    expect(isTestFile('src/foo.test.ts')).toBe(true);
+    expect(isTestFile('src/foo.spec.ts')).toBe(true);
+    expect(isTestFile('com/example/FooTest.java')).toBe(true);
+    expect(isTestFile('com/example/FooTestCase.java')).toBe(true);
+    expect(isTestFile('project/__tests__/foo.ts')).toBe(true);
+    expect(isTestFile('project/tests/foo.rb')).toBe(true);
+  });
+
+  it('does NOT flag production files that merely contain "test" lowercase', () => {
+    // The fix is capital-led so camelCase boundaries distinguish these.
+    expect(isTestFile('src/latest/loader.kt')).toBe(false);
+    expect(isTestFile('lib/manifest.kt')).toBe(false);
+    expect(isTestFile('okhttp/src/jvmMain/kotlin/okhttp3/internal/connection/RealCall.kt')).toBe(false);
+    expect(isTestFile('src/contestEntry.ts')).toBe(false);
+    expect(isTestFile('pkg/greatest.go')).toBe(false);
+  });
+
+  it('does NOT flag ordinary production source', () => {
+    expect(isTestFile('src/flask/app.py')).toBe(false);
+    expect(isTestFile('src/vs/workbench/api/common/extensionHostMain.ts')).toBe(false);
+    expect(isTestFile('okhttp/src/commonJvmAndroid/kotlin/okhttp3/OkHttpClient.kt')).toBe(false);
+  });
+});
@@ -0,0 +1,131 @@
+# Running the agent-behavior test (how agents actually use codegraph)
+
+This explains how to measure **how a Claude Code agent uses the codegraph MCP
+tools** on a real repo — which tools it calls (does it lead with
+`codegraph_explore`?), how many follow-up `Read`/`Grep`s it does, and the token
+cost. Use it when changing tool guidance (`server-instructions.ts`,
+`instructions-template.ts`, tool descriptions) or retrieval, to verify the
+change actually shifts agent behavior.
+
+Scripts live in `scripts/agent-eval/`.
+
+## Why two harnesses (read this first)
+
+| | Interactive (`itrun.sh`) | Headless (`run-agent.sh`) |
+|---|---|---|
+| Drives | the real TUI via tmux | `claude -p` print mode |
+| Subagent it picks | **Explore** (matches real UX) | general-purpose (diverges) |
+| Metrics | tool breakdown (from session logs) + `Done(…)` token summary | exact per-tool calls + tokens/cost (stream-json) |
+| Cost | Claude Max subscription | API $ (`total_cost_usd`) |
+
+**Headless `claude -p` does NOT reproduce what users see** — it silently picks
+the general-purpose subagent, while interactive sessions delegate to the
+read-first **Explore** subagent. So for "what does my session actually do," use
+the interactive harness. For a clean per-tool/token breakdown in one shot, use
+headless (and ask for the Explore subagent in the prompt if you want that path).
+
+## Prerequisites
+
+- **tmux 3.0+**
+- A logged-in `claude` CLI (Claude Max or API).
+- codegraph configured as an MCP server (`claude mcp list` shows `codegraph`).
+  The interactive harness uses your global config, so it runs whatever
+  `codegraph` resolves to — point that at your dev build (`npm link` / the
+  symlinked global) to test local changes.
+- A target repo, cloned and indexed:
+  ```bash
+  git clone --depth 1 https://github.com/square/okhttp /tmp/corpus/okhttp
+  cd /tmp/corpus/okhttp && codegraph init -i
+  ```
+  Good scale spread for a sweep: Alamofire (~100 files), Excalidraw (~600),
+  OkHttp (~640), VS Code (~10k).
+
+## Interactive test (the faithful one)
+
+```bash
+scripts/agent-eval/itrun.sh <repo-path> <label> "<question>"
+```
+
+Example:
+```bash
+scripts/agent-eval/itrun.sh /tmp/corpus/vscode vscode \
+  "How does the extension host communicate with the main process?"
+```
+
+It opens `claude` in a tmux session, types the question, waits for the agent to
+finish, then prints:
+- the `Done (N tool uses · Xk tokens · Ym)` subagent summary (from the pane),
+- the `Context Xk/1.0M` main-session size,
+- a **tool breakdown** parsed from the session logs (main + subagents), ending
+  in a `VERDICT: codegraph_explore used Nx | Read N | Grep/Bash N` line.
+
+### Startup robustness (so unattended runs don't silently no-op)
+
+Two things bite an unattended driver before the prompt even runs:
+- **The `❯` glyph is drawn ~6s before the input accepts keystrokes.** Waiting
+  for `❯` is necessary but not sufficient. The harness sends the prompt, then
+  **verifies a chunk of it actually landed in the input box**, retrying until it
+  does — so it can't type into a not-yet-live input and submit nothing.
+- **First time claude opens a repo it shows "Is this a project you trust?"**
+  (which also contains `❯`). The harness detects that dialog and presses Enter
+  to accept it before typing.
+
+If the prompt never lands or work never starts, the harness now **fails loudly**
+(non-zero exit) instead of capturing an empty pane and reporting a bogus run.
+
+### How completion is detected (the tricky part)
+
+Claude's TUI redraws in place, so you can't just wait for output to stop. The
+harness polls `tmux capture-pane` and treats the pane as **busy** when it shows
+the spinner's elapsed-time-in-parens — `(8s · …)` / `(1m 3s · …)`, matched by
+`\(([0-9]+m )?[0-9]+s ·`. That's the *universal* working signal: it shows during
+the pre-stream **thinking** phase (`(8s · thinking with max effort)`, which has
+no token arrow yet) *and* during streaming. The `↓ N`/`↑ N` token arrow,
+`esc to interrupt`, and `Initializing…` are OR'd in as belt-and-braces (some TUI
+versions show one but not the others). It declares **idle** when the `❯` prompt
+is present and not busy for 10 consecutive polls (~5s, long enough to ride out
+mid-conversation thinking gaps that briefly drop the spinner). (Technique
+adapted from devpit's `WaitForIdle`.)
+
+### Where the breakdown comes from
+
+`parse-session.mjs` reads the newest session log under
+`~/.claude/projects/<escaped-cwd>/<session>.jsonl` and its subagent transcripts
+under `<session>/subagents/*.jsonl`. The **subagent** file is where the real
+tool calls are — the main log only shows the `Agent` delegation. You can run it
+standalone:
+```bash
+node scripts/agent-eval/parse-session.mjs /tmp/corpus/vscode
+```
+
+## Headless test (clean tokens, forceable Explore path)
+
+```bash
+scripts/agent-eval/run-agent.sh <repo-path> <label> "<question>"
+```
+Writes stream-json and prints the tool sequence + exact tokens/cost. To
+reproduce the Explore-subagent path headlessly, ask for it:
+`"Use an Explore subagent to investigate, then answer: …"`.
+
+## Running a sweep
+
+Single runs vary a lot (the VS Code question has ranged 26–37 tool uses /
+88–105k tokens across runs). For a real signal, run N≥3 and take the median:
+```bash
+for i in 1 2 3; do
+  scripts/agent-eval/itrun.sh /tmp/corpus/vscode "vscode-$i" "<question>"
+done
+```
+
+## What "good" looks like
+
+After the explore-first guidance (PR #191), an understanding question should
+show the agent **leading with `codegraph_explore`** and using `search`/`node`
+to fill gaps — not a wall of `Read`/`Grep`. Example faithful run:
+`VERDICT: codegraph_explore used 3x | Read 8 | Grep/Bash 1`. If `explore` is 0
+and `Read`/`Grep` dominate, the guidance regressed.
+
+## Output artifacts
+
+Transcripts and logs go to `$AGENT_EVAL_OUT` (default `/tmp/agent-eval/`):
+`itrun-<label>.txt` (pane capture), `run-<label>.jsonl` (headless stream-json).
@@ -0,0 +1,107 @@
+#!/usr/bin/env bash
+# Drive an INTERACTIVE Claude Code session in tmux, send a prompt, wait for the
+# agent to finish, then print the tool-call breakdown from the session logs.
+#
+# Why interactive (not `claude -p`): headless print-mode picks the
+# general-purpose subagent, while real interactive sessions delegate to the
+# Explore subagent (or drive codegraph from the main thread). Only the
+# interactive TUI reproduces the behavior users actually see. (Idle-detection
+# technique borrowed from devpit's WaitForIdle.)
+#
+# Usage: itrun.sh <repo-path> <label> "<prompt>"
+# Output dir: $AGENT_EVAL_OUT (default /tmp/agent-eval)
+# Requires: tmux 3.0+, a logged-in `claude` CLI, codegraph MCP configured.
+set -uo pipefail
+REPO="$1"; LABEL="$2"; PROMPT="$3"
+SESSION="cgt_${LABEL}"
+OUT_DIR="${AGENT_EVAL_OUT:-/tmp/agent-eval}"; mkdir -p "$OUT_DIR"
+OUT="$OUT_DIR/itrun-${LABEL}.txt"
+HERE="$(cd "$(dirname "$0")" && pwd)"
+
+cap() { tmux capture-pane -p -t "$SESSION" -S -40; }
+
+tmux kill-session -t "$SESSION" 2>/dev/null
+
+# Wide pane so the TUI doesn't hard-wrap tool lines.
+tmux new-session -d -s "$SESSION" -x 230 -y 60
+tmux send-keys -t "$SESSION" "cd $REPO && claude --dangerously-skip-permissions" Enter
+
+# Wait for the ❯ prompt (claude drew its UI), up to 60s. NOTE: ❯ appears on the
+# welcome screen seconds before the input actually accepts keystrokes, so this is
+# necessary but NOT sufficient — the type-and-verify loop below is what proves
+# the input is live.
+ready=0
+for _ in $(seq 1 120); do
+  cap | grep -q "❯" && { ready=1; break; }
+  sleep 0.5
+done
+[ "$ready" = 1 ] || { echo "claude never drew its UI"; cap; tmux kill-session -t "$SESSION" 2>/dev/null; exit 1; }
+
+# Accept the per-folder "Is this a project you trust?" dialog if it shows (first
+# time claude opens a given repo). Option 1 ("Yes, I trust this folder") is
+# pre-selected, so Enter accepts. This dialog also contains ❯, so it must be
+# cleared before the type-and-verify loop or keystrokes land on the menu.
+for _ in $(seq 1 20); do
+  cap | grep -q "trust this folder" || break
+  tmux send-keys -t "$SESSION" Enter
+  sleep 1
+done
+
+# Type-and-verify: send the prompt, confirm a distinctive chunk of it actually
+# landed in the input box, retry if it didn't (handles the early-❯ race where
+# the welcome screen shows the prompt glyph but MCP init is still eating keys).
+needle="${PROMPT:0:24}"
+typed=0
+for _ in $(seq 1 30); do
+  tmux send-keys -l -t "$SESSION" "$PROMPT"
+  sleep 1
+  if cap | grep -Fq "$needle"; then typed=1; break; fi
+  # Clear whatever partial text may have landed, then retry.
+  tmux send-keys -t "$SESSION" C-u
+  sleep 1
+done
+[ "$typed" = 1 ] || { echo "prompt never landed in the input box"; cap; tmux kill-session -t "$SESSION" 2>/dev/null; exit 1; }
+sleep 0.5
+tmux send-keys -t "$SESSION" Enter
+
+# Busy signals. The robust one is the spinner's elapsed-time-in-parens, which
+# EVERY working state shows — both the pre-stream thinking phase
+# "(8s · thinking with max effort)" and the streaming phase
+# "(24s · ↑ 2.5k tokens · …)", and it survives the 32s→"1m 3s" rollover. We OR
+# in the token arrows, "esc to interrupt", and "Initializing" as belt-and-braces
+# (some TUI versions/states show one but not the others).
+BUSY_RE='esc to interrupt|↓ [0-9]|↑ [0-9]|Initializing|\(([0-9]+m )?[0-9]+s ·'
+
+# Wait for work to START (busy indicator appears), up to 60s. If it never starts,
+# fail loudly rather than silently reporting an empty run.
+started=0
+for _ in $(seq 1 120); do
+  cap | grep -qE "$BUSY_RE" && { started=1; break; }
+  sleep 0.5
+done
+[ "$started" = 1 ] || { echo "agent never started working"; cap; tmux kill-session -t "$SESSION" 2>/dev/null; exit 1; }
+
+# Poll for idle: not busy AND ❯ present, for 10 consecutive polls (~5s) to ride
+# out mid-conversation thinking gaps that briefly drop the spinner. Up to ~15min.
+consec=0
+for _ in $(seq 1 1800); do
+  pane=$(cap)
+  if echo "$pane" | grep -qE "$BUSY_RE"; then
+    consec=0
+  elif echo "$pane" | grep -q "❯"; then
+    consec=$((consec+1)); [ "$consec" -ge 10 ] && break
+  else
+    consec=0
+  fi
+  sleep 0.5
+done
+sleep 1
+
+tmux capture-pane -p -t "$SESSION" -S - > "$OUT"
+echo "captured $(wc -l < "$OUT") lines -> $OUT"
+grep -oE "Done \([^)]*\)" "$OUT" | tail -1
+grep -oE "[0-9.]+k?/[0-9.]+M" "$OUT" | tail -1 | sed 's/^/Context /'
+tmux kill-session -t "$SESSION" 2>/dev/null
+
+# Clean tool breakdown from the session logs (main + subagents).
+node "$HERE/parse-session.mjs" "$REPO" 2>/dev/null || true
@@ -0,0 +1,45 @@
+#!/usr/bin/env node
+// Parse a Claude Code stream-json run log: tool-call sequence + token usage.
+import { readFileSync } from 'fs';
+const file = process.argv[2];
+const lines = readFileSync(file, 'utf8').split('\n').filter(Boolean);
+
+const toolCalls = [];
+let result = null;
+let initTools = null;
+
+for (const line of lines) {
+  let ev;
+  try { ev = JSON.parse(line); } catch { continue; }
+  if (ev.type === 'system' && ev.subtype === 'init') {
+    initTools = (ev.tools || []).filter(t => /codegraph/.test(t));
+  }
+  if (ev.type === 'assistant' && ev.message?.content) {
+    for (const block of ev.message.content) {
+      if (block.type === 'tool_use') {
+        let detail = '';
+        if (block.name === 'Task') detail = ` [subagent_type=${block.input?.subagent_type ?? '?'}] ${(block.input?.description ?? '').slice(0,40)}`;
+        else if (/codegraph/.test(block.name)) detail = ` ${JSON.stringify(block.input?.query ?? block.input?.task ?? block.input?.symbol ?? '').slice(0,60)}`;
+        else if (block.name === 'Bash') detail = ` ${(block.input?.command ?? '').slice(0,50)}`;
+        else if (block.name === 'Read') detail = ` ${(block.input?.file_path ?? '').split('/').slice(-1)[0]}`;
+        toolCalls.push(`${block.name}${detail}`);
+      }
+    }
+  }
+  if (ev.type === 'result') result = ev;
+}
+
+console.log(`\n=== ${file.split('/').pop()} ===`);
+console.log(`codegraph tools exposed: ${initTools ? initTools.length : '?'}`);
+console.log(`\nTool calls (${toolCalls.length}):`);
+const counts = {};
+for (const tc of toolCalls) { const n = tc.split(' ')[0]; counts[n] = (counts[n]||0)+1; }
+console.log('  by type:', JSON.stringify(counts));
+toolCalls.forEach((tc, i) => console.log(`  ${i+1}. ${tc}`));
+
+if (result) {
+  const u = result.usage || {};
+  const totalIn = (u.input_tokens||0) + (u.cache_read_input_tokens||0) + (u.cache_creation_input_tokens||0);
+  console.log(`\nResult: ${result.subtype} | duration ${(result.duration_ms/1000).toFixed(0)}s | turns ${result.num_turns}`);
+  console.log(`  tokens: in=${totalIn} out=${u.output_tokens||0} | cost $${(result.total_cost_usd||0).toFixed(3)}`);
+}
@@ -0,0 +1,93 @@
+#!/usr/bin/env node
+// Parse the newest Claude Code session log for a project + its subagent logs,
+// and report the tool-call breakdown (main + subagents). Works for interactive
+// runs (driven via itrun.sh) — Claude Code writes full transcripts to
+// ~/.claude/projects/<escaped-cwd>/<session>.jsonl with subagents/ alongside.
+import { readFileSync, readdirSync, statSync, existsSync, realpathSync } from 'fs';
+import { join } from 'path';
+import { homedir } from 'os';
+
+const projectArg = process.argv[2];
+if (!projectArg) { console.error('usage: parse-session.mjs <project-dir>'); process.exit(1); }
+
+// Claude Code escapes the (real) cwd by replacing every "/" with "-".
+const real = realpathSync(projectArg);
+const escaped = real.replace(/\//g, '-');
+const projDir = join(homedir(), '.claude', 'projects', escaped);
+if (!existsSync(projDir)) { console.error('no session logs at', projDir); process.exit(1); }
+
+// Newest top-level session .jsonl
+const sessions = readdirSync(projDir)
+  .filter(f => f.endsWith('.jsonl'))
+  .map(f => ({ f, m: statSync(join(projDir, f)).mtimeMs }))
+  .sort((a, b) => b.m - a.m);
+if (sessions.length === 0) { console.error('no .jsonl sessions in', projDir); process.exit(1); }
+const sessionId = sessions[0].f.replace('.jsonl', '');
+
+function tally(file) {
+  const counts = {};
+  for (const line of readFileSync(file, 'utf8').split('\n')) {
+    if (!line) continue;
+    let ev; try { ev = JSON.parse(line); } catch { continue; }
+    const content = ev.message?.content;
+    if (!Array.isArray(content)) continue;
+    for (const b of content) {
+      if (b.type === 'tool_use') counts[b.name] = (counts[b.name] || 0) + 1;
+    }
+  }
+  return counts;
+}
+
+// Sum token usage from a transcript. The TUI's "Done (…Xk tokens…)" line only
+// covers a subagent's throughput; this works for main-thread runs too and is
+// consistent across both paths. `gen` = output, `fresh` = uncached input
+// (input + cache_creation), `cached` = cache reads (≈free), `total` = all.
+function sumTokens(file) {
+  const t = { gen: 0, fresh: 0, cached: 0 };
+  for (const line of readFileSync(file, 'utf8').split('\n')) {
+    if (!line) continue;
+    let ev; try { ev = JSON.parse(line); } catch { continue; }
+    const u = ev.message?.usage;
+    if (!u) continue;
+    t.gen += u.output_tokens || 0;
+    t.fresh += (u.input_tokens || 0) + (u.cache_creation_input_tokens || 0);
+    t.cached += u.cache_read_input_tokens || 0;
+  }
+  return t;
+}
+
+const mainCounts = tally(join(projDir, sessionId + '.jsonl'));
+
+// Subagent transcripts live under <session>/subagents/*.jsonl
+const subDir = join(projDir, sessionId, 'subagents');
+const subCounts = {};
+let subAgentFiles = 0;
+if (existsSync(subDir)) {
+  for (const f of readdirSync(subDir).filter(f => f.endsWith('.jsonl'))) {
+    subAgentFiles++;
+    const c = tally(join(subDir, f));
+    for (const [k, v] of Object.entries(c)) subCounts[k] = (subCounts[k] || 0) + v;
+  }
+}
+
+const fmt = (counts) => Object.entries(counts).sort((a, b) => b[1] - a[1])
+  .map(([k, v]) => `    ${String(v).padStart(3)}  ${k}`).join('\n') || '    (none)';
+
+console.log(`session: ${sessionId}`);
+console.log(`\nMAIN thread tools:\n${fmt(mainCounts)}`);
+console.log(`\nSUBAGENT tools (${subAgentFiles} subagent transcript${subAgentFiles === 1 ? '' : 's'}):\n${fmt(subCounts)}`);
+
+const explore = subCounts['mcp__codegraph__codegraph_explore'] || mainCounts['mcp__codegraph__codegraph_explore'] || 0;
+const reads = (subCounts['Read'] || 0) + (mainCounts['Read'] || 0);
+const greps = (subCounts['Grep'] || 0) + (mainCounts['Grep'] || 0) + (subCounts['Bash'] || 0) + (mainCounts['Bash'] || 0);
+console.log(`\nVERDICT: codegraph_explore used ${explore}x | Read ${reads} | Grep/Bash ${greps}`);
+
+// Token totals (main + subagents), consistent across main-thread and subagent runs.
+const tok = { gen: 0, fresh: 0, cached: 0 };
+const addTok = (t) => { tok.gen += t.gen; tok.fresh += t.fresh; tok.cached += t.cached; };
+addTok(sumTokens(join(projDir, sessionId + '.jsonl')));
+if (existsSync(subDir)) {
+  for (const f of readdirSync(subDir).filter(f => f.endsWith('.jsonl'))) addTok(sumTokens(join(subDir, f)));
+}
+const k = (n) => (n / 1000).toFixed(1) + 'k';
+console.log(`TOKENS: gen ${k(tok.gen)} | fresh-in ${k(tok.fresh)} | cached-in ${k(tok.cached)} | billable≈ ${k(tok.gen + tok.fresh)}`);
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# Headless Claude Code run against a repo with codegraph MCP, capturing the
+# full stream-json so we can see tool calls + token usage. Complements the
+# interactive itrun.sh: headless gives a clean per-tool breakdown + exact
+# tokens/cost, but defaults to the general-purpose subagent (not Explore).
+# To force the Explore path, ask for it in the prompt.
+#
+# Usage: run-agent.sh <repo-path> <label> "<prompt>"
+# Env: AGENT_EVAL_OUT (default /tmp/agent-eval), CG_BIN (codegraph dist binary)
+set -uo pipefail
+
+REPO="$1"; LABEL="$2"; PROMPT="$3"
+CG_BIN="${CG_BIN:-$(command -v codegraph || echo /usr/local/bin/codegraph)}"
+OUT_DIR="${AGENT_EVAL_OUT:-/tmp/agent-eval}"; mkdir -p "$OUT_DIR"
+OUT="$OUT_DIR/run-${LABEL}.jsonl"
+
+MCP_CONFIG=$(cat <<JSON
+{"mcpServers":{"codegraph":{"command":"${CG_BIN}","args":["serve","--mcp","--path","${REPO}"]}}}
+JSON
+)
+
+echo "→ running [$LABEL] in $REPO"
+cd "$REPO" || exit 1
+
+claude -p "$PROMPT" \
+  --output-format stream-json --verbose \
+  --permission-mode bypassPermissions \
+  --model opus \
+  --max-budget-usd 2 \
+  --strict-mcp-config --mcp-config "$MCP_CONFIG" \
+  > "$OUT" 2>"$OUT_DIR/run-${LABEL}.err"
+
+echo "exit: $? | wrote $OUT ($(wc -l < "$OUT") lines)"
+node "$(cd "$(dirname "$0")" && pwd)/parse-run.mjs" "$OUT" 2>/dev/null || true
@@ -31,22 +31,22 @@ Use codegraph for **structural** questions — what calls what, what would break

 | Question | Tool |
 |---|---|
+| "How does X work? / trace X / explain a system / architecture" | \`codegraph_explore\` (seed with symbol names) |
 | "Where is X defined?" / "Find symbol named X" | \`codegraph_search\` |
 | "What calls function Y?" | \`codegraph_callers\` |
 | "What does Y call?" | \`codegraph_callees\` |
 | "What would break if I changed Z?" | \`codegraph_impact\` |
 | "Show me Y's signature / source / docstring" | \`codegraph_node\` |
 | "Give me focused context for a task/area" | \`codegraph_context\` |
-| "Survey an unfamiliar module/topic" | \`codegraph_explore\` |
 | "What files exist under path/" | \`codegraph_files\` |
 | "Is the index healthy?" | \`codegraph_status\` |

 ### Rules of thumb

+- **\`codegraph_explore\` is the workhorse for understanding questions** ("how does X work", "trace…", "explain the Y system"). Feed it the key symbol/file names and read its output (line-numbered source from many files in one call). If the question names nothing concrete, do one quick \`codegraph_search\`/\`codegraph_context\` to surface the names, then explore with them. Fill gaps with \`codegraph_node\`/Read — don't grep-and-read your way through; that's the loop explore replaces.
+- **Delegating exploration to a subagent?** Tell it to call \`codegraph_explore\` first and trust the result. A generic "explore"-style agent defaults to grep+Read and treats codegraph as just a search index, throwing away the token savings.
 - **Trust codegraph results.** They come from a full AST parse. Do NOT re-verify them with grep — that's slower, less accurate, and wastes context.
 - **Don't grep first** when looking up a symbol by name. \`codegraph_search\` is faster and returns kind + location + signature in one call.
- **Don't chain \`codegraph_search\` + \`codegraph_node\`** when you just want context — \`codegraph_context\` is one call.
- **\`codegraph_explore\` is the heavy hitter** for unfamiliar areas — it returns full source from all relevant files in one call, but is token-heavy. If your harness supports parallel subagents (e.g., Claude Code's Task tool), spawn one for explore-class questions to keep main session context clean.
 - **Index lag**: the file watcher debounces ~500ms behind writes; don't re-query immediately after editing a file in the same turn.

 ### If \`.codegraph/\` doesn't exist
@@ -24,27 +24,27 @@ editing code, not during.

 ## Tool selection by intent

- **"What is the symbol named X?"** → \`codegraph_search\`
- **"What's the deal with this task / feature / area?"** → \`codegraph_context\` (PRIMARY — composes search + node + callers + callees in one call)
+- **"How does X work? / trace X end to end / explain the Y system / architecture?"** → \`codegraph_explore\` (PRIMARY for understanding — seed it with the key symbol names, read its output, don't grep+Read your way there)
+- **"What is the symbol named X? / where is X defined?"** → \`codegraph_search\` (pinpoint lookups)
+- **"What's the deal with this task / feature / area?"** → \`codegraph_context\` (lighter composed view of search + node + callers + callees)
 - **"What calls this?"** → \`codegraph_callers\`
 - **"What does this call?"** → \`codegraph_callees\`
 - **"What would changing this break?"** → \`codegraph_impact\`
 - **"Show me this symbol's source / signature / docstring."** → \`codegraph_node\`
- **"Survey an unfamiliar topic / pattern / module."** → \`codegraph_explore\` (heavier; deep dive)
 - **"What's in directory X?"** → \`codegraph_files\`
 - **"Is the index ready / what's its size?"** → \`codegraph_status\`

 ## Common chains

- **Onboarding**: \`codegraph_context\` first. If still unclear, \`codegraph_explore\` for breadth, then \`codegraph_node\` on specific symbols.
+- **Understanding / onboarding**: feed \`codegraph_explore\` the key symbol/file names and read its output (line-numbered source from many files in one call). If the question names nothing concrete, do ONE quick \`codegraph_search\` / \`codegraph_context\` to surface the names, then explore with them. Fill remaining gaps with \`codegraph_node\` / Read — don't drop back to grep+Read for the whole topic.
 - **Refactor planning**: \`codegraph_search\` → \`codegraph_callers\` → \`codegraph_impact\`. The blast-radius answer comes from impact, not from walking callers manually.
 - **Debugging a regression**: \`codegraph_callers\` of the suspected symbol; widen with \`codegraph_impact\` if an unexpected call appears.

 ## Anti-patterns

+- **Don't search-then-Read your way through an understanding question** — feed the names you find into \`codegraph_explore\` instead of Reading the files one by one; it does that whole loop in one call and returns line numbers you can cite directly.
 - **Don't grep first** when looking up a symbol by name — \`codegraph_search\` is faster and returns kind + location + signature.
- **Don't chain \`codegraph_search\` + \`codegraph_node\`** when you just want context — \`codegraph_context\` is one round-trip.
- **Don't use \`codegraph_explore\` for narrow questions** — it's a multi-call deep dive, expensive in tokens. Save it for genuine "I'm new here" surveys.
+- **Don't reach for \`codegraph_explore\` on a pinpoint "where is X defined" lookup** — \`codegraph_search\` is one cheap call.
 - **Don't query the index immediately after editing a file** — the watcher needs ~500ms to debounce + sync. Wait for the next turn.

 ## Limitations
@@ -238,7 +238,7 @@ const projectPathProperty: PropertySchema = {
 export const tools: ToolDefinition[] = [
  {
    name: 'codegraph_search',
-    description: 'Quick symbol search by name. Returns locations only (no code). Use codegraph_context instead for comprehensive task context.',
+    description: 'Quick symbol search by name. Returns locations only (no code) — best for pinpoint "where is X defined / find the symbol named X" lookups. For understanding how something works or tracing a flow, lead with codegraph_explore instead of searching then reading.',
    inputSchema: {
      type: 'object',
      properties: {
@@ -368,13 +368,13 @@ export const tools: ToolDefinition[] = [
  },
  {
    name: 'codegraph_explore',
-    description: 'Deep exploration tool — returns comprehensive context for a topic in a SINGLE call. Groups all relevant source code by file (contiguous sections, not snippets), includes a relationship map, and uses deeper graph traversal. Designed to replace multiple codegraph_node + file Read calls. Use this instead of codegraph_context when you need thorough understanding. IMPORTANT: Use specific symbol names, file names, or short code terms in your query — NOT natural language sentences. Before calling this, use codegraph_search to discover relevant symbol names, then include those names in your query. Bad: "how are agent prompts loaded and passed to the CLI". Good: "readAgentsFromDirectory createClaudeSession chat-manager agents.ts".',
+    description: 'PRIMARY TOOL for understanding questions — "how does X work", "trace X end to end", "explain the Y system", architecture/onboarding. Returns comprehensive context in a SINGLE call: relevant source grouped by file (contiguous, line-numbered sections, not snippets) + a relationship map + deep graph traversal. It REPLACES the grep+Read exploration loop: feed it the key symbol/file names and read its output — do NOT Read the files one by one. It works best when your query names the relevant symbols (e.g. "readAgentsFromDirectory createClaudeSession chat-manager agents.ts"); if the question is a plain sentence that names nothing concrete, do ONE quick codegraph_search or codegraph_context to surface the names, then call this with them. After exploring, use codegraph_node / Read only to fill specific gaps it did not cover. Prefer codegraph_search over this only for a pinpoint "where is X defined" lookup.',
    inputSchema: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
-          description: 'Symbol names, file names, or short code terms to explore (e.g., "AuthService loginUser session-manager", "GraphTraverser BFS impact traversal.ts"). Use codegraph_search first to find relevant names.',
+          description: 'What to explore. A short list of symbol/file/keyword terms works best (e.g., "AuthService loginUser session-manager", "GraphTraverser BFS impact traversal.ts"), but a plain-language phrase also works — the tool runs its own retrieval. No need to codegraph_search first.',
        },
        maxFiles: {
          type: 'number',
@@ -207,36 +207,43 @@ export function scorePathRelevance(filePath: string, query: string): number {
 */
 export function isTestFile(filePath: string): boolean {
  const lower = filePath.toLowerCase();
-  const fileName = path.basename(lower);
+  const fileName = path.basename(filePath);   // original case — needed for camelCase boundaries
+  const lowerName = fileName.toLowerCase();

-  // Common test file patterns
-  return (
-    fileName.startsWith('test_') ||
-    fileName.startsWith('test.') ||
-    fileName.endsWith('.test.ts') ||
-    fileName.endsWith('.test.js') ||
-    fileName.endsWith('.test.tsx') ||
-    fileName.endsWith('.test.jsx') ||
-    fileName.endsWith('.spec.ts') ||
-    fileName.endsWith('.spec.js') ||
-    fileName.endsWith('_test.go') ||
-    fileName.endsWith('_test.py') ||
-    fileName.endsWith('_test.rs') ||
-    fileName.endsWith('Tests.java') ||
-    fileName.endsWith('Test.java') ||
-    fileName.endsWith('Tester.java') ||
-    fileName.endsWith('TestCase.java') ||
-    lower.includes('/tests/') ||
-    lower.includes('/test/') ||
-    lower.includes('/__tests__/') ||
-    lower.includes('/spec/') ||
-    lower.includes('/testlib/') ||
+  // --- Filename patterns ---
+  if (
+    lowerName.startsWith('test_') ||                              // python: test_foo.py
+    lowerName.startsWith('test.') ||
+    // separator-delimited: foo_test.go, foo.test.ts, foo-spec.rb, bar_spec.py
+    /[._-](test|tests|spec|specs)\.[a-z0-9]+$/.test(lowerName) ||
+    // CamelCase suffix (Java/Kotlin/Swift/C#/Scala): FooTest.kt, BarTests.swift,
+    // BazSpec.scala, QuxTestCase.java. Capital-led so "latest.kt"/"manifest.kt"
+    // (lowercase "test") are NOT matched.
+    /(?:Test|Tests|TestCase|Tester|Spec|Specs)\.[A-Za-z0-9]+$/.test(fileName)
+  ) {
+    return true;
+  }
+
+  // --- Directory patterns ---
+  if (
+    lower.includes('/tests/') || lower.includes('/test/') ||
+    lower.includes('/__tests__/') || lower.includes('/spec/') ||
+    lower.includes('/specs/') || lower.includes('/testlib/') ||
    lower.includes('/testing/') ||
-    // Non-production directories: examples, samples, benchmarks, fixtures, demos.
-    // Check both mid-path (/integration/) and start-of-path (integration/) since
-    // file paths may be stored as relative paths without a leading slash.
-    matchesNonProductionDir(lower)
-  );
+    lower.startsWith('test/') || lower.startsWith('tests/') ||
+    lower.startsWith('spec/') || lower.startsWith('specs/') ||
+    // CamelCase test source-set dirs (Kotlin Multiplatform / Gradle / Xcode):
+    // jvmTest/, commonTest/, androidTest/, iosTest/, integrationTest/. Capital-led
+    // so "latest/" / "manifest/" are not matched.
+    /(?:^|\/)[A-Za-z0-9]*(?:Test|Tests|Spec)\//.test(filePath)
+  ) {
+    return true;
+  }
+
+  // Non-production directories: examples, samples, benchmarks, fixtures, demos.
+  // Check both mid-path (/integration/) and start-of-path (integration/) since
+  // file paths may be stored as relative paths without a leading slash.
+  return matchesNonProductionDir(lower);
 }

 /**