12 Commits

Author SHA1 Message Date
Colby Mchenry 2268a09b24 fix(sync): recover index freshness after lock contention (#1959) (#2043)
* fix(lock): keep a live writer lock regardless of file age (#1959)

* fix(watcher): re-arm after lock contention and reconcile before trusting index (#1959)

* feat(mcp): report exact index freshness without blocking the daemon (#1959)

* fix(mcp): refuse drifted explore source after watcher degradation (#1959)

* fix(mcp): refuse graph answers that name a drifted file after watcher degradation (#1959)

Explore already refuses to render source from a file that changed while
auto-sync was off. search, callers, callees and impact answer from the same
frozen graph but kept replying, naming the changed file as if its edges were
current. They now take the same refusal: the indexed files an answer names
as locations are hashed against the index, and a drifted one is named
instead of the answer. codegraph_node keeps its own drift gate, which serves
the changed file's current bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(sync): recover index freshness after lock contention (#1959)

Keep a live writer's lock regardless of its age.
Re-arm lock-degraded watchers with throttling and retain stale state until full reconciliation succeeds.
Validate contributing files through structured paths and hashes, refusing incomplete checks without tool errors.
Report bounded freshness measurements and cover Git fallback exclusions.

Co-authored-by: danusha2345 <danusha2345@users.noreply.github.com>
Co-authored-by: danusha2345 <ewidusoc498@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: danusha2345 <danusha2345@users.noreply.github.com>
Co-authored-by: danusha2345 <ewidusoc498@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 14:16:48 +00:00
Colby McHenry 16e17495f4 feat(extraction): content-based generated-file detection (CG-5, #1500)
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT
marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one.
A Go monorepo with generated CRUD in ordinarily-named files sitting beside
hand-written use-cases was therefore invisible to every generated-file
down-rank in the codebase — that is #1500.

Measured on kubernetes/client-go (2,453 Go files): the canonical banner
appears in 2,001 of them, the path check flags 0, the new content check
flags exactly those 2,001 — no false positives, no misses.

Design: decide at INDEX time (content is already in memory for parsing),
persist on `files.generated`, read from the DB. Explore never reads file
headers per request.

- `hasGeneratedHeader(content)` recognizes the standard banners — Go's,
  protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator,
  FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an
  8KB/60-line header window, a comment-line requirement (leader or open
  block comment), and markers tight enough that prose can't trip them. A
  generator's own source, holding the banner as a string constant in its
  body, is not flagged; neither is this module itself (pinned by test).
- `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback.
- Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no
  backfill: the flag derives from content the migration cannot see, so
  rows stay 0 until a re-index and every reader unions the flag with the
  path check — an un-migrated index keeps pre-#1500 behavior rather than
  regressing. Re-index required; noted in the CHANGELOG.
- `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1)
  lookups. Bounded, not cached: no invalidation, so a ranking call can
  never serve a verdict the last sync already replaced. Wired into explore
  ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the
  context formatter, and the dominant-file/route-file hygiene filters.

Cost (acceptance bar was no measurable index-time regression): a single
unanchored `/generat/i` test over the header rejects ~every hand-written
file before any line splitting. 4.6 µs/file on client-go (worst case —
82% generated). End-to-end `codegraph init` on client-go, n=3 alternating
arms: 5.73s median with detection vs 5.76s path-only baseline; the arms
cross over between runs, so the difference is inside run-to-run noise.

Scope note: generated status remains a stable TIEBREAK at equal score,
exactly where it was. Making it a strong negative signal is CG-10, which
this unblocks by making the signal correct and available.

Two pre-existing tests hard-coded schema version 8; both now track
CURRENT_SCHEMA_VERSION (or the migration table) so future migrations
don't require editing them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 23:13:59 -05:00
Colby Mchenry 6459ead6aa fix(extraction): index files reached through in-root symlinks that point outside the repo (#935) (#956)
The directory walk deliberately follows an in-root symlink whose target
lives outside the repo root (the standard Dota custom-game layout, where
`game/` and `content/` link into the SDK tree) and enumerates the files
under it. But the read path then rejected every one of them via the
strict symlink-escape guard, logging `Path traversal blocked in batch
reader` and indexing nothing — discovery and the reader disagreed.

Add an opt-in `allowSymlinkEscape` to validatePathWithinRoot that waives
only the realpath-escape rejection (the lexical `../` guard still
applies) and pass it at the three indexing read sites (batch reader,
indexFile, indexFileWithContent). The content-serving sinks
(ContextBuilder, MCP tools) keep the strict guard, so this stays inside
the #527 model: indexing now follows the symlink, getCode still refuses
to serve out-of-root contents.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 17:23:17 -05:00
Colby Mchenry 7fd8b4c185 fix(security): resolve symlinks in path validation to block out-of-root reads (#527) (#724)
* fix(security): resolve symlinks in path validation to block out-of-root reads (#527)

validatePathWithinRoot was purely lexical (path.resolve + startsWith), so an
in-repo symlink whose logical path is inside the project root but whose real
target escapes it passed validation — and both content-serving read sinks
(codegraph_node includeCode, codegraph_explore source) then readFileSync'd it,
leaking out-of-root file contents (e.g. ~/.ssh, /etc) to the agent.

Add a realpath layer: after the lexical check, resolve symlinks on both the
candidate path and the root and re-compare, rejecting anything whose real path
escapes the root. An in-root symlink is still allowed (no over-blocking).
Comparison is case-insensitive on Windows (NTFS + realpath casing). Not-yet-
existing paths (ENOENT) fall back to the lexical result so about-to-be-written
files still validate; other resolution errors reject.

Removes the dead, never-called isPathWithinRoot / isPathWithinRootReal helpers
(the latter a footgun — it returned true on realpath failure). Adds RED->GREEN
tests: in->out file/dir symlinks rejected, in->in allowed, ../ rejected, ENOENT
allowed, plus an end-to-end test proving getCode no longer serves an out-of-root
file reached through a dir symlink.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): note the #527 symlink path-escape fix

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 01:02:34 -04:00
Colby Mchenry 68eaf0dbd8 feat(mcp): codegraph_explore as the sole primary tool + store coverage + overload disambiguation (#647)
## Summary

Completes the explore-overhaul arc: `codegraph_explore` becomes the single primary tool an agent reaches for, and its coverage + output shape are tuned so flow/architecture questions resolve with near-zero Read/Grep.

### What changed
- **explore is the sole primary tool** — removed `codegraph_context` (the fuzzy-input Read-trigger) and `codegraph_trace` (under-picked by agents); explore already surfaces the call flow among the symbols you name. A plain natural-language question now works as the query.
- **Store/handler coverage** — functions defined inside object literals (Zustand `create((set, get) => ({ … }))`, Redux/Pinia/MobX, exported handler/route maps) are indexed as real symbols, including calls through `useStore.getState().fn()` and destructured `const { fn } = useStore.getState()`. A general AST rule, not a per-lib hack.
- **Overload disambiguation** — explore leads with the *right* definition when a method name is overloaded across types (a PascalCase type token in the query biases to that type's own def); `codegraph_node` returns *every* overload's body in one call, with an optional `file`/`line` selector to pin one.
- **Method-atomic render** — explore never returns half a method; at the size budget it drops whole methods/files (and lists what it dropped) instead of truncating a body mid-method.
- **Native-read-shaped output** — per-call output is capped to ~24K with a 25K hard ceiling and concentrated into ~150–250-line flow windows, mirroring how the agent natively reads; repo size scales the *call* budget, not the per-call size (a larger response just gets externalized to a file the host Reads back).
- **Blast radius** folded into explore (dependents + covering tests, locations only).

### Benchmark (refreshed on this build)
Re-validated the 7-repo A/B on 2026-06-02 (Opus 4.8, effort=high, median of 4). WITH arm re-measured on this build, WITHOUT reused:

**~16% cheaper · 47% fewer tokens · 22% faster · 58% fewer tool calls** — 0 file reads on 6 of 7 repos (Gin ~1).

The arc trades larger, cache-heavy explore responses for guaranteed near-zero reads, so cost/token margins soften vs the prior build (Excalidraw and Tokio land at cost break-even) while time and tool-calls stay clear wins everywhere — consistent with the project's stated optimization target (latency + tool-calls, not token cost).

### Validation
- Full suite green: **1112 passed, 2 skipped**.
- 28/28 plain WITH runs across the 7 README repos completed clean; reads median 0 on 6/7.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-02 10:15:27 -05:00
Aditya Rawat 02ea482b37 fix: validate projectPath in MCP handler to block sensitive directories (#230)
Validate projectPath in getCodeGraph so MCP clients can't open a codegraph in a
sensitive system directory. Guarded with existsSync so nested/not-yet-created
sub-paths still resolve up to the default project (preserves issue #238). Adds
MCP-handler rejection tests (POSIX + Windows-gated); validated on a real
Windows 11 VM.

Closes #230
2026-05-22 14:15:02 -05:00
Leon.C 7d5dd4cda7 fix: remove dead try/catch in insertNode; fix SENSITIVE_PATHS case-sensitivity (#327)
Drop the no-op try/catch around insertNode.run, and lowercase the Windows
SENSITIVE_PATHS entries so validateProjectPath's case-insensitive check
actually blocks c:\windows. Adds a validateProjectPath test (POSIX +
Windows-gated); the Windows-gated case was validated on a real Windows 11 VM.

Closes #327
2026-05-22 14:13:42 -05:00
Baijack-star 23ad4ea923 fix(mcp): cap codegraph_context output to prevent context bloat (#296)
Route handleContext's output through the shared truncateOutput cap
(MAX_OUTPUT_LENGTH) so codegraph_context can no longer blow past the context
budget — every sibling MCP tool already truncates; this was the one uncapped
output path.

Closes #296

Co-authored-by: Baijack-star <71923891+Baijack-star@users.noreply.github.com>
2026-05-22 13:20:08 -05:00
Colby Mchenry f6772dac7c feat: zero-config indexing driven by .gitignore (#283) (#285)
Remove .codegraph/config.json and the entire config surface. CodeGraph now
indexes every file whose extension maps to a supported language and respects
.gitignore everywhere — git repos via git itself, non-git projects via the
`ignore` library (root + nested .gitignore files, the same way git does).

- Remove CodeGraphConfig/DEFAULT_CONFIG, src/config.ts, and the public config
  API (the `config` option on init, getConfig/updateConfig/getConfigPath).
- Derive the source-file allowlist from EXTENSION_MAP (isSourceFile); maxFileSize
  is now a constant. Drop the .codegraphignore marker.
- Behavior change: committed, non-gitignored dirs (vendor/, a committed dist/)
  are now indexed — .gitignore is the single source of truth.

Earlier inert fields (languages, frameworks, extractDocstrings, trackCallSites,
customPatterns) and their dead helpers are removed as part of this.

Resolves #283.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 18:06:02 -05:00
@aaronjmars cda42c8222 fix(security): refuse to follow symlinks when writing /tmp session marker (#280)
`markSessionConsulted` writes `${tmpdir()}/codegraph-consulted-${hash}` on
every `codegraph_context` call so external tooling can detect that an MCP
session has consulted CodeGraph. The old `writeFileSync` followed symlinks
unconditionally, so on a multi-user system any other local user could
pre-create that marker path as a symlink pointing at a victim-writable
file — the next codegraph context call would then overwrite the target's
contents with the ISO timestamp string (CWE-59).

The session-id hash gates predictability and makes opportunistic exploit
infeasible on its own, but tmpdir() is world-writable (mode 1777 on Linux)
and the proper pattern is to never follow links into a shared-prefix
tmpfile. Switch to `openSync` with O_NOFOLLOW + mode 0o600. ELOOP from a
planted symlink lands in the existing silent-fail catch — refuse to write
rather than touch an attacker-chosen target.

Detected by Aeon + manual review.
Severity: medium
CWE-59 (link following), CWE-732 (incorrect permission for critical resource)

Co-authored-by: aaronjmars <aaron@aeon.local>
2026-05-21 16:58:41 -05:00
Martin Oehlert 399d78b938 security: path validation, ReDoS prevention, picomatch, PID-based file lock
- Add validateProjectPath() to reject sensitive system directories
- Add isPathWithinRoot/isPathWithinRootReal for symlink-aware path checks
- Replace hand-rolled glob-to-regex with picomatch to prevent ReDoS
- Add isSafeRegex() to reject custom patterns with nested quantifiers
- Replace FileLock with PID-tracking version that detects stale locks
- Add symlink detection in removeDirectory/listDirectoryContents
- Add subdirectory name validation in ensureSubdirectory
- Add atomicWriteFileSync and corrupted file backup in config-writer
- Add MCP input validation (validateString) for all tool handlers
- Fix CLAUDE.md section replacement to handle ### subsections correctly
2026-02-10 11:22:04 +01:00
Colby McHenry 41c49ce83d Add security hardening test suite (28 tests)
Tests cover all security utilities introduced in the previous commit:
- validatePathWithinRoot: path traversal prevention (7 tests)
- safeJsonParse: corrupted JSON fallback handling (6 tests)
- clamp: input range clamping (5 tests)
- FileLock: cross-process file locking (7 tests)
- Atomic config writes: temp file + rename pattern (3 tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:21:36 -06:00