Commit Graph

2 Commits

Author SHA1 Message Date
Colby McHenry cbf84855e7 fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581)
A C/C++ (or any other kernel-routed) file with extremely deep nesting —
clang's 16,384-brace `parser_overflow.c`, fuzzer corpora — parsed fine
(tree-sitter is iterative) and then overflowed the native stack of the
kernel's recursive walker. A native overflow is uncatchable: the parse
worker is a thread of the `codegraph` process, so the SIGSEGV took the
whole indexer down with no message, no partial index and no per-file
fallback. Worker threads get Node's 4 MiB default stack; the 8 MiB main
thread only moved the cliff (100k levels still died), so a bigger
`resourceLimits.stackSizeMb` was never a fix.

The walkers now guard their own recursion against the CALLING THREAD's
real stack bounds (`codegraph-kernel/src/stack.rs`: glibc/musl
`pthread_getattr_np`, macOS `pthread_get_stackaddr_np`, Win32
`GetCurrentThreadStackLimits`; one thread-local load + one compare per
recursive entry, inserted by the `stack_guard!` macro at all 150
self-recursive / on-cycle walker functions). Within 256 KiB of the limit
the walk stops descending and latches a flag; `stack::run_guarded` turns
a tripped walk into the kernel's existing `defer:` routing signal, so the
file takes the wasm path — whose walker catches its own JS `RangeError`
per file — and lands as a partial result with a recorded parse error
while the rest of the repository indexes normally. Platforms without a
bounds query fall back to a fixed descent budget that is safe on any
stack ≥ 2 MiB. No Worker stack bump; no new crates beyond `libc`
(already in the lock file transitively).

Validated: the reporter's `deep.c` inside a default 4 MiB worker goes
from rc=132/139 to a clean `deferred` exit; `codegraph init` on a repo
holding it exits 0 with the file recorded; 60k-deep expressions in every
default-routed language survive on the main thread and in a worker;
Rust unit tests drive the walkers on a 1 MiB thread; all 15 existing
kernel parity suites unchanged; index wall-clock on express and redis
within run-to-run noise with identical node/edge counts; Linux verified
in Docker (node:22-bookworm, glibc bounds path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
2026-08-22 11:01:26 -07:00
Colby Mchenry 1909931238 feat(kernel): R7b Ruby walker — ruby module, tree-sitter-ruby 0.23.1 bump, ref-flag wire slot, ruby default-routed (#1379)
Third R7b port, checklist-first recipe (docs/design/ruby-kernel-port-checklist.md).

Grammar bump first, validated standalone (the rust pattern): tree-sitter-ruby
^0.20.1 (tree-sitter-wasms, 2024-02) → v0.23.1 — crate pinned =0.23.1, wasm
built from tag 71bd32f's checked-in parser.c/scanner.c (both sha-matched
against the crates.io tarball; content bump, ABI stays 14). Old-vs-new
full-init dumps: sinatra/jekyll byte-identical; rails = exactly the one
classified hunk (the `recv&.!=` safe-nav operator misparse fix,
`table_name.!` → `table_name.!=`, precision-positive).

Walker (python.rs chassis + the six ruby divergences) preserves bug-for-bug:
the importTypes:['call'] funnel (class-body DSL — attr_accessor, has_many,
define_method incl. its block, sinatra route blocks — emits NOTHING at
non-body scope), hook-handled module multiply-capture (nested modules re-scan
their subtree per level after popping — `this.hooked` fn-refs from class AND
module AND file), the sibling-scan visibility trio (bare `private` invisible;
`private :sym`/`private def` poison all later defs; the inner def stays
public), bare-call statements (do…end body_statement emits, brace-block
block_body doesn't), `.new` instantiates with last-`::`-segment names,
constant-receiver references refs, require/require_relative path refs
(posix-normalized, `.rb`-suffixed, `Kernel.require` and interpolated-path
quirks included), `=begin` docstring marker survival, and the reverse-order
value-ref DFS.

Wire v2: the hook's mixin `implements` refs carry `filePath: ctx.filePath` —
the ONE extraction-ref denormalized field (php's trait-use refs share the
shape). RefRow's first pad byte becomes a flags slot (REF_FLAG_FILE_PATH);
decode re-attaches its own filePath parameter; KERNEL_ABI_VERSION 1→2 on both
sides (mismatched dist/.node pairs degrade to wasm, as designed).

Gates: sweeps 0-diff sinatra 147/147, jekyll 164/164, rails 3452/3452 (3,763
files, 0 deferrals — ruby error incidence 0.00%, any deferral = walker bug);
full-init dumps byte-identical ×3 (7.2k/9.4k/375.6k lines); kernel-ruby-parity
suite (torture + CRLF + wire-flag pin + defer) + ruby grammar-parity row;
full suite 2,613 green ×2 under CODEGRAPH_KERNEL_EXPECT=1 (one unrelated
mcp-initialize timing flake under parallel load, passes solo 3/3 ×3).
DEFAULT_ROUTED += ruby (12 langs).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 15:44:13 -05:00