The vendored grammar lexed the member modifiers as one prec-10 token and
`Sub New` as one prec-100 token. Neither went through keyword extraction,
and both outranked the prec(-1) identifier, so the lexer stopped at the end
of a modifier spelled at the start of a name: `Public SharedCache` indexed a
field `Cache`, `Public Dimension` a field `ension`, `Public Shared Shared1`
was lost to a parse error, and `Sub NewItem()` became a constructor. `Dim`
and `Const` were part of that token, which also kept the local-declaration
`Dim`/`Const` out of keyword extraction, so `ConstVBV.Value = False` and
`ConstructPath()` inside a method parsed as `Const` declarations.
The modifiers and `Sub New` are now plain keywords. The old tokens also
folded a newline into the keyword after it, which two parses relied on by
accident: `Option` lines under a comment banner may now follow newlines, and
upstream's top-level `file_attribute_section` is gone, so attribute lines
above a top-level declaration belong to it. `Protected Friend` and
`Private Protected` are two modifiers now; the extractor reads them as
protected and private.
Rebuilt with tree-sitter-cli 0.25.10 and emscripten 4.0.4. On SCrawler and
staxrip, 3 fields get their whole names, staxrip recovers one method, and
every other graph difference is an `imports` edge or a node start now on its
own line instead of the comment line above.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cobol): a file ending inside the sequence area no longer hangs the parser
The vendored COBOL grammar's external scanner skips the fixed-format
sequence area (columns 1-6) with `while (get_column() <= 5) advance()`.
At end of input advance() is a no-op, so a file whose last line stops in
that area (` .`, `X\n .`, a sentence closed by a short ` .` line)
never reached column 7 and the scanner spun forever inside wasm. Indexing
hit the parse worker's hard timeout three times (~93 s) and stored the
file with zero symbols; the viewer's highlighter froze on the same text.
The loop now also stops at eof(). It is upstream code (present at
e99dbdc3), so the change is a standalone upstreamable fix; inputs that
parsed before are unaffected (412 NIST + ocesql test sources give
byte-identical trees with the old and new wasm, and the upstream corpus
keeps its single pre-existing failure).
Regenerates docs/grammars/tree-sitter-cobol.patch, rebuilds the wasm per
docs/grammars/tree-sitter-cobol.md, and adds an out-of-process regression
test under a hard deadline (a regression is an uninterruptible wasm loop).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(viewer): force the highlight deadline instead of relying on a grammar bug
The bounded-highlight test used the COBOL ` .` slice as its slice that
never finishes; the grammar fix makes it parse, so the deadline is now
forced through tokenizeBounded's deadline parameter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
tree-sitter-kotlin 0.3.8's external scanner inserted an automatic
semicolon on the same line before any word starting with `e` other than
`else`, so Exposed's `Users.id eq id1` ended its statement early and
error recovery could swallow the class around it (r2dbc UpsertTests
became one loose function nesting every test).
Patch scanner.c (same line: never a semicolon before an e-word) in both
the vendored kernel C and the wasm build; provenance, shas and rebuild
recipe in docs/grammars/tree-sitter-kotlin.md. Exposed files with a parse
error 229 -> 71 of 1,004; koin, nowinandroid, moshi, okio, okhttp and
kotlinx.coroutines unchanged (one okhttp file fixed, none broken);
kernel-parity 0 diffs.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch
+ new external scanner for XML literals and multi-line LINQ continuation;
provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md),
vbnet extractor with VB-specific call/index disambiguation, Inherits/
Implements heritage, As New instantiation, events, Declare P/Invoke, and
MustOverride abstract members.
Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%,
staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%).
Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Programs, sections/paragraphs (reconstructed extents over the grammar's
flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook
imports incl. standalone .cpy fragments, DATA DIVISION records/fields/
88-levels with write-site impact references, and CICS flows: EXEC
LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL
INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved
to the owning program via a CICS framework resolver. Fixed and free
source format (free format via a scanner wide-mode sentinel).
Grammar: vendored wasm built from a patched yutaro-sakamoto/
tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook
fragment entry point, single-quote continuation, COPY REPLACING
pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated
relations, COBOL-2002 usages, and more). Patch + provenance + upstream
PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native
(upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft
free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382.
Copybook members resolve to files like C includes (basename index,
name-matcher short-circuit so compiler-supplied members stay honestly
unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL
(CVACT01Y copybook) surfaces its 4 writer programs cross-file.
Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B
arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Colby Mchenry