tree-sitter-cpp has no preprocessor, so an attribute macro where it
expects a name or a type misparses the declaration around it. The C++
preParse already blanked one export macro before a class name, an inline
macro before a return type, and a lone macro line before a declaration.
It missed:
- several macros, or one with arguments, between `class` and the name
(protobuf's generated `class PROTOBUF_EXPORT
PROTOBUF_FUTURE_ADD_EARLY_WARN_UNUSED Any final : public Message`,
rocksdb's `struct ALIGN_AS(64U) HandleImpl`), and a macro before a
partial specialization's name;
- a macro between a pointer and the declared name (`const Descriptor*
PROTOBUF_NONNULL descriptor()`), after a parameter list
(`unknown_fields() const ABSL_ATTRIBUTE_LIFETIME_BOUND {`, leveldb's
`LOCKS_EXCLUDED(mu_) {`), after a declared name (`int count_
GUARDED_BY(mu_);`), or opening a declaration
(`PROTOBUF_FUTURE_ADD_EARLY_NODISCARD absl::string_view name() const`);
- a lone macro line with a comment under it ({fmt}'s
`FMT_BEGIN_EXPORT` above `// A generic formatting context ...`).
Every generated protobuf message class misparsed, so its members were
indexed as namespace-level functions; leveldb's annotated methods became
phantoms named after the annotation.
The new passes match on the code alone (comments and directives as
spaces, literals as placeholders), blank only the macro tokens, and keep
every offset. They run in the hoisted preParse, so files that now parse
clean go through the native kernel, at parity. The passes share one mask
per file: a pass that blanks nothing hands the next the same string, and
one that blanks hands it the mask blanked the same way.
#2426's brace-scope fixtures misparsed only through
PROTOBUF_FUTURE_ADD_EARLY_NODISCARD and ALIGN_AS(64U), which the preParse
now blanks. They now use macros it doesn't know (NODISCARD,
cacheline_aligned(64U)), with the same expected scopes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tree-sitter recovers from a construct it can't parse by inserting the token
it expected, and inside a class body that is often a `}`: an unknown macro
in front of a member (`PROTOBUF_FUTURE_ADD_EARLY_NODISCARD absl::string_view
name() const`) or in a class head (`struct ALIGN_AS(64U) HandleImpl`) closed
the class there, and every `}` after it closed the scope one level out.
Recovery also ran the other way, keeping a scope open past its own `}` once
an ERROR swallowed it. The walker scoped declarations by tree nesting, so
qualified names lost their namespace and class segments, or gained ones they
don't belong to: protobuf's `FieldDescriptor`, `MapFieldBase` and
`MutableRepeatedFieldProxyImpl`, fmt's `detail::` buffers (and
`fmt::format_int` put in `detail`), rocksdb's clock_cache classes and the
`Opts` structs nested in them, gtest's `PrettyUnitTestResultPrinter`.
Every such file parses with errors, and the kernel defers those to wasm, so
the wasm walker produced all of them; the kernel's walker has the same
tree-nested stack and reaches erroring trees only under the
CODEGRAPH_KERNEL_CCPP_ERROR_EXTRACT sweep hatch.
For a C++ file whose tree has errors, the walker now takes each
declaration's namespaces, and at declaration level its enclosing classes,
from the source's braces (languages/cpp-brace-scopes.ts: comments,
literals, raw strings, digit separators and preprocessor lines skipped,
each #if branch read from the braces open at the #if). A class-like node
ends at its body's `}`, and a class the tree glued into a namespace-level
declaration's type is walked as a class. A file whose braces don't balance,
and every file that parses cleanly, keeps the tree's scopes. Ported to the
kernel (ccpp/brace_scopes.rs) so the hatch stays parity-true: the same
erroring files diverge before and after (UTF-8/UTF-16 recovery), every
other one is byte-identical.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Colby Mchenry