mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-17 05:07:37 +08:00
fef37765c1
* perf(resolution): read JS/TS parameter lists without exponential backtracking isLocallyBoundJsName (#1759,cd4e65b5) scans a file for a parameter that binds a bare-called name. Its per-parameter pattern `(type)?(default)?\s*` let one `name: T = value` text split several ways, so when the name was absent the search backtracked through every combination. On vscode's markersModel.test.ts, whose helper takes nine typed, defaulted parameters, each `suite`/`test` scan took 30-40s — one pool worker held a whole resolution batch while the rest idled, a 70-400s stall per run. Each parameter now has exactly one parse: its first non-space character after the identifier picks the type, default or bare alternative. The alternatives match exactly the strings the old optional groups did, so the result is identical for every input (4M generated cases and 5.6k real vscode file/name pairs agree); the scan is 0.1-0.3ms on that file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): memoize Python module-file lookups per resolution pass #2034 (5eaa6fed) sent every Python named import whose path the import resolver cannot place — stdlib and third-party modules such as `from unittest import mock` — through findPythonModuleFile, which scans every `__init__.py` file node for a path suffix. It ran once per ref: on django that scan alone went from 0.6s to 1.4s of resolution CPU. The candidates for a module depend only on the module path, so they are now collected once per resolution context, in the same name-lookup order; the importing file is still excluded per call, so the first survivor is the node the scan returned. The memo drops with the other import-resolver memos. Resolution output on django is hash-identical; single-threaded resolution CPU there goes from 6.5-6.8s to 4.9s (v1.6.0: 4.5s). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): test sealed-module candidates cheapest-first isSealedModule (#1719,bffd50e4) masks a candidate file's comments and strings before asking whether it is a module that exports nothing. Import refs call it for every same-named candidate across the tree, so on vscode it masked nearly every TypeScript file once per resolver: 3.3s of masking in a single-threaded pass, paid again in each pool worker. The verdict is a pure conjunction, so it now asks the cheap questions first: the source contains `import`, no node in the file is exported, and there is no CommonJS export. Masking runs only for files that pass all three, which in practice are the few that can be sealed. The masker blanks text and never deletes it, so a file without `import` in its source has none in its masked code either. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): scan a class for a field's declared type once per field matchTsThisFieldCall (#1496,cece0720) resolves `this.<field>.<method>()` by finding the field's declaration in the enclosing class. It re-split the whole file, stripped comments line by line and compiled three patterns for every such call, although the answer depends only on the class and the field: 16.5s of a 148s single-threaded resolution pass on vscode. The first declaring line — `: typeof X`, `: X` or `= new X`, in that order per line — is now memoized per (class node, field), and the last few classes' stripped lines are kept, since calls arrive file by file. Owners are still tried in the same order and everything after the declaration lookup is unchanged. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(synthesis): start cross-tier receiver chains only at a word's start The cross-tier pass (b1f40c57) scans every non-test JS/TS file whose text has a `.add(`, `.get(` or `.on(` — nearly all of vscode — with receiver- chain patterns such as `a.b.c.add('job')`. A chain could begin at any letter of an identifier, so each letter re-read the dotted chain behind it; on vscode the pass ran 35s on one pool worker and set the synthesis phase's wall. A match that starts mid-word implies one from the word's first letter, and every match of these patterns ends on a non-word character, so a scan never resumes mid-word: a lookbehind that rejects those starts skips only attempts bound to fail. Over all 14,135 vscode JS/TS files the five patterns return identical matches (positions and groups) before and after, in 8.8s instead of 32.8s; the pass itself drops from ~9.8s to ~5.6s CPU standalone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): filter same-named candidates in one pass, cheap checks first Interface members (#1638) and nested const-bound functions (#1669) made common names — `editor`, `model`, `dispose` — carry thousands of nodes. matchByExactName ran ten chained filters over that list for every reference, copying it once per rule before the source-reading checks ran, and the method-call strategies copied it again to put the call site's file first before looking at only the class-like or same-file entries. Each rule judges one candidate on its own, so the exact-name rules now run as a single pass with the kind and language checks ahead of the lexical, sealed-module and local-binding checks; the class strategies narrow to owner kinds before ordering by file (the same nodes in the same order), and the object-literal strategy, which keeps same-file holders only, skips the reordering altogether. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(synthesis): scope language-specific node scans in SQL Eleven synthesis passes stream every node of a kind and keep only their language's (Go, Java/Kotlin, XML, native, ArkTS, Nix, Erlang). On a TypeScript monorepo the Go passes materialized all 113k vscode methods to find two Go ones — goMethodContains alone went from 0.5s to ~2.9s on the main thread after interface members (#1638) grew the method count and the canonical ORDER BY (#1988) landed. iterateNodesByKindIn adds the language predicate to the same query. Its ORDER BY (file_path, start_line, id) is a total order, so it yields exactly the nodes the callers kept, in the same sequence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(synthesis): run the synthesis-site index and source-gate scan beside the passes Two serial main-thread steps landed with the incremental synthesis refresh (#1988, #2038). The bulk-edge window rebuilt idx_edges_synthesis_site before synthesis — its partial predicate runs json_valid over the metadata of every edge (~1.5M rows, 2.7s on vscode) — and after the merged insert synthesis re-read every file in the project to record its source gates (~1.9s). Neither result is read until the index is finished: only sync consults the index, and the gates depend only on the files. The window's end now leaves that one index for later, and the resolver builds it while the pool computes the passes, when the main thread was only waiting; the source-gate scan runs in the same window. Both still happen before resolution returns, whatever path synthesis takes, and a crash in between is healed on the next open as before. On vscode edge-index-recreate drops from 8.3s to 3.6s of serial time; the graph, synthesis_inputs and the final index set are identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): stop re-reading resolved targets and re-probing missing files Every resolved reference now re-reads its target node several times — the kind gate (#1796), the cross-family language gate (#1986), alias following (#1482) and edge creation — through the query layer's 1,000-entry node cache, while targets recur ~5x (1.46M vscode edges onto 293k nodes). The language gate alone spent ~6s of single-threaded resolution on vscode fetching whole nodes to read one field, and fileExists re-probed the same missing candidate paths (every extension of a `.js` specifier) from every importing file, now under a path.resolve containment check (#1631). The resolver keeps its own id → node cache and a memo of filesystem probes, both in the stable window of its other caches and dropped by clearCaches; the language gate returns early when the reference's language has no code family (no target can cross one) and otherwise memoizes each target's language. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): cheaper sealed-module and receiver-less call checks isSealedModule read and decoded a candidate file — its source and every node — to learn whether anything is exported, though nearly every module does. It now asks one indexed EXISTS probe first and reads the source only for files that export nothing (the conjunction is unchanged). isReceiverLessCall (#1759, #1857) compiled a pattern per call and end-anchored two more over the line's prefix, which V8 scans from the start. It now matches the name literally with a sticky `\s*[(<]`, and reads the prefix backwards: the last non-space character decides whether a receiver precedes the call, and the word run ending there decides the keyword exception — exactly what `[.\w$\])]\s*$` and `\b(?:return|...)\s*$` accepted. Resolution output is hash-identical on vscode and django. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(synthesis): pull passes onto workers heaviest-first; cheaper registry and tier scans The pooled fan-out handed every pass out at once, so each worker held a count-based share and a heavy pass finished only with its worker's whole share (vscode's registry pass ended ~16s in). Workers now pull one pass at a time, heaviest first; edges still merge by registry index, so the output is unchanged. vscode synthesis: 14.5s vs 19.6-24.4s. The registry pass retried its greedy name at every character of every identifier; both scans now start a name only at an identifier's first character or a `this.` (a mid-identifier resume after a `.method` dispatch falls back to the unguarded pattern), and dispatch lines come from a newline-offset lookup instead of splitting the prefix per match. Checked match-for-match against the old patterns on every vscode JS/TS file plus 300k fuzzed strings. The tier pass reads a file's nodes only when a site needs them. Standalone on vscode: registry 5.6s -> 4.5s, tier 5.1s -> 4.4s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): lean export indexes; no per-name regex compiles in field and local-binding checks Three per-lookup costs the resolver pool paid once per worker: - A file's export index decoded every node in the file and read its source, for each file an import reached, in every worker. It now reads only the exported rows (same `start_line, id` order); the default-export binding and names a local `export { a as b }` clause introduces are read on first use, from name-targeted rows. findExportedSymbol across vscode's workers: 12.5s -> 3.7s. - The `this.<field>` declaration scan compiled three patterns per field of every class and walked every class line. Each pattern needs the field as a whole `[\w$#]` token, so a class's lines are indexed by token once and three sticky patterns, compiled once, are tried at the field's token occurrences. Identical on 721k vscode (file, field) pairs; 56.7s -> 1.1s standalone. - isLocallyBoundJsName searched the whole file with four patterns per name. Each can only start at a `const`/`let`/`var` or `function`/`class` keyword, an `=>`, or the `(` opening a list before the name, so the patterns (compiled once per name, sticky) are tried there. Identical on 198k vscode (file, name) pairs; 68.1s -> 10.7s standalone. Replaying every vscode (2.19M) and django (270k) reference single-threaded gives the same result hash before and after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): prepare kind-filtered edge reads once; memoize the inherited this-member walk - getOutgoingEdges / getIncomingEdges with a kind filter prepared a fresh statement on every call, a third of the read's cost; they are issued per node by supertype walks and the member-lookup synthesis passes. Each SQL shape is now prepared once per connection (dropped on rebind). - The post-loop `this.<member>` pass walked the same supertypes for every reference, reading each one's contains edges and decoding members one by one until a name matched. Nothing is written until the pass ends, so each type's supertype edges and each supertype's callable members (by name, in edge order) are read once. vscode: 4.1s -> 1.0s. - Express's mount pass decoded every JS/TS file to look for `.use(`; it now checks the raw bytes first (an ASCII needle survives UTF-8 decoding unchanged) and decodes only files that have it. A full vscode init dumps the identical graph before and after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(resolution): fan the next batch out before inserting edges resolution never reads The batch loop inserted every batch's edges before fanning out the next one, so the pool sat idle through each insert. That ordering exists for one reason: later batches walk supertype chains over the `extends` / `implements` edges earlier batches resolved. Those are the only edges resolution reads, and the prerequisite phase drains every such ref first, so after it no batch makes one. A batch without them now fans the next batch out first and inserts while it resolves: the same writes in the same order, and nothing the workers read differs. The kinds live in one constant, shared with the supertype walk. vscode resolution, three interleaved rounds against the parent commit: 50.3-51.0s vs 56.8-59.1s wall (the pool's wait per batch halves; the overlapped inserts cost ~7s more CPU). Full vscode and django inits dump identical graphs before and after, and identical to main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(db): prepare the edge-endpoint existence check once Every edge insert checks its endpoints with a 500-id IN query, a chunk at a time, and prepared that statement afresh for each chunk. The batch loop now runs these inserts on the main thread while the pool resolves, so the full-size statement is prepared once; only a final partial chunk prepares ad hoc (as deleteReferencesByRowIds already does). Same SQL, same result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
188 lines
7.5 KiB
TypeScript
188 lines
7.5 KiB
TypeScript
/**
|
|
* In JS/TS a receiver-less call can never bind to a class method: `serialize(x)`
|
|
* inside `Record.serialize` means the module-scope function, and the method
|
|
* itself — which the same-file proximity term used to pick, producing a
|
|
* self-edge — is not a candidate (#1714). `this.serialize(x)` still is.
|
|
*/
|
|
|
|
import { describe, it, expect, afterEach } from 'vitest';
|
|
import * as fs from 'fs';
|
|
import * as os from 'os';
|
|
import * as path from 'path';
|
|
import CodeGraph from '../src/index';
|
|
|
|
let tempDir: string;
|
|
let cg: CodeGraph | null = null;
|
|
|
|
async function callsFromMethod(source: string, methodName: string): Promise<string[]> {
|
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1714-'));
|
|
fs.writeFileSync(path.join(tempDir, 'record.ts'), source);
|
|
cg = await CodeGraph.init(tempDir, { index: true });
|
|
cg.resolveReferences();
|
|
const from = cg.getNodesByKind('method').find((n) => n.name === methodName)!;
|
|
expect(from).toBeDefined();
|
|
return cg
|
|
.getOutgoingEdges(from.id)
|
|
.filter((e) => e.kind === 'calls')
|
|
.map((e) => cg!.getNode(e.target))
|
|
.filter((n): n is NonNullable<typeof n> => !!n)
|
|
.map((n) => `${n.kind}:${n.qualifiedName ?? n.name}`);
|
|
}
|
|
|
|
afterEach(() => {
|
|
cg?.close();
|
|
cg = null;
|
|
fs.rmSync(tempDir, { recursive: true, force: true });
|
|
});
|
|
|
|
describe('a receiver-less JS/TS call never binds to a method (#1714)', () => {
|
|
it('resolves the bare call onto the module-scope function, not the enclosing method', async () => {
|
|
const callees = await callsFromMethod(
|
|
[
|
|
'function serialize(value: string): string {',
|
|
' return value.trim();',
|
|
'}',
|
|
'',
|
|
'export class Record {',
|
|
' constructor(private readonly raw: string) {}',
|
|
' serialize(): string {',
|
|
' return serialize(this.raw);',
|
|
' }',
|
|
'}',
|
|
'',
|
|
].join('\n'),
|
|
'serialize'
|
|
);
|
|
expect(callees).toContain('function:serialize');
|
|
expect(callees).not.toContain('method:Record::serialize');
|
|
});
|
|
|
|
it('keeps `this.serialize()` — a real recursive self-call', async () => {
|
|
const callees = await callsFromMethod(
|
|
[
|
|
'function serialize(value: string): string {',
|
|
' return value.trim();',
|
|
'}',
|
|
'',
|
|
'export class Record {',
|
|
' constructor(private readonly raw: string, private depth = 0) {}',
|
|
' serialize(): string {',
|
|
' if (this.depth > 0) return this.serialize();',
|
|
' return this.raw;',
|
|
' }',
|
|
'}',
|
|
'',
|
|
].join('\n'),
|
|
'serialize'
|
|
);
|
|
expect(callees).toContain('method:Record::serialize');
|
|
});
|
|
|
|
it('a bare call to a name the file binds itself has no cross-file candidate', async () => {
|
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1714-'));
|
|
fs.writeFileSync(path.join(tempDir, 'config.ts'), 'export function resolve(p: string) { return p; }\nexport function transform(c: string) { return c; }\nexport function now() { return 0; }\n');
|
|
fs.writeFileSync(
|
|
path.join(tempDir, 'client.ts'),
|
|
[
|
|
'const transform = makeTransform();',
|
|
'export function ping(): Promise<void> {',
|
|
' return new Promise((resolve, reject) => {',
|
|
' setTimeout(() => resolve(), 10);',
|
|
' });',
|
|
'}',
|
|
'export function run(options: { now?: () => number }) {',
|
|
' const now = options.now || (() => Date.now());',
|
|
' return now() + transform("x").length;',
|
|
'}',
|
|
'',
|
|
].join('\n')
|
|
);
|
|
cg = await CodeGraph.init(tempDir, { index: true });
|
|
cg.resolveReferences();
|
|
const targets = cg.getNodesByKind('function').filter((n) => n.filePath === 'config.ts').map((n) => n.id);
|
|
const callers = cg.getNodesByKind('function').filter((n) => n.filePath === 'client.ts');
|
|
const crossFile = callers.flatMap((c) => cg!.getOutgoingEdges(c.id)).filter((e) => e.kind === 'calls' && targets.includes(e.target));
|
|
expect(crossFile).toEqual([]);
|
|
});
|
|
|
|
it('a destructured require or a string mentioning the name is not a local binding', async () => {
|
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1714-'));
|
|
fs.writeFileSync(path.join(tempDir, 'public-ip.js'), 'function lookupPublicIPv4() { return "1.2.3.4"; }\nfunction test(name, fn) { return fn(); }\nmodule.exports = { lookupPublicIPv4, test };\n');
|
|
fs.writeFileSync(
|
|
path.join(tempDir, 'main.js'),
|
|
[
|
|
'const { lookupPublicIPv4 } = require("./public-ip");',
|
|
'const { test } = require("./public-ip");',
|
|
'async function prepare() {',
|
|
' const ip = await lookupPublicIPv4();',
|
|
' test("a test of the thing", () => {});',
|
|
' return ip;',
|
|
'}',
|
|
'module.exports = { prepare };',
|
|
'',
|
|
].join('\n')
|
|
);
|
|
cg = await CodeGraph.init(tempDir, { index: true });
|
|
cg.resolveReferences();
|
|
const prepare = cg.getNodesByKind('function').find((n) => n.name === 'prepare')!;
|
|
const names = cg.getOutgoingEdges(prepare.id).filter((e) => e.kind === 'calls').map((e) => cg!.getNode(e.target)?.name);
|
|
expect(names).toContain('lookupPublicIPv4');
|
|
expect(names).toContain('test');
|
|
});
|
|
|
|
it('reads typed, defaulted parameter lists in linear time', async () => {
|
|
// A helper with many `name: T = value` parameters made the parameter scan
|
|
// backtrack through every way to split each type from its default — tens
|
|
// of seconds per called name on vscode's markersModel.test.ts, which
|
|
// stalled a whole resolution batch.
|
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1714-'));
|
|
fs.writeFileSync(
|
|
path.join(tempDir, 'harness.ts'),
|
|
'export function suite(name: string, fn: () => void) { fn(); }\nexport function resolve(p: string) { return p; }\n'
|
|
);
|
|
const params = Array.from({ length: 14 }, (_, i) => `p${i}: number = ${i} + ${i}`).join(', ');
|
|
fs.writeFileSync(
|
|
path.join(tempDir, 'markers.test.ts'),
|
|
[
|
|
`function aMarker(${params}, related?: string[]): number {`,
|
|
' return p0;',
|
|
'}',
|
|
'function settle(value: number = 1, resolve: Handler = noop) {',
|
|
' resolve(value);',
|
|
'}',
|
|
'suite("markers", () => {',
|
|
' aMarker(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14);',
|
|
' settle();',
|
|
'});',
|
|
'',
|
|
].join('\n')
|
|
);
|
|
cg = await CodeGraph.init(tempDir, { index: true });
|
|
cg.resolveReferences();
|
|
const harness = cg.getNodesByKind('function').filter((n) => n.filePath === 'harness.ts');
|
|
const suiteFn = harness.find((n) => n.name === 'suite')!;
|
|
const resolveFn = harness.find((n) => n.name === 'resolve')!;
|
|
const settle = cg.getNodesByKind('function').find((n) => n.name === 'settle')!;
|
|
// `suite` is bound nowhere in the test file, so it still resolves across files.
|
|
expect(cg.getIncomingEdges(suiteFn.id).some((e) => e.kind === 'calls')).toBe(true);
|
|
// `resolve` is a defaulted parameter of `settle` — a local binding.
|
|
expect(cg.getOutgoingEdges(settle.id).filter((e) => e.kind === 'calls').map((e) => e.target)).not.toContain(resolveFn.id);
|
|
}, 30_000);
|
|
|
|
it('keeps `other.serialize()` — a call through a receiver', async () => {
|
|
const callees = await callsFromMethod(
|
|
[
|
|
'export class Record {',
|
|
' serialize(): string { return ""; }',
|
|
' copyOf(other: Record): string {',
|
|
' return other.serialize();',
|
|
' }',
|
|
'}',
|
|
'',
|
|
].join('\n'),
|
|
'copyOf'
|
|
);
|
|
expect(callees).toContain('method:Record::serialize');
|
|
});
|
|
});
|