mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-17 05:07:37 +08:00
perf(resolution): remove the JS indexing regressions since 1.6.2; the summary time covers the whole run (#2334) (#2362)
Indexing pretix took 2.2x as long since #2163 (issue #2334), and the bundled minified d3 was what #2344 left of CPython's 1.6.2 slowdown. Two JS/TS checks that first shipped in 1.6.2 redid whole-file work for every reference: - matchDestructuredCallResult (#2163) ran for every bare call in a file holding `const {` / `let {` / `var {`: per call it re-tested that pattern on the source, then stripped, blanked and brace-walked every line above the call and matched the binding regex over it. - jsFunctionLocalScope (#2226) re-stripped the calling function's lines above each reference. In a minified script every function's text runs to the end of its one line, so each strip was the whole file. CPU profiles, main -> this change: matchDestructuredCallResult 82.4 s -> 1.7 s and jsFunctionLocalScope 12.3 s -> 5.2 s on pretix; 35.4 s -> 0.37 s and 23.3 s -> 1.0 s on CPython's Lib/profiling + Doc/_static. Now: - A file's `const { … } = f(…)` bindings are read once per resolution: the blanked code, each binding's end, callee and keys, the block it sits in and where that block closes, and the spans the blanker read as regex literals. A call looks up its own name; only a call through a destructured name scans the code between that binding and itself for shadowing, and only once the callee is found to return the key (a `require(…)` destructuring, for one, needs no scan). - Blanking the text above a call on its own gives the file's blanked code up to the call: the stripper and blanker look past a character only for a comment opener's second character and for a regex literal's closing `/`. So the prefix's bindings are the file's that end at or before the call (a match never reads past its `(`, and one running past the call holds no `{`…`(` to start another), and its brace stack follows from where each block closes. For a call inside a span the file reads as a regex literal (a division, in minified code) the prefix is the code up to the `/` plus the short tail blanked on its own; where a binding statement could run across that `/` (inside its `{ … }` or `< … >`), and for out-of-range positions, the old per-call scan still runs. - jsFunctionLocalScope strips a function's lines once and cuts at the reference's line end; functions that start on the same line share the regex answer by (file, first line, reference line, name). - The new memos, and JS_FN_LOCAL_MEMO (never cleared before), drop with the other per-file memos in clearNameMatcherMemos. The summary line printed orchestrator.indexAll's durationMs, which covers extraction only, beside node and edge totals recomputed after resolution: the issue's 19.9 s run printed "in 1.7s". CodeGraph.indexAll and sync now report the whole run's wall time, which `codegraph init`, `index` and `sync`, the MCP auto-sync log and the telemetry duration bucket read. Verification: nodes, edges and unresolved_refs dumps keyed by file|kind|qualified_name|start_line are byte-identical to origin/main8998697f(with #2357's new destructuring and Vue template refs) on all 24 runs: pretix, mealie and CPython's Lib/profiling + Doc/_static, 8 each; and todeac7710(#2358, #2360, #2361) on one more round of each. A differential harness ran old and new matchDestructuredCallResult on every call position of mealie (10,330) and the CPython assets (9,170, 742 inside misread regex literals) and on 16,000 random token-soup sources (1.33M positions), and old and new jsFunctionLocalScope on ~500K (function, line, name) triples: no difference; it reports one for each deliberate mutation of the new code that is not provably equivalent. Medians of 4 interleaved runs on a loaded 16-thread Windows box, process CPU / wall / peak RSS: pretix 160.2 s / 75.9 s / 2.47 GB -> 88.7 s / 43.5 s / 1.93 GB; CPython assets 75.1 s / 37.6 s / 1.23 GB -> 32.1 s / 20.4 s / 0.81 GB; mealie (no bundles, the control) 39.2 s -> 38.6 s CPU. __tests__/js-resolution-work.test.ts counts strips: a file with a destructured binding and 40 bare calls is stripped for the check once (41 on main), a 40-line function once (40 on main); each count fails when only its own half of the fix is reverted. full-pipeline.test.ts requires indexAll's and sync's durationMs to cover a linking pass delayed 250 ms past the orchestrator's time (on main: 218 < 468). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -35,6 +35,8 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
- In Dart, a type is now linked from more of the places it is written, not only from parameters and return types: the type an `extension … on` targets, a field's type, a top-level variable's type, generic arguments such as `Future<Report?>.value(null)` or Riverpod's `final reportProvider = Family<Report?, String>()`, and a local variable's type, a cast or a type check inside a function. So `callers` and impact for a model class now reach the extensions, models and providers that use it. Thanks @mg-mg-mg for the report. (#2327)
|
||||
- In Go, calls through a method receiver or parameter of an unexported type, the usual shape of gRPC and HTTP handlers (`s.service.AddItem()` inside `func (s *server) Create()`), now resolve, and always within that type's own package: another package's `server` with a same-named method no longer takes the call, and a method the type gets from a struct or interface it embeds is found too. Re-index Go projects after upgrading. Thanks @GoDiao for the report and the fix. (#2323)
|
||||
- In Go, calls into a module whose `go.mod` is not at the project root now resolve, whether it is a `server/` backend next to a `web/` frontend or one of several modules side by side as in etcd, so calls like `store.New()` and `s.db.CreateItem()` find their targets instead of being treated as calls into a third-party package. A name written through a package, like a `job.OPCommand` result type or a field of type `artifact.Manager`, now links to that package's symbol rather than a same-named one elsewhere, which also corrects links in projects with a single `go.mod`. Re-index Go projects after upgrading. Thanks @GoDiao for the report and @danusha2345 for the fix. (#2322)
|
||||
- Indexing a project that includes large bundled JavaScript files, such as a copy of pdf.js or d3, is fast again: since 1.6.2, resolving the calls in a JavaScript or TypeScript file re-read the file's text above each call, so a single bundled library could add many seconds to an index. The graph it builds is unchanged. Thanks @bompus for the report. (#2334)
|
||||
- The time `codegraph init` and `codegraph index` print beside the node and edge counts now covers the whole run, resolving references and linking included, and `codegraph sync` reports its whole run the same way. Before, it counted only reading and parsing the files, which can be a small part of an index, so a slow index looked fast. Thanks @bompus for the report. (#2334)
|
||||
|
||||
## [1.6.2] - 2026-10-03
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
* crawl).
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
@@ -269,4 +269,53 @@ describe('Integration: full pipeline', () => {
|
||||
cg.destroy();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
it('reports durationMs for the whole run, resolution and linking included (#2334)', async () => {
|
||||
// The CLI summary printed the orchestrator's time — extraction alone —
|
||||
// beside the totals of the whole run, so slow resolution never showed.
|
||||
generateSyntheticProject(tempDir, 10);
|
||||
const cg = await CodeGraph.init(tempDir, {
|
||||
config: { include: ['**/*.ts'], exclude: [] },
|
||||
});
|
||||
const internals = cg as unknown as {
|
||||
orchestrator: Record<'indexAll' | 'sync', (...args: unknown[]) => Promise<{ durationMs: number }>>;
|
||||
resolver: { resolveDeferredThisMemberRefs: () => Promise<number> };
|
||||
};
|
||||
// What the orchestrator measured; and a linking pass that takes a known while.
|
||||
let extraction = -1;
|
||||
const measure = (method: 'indexAll' | 'sync') => {
|
||||
const original = internals.orchestrator[method].bind(internals.orchestrator);
|
||||
vi.spyOn(internals.orchestrator, method).mockImplementation(async (...args: unknown[]) => {
|
||||
const result = await original(...args);
|
||||
extraction = result.durationMs;
|
||||
return result;
|
||||
});
|
||||
};
|
||||
const LINKING_MS = 250;
|
||||
const linking = internals.resolver.resolveDeferredThisMemberRefs.bind(internals.resolver);
|
||||
vi.spyOn(internals.resolver, 'resolveDeferredThisMemberRefs').mockImplementation(async () => {
|
||||
const until = Date.now() + LINKING_MS;
|
||||
while (Date.now() < until) await new Promise((r) => setTimeout(r, 10));
|
||||
return linking();
|
||||
});
|
||||
|
||||
try {
|
||||
measure('indexAll');
|
||||
const indexed = await cg.indexAll();
|
||||
expect(extraction).toBeGreaterThanOrEqual(0);
|
||||
expect(indexed.durationMs).toBeGreaterThanOrEqual(extraction + LINKING_MS);
|
||||
|
||||
// Likewise a sync that re-resolves a changed file.
|
||||
fs.appendFileSync(path.join(tempDir, 'src', 'index.ts'), 'export function again(): number { return entry(); }\n');
|
||||
extraction = -1;
|
||||
measure('sync');
|
||||
const synced = await cg.sync();
|
||||
expect(synced.filesModified).toBe(1);
|
||||
expect(extraction).toBeGreaterThanOrEqual(0);
|
||||
expect(synced.durationMs).toBeGreaterThanOrEqual(extraction + LINKING_MS);
|
||||
} finally {
|
||||
vi.restoreAllMocks();
|
||||
cg.destroy();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { CodeGraph } from '../src';
|
||||
import { stripCommentsForRegex } from '../src/resolution/strip-comments';
|
||||
|
||||
// Pass-through, so a test can count how often text is stripped.
|
||||
vi.mock('../src/resolution/strip-comments', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../src/resolution/strip-comments')>();
|
||||
return { ...actual, stripCommentsForRegex: vi.fn(actual.stripCommentsForRegex) };
|
||||
});
|
||||
|
||||
/**
|
||||
* #2334: resolving JS/TS does a bounded amount of work per file. In a file
|
||||
* with destructuring, the check for a call through a destructured name
|
||||
* stripped and scanned every line above each bare call; the check for a name
|
||||
* the calling function binds itself stripped that function's lines above each
|
||||
* reference again. A bundled library took time in the square of its size.
|
||||
* Counted, never timed.
|
||||
*/
|
||||
describe('JS resolution work (#2334)', () => {
|
||||
let tmpDir: string | undefined;
|
||||
let cg: CodeGraph | undefined;
|
||||
|
||||
afterEach(() => {
|
||||
vi.mocked(stripCommentsForRegex).mockClear();
|
||||
cg?.close();
|
||||
cg = undefined;
|
||||
if (tmpDir) fs.rmSync(tmpDir, { recursive: true, force: true, maxRetries: 5 });
|
||||
tmpDir = undefined;
|
||||
});
|
||||
|
||||
async function index(files: Record<string, string>): Promise<CodeGraph> {
|
||||
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-js-work-'));
|
||||
for (const [file, content] of Object.entries(files)) {
|
||||
fs.mkdirSync(path.dirname(path.join(tmpDir, file)), { recursive: true });
|
||||
fs.writeFileSync(path.join(tmpDir, file), content);
|
||||
}
|
||||
cg = CodeGraph.initSync(tmpDir);
|
||||
await cg.indexAll();
|
||||
return cg;
|
||||
}
|
||||
|
||||
/** How often a non-empty start of `text` was stripped as `lang`. */
|
||||
const strips = (lang: string, text: string) => vi.mocked(stripCommentsForRegex).mock.calls
|
||||
.filter(([stripped, as]) => as === lang && stripped.length > 0 && text.startsWith(stripped)).length;
|
||||
|
||||
const callers = (graph: CodeGraph, file: string, name: string) => {
|
||||
const target = graph.getNodesByName(name).find((n) => n.filePath === file && n.kind === 'function')!;
|
||||
return [...new Set(graph.getIncomingEdges(target.id).filter((e) => e.kind === 'calls')
|
||||
.map((e) => graph.getNode(e.source)?.name))];
|
||||
};
|
||||
|
||||
it('reads a file with destructuring once, not once per call', async () => {
|
||||
const CALLS = 40;
|
||||
const app = `import { useAuth } from './auth';\n` +
|
||||
`const { login } = useAuth();\n` +
|
||||
Array.from({ length: CALLS }, (_, i) => `function step${i}() { return ${i}; }\n`).join('') +
|
||||
`export function run() {\n` +
|
||||
Array.from({ length: CALLS }, (_, i) => ` step${i}();\n`).join('') +
|
||||
// A division the blanking reads as a regex literal around the call.
|
||||
` return (${CALLS}) / login() / 2;\n}\n`;
|
||||
const graph = await index({
|
||||
'auth.js': 'export function useAuth() {\n function login() {\n return 1;\n }\n return { login };\n}\n',
|
||||
'app.js': app,
|
||||
});
|
||||
// The call through the destructured name reaches the function the hook returns.
|
||||
expect(callers(graph, 'auth.js', 'login')).toEqual(['run']);
|
||||
expect(callers(graph, 'app.js', 'step7')).toEqual(['run']);
|
||||
// Each bare call was checked; the file was stripped for that once, not once per call.
|
||||
expect(strips('typescript', app)).toBeLessThan(CALLS / 4);
|
||||
});
|
||||
|
||||
it("strips a function's lines once, not once per reference", async () => {
|
||||
const LINES = 40;
|
||||
const host = `export function host() {\n` +
|
||||
Array.from({ length: LINES }, (_, i) => ` helper(${i});\n`).join('') + `}\n`;
|
||||
const graph = await index({ 'lib.js': `export function helper(n) {\n return n;\n}\n${host}` });
|
||||
// Each call resolved, so each asked whether `host` binds `helper` itself.
|
||||
expect(callers(graph, 'lib.js', 'helper')).toEqual(['host']);
|
||||
expect(strips('javascript', host)).toBeLessThan(LINES / 4);
|
||||
});
|
||||
});
|
||||
@@ -114,6 +114,12 @@ export interface IndexResult {
|
||||
nodesCreated: number;
|
||||
edgesCreated: number;
|
||||
errors: ExtractionError[];
|
||||
/**
|
||||
* Wall time in milliseconds. `CodeGraph.indexAll` reports the whole run —
|
||||
* scanning, parsing and storing, then resolving references and linking —
|
||||
* as it does the node and edge totals; the orchestrator's own result
|
||||
* covers only the files' extraction.
|
||||
*/
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
@@ -132,6 +138,11 @@ export interface SyncResult {
|
||||
filesModified: number;
|
||||
filesRemoved: number;
|
||||
nodesUpdated: number;
|
||||
/**
|
||||
* Wall time in milliseconds. `CodeGraph.sync` reports the whole sync,
|
||||
* resolution and linking included; the orchestrator's own result covers
|
||||
* only reconciling and re-extracting the files.
|
||||
*/
|
||||
durationMs: number;
|
||||
changedFilePaths?: string[];
|
||||
/** Paths not absorbed because reading or extraction failed; retain for status/retry. */
|
||||
|
||||
@@ -514,6 +514,7 @@ export class CodeGraph {
|
||||
*/
|
||||
async indexAll(options: IndexOptions = {}): Promise<IndexResult> {
|
||||
return this.indexMutex.withLock(async () => {
|
||||
const startedAt = Date.now();
|
||||
try {
|
||||
this.fileLock.acquire();
|
||||
} catch {
|
||||
@@ -775,6 +776,10 @@ export class CodeGraph {
|
||||
}
|
||||
} catch { /* metadata is advisory — never fail an index over it */ }
|
||||
|
||||
// The time covers the whole run, like the totals above. The
|
||||
// orchestrator's covers extraction alone, and the summary printed it
|
||||
// as the run's, leaving out resolution and linking (#2334).
|
||||
result.durationMs = Date.now() - startedAt;
|
||||
return result;
|
||||
} finally {
|
||||
// Restore the auto-checkpoint interval AFTER the fold-up above so the
|
||||
@@ -827,6 +832,7 @@ export class CodeGraph {
|
||||
*/
|
||||
async sync(options: IndexOptions = {}): Promise<SyncResult> {
|
||||
return this.indexMutex.withLock(async () => {
|
||||
const startedAt = Date.now();
|
||||
try {
|
||||
this.fileLock.acquire();
|
||||
} catch (err) {
|
||||
@@ -1146,6 +1152,8 @@ export class CodeGraph {
|
||||
this.orchestrator.finishGitIndexState(gitState, fullReconcile, result.failedFilePaths);
|
||||
|
||||
if (fullReconcile && result.filesChecked > 0) this.pendingFullReconcile = false;
|
||||
// The whole sync, as for indexAll: resolution and linking included (#2334).
|
||||
result.durationMs = Date.now() - startedAt;
|
||||
return result;
|
||||
} finally {
|
||||
// Mirror indexAll's teardown: stop the valve, then restore the
|
||||
|
||||
+323
-41
@@ -5509,9 +5509,30 @@ function jsFunctionLocalScope(name: string, ref: UnresolvedRef, context: Resolut
|
||||
if (hit !== undefined) return hit;
|
||||
let scope: { start: number; end: number } | null = null;
|
||||
const fn = context.getNodeById?.(ref.fromNodeId);
|
||||
if (fn && (fn.kind === 'function' || fn.kind === 'method') && fn.startLine <= ref.line && fn.endLine >= ref.line) {
|
||||
const lines = context.getFileLines?.(ref.filePath) ?? context.readFile(ref.filePath)?.split(/\r?\n/) ?? [];
|
||||
const text = stripCommentsForRegex(lines.slice(fn.startLine - 1, ref.line).join('\n'), 'javascript');
|
||||
if (fn && (fn.kind === 'function' || fn.kind === 'method') && fn.startLine <= ref.line && fn.endLine >= ref.line &&
|
||||
jsCodeBindsName(name, fn, ref, context)) {
|
||||
scope = { start: fn.startLine, end: fn.endLine };
|
||||
}
|
||||
memo.set(key, scope);
|
||||
return scope;
|
||||
}
|
||||
|
||||
/**
|
||||
* Per context, by `file\0first line\0reference line\0name`: whether the code
|
||||
* from a function's first line through a reference's binds the name. Every
|
||||
* function that starts on the same line has that code — all of a minified
|
||||
* script's do — so they share the answer (#2334).
|
||||
*/
|
||||
const JS_CODE_BINDS = new WeakMap<ResolutionContext, Map<string, boolean>>();
|
||||
|
||||
/** Whether the function's code through the reference's line declares `name`, or names it in a parameter list. */
|
||||
function jsCodeBindsName(name: string, fn: Node, ref: UnresolvedRef, context: ResolutionContext): boolean {
|
||||
let memo = JS_CODE_BINDS.get(context);
|
||||
if (!memo) JS_CODE_BINDS.set(context, (memo = new Map()));
|
||||
const key = `${ref.filePath}\0${fn.startLine}\0${ref.line}\0${name}`;
|
||||
let binds = memo.get(key);
|
||||
if (binds === undefined) {
|
||||
const text = jsFunctionCodeThrough(fn, ref, context);
|
||||
const { param } = localBindingPatterns(name, 'g');
|
||||
const n = name.replace(/\$/g, '\\$');
|
||||
// A plain declaration. Destructuring re-binds what a call returns under
|
||||
@@ -5521,10 +5542,46 @@ function jsFunctionLocalScope(name: string, ref: UnresolvedRef, context: Resolut
|
||||
// A parameter list — never a control-flow head (`if (openMarkerClose) {`).
|
||||
// A return type stays on its line, never a ternary's `: data.slice()` below `filter(canRowExpand)`.
|
||||
const parameter = new RegExp(`(?<!\\b(?:if|while|for|switch|with)\\s*)${param.source.replace('(?::[^=;{]*)?', '(?::[^=;{}()\\n]*)?')}`);
|
||||
if (declared || parameter.test(text)) scope = { start: fn.startLine, end: fn.endLine };
|
||||
binds = declared || parameter.test(text);
|
||||
memo.set(key, binds);
|
||||
}
|
||||
memo.set(key, scope);
|
||||
return scope;
|
||||
return binds;
|
||||
}
|
||||
|
||||
/**
|
||||
* Per context: the comment-stripped lines of the functions jsFunctionLocalScope
|
||||
* read last, and where each line ends in that text, most recent last.
|
||||
*/
|
||||
const JS_FN_CODE = new WeakMap<ResolutionContext, Map<string, { code: string; lineEnds: number[] }>>();
|
||||
const JS_FN_CODE_KEEP = 64;
|
||||
|
||||
/**
|
||||
* The function's lines from its first through the reference's, comments
|
||||
* blanked. Stripping looks at most one character ahead — past a line's end,
|
||||
* a newline, which completes no comment marker — so this is the whole
|
||||
* function's stripped text cut at that line's end. Stripping the lines again
|
||||
* for every reference took time in the square of a large function's length;
|
||||
* in a minified script, every function's text runs to the end of its one
|
||||
* line (#2334).
|
||||
*/
|
||||
function jsFunctionCodeThrough(fn: Node, ref: UnresolvedRef, context: ResolutionContext): string {
|
||||
let fns = JS_FN_CODE.get(context);
|
||||
if (!fns) JS_FN_CODE.set(context, (fns = new Map()));
|
||||
const key = `${ref.filePath}\0${fn.startLine}\0${fn.endLine}`;
|
||||
let own = fns.get(key);
|
||||
if (own) {
|
||||
fns.delete(key);
|
||||
} else {
|
||||
const lines = (context.getFileLines?.(ref.filePath) ?? context.readFile(ref.filePath)?.split(/\r?\n/) ?? [])
|
||||
.slice(fn.startLine - 1, fn.endLine);
|
||||
let end = -1;
|
||||
own = { code: stripCommentsForRegex(lines.join('\n'), 'javascript'), lineEnds: lines.map((line) => (end += line.length + 1)) };
|
||||
if (fns.size >= JS_FN_CODE_KEEP) fns.delete(fns.keys().next().value!);
|
||||
}
|
||||
fns.set(key, own);
|
||||
// Through the reference's line, or the file's last when it ends sooner.
|
||||
const last = Math.min(ref.line, fn.startLine - 1 + own.lineEnds.length) - fn.startLine;
|
||||
return last < 0 ? '' : own.code.slice(0, own.lineEnds[last]);
|
||||
}
|
||||
|
||||
const LOCAL_BINDING_MEMO = new WeakMap<ResolutionContext, Map<string, boolean>>();
|
||||
@@ -7017,6 +7074,10 @@ export function clearNameMatcherMemos(context: ResolutionContext): void {
|
||||
LOCAL_BINDING_SITES.delete(context);
|
||||
SELECTOR_NAMES.delete(context);
|
||||
GET_STATE_FILES.delete(context);
|
||||
DESTRUCTURED_FILES.delete(context);
|
||||
JS_FN_LOCAL_MEMO.delete(context);
|
||||
JS_CODE_BINDS.delete(context);
|
||||
JS_FN_CODE.delete(context);
|
||||
TS_FIELD_DECL_MEMO.delete(context);
|
||||
TS_CLASS_LINES.delete(context);
|
||||
TARGET_LANGUAGE.delete(context);
|
||||
@@ -9366,15 +9427,225 @@ function matchDestructuredStoreCall(ref: UnresolvedRef, context: ResolutionConte
|
||||
* and its source must return the key; a later declaration of the name at the
|
||||
* call's scope shadows the binding. The local binding otherwise ruled out
|
||||
* every cross-file candidate, so the call resolved to nothing.
|
||||
*
|
||||
* The file is read once (#2334): stripping and scanning all the text above
|
||||
* each call took time in the square of the file's size — seconds for each
|
||||
* bundled library (pdf.js, d3) a project ships. The text above a call,
|
||||
* blanked on its own, is the file's blanked code up to the call: blanking
|
||||
* reads past a character only for a comment opener's second character (a
|
||||
* call's name never starts with one) and for the `/` that closes a regex
|
||||
* literal (calls inside one: matchDestructuredCallInLiteral). Its bindings
|
||||
* are then the file's that end at or before the call: the pattern finds a
|
||||
* statement the same way wherever the two texts agree up to its `(`, and
|
||||
* one running past the call holds no `{` followed by a `(` to start another.
|
||||
*/
|
||||
function matchDestructuredCallResult(ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null {
|
||||
const source = context.readFile(ref.filePath);
|
||||
if (!source || !/\b(?:const|let|var)\s*\{/.test(source)) return null;
|
||||
const name = ref.referenceName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
if (!source) return null;
|
||||
const file = destructuredBindings(ref.filePath, source, context);
|
||||
if (!file) return null;
|
||||
const cut = destructuredCallOffset(file, source, ref);
|
||||
if (cut < 0) return matchDestructuredCallAbove(ref, source, context);
|
||||
const literal = regexLiteralAround(file.literals, cut);
|
||||
if (literal >= 0) {
|
||||
const depth = file.literalDepths[literal]!;
|
||||
return depth < 0 ? matchDestructuredCallAbove(ref, source, context)
|
||||
: matchDestructuredCallInLiteral(ref, file, file.literals[2 * literal]!, depth, cut, context);
|
||||
}
|
||||
const bindings = file.byName.get(ref.referenceName) ?? [];
|
||||
// The nearest binding above the call whose block is still open at the call.
|
||||
for (let i = bindings.length - 1; i >= 0; i--) {
|
||||
const { binding, key } = bindings[i]!;
|
||||
if (binding.end > cut || binding.scopeEnd < cut) continue;
|
||||
return destructuredCallTarget(binding.callee, key, file.code.slice(binding.end, cut), ref, context);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** A `const { … } = f(…)` statement. */
|
||||
interface DestructuredBinding {
|
||||
/** Just past its `(`. */
|
||||
end: number;
|
||||
/** The function it calls. */
|
||||
callee: string;
|
||||
/** The `{` of the block it sits in: its place on the stack of open blocks (-1 at top level), and where its `}` is (Infinity if nowhere). */
|
||||
depth: number;
|
||||
scopeEnd: number;
|
||||
}
|
||||
|
||||
/** What matchDestructuredCallResult needs of a file, read once. */
|
||||
interface DestructuredFile {
|
||||
/** The file with comments and string contents blanked. */
|
||||
code: string;
|
||||
/** Where each line starts. */
|
||||
lineStarts: number[];
|
||||
/** Each name a binding introduces → those bindings, in file order, with the key it reads. */
|
||||
byName: Map<string, Array<{ binding: DestructuredBinding; key: string }>>;
|
||||
/** The opening and closing offset of each span the blanking read as a regex literal, in order. */
|
||||
literals: number[];
|
||||
/** Per literal: how many blocks are open at its `/`, or -1 when a binding statement could run across that `/`. */
|
||||
literalDepths: number[];
|
||||
}
|
||||
|
||||
const DESTRUCTURED_BINDING = /\b(?:const|let|var)\s*\{([^{}]*)\}\s*=\s*(?:await\s+)?([A-Za-z_$][\w$]*)\s*(?:<[^<>()]*>)?\s*\(/g;
|
||||
/** Per context: the files matchDestructuredCallResult read last (calls arrive file by file); null for one without destructuring. */
|
||||
const DESTRUCTURED_FILES = new WeakMap<ResolutionContext, Map<string, DestructuredFile | null>>();
|
||||
const DESTRUCTURED_FILES_KEEP = 16;
|
||||
|
||||
function destructuredBindings(filePath: string, source: string, context: ResolutionContext): DestructuredFile | null {
|
||||
let files = DESTRUCTURED_FILES.get(context);
|
||||
if (!files) DESTRUCTURED_FILES.set(context, (files = new Map()));
|
||||
let file = files.get(filePath);
|
||||
if (file !== undefined) return file;
|
||||
file = /\b(?:const|let|var)\s*\{/.test(source) ? readDestructuredBindings(source) : null;
|
||||
if (files.size >= DESTRUCTURED_FILES_KEEP) files.delete(files.keys().next().value!);
|
||||
files.set(filePath, file);
|
||||
return file;
|
||||
}
|
||||
|
||||
function readDestructuredBindings(source: string): DestructuredFile {
|
||||
const literals: number[] = [];
|
||||
const code = blankStringContents(stripCommentsForRegex(source, 'typescript'), literals);
|
||||
const lineStarts = [0];
|
||||
for (let at = source.indexOf('\n'); at !== -1; at = source.indexOf('\n', at + 1)) lineStarts.push(at + 1);
|
||||
const found = Array.from(code.matchAll(DESTRUCTURED_BINDING), (m) => ({
|
||||
index: m.index!,
|
||||
binding: { end: m.index! + m[0].length, callee: m[2]!, depth: -1, scopeEnd: Infinity },
|
||||
keys: destructuredKeys(m[1]!),
|
||||
}));
|
||||
// The `{` after every `const`, `let` and `var`: where a binding statement's pattern opens.
|
||||
const patterns = new Set(Array.from(code.matchAll(/\b(?:const|let|var)\s*\{/g), (m) => m.index! + m[0].length - 1));
|
||||
const literalDepths: number[] = [];
|
||||
const stack: number[] = [];
|
||||
const scoped = new Map<number, DestructuredBinding[]>();
|
||||
let lastBrace = -1;
|
||||
let lastBracket = -1; // the last `<`, `>`, `(` or `)`
|
||||
for (let i = 0, next = 0, literal = 0; i < code.length; i++) {
|
||||
if (found[next]?.index === i) {
|
||||
const { binding } = found[next++]!;
|
||||
// In scope until the `}` that closes the block it sits in.
|
||||
binding.depth = stack.length - 1;
|
||||
const open = stack[binding.depth];
|
||||
if (open !== undefined) {
|
||||
const list = scoped.get(open);
|
||||
if (list) list.push(binding);
|
||||
else scoped.set(open, [binding]);
|
||||
}
|
||||
}
|
||||
if (literals[2 * literal] === i) {
|
||||
// A binding statement holds a `/` only in its pattern `{ … }` or its type arguments `< … >`.
|
||||
literalDepths.push(patterns.has(lastBrace) || code[lastBracket] === '<' ? -1 : stack.length);
|
||||
literal++;
|
||||
}
|
||||
const c = code.charCodeAt(i);
|
||||
if (c === 123 /* { */) {
|
||||
stack.push(i);
|
||||
lastBrace = i;
|
||||
} else if (c === 125 /* } */) {
|
||||
lastBrace = i;
|
||||
const open = stack.pop();
|
||||
for (const binding of (open !== undefined && scoped.get(open)) || []) binding.scopeEnd = i;
|
||||
} else if (c === 60 /* < */ || c === 62 /* > */ || c === 40 /* ( */ || c === 41 /* ) */) {
|
||||
lastBracket = i;
|
||||
}
|
||||
}
|
||||
const byName = new Map<string, Array<{ binding: DestructuredBinding; key: string }>>();
|
||||
for (const { binding, keys } of found) {
|
||||
for (const [name, key] of keys) {
|
||||
const list = byName.get(name);
|
||||
if (list) list.push({ binding, key });
|
||||
else byName.set(name, [{ binding, key }]);
|
||||
}
|
||||
}
|
||||
return { code, lineStarts, byName, literals, literalDepths };
|
||||
}
|
||||
|
||||
/** Each name a destructuring pattern binds → the key it reads (`{ a, b: c = 1 }`); a later entry for a name wins. */
|
||||
function destructuredKeys(pattern: string): Map<string, string> {
|
||||
const keys = new Map<string, string>();
|
||||
for (const part of pattern.split(',')) {
|
||||
const [k, v] = part.split(':').map((x) => x.trim().replace(/\s*=.*$/, ''));
|
||||
if (/^[A-Za-z_$][\w$]*$/.test(k ?? '')) keys.set(v ?? k!, k!);
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
|
||||
/**
|
||||
* Where the text above a call ends in `source` (that text is
|
||||
* `source.slice(0, cut)`), or -1 when the call's line or column lies outside
|
||||
* the file, or that text ends in a `/` that opens a comment in the file.
|
||||
*/
|
||||
function destructuredCallOffset(file: DestructuredFile, source: string, ref: UnresolvedRef): number {
|
||||
const { line, column } = ref;
|
||||
const starts = file.lineStarts;
|
||||
if (!Number.isInteger(line) || !Number.isInteger(column) || line < 1 || line > starts.length || column < 0) return -1;
|
||||
const cut = Math.min(starts[line - 1]! + column, line < starts.length ? starts[line]! - 1 : source.length);
|
||||
return source[cut - 1] === '/' && (source[cut] === '/' || source[cut] === '*') ? -1 : cut;
|
||||
}
|
||||
|
||||
/** The regex literal (its index) that opens before `cut` and closes at or after it, or -1. */
|
||||
function regexLiteralAround(literals: number[], cut: number): number {
|
||||
let lo = 0;
|
||||
let hi = literals.length / 2;
|
||||
while (lo < hi) {
|
||||
const mid = (lo + hi) >> 1;
|
||||
if (literals[2 * mid]! < cut) lo = mid + 1;
|
||||
else hi = mid;
|
||||
}
|
||||
return lo > 0 && literals[2 * lo - 1]! >= cut ? lo - 1 : -1;
|
||||
}
|
||||
|
||||
/**
|
||||
* matchDestructuredCallResult for a call inside what the file's blanking read
|
||||
* as a regex literal opening at `start`. Blanked on its own, the text above
|
||||
* the call reads that `/` as division: it is the file's code up to there,
|
||||
* then `tail` — the rest, blanked as text of its own. No binding statement
|
||||
* runs across the `/` (literalDepths), so the call sees the file's bindings
|
||||
* that end before it, then any in `tail`; and of the `depth` blocks open at
|
||||
* the `/`, those `tail` does not close.
|
||||
*/
|
||||
function matchDestructuredCallInLiteral(
|
||||
ref: UnresolvedRef, file: DestructuredFile, start: number, depth: number, cut: number, context: ResolutionContext
|
||||
): ResolvedRef | null {
|
||||
const tail = blankStringContents(file.code.slice(start, cut));
|
||||
const atCall = blocksThrough(tail, depth);
|
||||
for (const m of Array.from(tail.matchAll(DESTRUCTURED_BINDING)).reverse()) {
|
||||
const key = destructuredKeys(m[1]!).get(ref.referenceName);
|
||||
if (!key) continue;
|
||||
const atBinding = blocksThrough(tail.slice(0, m.index), depth);
|
||||
const block = atBinding.opened[atBinding.opened.length - 1];
|
||||
if (block !== undefined ? !atCall.opened.includes(block) : atBinding.enclosing > atCall.enclosing) continue;
|
||||
return destructuredCallTarget(m[2]!, key, tail.slice(m.index! + m[0].length), ref, context);
|
||||
}
|
||||
const bindings = file.byName.get(ref.referenceName) ?? [];
|
||||
for (let i = bindings.length - 1; i >= 0; i--) {
|
||||
const { binding, key } = bindings[i]!;
|
||||
if (binding.end > start) continue;
|
||||
if (binding.depth >= 0 && (binding.scopeEnd < start || binding.depth >= atCall.enclosing)) continue;
|
||||
return destructuredCallTarget(binding.callee, key, file.code.slice(binding.end, start) + tail, ref, context);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** After `text`'s braces: how many of the `enclosing` blocks open before it are still open, and which it opened are. */
|
||||
function blocksThrough(text: string, enclosing: number): { enclosing: number; opened: number[] } {
|
||||
const opened: number[] = [];
|
||||
for (let i = 0; i < text.length; i++) {
|
||||
const c = text.charCodeAt(i);
|
||||
if (c === 123 /* { */) opened.push(i);
|
||||
else if (c === 125 /* } */) {
|
||||
if (opened.length > 0) opened.pop();
|
||||
else if (enclosing > 0) enclosing--;
|
||||
}
|
||||
}
|
||||
return { enclosing, opened };
|
||||
}
|
||||
|
||||
/** matchDestructuredCallResult by scanning the text above the call, where the whole file's reading may not hold. */
|
||||
function matchDestructuredCallAbove(ref: UnresolvedRef, source: string, context: ResolutionContext): ResolvedRef | null {
|
||||
const lines = source.split('\n');
|
||||
const before = lines.slice(0, ref.line - 1).concat(lines[ref.line - 1]?.slice(0, ref.column) ?? '').join('\n');
|
||||
const code = blankStringContents(stripCommentsForRegex(before, 'typescript'));
|
||||
const binding = /\b(?:const|let|var)\s*\{([^{}]*)\}\s*=\s*(?:await\s+)?([A-Za-z_$][\w$]*)\s*(?:<[^<>()]*>)?\s*\(/g;
|
||||
const stackAt = (end: number): number[] => {
|
||||
const stack: number[] = [];
|
||||
for (let i = 0; i < end; i++) {
|
||||
@@ -9384,44 +9655,55 @@ function matchDestructuredCallResult(ref: UnresolvedRef, context: ResolutionCont
|
||||
return stack;
|
||||
};
|
||||
const callScope = stackAt(code.length);
|
||||
for (const m of [...code.matchAll(binding)].reverse()) {
|
||||
let key: string | null = null;
|
||||
for (const part of m[1]!.split(',')) {
|
||||
const [k, v] = part.split(':').map((x) => x.trim().replace(/\s*=.*$/, ''));
|
||||
if ((v ?? k) === ref.referenceName && /^[A-Za-z_$][\w$]*$/.test(k ?? '')) key = k!;
|
||||
}
|
||||
for (const m of [...code.matchAll(DESTRUCTURED_BINDING)].reverse()) {
|
||||
const key = destructuredKeys(m[1]!).get(ref.referenceName);
|
||||
if (!key) continue;
|
||||
if (!stackAt(m.index!).every((pos, i) => callScope[i] === pos)) continue;
|
||||
const rest = code.slice(m.index! + m[0].length);
|
||||
if (new RegExp(`\\b(?:const|let|var|function|class)\\s+(?:${name}\\b|\\{[^}]*\\b${name}\\b)`).test(rest)) return null;
|
||||
const calleeName = m[2]!;
|
||||
const imported = context.resolveImport?.({ ...ref, referenceName: calleeName, referenceKind: 'calls' });
|
||||
// Through the import; else the same file's; else the one function of that
|
||||
// name in the project (an alias the import resolver can't follow, like
|
||||
// Nuxt's `~/composables/…`) — the returned key is checked below either way.
|
||||
const holders = context.getNodesByName(calleeName).filter((n) =>
|
||||
(n.kind === 'function' || n.kind === 'constant' || n.kind === 'variable') && sameLanguageFamily(n.language, ref.language));
|
||||
const callee = (imported && context.getNodeById?.(imported.targetNodeId)) ??
|
||||
holders.find((n) => n.filePath === ref.filePath) ??
|
||||
(holders.length === 1 ? holders[0] : undefined);
|
||||
if (!callee || !sameLanguageFamily(callee.language, ref.language)) return null;
|
||||
const calleeText = (context.getFileLines?.(callee.filePath) ?? context.readFile(callee.filePath)?.split('\n') ?? [])
|
||||
.slice(callee.startLine - 1, callee.endLine).join('\n');
|
||||
if (!new RegExp(`\\breturn\\s*\\{[^]*?\\b${key}\\b`).test(calleeText)) return null;
|
||||
const callable = (n: Node) => n.kind === 'function' || n.kind === 'method' || n.kind === 'constant' || n.kind === 'variable';
|
||||
const inFile = context.getNodesInFile(callee.filePath);
|
||||
const inner = inFile.filter((n) => n.name === key && callable(n) && n.id !== callee.id && rangeWithin(n, callee) &&
|
||||
!inFile.some((f) => f.id !== callee.id && f.id !== n.id && (f.kind === 'function' || f.kind === 'method') &&
|
||||
rangeWithin(f, callee) && rangeWithin(n, f) && !sameRange(f, n)));
|
||||
const top = inner.length > 0 ? inner : inFile.filter((n) => n.name === key && callable(n) && !n.qualifiedName.includes('::') &&
|
||||
!inFile.some((f) => (f.kind === 'function' || f.kind === 'method') && f.id !== n.id && rangeWithin(n, f) && !sameRange(f, n)));
|
||||
const target = top.sort((a, b) => Number(b.kind === 'function') - Number(a.kind === 'function'))[0];
|
||||
if (!target) return null;
|
||||
return { original: ref, targetNodeId: target.id, confidence: 0.85, resolvedBy: 'instance-method' };
|
||||
return destructuredCallTarget(m[2]!, key, code.slice(m.index! + m[0].length), ref, context);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The function `calleeName` returns under `key`, unless a declaration in
|
||||
* `rest` (the code between the binding and the call) shadows the binding.
|
||||
* The callee is resolved first: most callees (`require`) return no such
|
||||
* function, and then the shadowing scan is not needed.
|
||||
*/
|
||||
function destructuredCallTarget(calleeName: string, key: string, rest: string, ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null {
|
||||
const target = destructuredKeyTarget(calleeName, key, ref, context);
|
||||
if (!target) return null;
|
||||
const name = ref.referenceName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
return new RegExp(`\\b(?:const|let|var|function|class)\\s+(?:${name}\\b|\\{[^}]*\\b${name}\\b)`).test(rest) ? null : target;
|
||||
}
|
||||
|
||||
/** The function `calleeName` returns under `key`: one declared in its body, else a top-level one of its module. */
|
||||
function destructuredKeyTarget(calleeName: string, key: string, ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null {
|
||||
const imported = context.resolveImport?.({ ...ref, referenceName: calleeName, referenceKind: 'calls' });
|
||||
// Through the import; else the same file's; else the one function of that
|
||||
// name in the project (an alias the import resolver can't follow, like
|
||||
// Nuxt's `~/composables/…`) — the returned key is checked below either way.
|
||||
const holders = context.getNodesByName(calleeName).filter((n) =>
|
||||
(n.kind === 'function' || n.kind === 'constant' || n.kind === 'variable') && sameLanguageFamily(n.language, ref.language));
|
||||
const callee = (imported && context.getNodeById?.(imported.targetNodeId)) ??
|
||||
holders.find((n) => n.filePath === ref.filePath) ??
|
||||
(holders.length === 1 ? holders[0] : undefined);
|
||||
if (!callee || !sameLanguageFamily(callee.language, ref.language)) return null;
|
||||
const calleeText = (context.getFileLines?.(callee.filePath) ?? context.readFile(callee.filePath)?.split('\n') ?? [])
|
||||
.slice(callee.startLine - 1, callee.endLine).join('\n');
|
||||
if (!new RegExp(`\\breturn\\s*\\{[^]*?\\b${key}\\b`).test(calleeText)) return null;
|
||||
const callable = (n: Node) => n.kind === 'function' || n.kind === 'method' || n.kind === 'constant' || n.kind === 'variable';
|
||||
const inFile = context.getNodesInFile(callee.filePath);
|
||||
const inner = inFile.filter((n) => n.name === key && callable(n) && n.id !== callee.id && rangeWithin(n, callee) &&
|
||||
!inFile.some((f) => f.id !== callee.id && f.id !== n.id && (f.kind === 'function' || f.kind === 'method') &&
|
||||
rangeWithin(f, callee) && rangeWithin(n, f) && !sameRange(f, n)));
|
||||
const top = inner.length > 0 ? inner : inFile.filter((n) => n.name === key && callable(n) && !n.qualifiedName.includes('::') &&
|
||||
!inFile.some((f) => (f.kind === 'function' || f.kind === 'method') && f.id !== n.id && rangeWithin(n, f) && !sameRange(f, n)));
|
||||
const target = top.sort((a, b) => Number(b.kind === 'function') - Number(a.kind === 'function'))[0];
|
||||
if (!target) return null;
|
||||
return { original: ref, targetNodeId: target.id, confidence: 0.85, resolvedBy: 'instance-method' };
|
||||
}
|
||||
|
||||
/** Bound action names need not have a same-named definition (selectors may
|
||||
* rename them). The resolver's symbol-existence prefilter must allow them. */
|
||||
export function matchJsStoreBindingCall(ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null {
|
||||
|
||||
@@ -27,8 +27,10 @@
|
||||
* Blank string contents while preserving quotes and offsets. Template
|
||||
* interpolations are blanked too; callers checking executable expressions
|
||||
* must conservatively inspect those expressions in the original source.
|
||||
* `regexLiterals`, when given, receives the offsets of the opening and
|
||||
* closing `/` of every span read as a regex literal (left as is).
|
||||
*/
|
||||
export function blankStringContents(text: string): string {
|
||||
export function blankStringContents(text: string, regexLiterals?: number[]): string {
|
||||
const out = text.split('');
|
||||
let i = 0;
|
||||
const n = text.length;
|
||||
@@ -45,7 +47,11 @@ export function blankStringContents(text: string): string {
|
||||
if (text[end] === ']') inClass = false;
|
||||
if (text[end] === '/' && !inClass) break;
|
||||
}
|
||||
if (end < n && text[end] === '/') { i = end + 1; continue; }
|
||||
if (end < n && text[end] === '/') {
|
||||
regexLiterals?.push(i, end);
|
||||
i = end + 1;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if (c === '"' || c === "'" || c === '`') {
|
||||
const quote = c;
|
||||
|
||||
Reference in New Issue
Block a user