mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-16 20:53:57 +08:00
fix(ts): a re-export imports its module, and sync follows barrels of barrels (#2526)
An `export … from './x'` statement emitted one `imports` reference per re-exported name and nothing for the module itself; `export * from` emitted nothing at all. So a barrel made only of `export *` lines had no dependency on what it re-exports (no dependent for `affected`), and sync could not find an `index.ts` that re-exports another barrel: when the inner one was deleted or added, the outer one's importers kept their stale or name-matched edges. A re-export now imports its module the way `import … from` does, in both extractors (TS and the Rust kernel): an import node named by the specifier and a module reference that links the barrel to the file it names. Sync's importer discovery follows such modules to their own importers, carrying the names each passes on (`export *`, a named or renamed re-export, an `export * as ns`), as deep as the resolver chases a re-export; an ES namespace import reads whatever its module passes on. A re-export's import node does not count as a symbol for the "parse produced no symbols" warning, so a barrel with an unparseable line still reports it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -81,6 +81,8 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
- `codegraph sync` and the file watcher now do the same for a Python import under a name of its own, like `from .models import User as UserModel`, `from . import helpers as h` or `from app.pkg import sub as subpkg`: when the module or package is added after the importing file was indexed, comes back after being deleted, or gets the imported name in a later edit, the import and every use of the alias, such as a call, a base class or the class passed as a value, now link to it. Before, they stayed unlinked until the importing file changed or the project was re-indexed. Re-index Python projects after upgrading to pick up links an earlier version missed.
|
||||
- `codegraph sync` and the file watcher now also link a Python import of a name a package re-exports, like `from fastapi import FastAPI` through `fastapi/__init__.py`, when that `__init__.py` is added after the importing files were indexed or gains the re-export in a later edit. Before, calls through the name stayed unlinked until each importing file changed or the project was re-indexed. Large projects gain the most: sync also used to skip, in any language, a module or package that more than 500 references were waiting for, and now restoring Django's `django/urls/__init__.py` relinks every file that imports `path` from it. Re-index Python projects after upgrading to pick up links an earlier version missed.
|
||||
- `codegraph sync` and the file watcher now update the files that import through a module that re-exports what it imports, like a package's `__init__.py` or an `index.ts` barrel, when that module is deleted or added, and in Python also the files that reach it through another package re-exporting it. Before, deleting `shop/__init__.py` left every `from shop import Order` linked to the class it re-exported, and deleting `components/index.ts` left every `import { Button } from './components'` linked, until each importing file changed or the project was re-indexed. When such a module came back, the imports stayed linked the way they were while it was missing, and in a large Python project an import of a module whose name is as common as `models` or `test`, like `from django.test import TestCase`, stayed unlinked. Python, JavaScript, TypeScript, Vue, Svelte and Astro.
|
||||
- In JavaScript and TypeScript, a file that re-exports a module, like an `index.ts` with `export * from './button'` or `export { Button } from './button'`, now counts as importing that module, the way an `import` statement does. `codegraph affected` and a module's dependents now include the barrels that re-export it; before, a barrel made only of `export *` lines showed no dependency at all. Re-index JavaScript and TypeScript projects after upgrading.
|
||||
- `codegraph sync` and the file watcher now also update the files that import through an `index.ts` re-exporting another barrel, like `export * from './dropdown-menu'`, when that inner barrel is deleted or added. Before, their imports stayed linked through the deleted barrel, or stayed linked the way they were while it was missing, until each importing file changed or the project was re-indexed.
|
||||
- In C, C++ and Objective-C, a struct or class member declared as a pointer, reference, array or function, like `SharedState* shared;`, `jv elements[];` or `virtual Status Put(…) = 0;`, no longer makes the struct look like it inherits from the member's type. Before, `codegraph_explore` and impact listed those types as base classes, every struct holding a pointer to a type as one of its subclasses, and a method of that struct as an override of the type's method with the same name. Real base classes like `class Derived : public Base`, Objective-C superclasses and Go's embedded fields are unchanged. Re-index C, C++ and Objective-C projects after upgrading.
|
||||
- In C++, a call on a variable, parameter or member declared through a `typedef` or `using` alias now links to the method of the type the alias names, with the alias looked up the way C++ does it: in the calling function, its class and the classes that class inherits from, then the namespaces around it. In google/leveldb, `table_.Insert(buf)` on a `Table table_;` declared next to `typedef SkipList<const char*, KeyComparator> Table;` now reaches `SkipList::Insert` instead of the unrelated `HandleTable::Insert`, and `Table::Iterator iter(&table_);` reaches `SkipList::Iterator`'s constructor. A class that only shares the alias's name is no longer taken for it, and when the aliased type has no such method, as with an alias of `std::vector` or of a template parameter, no method is guessed from the receiver's name. Re-index C++ projects after upgrading.
|
||||
- In C++, a call on a variable, parameter or member declared with template arguments nested inside its template arguments, like `autovector<std::pair<int, FileMetaData*>>` or `Striped<CacheAlignedWrapper<port::Mutex>>`, now links to the method of the class it is declared as. Before, the nested arguments hid that class: in RocksDB, `files_marked_for_compaction_.clear()` on such an `autovector` linked to an unrelated `CompactionInputFiles::clear` that only shares words with the variable's name, and `mutex_.Get(key)` on a `Striped` linked to nothing. Re-index C++ projects after upgrading.
|
||||
|
||||
@@ -10555,6 +10555,34 @@ export const registry = [widget];
|
||||
expect(names).toContain('beta');
|
||||
});
|
||||
|
||||
it('a re-export imports its source module, as an import statement does', () => {
|
||||
const result = extractFromSource(
|
||||
'barrel.ts',
|
||||
`export * from './star';\nexport { alpha } from './named';\nexport * as ns from './space';\n`
|
||||
);
|
||||
const modules = ['./star', './named', './space'];
|
||||
expect(result.nodes.filter((n) => n.kind === 'import').map((n) => n.name)).toEqual(modules);
|
||||
const refs = result.unresolvedReferences.filter((r) => r.referenceKind === 'imports').map((r) => r.referenceName);
|
||||
for (const module of modules) expect(refs).toContain(module);
|
||||
expect(refs).toContain('alpha');
|
||||
});
|
||||
|
||||
it('a barrel that only re-exports everything is a dependent of the module', async () => {
|
||||
const dir = createTempDir();
|
||||
try {
|
||||
fs.mkdirSync(path.join(dir, 'src'), { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, 'src', 'foo.ts'), `export function helper(): void {}\n`);
|
||||
fs.writeFileSync(path.join(dir, 'src', 'index.ts'), `export * from './foo';\n`);
|
||||
const cg = CodeGraph.initSync(dir, { config: { include: ['src/**/*.ts'], exclude: [] } });
|
||||
await cg.indexAll();
|
||||
cg.resolveReferences();
|
||||
expect(cg.getFileDependents('src/foo.ts')).toContain('src/index.ts');
|
||||
cg.destroy();
|
||||
} finally {
|
||||
cleanupTempDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it('a value imported/re-exported but never called still makes the importer a dependent', async () => {
|
||||
const dir = createTempDir();
|
||||
try {
|
||||
|
||||
@@ -53,13 +53,22 @@ const PYTHON: Files = {
|
||||
};
|
||||
const PYTHON_BARRELS = ['app/shop/__init__.py', 'lib/api.py', 'app/orm/__init__.py'];
|
||||
|
||||
/** ES modules: an `index.ts` re-exporting under the same name and under another. */
|
||||
/**
|
||||
* ES modules: an `index.ts` re-exporting under the same name and under
|
||||
* another, and modules re-exporting it in turn — an outer `index.ts`, all of
|
||||
* it and one name renamed, and a module passing on one name, read through a
|
||||
* namespace import — imported in its stead.
|
||||
*/
|
||||
const TS: Files = {
|
||||
'package.json': '{"name":"app"}',
|
||||
'src/components/Button.ts': 'export function Button() { return 1; }\n',
|
||||
'src/components/Icon.ts': 'export function BaseIcon() { return 2; }\n',
|
||||
'src/components/index.ts': "export { Button } from './Button';\nexport { BaseIcon as Icon } from './Icon';\n",
|
||||
'src/app.ts': "import { Button, Icon } from './components';\nexport function render() { return [Button(), Icon()]; }\n",
|
||||
'src/ui/index.ts': "export * from '../components';\nexport { Icon as UiIcon } from '../components';\n",
|
||||
'src/page.ts': "import { Button, UiIcon } from './ui';\nexport function page() { return [Button(), UiIcon()]; }\n",
|
||||
'src/kit.ts': "export { Button as KitButton } from './components';\n",
|
||||
'src/screen.ts': "import * as kit from './kit';\nexport function screen() { return kit.KitButton(); }\n",
|
||||
};
|
||||
const TS_BARRELS = ['src/components/index.ts'];
|
||||
|
||||
|
||||
@@ -721,7 +721,10 @@ impl<'t> Walker<'t> {
|
||||
} else if kind == "import_statement" {
|
||||
self.extract_import(node);
|
||||
} else if kind == "export_statement" && node.child_by_field_name("source").is_some() {
|
||||
// Re-export: `export { X } from './y'`.
|
||||
// Re-export: `export { X } from './y'`, `export * from './y'`. It
|
||||
// imports the module as `import … from` does (an import node and a
|
||||
// reference to the module), then links each re-exported name.
|
||||
self.extract_import(node);
|
||||
self.emit_re_export_refs(node);
|
||||
} else if kind == "export_statement" && self.looks_like_vue_store_file() {
|
||||
// Vuex MODULE default export (`export default { actions: {…} }`).
|
||||
|
||||
@@ -564,6 +564,8 @@ export class TreeSitterExtractor {
|
||||
private fileScopeValueCounts = new Map<string, number>(); // file-scope nodes per name (conditional-def detection)
|
||||
private valueRefScopes: Array<{ id: string; node: SyntaxNode; name: string }> = [];
|
||||
private errors: ExtractionError[] = [];
|
||||
// Import nodes made for `export … from` statements: no symbols of the file's own.
|
||||
private reExportImports = 0;
|
||||
private extractor: LanguageExtractor | null = null;
|
||||
private nodeStack: string[] = []; // Stack of parent node IDs
|
||||
// C/C++ enclosing `namespace ns { … }` names, prepended to every contained
|
||||
@@ -733,7 +735,9 @@ export class TreeSitterExtractor {
|
||||
this.unresolvedReferences.push(...commonJsRequireRefs(this.filePath, this.source, this.language));
|
||||
|
||||
// hasError is routine for several grammars; warn only when no symbols survived.
|
||||
const symbolCount = this.nodes.filter((n) => n.kind !== 'file').length;
|
||||
// (A re-export's import of its module is none: a barrel whose other lines
|
||||
// failed to parse has lost what it passes on.)
|
||||
const symbolCount = this.nodes.filter((n) => n.kind !== 'file').length - this.reExportImports;
|
||||
if (this.tree?.rootNode.hasError && symbolCount === 0) {
|
||||
this.errors.push({
|
||||
message:
|
||||
@@ -1577,13 +1581,16 @@ export class TreeSitterExtractor {
|
||||
else if (this.extractor.importTypes.includes(nodeType)) {
|
||||
this.extractImport(node);
|
||||
}
|
||||
// Re-export from another module — `export { X } from './y'` (TS/JS). A
|
||||
// re-export is a dependency on the source module just like an import, but
|
||||
// the export_statement is otherwise only descended into (no declaration to
|
||||
// extract), so a barrel that ONLY re-exports produced zero edges and showed
|
||||
// 0 dependents. Link each re-exported name to its definition. Children are
|
||||
// still visited (a non-re-export `export const X = …` has no `source` and
|
||||
// falls through to its normal declaration extraction).
|
||||
// Re-export from another module — `export { X } from './y'`, `export * from
|
||||
// './y'` (TS/JS). A re-export is a dependency on the source module just like
|
||||
// an import, but the export_statement is otherwise only descended into (no
|
||||
// declaration to extract), so a barrel that ONLY re-exports produced zero
|
||||
// edges and showed 0 dependents. It imports the module as `import … from`
|
||||
// does — an import node and a reference to the module, which links the
|
||||
// barrel to the file it names (so a barrel of barrels can be followed) —
|
||||
// and links each re-exported name to its definition. Children are still
|
||||
// visited (a non-re-export `export const X = …` has no `source` and falls
|
||||
// through to its normal declaration extraction).
|
||||
else if (
|
||||
nodeType === 'export_statement' &&
|
||||
(this.language === 'typescript' || this.language === 'tsx' ||
|
||||
@@ -1591,6 +1598,9 @@ export class TreeSitterExtractor {
|
||||
this.language === 'arkts') &&
|
||||
getChildByField(node, 'source')
|
||||
) {
|
||||
const before = this.nodes.length;
|
||||
this.extractImport(node);
|
||||
this.reExportImports += this.nodes.length - before;
|
||||
const parentId = this.nodeStack[this.nodeStack.length - 1];
|
||||
if (parentId) this.emitReExportRefs(node, parentId);
|
||||
}
|
||||
|
||||
@@ -2732,7 +2732,7 @@ function findGoPackageMember(
|
||||
/** Recursive depth cap for re-export chain following. Real codebases
|
||||
* rarely chain barrels more than 2–3 deep; 8 is a generous safety
|
||||
* net that still bounds worst-case work. */
|
||||
const REEXPORT_MAX_DEPTH = 8;
|
||||
export const REEXPORT_MAX_DEPTH = 8;
|
||||
|
||||
/**
|
||||
* Find an exported symbol in `filePath`, following `export { x } from
|
||||
|
||||
@@ -11,20 +11,18 @@
|
||||
* binds them through the module when it is there, and another way (by name)
|
||||
* or not at all when it is not.
|
||||
*
|
||||
* A Python module that re-exports another one (`from .shop import *`, or a
|
||||
* package's `__init__.py` importing names from it) is the way in for its own
|
||||
* importers, so theirs are followed too, as deep as the resolver chases a
|
||||
* name through such modules. An ES module's `export … from` leaves no import
|
||||
* edge or node to find it by, so an `index.ts` re-exporting another is not
|
||||
* followed.
|
||||
* A module that re-exports another one (`export * from './components'` in an
|
||||
* `index.ts`, `from .shop import *` or a package's `__init__.py` importing
|
||||
* names from it) is the way in for its own importers, so theirs are followed
|
||||
* too, as deep as the resolver chases a name through such modules.
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import type { QueryBuilder } from '../db/queries';
|
||||
import type { Language, UnresolvedReference } from '../types';
|
||||
import type { ImportMapping } from './types';
|
||||
import { extractImportMappings, pythonModulePath } from './import-resolver';
|
||||
import type { ImportMapping, ReExport } from './types';
|
||||
import { extractImportMappings, extractReExports, pythonModulePath, REEXPORT_MAX_DEPTH } from './import-resolver';
|
||||
import { resurrectRefFromDroppedEdge } from './resurrect-ref';
|
||||
|
||||
/**
|
||||
@@ -37,11 +35,12 @@ const BINDING_LANGUAGES = new Set<Language>([
|
||||
]);
|
||||
|
||||
/**
|
||||
* How many re-exporting Python modules deep the importers of a file are
|
||||
* followed: as deep as the resolver chases a name through star imports and
|
||||
* re-exports (`pythonModuleSymbol`).
|
||||
* How many re-exporting modules deep the importers of a file are followed: as
|
||||
* deep as the resolver chases a name through them, three for Python's star
|
||||
* imports and re-exports (`pythonModuleSymbol`), `REEXPORT_MAX_DEPTH` for
|
||||
* ES-module `export … from`.
|
||||
*/
|
||||
const PYTHON_REEXPORT_DEPTH = 3;
|
||||
const reExportDepth = (language: Language): number => (language === 'python' ? 3 : REEXPORT_MAX_DEPTH);
|
||||
|
||||
/** A Python star import, read the way the resolver reads one. */
|
||||
const PYTHON_STAR_IMPORT = /^\s*from\s+([\w.]+)\s+import\s+\*/gm;
|
||||
@@ -72,12 +71,14 @@ type AddImporter = (importer: string, language: Language, source: string, module
|
||||
export interface ImportReader {
|
||||
/** Its import mappings. */
|
||||
mappings(file: string, language: Language): ImportMapping[];
|
||||
/** Its ES-module `export … from` statements, which have no import mapping. */
|
||||
reExports(file: string, language: Language): ReExport[];
|
||||
/** The modules a Python file star-imports, which have no import mapping. */
|
||||
starImports(file: string, language: Language): string[];
|
||||
}
|
||||
|
||||
export function createImportReader(rootDir: string): ImportReader {
|
||||
const memo = new Map<string, { mappings: ImportMapping[]; starImports: string[] }>();
|
||||
const memo = new Map<string, { mappings: ImportMapping[]; reExports: ReExport[]; starImports: string[] }>();
|
||||
const read = (file: string, language: Language) => {
|
||||
let entry = memo.get(file);
|
||||
if (!entry) {
|
||||
@@ -89,6 +90,7 @@ export function createImportReader(rootDir: string): ImportReader {
|
||||
}
|
||||
entry = {
|
||||
mappings: content ? extractImportMappings(file, content, language) : [],
|
||||
reExports: content ? extractReExports(content, language) : [],
|
||||
starImports: language === 'python' ? [...content.matchAll(PYTHON_STAR_IMPORT)].map((m) => m[1]!) : [],
|
||||
};
|
||||
memo.set(file, entry);
|
||||
@@ -97,6 +99,7 @@ export function createImportReader(rootDir: string): ImportReader {
|
||||
};
|
||||
return {
|
||||
mappings: (file, language) => read(file, language).mappings,
|
||||
reExports: (file, language) => read(file, language).reExports,
|
||||
starImports: (file, language) => read(file, language).starImports,
|
||||
};
|
||||
}
|
||||
@@ -177,12 +180,11 @@ function importersOf(
|
||||
}
|
||||
const python = level.filter((module) => module.language === 'python');
|
||||
if (python.length > 0) addPythonImporters(queries, python, reader, add);
|
||||
// The next level: the Python importers that pass on what they import from these.
|
||||
if (depth > PYTHON_REEXPORT_DEPTH) break;
|
||||
// The next level: the importers that pass on what they import from these.
|
||||
const next: Module[] = [];
|
||||
for (const [importer, { language, statements }] of naming) {
|
||||
if (language !== 'python' || seen.has(importer)) continue;
|
||||
const passes = pythonPassedOn(importer, statements, reader);
|
||||
if (seen.has(importer) || depth > reExportDepth(language)) continue;
|
||||
const passes = passedOn(importer, language, statements, reader);
|
||||
if (passes === undefined) continue;
|
||||
seen.add(importer);
|
||||
next.push({ path: importer, language, passes });
|
||||
@@ -193,12 +195,14 @@ function importersOf(
|
||||
}
|
||||
|
||||
/**
|
||||
* What the Python module `importer` passes on of what its `statements`
|
||||
* import, or undefined when the resolver reads no name through it: what it
|
||||
* star-imports, and for a package's `__init__.py`, the names it imports too.
|
||||
* What `importer` passes on of what its `statements` import, or undefined
|
||||
* when the resolver reads no name through it: an ES module what it `export …
|
||||
* from`s; a Python module what it star-imports, and a package's `__init__.py`
|
||||
* the names it imports too.
|
||||
*/
|
||||
function pythonPassedOn(
|
||||
function passedOn(
|
||||
importer: string,
|
||||
language: Language,
|
||||
statements: ReadonlyArray<{ source: string; module: Module; onlyName?: string }>,
|
||||
reader: ImportReader,
|
||||
): Set<string> | null | undefined {
|
||||
@@ -206,15 +210,25 @@ function pythonPassedOn(
|
||||
const pass = (names: Set<string> | null): void => {
|
||||
passes = passes === null || names === null ? null : new Set([...(passes ?? []), ...names]);
|
||||
};
|
||||
const stars = new Set(reader.starImports(importer, 'python'));
|
||||
const passing = (module: Module, name: string): boolean => module.passes === null || module.passes.has(name);
|
||||
if (language !== 'python') {
|
||||
for (const rex of reader.reExports(importer, language)) {
|
||||
for (const { source, module } of statements) {
|
||||
if (source !== rex.source) continue;
|
||||
if (rex.kind === 'wildcard') pass(module.passes);
|
||||
else if (rex.kind === 'namespace') pass(new Set([rex.exportedName]));
|
||||
else if (passing(module, rex.originalName)) pass(new Set([rex.exportedName]));
|
||||
}
|
||||
}
|
||||
return passes;
|
||||
}
|
||||
const stars = new Set(reader.starImports(importer, language));
|
||||
const isPackage = importer === '__init__.py' || importer.endsWith('/__init__.py');
|
||||
for (const { source, module, onlyName } of statements) {
|
||||
if (onlyName === undefined && stars.has(source)) pass(module.passes);
|
||||
if (!isPackage) continue;
|
||||
const names = reader.mappings(importer, 'python')
|
||||
.filter((m) => m.source === source && (onlyName !== undefined
|
||||
? m.exportedName === onlyName
|
||||
: module.passes === null || module.passes.has(m.exportedName)))
|
||||
const names = reader.mappings(importer, language)
|
||||
.filter((m) => m.source === source && (onlyName !== undefined ? m.exportedName === onlyName : passing(module, m.exportedName)))
|
||||
.map((m) => m.localName);
|
||||
if (names.length > 0) pass(new Set(names));
|
||||
}
|
||||
@@ -288,7 +302,8 @@ function addPythonImporters(
|
||||
|
||||
/**
|
||||
* Per importer, what its statements naming the modules import: the modules
|
||||
* named whole (not the package around one), and the local names bound.
|
||||
* named whole (not the package around one), and the local names bound. An ES
|
||||
* namespace import (`import * as z`) reads whatever its module passes on.
|
||||
*/
|
||||
function bindingsOf(importers: Importers, reader: ImportReader): Map<string, { modules: string[]; names: string[] }> {
|
||||
const bindings = new Map<string, { modules: string[]; names: string[] }>();
|
||||
@@ -297,7 +312,9 @@ function bindingsOf(importers: Importers, reader: ImportReader): Map<string, { m
|
||||
reader.mappings(file, language)
|
||||
.filter((m) => {
|
||||
const only = sources.get(m.source);
|
||||
return only === null || (only !== undefined && only.has(m.exportedName));
|
||||
if (only === null) return true;
|
||||
if (only === undefined) return false;
|
||||
return only.has(m.exportedName) || (m.isNamespace && language !== 'python' && only.size > 0);
|
||||
})
|
||||
.map((m) => m.localName)
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user