From 800e582d5513ce0de6648f10717582f20da3ebc6 Mon Sep 17 00:00:00 2001 From: Colby Mchenry Date: Thu, 22 Oct 2026 06:19:32 +0000 Subject: [PATCH] fix(go): conversions, external test packages and build-tag twins link as Go builds them (#2784) Three Go shapes recorded edges the source doesn't make. A conversion like Category(s), (*Pair)(p) or model.Kind(3) became a calls edge to the type, or an instantiates edge to a struct. A bare call in a package's external tests (package a_test) could bind to a same-named helper of package a in the same directory, which a_test can't see. And a name a package defines once per build target, like sep in sep_unix.go and sep_other.go, linked to whichever definition came first, so a Linux caller reached the !unix file. A Go call whose target is a type is now a references edge. Two files of one directory under different package clauses see each other's top-level names only through an import: qualified, or bare after a dot import and exported. A reference whose definition has build-constrained twins in its package reaches the one that builds for the most typical target its own file builds for: linux/amd64, then darwin/arm64, windows/amd64 and the rest. Constraints are read from the file name's GOOS/GOARCH suffix and the //go:build or // +build lines, with tags as go build sets them by default. Validation: 9 new tests in three files, all failing on main, and each part switched off fails its own file; the Go tests pass. A/B against main 0c3e12dd, edges changed as conversions / twin retargets: gin 56/0, etcd 556/89, prometheus 812/5270, sonic 49/0, x/sys 550/3072, gopsutil 31/179, fsnotify 3/7, kubernetes 16039/328. Nodes and refs are unchanged, and express and flask dumps are byte-identical. Every kind change sits on a conversion-shaped call, and the twin retargets read right by their files' build lines (prometheus labels to the default stringlabels file, x/sys syscalls to syscall_unix_gc.go). Indexing cost, wall/CPU seconds, median of 3: x/sys 6.7/20.5 -> 7.1/21.7, etcd 9.6/29.9 -> 9.8/30.0, prometheus 12.5/49.2 -> 12.9/50.2; kubernetes, 2 runs each, 108.1/385.8 -> 110.4/390.0. Fixes #2581 Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + __tests__/go-build-constraint-twins.test.ts | 110 ++++++++++ __tests__/go-conversion-references.test.ts | 114 ++++++++++ __tests__/go-external-test-package.test.ts | 145 +++++++++++++ __tests__/go-framework-own-package.test.ts | 3 +- src/resolution/index.ts | 16 +- src/resolution/name-matcher.ts | 220 +++++++++++++++++++- 7 files changed, 601 insertions(+), 8 deletions(-) create mode 100644 __tests__/go-build-constraint-twins.test.ts create mode 100644 __tests__/go-conversion-references.test.ts create mode 100644 __tests__/go-external-test-package.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 4251151a..3dedebbd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In Go, a call through a variable that holds the result of a type assertion, like `st.Fetch(key)` after `st := v.(storage.Store)` or `rc.Read(p)` after `rc, ok := v.(ReadCloser)`, now links to that method of the asserted type, as a call made through the assertion itself does, including one the type gets from a struct or interface it embeds, and to nothing when the type comes from outside your project. Before, such a call linked to whichever method shared the name or to nothing, and a variable named like a standard-library package, like grpc-go's `parser, ok := builder.(balancer.ConfigParser)`, was taken for that package. Re-index Go projects after upgrading. - In Go, a method called on an element of a slice, array or map, like kubectl's `schemas[ix].ValidateBytes(data)`, `pods[i].DeepCopy()`, `m.Items[len(m.Items)-1].Unmarshal(…)` in generated protobuf code or `(*in)[i].DeepCopyInto(&(*out)[i])` in every deepcopy file, now links to that method of the element's type. The type is read from what is indexed: a parameter, a variable, a struct field or a named slice type like `type ConjunctiveSchema []Schema`, and an element held by pointer, like `[]*Item`, reaches `Item`'s methods. Before, such a call linked to whichever method shared its name, often one in an unrelated package, or to nothing when many types declare it, as with Kubernetes' `DeepCopyInto`. An element whose type comes from outside your project, like `[]*net.IPNet`, or is a built-in one like `[]error`, now links to nothing rather than to a project method of that name. Re-index Go projects after upgrading. - In Go, a method called further along a chain now links to that method of the type the chain has reached, like `Where` and `First` in a GORM-generated query such as `query.Card.WithContext(ctx).Where(…).First()`, a method called on what a function returns, like the client from `store.New(…)`, or one called on a composite literal, like `GitOracle{}.Worktree()`. The type is read from a function's or method's results, a conversion, a literal or an embedded field. Before, such a call was matched by its method's name alone, so a query's `Where` linked to another table's query type, a literal's method to a type that only shares its name, and `time.Now().UTC().Format(…)` to the one project method named `Format`. A chain whose type comes from outside your project, or can't be worked out, now links to nothing rather than to a guess. Re-index Go projects after upgrading. Thanks @BX-D for the report and @inth3shadows for the extra cases. (#2543) +- In Go, three kinds of call now link where Go says they go. A conversion like `Category(s)`, `(*Pair)(p)` or `model.Kind(3)` is recorded as a use of the type rather than a call to it or a new instance of it. A call in a package's external tests (`package a_test`) reaches the helper those tests declare, not a same-named one in `package a` beside them, which they can't see. And a function, method or type a package defines once per platform or build tag, like `sep` in `sep_unix.go` and `sep_other.go`, now links to the definition a default build on Linux uses, or the one for the platform the calling file is written for, like Windows code reaching the Windows definition. Before, the second and third picked whichever definition came first, so a Linux caller could reach the Windows or `!unix` code. Re-index Go projects after upgrading. Thanks @inth3shadows for the report. (#2581) - In Go, a method a type declares in a file too large to index, like the `String`, `Marshal` and `Unmarshal` every Kubernetes API type has in `core/v1/generated.pb.go`, is no longer replaced by the method of that name a type it embeds has. Before, `pod.String()` on a `*v1.Pod` linked to `TypeMeta`'s `String`, a method the call never runs; it now links to nothing, since the pod's own method isn't in the graph. A method the type really gets from a type it embeds, like `GetName` from `ObjectMeta`, still links. Re-index Go projects after upgrading. - In Go, calls into a module whose `go.mod` is not at the project root now resolve, whether it is a `server/` backend next to a `web/` frontend or one of several modules side by side as in etcd, so calls like `store.New()` and `s.db.CreateItem()` find their targets instead of being treated as calls into a third-party package. A name written through a package, like a `job.OPCommand` result type or a field of type `artifact.Manager`, now links to that package's symbol rather than a same-named one elsewhere, which also corrects links in projects with a single `go.mod`. Re-index Go projects after upgrading. Thanks @GoDiao for the report and @danusha2345 for the fix. (#2322) - In Go, an exported type written without a package, like `Node` in `func Walk(v Visitor, node Node)`, now links to the type of that name in its own package, whether it is a struct, an interface or another kind of type. Before, it could link to a struct of the same name in an unrelated package, so prometheus's PromQL parser functions pointed at the Kubernetes discovery `Node` struct instead of the parser's own `Node` interface, or, for a result type like `Appender` in `func (f *fanout) Appender(…) Appender`, to the method itself. A type from a package outside your project, like `apiv1.Node` or `http.Handler`, no longer links to a struct of the same name in the file that uses it; a variadic parameter like `...storage.Filter` now links to that package's type; a route handler written as a method value, like `h.Follow` in an Echo or Gin app, links to that method rather than a model struct named `Follow`; and a method called on another call's result or on an indexed value, like `err[i].Error()` or a chained `.String()`, is no longer recorded as creating a struct of the same name. Re-index Go projects after upgrading. diff --git a/__tests__/go-build-constraint-twins.test.ts b/__tests__/go-build-constraint-twins.test.ts new file mode 100644 index 00000000..bf75ed6e --- /dev/null +++ b/__tests__/go-build-constraint-twins.test.ts @@ -0,0 +1,110 @@ +/** + * A Go package can define one name once per build target: `sep` in + * `sep_unix.go` (`//go:build unix`) and `sep_other.go` (`//go:build !unix`). + * Name matching took whichever came first, so a Linux caller reached the + * `!unix` definition (#2581). A reference now reaches the definition that + * builds for the most typical target its own file builds for: Linux on + * amd64, then a Mac, then Windows and the rest. The constraint is read from + * the `//go:build` line, the legacy `// +build` lines and the file name's + * `_GOOS` / `_GOARCH` suffix. + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import CodeGraph from '../src/index'; + +const GO_MOD = 'module example.com/app\n\ngo 1.22\n'; + +let tempDir: string; +let cg: CodeGraph | null = null; + +afterEach(() => { + cg?.close(); + cg = null; + if (tempDir) fs.rmSync(tempDir, { recursive: true, force: true }); +}); + +async function index(files: Record): Promise { + tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-2581-build-')); + for (const [rel, source] of Object.entries({ 'go.mod': GO_MOD, ...files })) { + fs.mkdirSync(path.dirname(path.join(tempDir, rel)), { recursive: true }); + fs.writeFileSync(path.join(tempDir, rel), source); + } + cg = await CodeGraph.init(tempDir, { index: true }); + cg.resolveReferences(); + return cg; +} + +/** ` ` of every call out of the function `fromName`, sorted. */ +function callsFrom(graph: CodeGraph, fromName: string): string[] { + const from = graph.getNodesByKind('function').find((n) => n.name === fromName); + expect(from, fromName).toBeDefined(); + return graph + .getOutgoingEdges(from!.id) + .filter((e) => e.kind === 'calls') + .map((e) => { + const target = graph.getNode(e.target); + return `${target?.qualifiedName} ${target?.filePath}`; + }) + .sort(); +} + +describe('Go definitions under complementary build constraints (#2581)', () => { + it("calls the issue's `unix` definition, not the `!unix` one", async () => { + const graph = await index({ + 'main.go': 'package main\n\nfunc main() { println(sep()) }\n', + 'sep_unix.go': '//go:build unix\n\npackage main\n\nfunc sep() string { return "/" }\n', + 'sep_other.go': '//go:build !unix\n\npackage main\n\nfunc sep() string { return "\\\\" }\n', + }); + expect(callsFrom(graph, 'main')).toEqual(['sep sep_unix.go']); + }); + + it('reads legacy `// +build` lines and `_GOOS` / `_GOARCH` file names', async () => { + const graph = await index({ + 'main.go': 'package main\n\nfunc main() { println(page(), dir(), cpu()) }\n', + // No system in these names: only the `+build` lines tell them apart. + 'page_small.go': '// +build plan9\n\npackage main\n\nfunc page() int { return 1 }\n', + 'page_std.go': '// +build !plan9\n\npackage main\n\nfunc page() int { return 4096 }\n', + 'dir_darwin.go': 'package main\n\nfunc dir() string { return "/Users" }\n', + 'dir_linux.go': 'package main\n\nfunc dir() string { return "/home" }\n', + 'cpu_386.go': 'package main\n\nfunc cpu() int { return 32 }\n', + 'cpu_amd64.go': 'package main\n\nfunc cpu() int { return 64 }\n', + }); + expect(callsFrom(graph, 'main')).toEqual(['cpu cpu_amd64.go', 'dir dir_linux.go', 'page page_std.go']); + }); + + it('follows the build target of the calling file, and falls back to a Mac', async () => { + const graph = await index({ + 'main.go': 'package main\n\nfunc main() { open(); notify() }\n', + 'run_windows.go': 'package main\n\nfunc run() { open() }\n', + 'open_other.go': '//go:build !windows\n\npackage main\n\nfunc open() {}\n', + 'open_windows.go': 'package main\n\nfunc open() {}\n', + 'notify_aix.go': 'package main\n\nfunc notify() {}\n', + 'notify_darwin.go': 'package main\n\nfunc notify() {}\n', + }); + expect(callsFrom(graph, 'main')).toEqual(['notify notify_darwin.go', 'open open_other.go']); + expect(callsFrom(graph, 'run')).toEqual(['open open_windows.go']); + }); + + it("applies to a method and to a call through another package's import", async () => { + const graph = await index({ + 'main.go': `package main + +import "example.com/app/osutil" + +func main() { + var f file + _ = f.close() + _ = osutil.Sep() +} +`, + 'file.go': 'package main\n\ntype file struct{}\n', + 'file_other.go': '//go:build !unix\n\npackage main\n\nfunc (f *file) close() error { return nil }\n', + 'file_unix.go': '//go:build unix\n\npackage main\n\nfunc (f *file) close() error { return nil }\n', + 'osutil/sep_other.go': '//go:build !unix\n\npackage osutil\n\nfunc Sep() string { return "\\\\" }\n', + 'osutil/sep_unix.go': '//go:build unix\n\npackage osutil\n\nfunc Sep() string { return "/" }\n', + }); + expect(callsFrom(graph, 'main')).toEqual(['Sep osutil/sep_unix.go', 'file::close file_unix.go']); + }); +}); diff --git a/__tests__/go-conversion-references.test.ts b/__tests__/go-conversion-references.test.ts new file mode 100644 index 00000000..6d500fe4 --- /dev/null +++ b/__tests__/go-conversion-references.test.ts @@ -0,0 +1,114 @@ +/** + * A Go call whose callee is a type is a conversion: `Category(s)` with + * `type Category string`, `(*Pair)(p)`, `model.Kind(3)`. It uses the type, + * and calls or instantiates nothing, so it is recorded as a `references` + * edge to the type. Before, it was a `calls` edge to a named non-struct type + * and an `instantiates` edge to a struct (#2581). + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import CodeGraph from '../src/index'; + +const GO_MOD = 'module example.com/app\n\ngo 1.22\n'; + +let tempDir: string; +let cg: CodeGraph | null = null; + +afterEach(() => { + cg?.close(); + cg = null; + if (tempDir) fs.rmSync(tempDir, { recursive: true, force: true }); +}); + +async function index(files: Record): Promise { + tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-2581-conv-')); + for (const [rel, source] of Object.entries({ 'go.mod': GO_MOD, ...files })) { + fs.mkdirSync(path.dirname(path.join(tempDir, rel)), { recursive: true }); + fs.writeFileSync(path.join(tempDir, rel), source); + } + cg = await CodeGraph.init(tempDir, { index: true }); + cg.resolveReferences(); + return cg; +} + +/** ` :` of every edge out of the function `fromName`, sorted. */ +function edgesFrom(graph: CodeGraph, fromName: string): string[] { + const from = graph.getNodesByKind('function').find((n) => n.name === fromName); + expect(from, fromName).toBeDefined(); + return graph + .getOutgoingEdges(from!.id) + .filter((e) => e.kind === 'calls' || e.kind === 'instantiates' || e.kind === 'references') + .map((e) => { + const target = graph.getNode(e.target); + return `${e.kind} ${target?.kind} ${target?.qualifiedName} :${e.line}`; + }) + .sort(); +} + +describe('Go type conversions (#2581)', () => { + it("records the issue's `Category(s)` as a use of the type, not a call", async () => { + const graph = await index({ + 'main.go': `package main + +type Category string + +func use(s string) Category { + return Category(s) // a conversion, not a call +} + +func main() { use("") } +`, + }); + expect(edgesFrom(graph, 'use')).toEqual([ + 'references type_alias Category :5', + 'references type_alias Category :6', + ]); + }); + + it('records conversions to struct, pointer, slice and imported types the same way, and keeps real calls', async () => { + const graph = await index({ + 'model/model.go': `package model + +type Kind int + +type Point struct{ X, Y int } + +func Make() Kind { return 0 } +`, + 'main.go': `package main + +import "example.com/app/model" + +type Celsius float64 + +type Pair struct{ X, Y int } + +type Names []string + +func convert(p *Pair, raw []string, v float64, pt model.Point) { + _ = Celsius(v) + _ = (*Pair)(p) + _ = Names(raw) + _ = model.Kind(3) + _ = Pair(pt) + _ = model.Make() + _ = Pair{X: 1} +} + +func main() { convert(nil, nil, 0, model.Point{}) } +`, + }); + const edges = edgesFrom(graph, 'convert'); + expect(edges.filter((e) => /:1[2-8]$/.test(e))).toEqual([ + 'calls function Make :17', + 'instantiates struct Pair :18', + 'references struct Pair :13', + 'references struct Pair :16', + 'references type_alias Celsius :12', + 'references type_alias Kind :15', + 'references type_alias Names :14', + ]); + }); +}); diff --git a/__tests__/go-external-test-package.test.ts b/__tests__/go-external-test-package.test.ts new file mode 100644 index 00000000..dfcb1c85 --- /dev/null +++ b/__tests__/go-external-test-package.test.ts @@ -0,0 +1,145 @@ +/** + * A package's external tests (`package a_test`) share its directory but are + * a package of their own. A name one of them declares at the top level + * reaches the other only through an import: written `a.Real`, or bare after + * `import . "…/a"`, and then only an exported one. Before, a bare call in an + * external test bound to package `a`'s test helper of the same name (#2581). + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import CodeGraph from '../src/index'; + +const GO_MOD = 'module example.com/a\n\ngo 1.22\n'; + +let tempDir: string; +let cg: CodeGraph | null = null; + +afterEach(() => { + cg?.close(); + cg = null; + if (tempDir) fs.rmSync(tempDir, { recursive: true, force: true }); +}); + +async function index(files: Record): Promise { + tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-2581-xtest-')); + for (const [rel, source] of Object.entries({ 'go.mod': GO_MOD, ...files })) { + fs.mkdirSync(path.dirname(path.join(tempDir, rel)), { recursive: true }); + fs.writeFileSync(path.join(tempDir, rel), source); + } + cg = await CodeGraph.init(tempDir, { index: true }); + cg.resolveReferences(); + return cg; +} + +/** ` ` of every call or type use out of the function `fromName`, sorted. */ +function edgesFrom(graph: CodeGraph, fromName: string): string[] { + const from = graph.getNodesByKind('function').find((n) => n.name === fromName); + expect(from, fromName).toBeDefined(); + return graph + .getOutgoingEdges(from!.id) + .filter((e) => e.kind === 'calls' || e.kind === 'references') + .map((e) => { + const target = graph.getNode(e.target); + return `${e.kind} ${target?.qualifiedName} ${target?.filePath}`; + }) + .sort(); +} + +const PACKAGE_A = `package a + +func Real() {} + +func helper() {} + +type fixture struct{} +`; + +describe('Go external test packages (#2581)', () => { + it("binds the issue's external test to its own package's helper", async () => { + const graph = await index({ + 'a.go': 'package a\n\nfunc Real() {}\n', + 'a_internal_test.go': `package a + +import "testing" + +func corpus(t *testing.T) []string { return nil } + +func TestInternal(t *testing.T) { _ = corpus(t) } +`, + 'b_ext_helpers_test.go': `package a_test + +import "testing" + +func corpus(t *testing.T, root string) []string { return nil } +`, + 'a_ext_test.go': `package a_test + +import "testing" + +func TestExt(t *testing.T) { _ = corpus(t, "") } +`, + }); + expect(edgesFrom(graph, 'TestExt')).toEqual(['calls corpus b_ext_helpers_test.go']); + expect(edgesFrom(graph, 'TestInternal')).toEqual(['calls corpus a_internal_test.go']); + }); + + it('reaches the package under test only through its import', async () => { + const graph = await index({ + 'a.go': PACKAGE_A, + 'qualified_test.go': `package a_test + +import ( + "testing" + + "example.com/a" +) + +func TestQualified(t *testing.T) { a.Real() } +`, + 'dot_test.go': `package a_test + +import ( + "testing" + + . "example.com/a" +) + +func TestDot(t *testing.T) { Real() } + +func TestUnexported(t *testing.T) { helper() } +`, + 'plain_test.go': `package a_test + +type fixture struct{ name string } +`, + 'use_test.go': `package a_test + +func check(f fixture) {} +`, + }); + expect(edgesFrom(graph, 'TestQualified')).toEqual(['calls Real a.go']); + expect(edgesFrom(graph, 'TestDot')).toEqual(['calls Real a.go']); + // A dot import brings only the exported names. + expect(edgesFrom(graph, 'TestUnexported')).toEqual([]); + expect(edgesFrom(graph, 'check')).toEqual(['references fixture plain_test.go']); + }); + + it("does not bind package a's code to a name only its external tests declare", async () => { + const graph = await index({ + 'a.go': 'package a\n\nfunc Real() {}\n', + 'a_internal_test.go': `package a + +import "testing" + +func TestInternal(t *testing.T) { extOnly() } +`, + 'ext_test.go': `package a_test + +func extOnly() {} +`, + }); + expect(edgesFrom(graph, 'TestInternal')).toEqual([]); + }); +}); diff --git a/__tests__/go-framework-own-package.test.ts b/__tests__/go-framework-own-package.test.ts index 137fa588..dacafd9c 100644 --- a/__tests__/go-framework-own-package.test.ts +++ b/__tests__/go-framework-own-package.test.ts @@ -190,7 +190,8 @@ describe('Go framework heuristics pick only in the package Go reads the name fro }); it('a conversion and a composite literal still reach their own package’s struct', () => { - expect(linksFrom('errors.go', 'Error', { name: '_' })).toEqual(['instantiates errors.go:Error']); + // `(*Error)(nil)` is a conversion: a use of the type, not an instantiation (#2581). + expect(linksFrom('errors.go', 'Error', { name: '_' })).toEqual(['references errors.go:Error']); expect(linksFrom('errors.go', 'Error', { name: 'newError' })).toEqual([ 'instantiates errors.go:Error', 'references errors.go:Error', diff --git a/src/resolution/index.ts b/src/resolution/index.ts index 5913fab8..1d07be21 100644 --- a/src/resolution/index.ts +++ b/src/resolution/index.ts @@ -21,7 +21,7 @@ import { isImportableKind, CPP_DEFINE_SIGNATURE, } from './types'; -import { isPythonSelfCall, matchJsStoreBindingCall, isUnresolvedJsMemberCall, matchObjectPathCall, thisScopeCaller, isVisibleAcrossFiles, matchReference, matchByFilePath, matchFunctionRef, matchDottedCallChain, matchScopedCallChain, matchMethodCall, sameLanguageFamily, crossesCodeBoundary, gateLanguageMatch, dumpNameMatcherProfile, clearNameMatcherMemos, isRustNameInScope, isRustForeignImplCallable, CASE_INSENSITIVE_LANGUAGES, isDartMemberRead, matchDartMemberRead, isDartChainLink, matchDartChainLink, isDartAnnotation, matchDartAnnotation, isStdMethodName, isGoUnknownQualified, isGoProjectLocalCall, GO_STDLIB_PACKAGES, isGoBareName, goTypePositionTarget, goDotImportedSupertype, GO_TYPE_KINDS, matchGoAssertedCall, matchGoIndexedCall, matchGoChainedCall, isGoAssertedLocal, isGoLocallyBound, isPythonFromImportModule, pythonAttrRefResolves, isImportShadowedByJsBinding } from './name-matcher'; +import { isPythonSelfCall, matchJsStoreBindingCall, isUnresolvedJsMemberCall, matchObjectPathCall, thisScopeCaller, isVisibleAcrossFiles, matchReference, matchByFilePath, matchFunctionRef, matchDottedCallChain, matchScopedCallChain, matchMethodCall, sameLanguageFamily, crossesCodeBoundary, gateLanguageMatch, dumpNameMatcherProfile, clearNameMatcherMemos, isRustNameInScope, isRustForeignImplCallable, CASE_INSENSITIVE_LANGUAGES, isDartMemberRead, matchDartMemberRead, isDartChainLink, matchDartChainLink, isDartAnnotation, matchDartAnnotation, isStdMethodName, isGoUnknownQualified, isGoProjectLocalCall, GO_STDLIB_PACKAGES, isGoBareName, goTypePositionTarget, goDotImportedSupertype, GO_TYPE_KINDS, matchGoAssertedCall, matchGoIndexedCall, matchGoChainedCall, preferGoBuildTwin, isGoAssertedLocal, isGoLocallyBound, isPythonFromImportModule, pythonAttrRefResolves, isImportShadowedByJsBinding } from './name-matcher'; import { isVisibleCppMacro, clearCppMacroVisibility } from './cpp-macro-visibility'; import { isCppConstructorRef, matchCppConstructor } from './cpp-constructor'; import { isCppSupertypeRef, matchCppSupertype, clearCppSupertypeMemos } from './cpp-supertypes'; @@ -1351,10 +1351,12 @@ export class ReferenceResolver { this.context, ); const scoped = this.gateRustScope(candidate, ref); - const resolved = this.gateSuperSelfCall( + // A Go name its package defines once per build target is the definition + // the reference's file builds with (see preferGoBuildTwin). + const resolved = preferGoBuildTwin(this.gateSuperSelfCall( scoped?.resolvedBy === 'framework' ? this.gateFrameworkLanguage(scoped, ref) : this.gateLanguage(scoped, ref), ref, - ); + ), ref, this.context); // A Dart class whose field holds its own type (`class Node { Node? next; }`) // names itself: a self-edge that says nothing (#2327). if (resolved && ref.language === 'dart' && ref.referenceKind === 'references' && @@ -1752,6 +1754,14 @@ export class ReferenceResolver { } } + // A Go call whose callee is a type is a conversion — `Category(s)`, + // `(*Pair)(p)`, `model.Kind(3)` — which uses the type, and calls or + // instantiates nothing (#2581). + if (kind === 'calls' && ref.original.language === 'go') { + const targetNode = this.nodeById(ref.targetNodeId); + if (targetNode && GO_TYPE_KINDS.has(targetNode.kind)) kind = 'references'; + } + // Promote "calls" to "instantiates" when the resolved target is a // class/struct/union. Languages without a `new` keyword (Python, Ruby) // express instantiation as `Foo()` — extraction can't tell that diff --git a/src/resolution/name-matcher.ts b/src/resolution/name-matcher.ts index d4a0d05a..e3523c96 100644 --- a/src/resolution/name-matcher.ts +++ b/src/resolution/name-matcher.ts @@ -3354,9 +3354,21 @@ function isSfcPrivate(n: Node, context: ResolutionContext): boolean { * exported name from anywhere, an unexported one only inside its own package, * which is a directory. By the name's first letter, not the extractor's flag: * the flag is unset for every Go method, exported or not. + * + * A package's external tests (`package a_test` beside `package a`) share its + * directory but are a package of their own: a name one of them declares at + * the top level reaches the other only through an import, written `a.Real`, + * or bare after `import . "…/a"`, and then only an exported one. A test + * helper `corpus` in `package a_test` is not package `a`'s `corpus` (#2581). */ -function isGoNameVisible(candidate: Node, ref: UnresolvedRef): boolean { - return /^[A-Z]/.test(candidate.name) || path.posix.dirname(candidate.filePath) === path.posix.dirname(ref.filePath); +function isGoNameVisible(candidate: Node, ref: UnresolvedRef, context: ResolutionContext): boolean { + const dir = goPackageDir(candidate.filePath); + if (dir !== goPackageDir(ref.filePath)) return /^[A-Z]/.test(candidate.name); + if (candidate.filePath === ref.filePath || TYPE_MEMBER_KINDS.has(candidate.kind) || ref.referenceName.includes('.')) return true; + const own = goPackageClause(ref.filePath, context); + const theirs = goPackageClause(candidate.filePath, context); + if (own === null || theirs === null || own === theirs) return true; + return /^[A-Z]/.test(candidate.name) && goDotImports(ref.filePath, context).dirs.includes(dir); } /** @@ -3457,7 +3469,7 @@ export function isVisibleAcrossFiles(candidate: Node, ref: UnresolvedRef, contex // no `.mm` file's `vector`. (A class is the runtime's, wherever it is // written.) if (isObjcFileLocalElsewhere(candidate, ref)) return false; - if (lang === 'go') return isGoNameVisible(candidate, ref); + if (lang === 'go') return isGoNameVisible(candidate, ref, context); if (lang === 'rust') { if (candidate.visibility !== 'private') return true; if (isRustTraitImplMethod(candidate, context)) return true; @@ -11805,7 +11817,7 @@ export function matchByExactName( // And a Go name that isn't exported, only its own package's: prometheus // tsdb's `[]*memSeries` is head.go's, not `tsdb/agent`'s, which scored the // same (a child package's directory shares as much of the path) and won. - (n.language !== 'go' || isGoNameVisible(n, ref)) && + (n.language !== 'go' || isGoNameVisible(n, ref, context)) && dartExtensionDecl(n, context)?.named !== false && // A Scala package object's member, only where it is in scope — ahead of // the ranking, so cats.laws' `Eq` can be the `cats` package object's. @@ -20449,6 +20461,206 @@ function goPackageClause(filePath: string, context: ResolutionContext): string | return name; } +/** What `go build` is asked to build for. */ +interface GoBuildTarget { + goos: string; + goarch: string; +} + +/** The operating systems and architectures go/build reads from a file name (its syslist). */ +const GO_KNOWN_OS: ReadonlySet = new Set([ + 'aix', 'android', 'darwin', 'dragonfly', 'freebsd', 'hurd', 'illumos', 'ios', 'js', 'linux', 'nacl', 'netbsd', + 'openbsd', 'plan9', 'solaris', 'wasip1', 'windows', 'zos', +]); +const GO_KNOWN_ARCH: ReadonlySet = new Set([ + '386', 'amd64', 'amd64p32', 'arm', 'armbe', 'arm64', 'arm64be', 'loong64', 'mips', 'mipsle', 'mips64', 'mips64le', + 'mips64p32', 'mips64p32le', 'ppc', 'ppc64', 'ppc64le', 'riscv', 'riscv64', 's390', 's390x', 'sparc', 'sparc64', 'wasm', +]); +/** The systems the `unix` build tag holds for. */ +const GO_UNIX_OS: ReadonlySet = new Set([ + 'aix', 'android', 'darwin', 'dragonfly', 'freebsd', 'hurd', 'illumos', 'ios', 'linux', 'netbsd', 'openbsd', 'solaris', +]); + +/** + * The targets a Go name with build-constrained twins is read for, the most + * typical first: a Linux server, a Mac, Windows, then every other system on + * its usual architecture and Linux on every other architecture. + */ +const GO_BUILD_TARGETS: readonly GoBuildTarget[] = (() => { + const usual: Record = { js: 'wasm', wasip1: 'wasm', ios: 'arm64', android: 'arm64', aix: 'ppc64', zos: 's390x', hurd: '386' }; + const targets: GoBuildTarget[] = [ + { goos: 'linux', goarch: 'amd64' }, { goos: 'darwin', goarch: 'arm64' }, { goos: 'windows', goarch: 'amd64' }, + { goos: 'linux', goarch: 'arm64' }, { goos: 'darwin', goarch: 'amd64' }, { goos: 'freebsd', goarch: 'amd64' }, + ]; + const add = (goos: string, goarch: string): void => { + if (!targets.some((t) => t.goos === goos && t.goarch === goarch)) targets.push({ goos, goarch }); + }; + for (const goos of GO_KNOWN_OS) add(goos, usual[goos] ?? 'amd64'); + for (const goarch of GO_KNOWN_ARCH) add('linux', goarch); + return targets; +})(); + +/** + * Whether a build tag holds for a target, as the go command sets them by + * default: the system and architecture (`android` is also `linux`, `ios` + * `darwin`, `illumos` `solaris`), `unix` on a Unix system, the `gc` compiler, + * cgo, and every `go1.N` release. Any other tag (`ignore`, `purego`, a + * project's `integration`) is unset. + */ +function goBuildTagHolds(tag: string, t: GoBuildTarget): boolean { + if (tag === t.goos || tag === t.goarch) return true; + switch (tag) { + case 'unix': return GO_UNIX_OS.has(t.goos); + case 'linux': return t.goos === 'android'; + case 'darwin': return t.goos === 'ios'; + case 'solaris': return t.goos === 'illumos'; + case 'gc': return true; + case 'cgo': return t.goos !== 'js' && t.goos !== 'wasip1'; + default: return /^go1\.\d+$/.test(tag); + } +} + +type GoBuildConstraint = (t: GoBuildTarget) => boolean; + +/** A `//go:build` expression (`unix && !(linux || darwin)`) as a test of a target, or null when it can't be read. */ +function goParseBuildExpr(text: string): GoBuildConstraint | null { + const tokens = text.match(/&&|\|\||[()!]|[A-Za-z0-9_.]+|\S/g) ?? []; + let i = 0; + const or = (): GoBuildConstraint | null => { + let left = and(); + while (left && tokens[i] === '||') { + i++; + const a = left, b = and(); + left = b && ((t) => a(t) || b(t)); + } + return left; + }; + const and = (): GoBuildConstraint | null => { + let left = not(); + while (left && tokens[i] === '&&') { + i++; + const a = left, b = not(); + left = b && ((t) => a(t) && b(t)); + } + return left; + }; + const not = (): GoBuildConstraint | null => { + const tok = tokens[i++]; + if (tok === '!') { + const inner = not(); + return inner && ((t) => !inner(t)); + } + if (tok === '(') { + const inner = or(); + return tokens[i++] === ')' ? inner : null; + } + return tok !== undefined && /^[A-Za-z0-9_.]+$/.test(tok) ? (t) => goBuildTagHolds(tok, t) : null; + }; + const expr = or(); + return expr && i === tokens.length ? expr : null; +} + +/** A legacy `// +build` line's options (`linux,amd64 darwin`, `!plan9`) as a test of a target, or null when it can't be read. */ +function goParsePlusBuild(text: string): GoBuildConstraint | null { + const options = text.trim().split(/\s+/).filter(Boolean).map((option) => option.split(',').map((term) => { + const neg = term.startsWith('!'); + const tag = neg ? term.slice(1) : term; + return /^[A-Za-z0-9_.]+$/.test(tag) ? { neg, tag } : null; + })); + if (options.length === 0 || options.some((terms) => terms.some((term) => term === null))) return null; + return (t) => options.some((terms) => terms.every((term) => goBuildTagHolds(term!.tag, t) !== term!.neg)); +} + +const GO_BUILD_CONSTRAINTS = new WeakMap>(); + +/** + * The targets a Go file builds for, as go/build reads them: a `_linux`, + * `_amd64` or `_windows_arm64` suffix on its name (before `_test`), and the + * `//go:build` line above its package clause, or the legacy `// +build` lines + * when it has none. Null for a file that builds everywhere, or whose + * constraint can't be read. + */ +function goFileConstraint(filePath: string, context: ResolutionContext): GoBuildConstraint | null { + let memo = GO_BUILD_CONSTRAINTS.get(context); + if (!memo) GO_BUILD_CONSTRAINTS.set(context, (memo = new Map())); + if (memo.has(filePath)) return memo.get(filePath)!; + const parts: GoBuildConstraint[] = []; + // go/build ignores what comes before the name's first `_`, so `linux.go` names no system. + const base = path.posix.basename(filePath.replace(/\\/g, '/')).replace(/\..*$/, ''); + const suffix = base.includes('_') ? base.slice(base.indexOf('_') + 1).split('_') : []; + if (suffix[suffix.length - 1] === 'test') suffix.pop(); + const last = suffix[suffix.length - 1]; + const prev = suffix[suffix.length - 2]; + if (prev !== undefined && GO_KNOWN_OS.has(prev) && last !== undefined && GO_KNOWN_ARCH.has(last)) { + parts.push((t) => goBuildTagHolds(prev, t) && goBuildTagHolds(last, t)); + } else if (last !== undefined && (GO_KNOWN_OS.has(last) || GO_KNOWN_ARCH.has(last))) { + parts.push((t) => goBuildTagHolds(last, t)); + } + // The constraint comes before the package clause, after only blank lines and line comments. + let goBuild: string | undefined; + const plusBuild: string[] = []; + for (const line of (context.getFileLines?.(filePath) ?? context.readFile(filePath)?.split(/\r?\n/) ?? []).slice(0, 200)) { + // (Past a byte-order mark.) + const text = (line.charCodeAt(0) === 0xfeff ? line.slice(1) : line).trim(); + if (text === '') continue; + if (!text.startsWith('//')) break; + const go = /^\/\/go:build\s+(.*)$/.exec(text); + if (go && goBuild === undefined) goBuild = go[1]; + const plus = /^\/\/\s*\+build\s+(.*)$/.exec(text); + if (plus) plusBuild.push(plus[1]!); + } + const header = goBuild !== undefined ? [goParseBuildExpr(goBuild)] : plusBuild.map(goParsePlusBuild); + let readable = true; + for (const c of header) { + if (c) parts.push(c); + else readable = false; + } + const constraint: GoBuildConstraint | null = !readable || parts.length === 0 ? null : (t) => parts.every((c) => c(t)); + memo.set(filePath, constraint); + return constraint; +} + +const GO_BUILD_TWINS = new WeakMap>(); + +/** + * The definition Go builds, of a name its package defines more than once for + * different targets — `sep` in `sep_unix.go` (`//go:build unix`) and + * `sep_other.go` (`//go:build !unix`), a method in `file_linux.go` and + * `file_windows.go`. Name matching took whichever came first, so a Linux + * caller's `sep()` reached the `!unix` file and the Unix definition showed no + * caller (#2581). The definition kept is the one that builds for the most + * typical target the reference's own file builds for: Linux on amd64 first, + * then a Mac, then Windows and the rest (GO_BUILD_TARGETS), so a file that + * only builds for Windows reaches the Windows definition. The result stands + * when no twin builds for those targets, or two do. + */ +export function preferGoBuildTwin(result: ResolvedRef | null, ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null { + if (!result || ref.language !== 'go') return result; + const target = context.getNodeById?.(result.targetNodeId); + // A file that builds everywhere can have no twin: the package would declare the name twice. + if (!target || target.language !== 'go' || !goFileConstraint(target.filePath, context)) return result; + let memo = GO_BUILD_TWINS.get(context); + if (!memo) GO_BUILD_TWINS.set(context, (memo = new Map())); + let twins = memo.get(target.id); + if (!twins) { + const dir = goPackageDir(target.filePath); + const clause = goPackageClause(target.filePath, context); + twins = context.getNodesByName(target.name).filter((n) => + n.language === 'go' && n.kind === target.kind && n.qualifiedName === target.qualifiedName && + goPackageDir(n.filePath) === dir && goPackageClause(n.filePath, context) === clause); + memo.set(target.id, twins); + } + if (twins.length < 2) return result; + const own = goFileConstraint(ref.filePath, context); + for (const t of GO_BUILD_TARGETS) { + if (own && !own(t)) continue; + const built = twins.filter((n) => goFileConstraint(n.filePath, context)?.(t) ?? true); + if (built.length === 1) return built[0]!.id === target.id ? result : { ...result, targetNodeId: built[0]!.id }; + if (built.length > 1) return result; + } + return result; +} + /** * The names a Go file may bind from a type assertion — the first name before * each `:=` or `=` on a line that asserts a type — so that only calls through