GithubStar/codegraph
1
0
Fork 0
mirror of https://github.com/colbymchenry/codegraph.git synced 2026-10-25 12:00:02 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(explore): names the query spells that the index never saw are said, not glossed over (#2789)

Browse Source 
Root cause: explore answers whatever the query's words match. When a name
the query spelled is not in the index at all, the answer is built from the
other words and says nothing about the missing ones. In the report, the
tool projects under a .NET repo's root build/ folder are skipped by
default. Asking for ReleaseNotesBuilder, ChangelogParser, PersistentData
and AddFeatures returned an unrelated Program.Main and a C++ Parser, and
the agent read the wrong code. The miss diagnostics (issue #2026) ran only
when nothing at all matched.

Fix:
- Query tokens written like code names (a camelCase hump, snake_case, or a
  qualified name) that resolve to no node at all are checked with a new
  bounded probe, getUnseenNames. It reports names the index has never seen:
  no node called that in any case, and no name, qualified name or path,
  signature or docstring word starting with it.
- A qualified name is judged by its member. Express's app.handle is indexed
  as handle, so the receiver alone never makes a name missing.
- Names that pass are listed in a note right under the summary line,
  success-shaped. The note points to codegraph status and never to Read.
  It names the .gitignore re-include for a default-skipped folder like
  build/ as the user's choice, since indexing scope is the user's call.
- A single word (Program, Main, How) is never named: it may be English.
- The total-miss answer gains the same note.
- The seeding's query tokenizer moved into one shared function, unchanged.

The skip itself is unchanged here. On main a root build/ folder is still
skipped by default: PRs 2681 and 2687 re-opened build folders below src/
only. The documented opt-in, !/build/ in .gitignore, indexes it, and sync
picks the opt-in up, verified on main and on 1.6.2.

Validation:
- New explore-unindexed-names.test.ts, 8 tests: 6 fail on main.
- All 38 explore suites pass, plus the tool-allowlist and dynamic-boundary
  suites (466 tests).
- The reporter's repo, newrelic/newrelic-dotnet-agent 418a0f2f, now commits
  the !/build/ workaround. With that line removed and the index synced,
  main answers the reporter's query with VersionFileGenerator/Program.cs
  Main and Sicily/Parser.cpp, saying nothing. The fix adds the note naming
  exactly the four missing names.
- Ordinary queries: 996 on express, flask, excalidraw, django, airflow and
  go-control-plane, all byte-identical to main.
- Miss probes: 120 queries mixing real names with invented ones, or only
  invented ones. Main never notes anything. The fix names every invented
  name in all 120 and no real name. The note costs about 5 source lines
  on a full response.

Fixes #2462

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-10-22 07:45:00 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 0953084bc5
commit 214ef50efe
5 changed files with 314 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/db/queries.ts
+37
View File
@@ -359,6 +359,7 @@ export class QueryBuilder {
getDominantFile?: SqliteStatement;
getChangeStamp?: SqliteStatement;
getTopRouteFile?: SqliteStatement;
unseenName?: SqliteStatement;
getRoutingManifest?: SqliteStatement;
insertNameSegment?: SqliteStatement;
} = {};
@@ -1881,6 +1882,42 @@ export class QueryBuilder {
};
}
/**
* Which of `names` the index has never seen: no node is called that, in any
* case, and no indexed name, qualified name, signature or docstring has a
* word starting with it. A file's path is its qualified name, so a folder or
* file named that counts as seen too. Explore uses this to say plainly that
* a name the query spelled is not in the index (#2462).
*
* A qualified name (`app.handle`, `pkg::run`) is asked about by its last
* segment: the index does not always record the receiver it was written on
* — express's `app.handle = function handle(…)` is indexed as `handle` — so
* only a member it has never seen makes the whole name unseen.
*
* Bounded like `getExploreMissDiagnostics`: two index probes per name (the
* lower(name) index and FTS postings), never a source scan, at most 16 names.
*/
getUnseenNames(names: readonly string[]): string[] {
const checked = [...new Set(names)].filter((n) => n.length > 0 && n.length <= 128).slice(0, 16);
if (checked.length === 0) return [];
if (!this.stmts.unseenName) {
this.stmts.unseenName = this.db.prepare(`
SELECT EXISTS (SELECT 1 FROM nodes WHERE lower(name) = lower(?))
OR EXISTS (SELECT 1 FROM nodes_fts WHERE nodes_fts MATCH ?) AS seen
`);
}
const stmt = this.stmts.unseenName;
return checked.filter((name) => {
const member = name.split(/::|\./).pop()!;
// FTS splits the phrase on everything but letters and digits, so a
// snake_case name is matched as the words it is made of.
const phrase = member.replace(/"/g, '');
if (phrase.length === 0) return false;
const row = stmt.get(member, `{name qualified_name signature docstring} : "${phrase}"*`) as { seen: number };
return !row.seen;
});
}
/**
* FTS5 search with prefix matching
*/
src/index.ts
+8
View File
@@ -2119,6 +2119,14 @@ export class CodeGraph {
return this.queries.getExploreMissDiagnostics(query);
}
/**
* Which of `names` the index has never seen, by name, path, signature or
* docstring — the names explore reports as not in the index (#2462).
*/
getUnseenNames(names: readonly string[]): string[] {
return this.queries.getUnseenNames(names);
}
/**
* The COBOL copybooks a query names (`CVACT01Y`, `MYCOPYBOOK`): each
* member's indexed copybook file(s) and every `COPY` / `EXEC SQL INCLUDE`
src/mcp/tools.ts
+72 -6
View File
@@ -1060,6 +1060,54 @@ export function elidedBodyNote(filePath: string, name: string, from: number, to:
return `\n… lines ${from}-${to} of \`${name}\` elided — codegraph_explore \`${filePath}:${from}-${to}\` returns them`;
}
/**
* The symbol-shaped tokens of an explore query, the way its named-symbol
* seeding reads them: split on spaces, commas and brackets, a source-file
* extension dropped (`Program.cs` names `Program`), at least 3 characters,
* plain or qualified identifiers only, deduped, at most 16.
*/
export function exploreQueryTokens(query: string): string[] {
const FILE_EXT = /\.(?:java|kt|kts|ts|tsx|js|jsx|mjs|cjs|cs|py|go|rb|php|swift|rs|cpp|cc|cxx|c|h|hpp|scala|lua|dart|vue|svelte|astro|erl|hrl)$/i;
return [...new Set(
query.split(/[\s,()[\]]+/)
.map((t) => t.replace(FILE_EXT, '').trim())
.filter((t) => t.length >= 3 && /^[A-Za-z_$][\w$]*(?:(?:::|\.)[\w$]+)*$/.test(t))
)].slice(0, 16);
}
/**
* A query token written the way code names things — `ChangelogParser`,
* `addFeatures`, `build_report`, `Session.request`, `pkg::run` — rather than
* a single word, which may just as well be English (`How`, `Parser`, `main`).
* Only these are named back as missing from the index (#2462): a question's
* ordinary words must never be reported as code the index lacks.
*/
export function isIdentifierShapedToken(token: string): boolean {
return /[a-z][A-Z]/.test(token)
|| /[A-Za-z0-9]_[A-Za-z0-9]/.test(token)
|| /[A-Za-z_$][\w$]*(?:\.|::)[A-Za-z_$]/.test(token);
}
/**
* The note explore gives for names the query spelled that the index has never
* seen (#2462). With results, they match the query's other words, and an agent
* reading them as the answer to the missing names reads the wrong code. It
* points at what decides the answer — whether those files are indexed — and
* never at Read. Including a skipped folder is worded as the user's choice:
* indexing scope is the user's call, never the agent's (AGENTS.md).
*/
export function notInIndexNote(names: readonly string[], withResults: boolean): string {
const one = names.length === 1;
const list = names.map((n) => `\`${n}\``).join(', ');
const effect = withResults
? `, so the code below matches the query's other words, not ${one ? 'it' : 'them'}.`
: '.';
return `> **Not in the index:** ${list}. No indexed symbol, file, signature or comment uses ${one ? 'this name' : 'these names'}${effect}`
+ ` If ${one ? 'it is' : 'they are'} defined in this project, the file${one ? '' : 's'} defining ${one ? 'it' : 'them'} ${one ? 'is' : 'are'} not indexed:`
+ ` \`codegraph status\` shows what is. If ${one ? 'it lives' : 'they live'} in a folder skipped by default, like \`build/\` or \`dist/\`,`
+ ' the user can include that folder by adding a re-include such as `!/build/` to `.gitignore`.';
}
/**
* Join rendered parts with gap markers that name whatever the trim skipped.
*
@@ -4330,6 +4378,12 @@ export class ToolHandler {
explanation += miss.candidates.length > 0
? `\nCandidates to retry with codegraph_explore (shared words, not confirmed answers): ${cappedList(miss.candidates, 350)}`
: '\nNo shared-word symbol candidates found; retry codegraph_explore with literal symbol/file names or code terms.';
// Names written like code that the index has never seen may live in
// files it does not index (#2462): say which, and where to look.
const named = exploreQueryTokens(matchQuery).filter(isIdentifierShapedToken);
let unseen: string[] = [];
try { unseen = named.length > 0 ? cg.getUnseenNames(named) : []; } catch { /* never fail an explore call */ }
if (unseen.length > 0) explanation += `\n\n${notInIndexNote(unseen, false)}`;
}
const empty = `No relevant code found for "${query}"${missNote}${explanation}`;
// Still an explore call, so it is still recorded: an empty answer spends a
@@ -4420,8 +4474,10 @@ export class ToolHandler {
// Files declaring a TYPE the query named by name — the counter-case guard
// for the declaration-only penalty (CG-28). Populated in the token loop.
const namedTypeFiles = new Set<string>();
// Tokens written like code names that resolved to no node at all, not even
// a camel-infix definer: candidates for the "not in the index" note (#2462).
const unresolvedNamedTokens: string[] = [];
{
const FILE_EXT = /\.(?:java|kt|kts|ts|tsx|js|jsx|mjs|cjs|cs|py|go|rb|php|swift|rs|cpp|cc|cxx|c|h|hpp|scala|lua|dart|vue|svelte|astro|erl|hrl)$/i;
const CALLABLE = new Set(['method', 'function', 'component', 'constructor']);
// Variables/constants seed too: in Svelte/React a `$state` variable
// (`chatAtBottom`, `feedAtBottom`) is exactly the kind of symbol an agent
@@ -4433,11 +4489,7 @@ export class ToolHandler {
const isTestPath = (p: string) => /(^|\/)(tests?|specs?|__tests__|testdata|mocks?|fixtures?)\//i.test(p) || /\.(test|spec)\.[a-z]+$/i.test(p);
const bodyLines = (n: Node) => Math.max(0, (n.endLine ?? n.startLine) - n.startLine);
const callerCount = (n: Node) => { try { return cg.getCallers(n.id).length; } catch { return 0; } };
const tokens = [...new Set(
matchQuery.split(/[\s,()[\]]+/)
.map((t) => t.replace(FILE_EXT, '').trim())
.filter((t) => t.length >= 3 && /^[A-Za-z_$][\w$]*(?:(?:::|\.)[\w$]+)*$/.test(t))
)].slice(0, 16);
const tokens = exploreQueryTokens(matchQuery);
// PascalCase tokens in the query are type/file disambiguators — when the
// agent writes "DataRequest task validate", the `task`/`validate` it wants
// are DataRequest's, NOT the same-named overloads in Validation.swift /
@@ -4548,6 +4600,7 @@ export class ToolHandler {
.sort((a, b) => a.name.length - b.name.length)
.slice(0, 3);
}
if (raw.length === 0 && cands.length === 0 && isIdentifierShapedToken(t)) unresolvedNamedTokens.push(t);
// Bare lowercase words only seed defs their query-siblings corroborate
// (see the NL-stopword guard above). Filtering CANDS (not picks) applies
// the guard uniformly to both branches below, including the >3-def
@@ -5120,6 +5173,19 @@ export class ToolHandler {
];
const summaryLineIdx = 2;
// Names the query spelled that the index has never seen (#2462): said
// first, under the summary, because the code below then answers the
// query's other words. A query for a root `build/` folder's tools, which
// the default skip list leaves unindexed, came back as an unrelated
// `Program.Main` and a C++ `Parser`, read as the answer.
let unseenNames: string[] = [];
if (unresolvedNamedTokens.length > 0) {
try {
unseenNames = cg.getUnseenNames(unresolvedNamedTokens);
} catch { /* the note must never fail an explore call */ }
}
if (unseenNames.length > 0) lines.push(notInIndexNote(unseenNames, true), '');
// A named COBOL copybook's include sites, ALL of them (#2342) — the source
// below has room for a few include windows, and "what includes X" is the
// question a copybook name asks.