GithubStar/codegraph
1
0
Fork 0
mirror of https://github.com/colbymchenry/codegraph.git synced 2026-10-26 04:19:33 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(scope): a folder git ignores matches only itself, not every folder with its name (#2858)

Browse Source 
The scope matcher that the indexer, sync and the file watcher share also
takes in the folders `git ls-files -o -i --exclude-standard --directory`
reports as ignored (PR #1754, for issue #1728). They were added to it as
gitignore patterns. git reports them relative to the root, but a pattern of
one segment, like `src/`, has no anchor in gitignore syntax and matches at
any depth. So an ignored root `src/`, holding only a gitignored jib agent,
hid every module's `src/` in a Maven monorepo, and the index came up empty.
As patterns, folder names were also read as syntax: `[draft]` as a
character class, `a[b` as a regex that throws on the first match (the git
listing then fell back to a filesystem walk, and the watcher threw on every
event), `#tmp` as a comment and `!keep` as a negation.

The reported folders now stay root-relative paths in a set, and a path is
out of scope when it is one of them or lies inside one. Nothing is parsed,
so there is nothing to anchor or escape. Escaping was not an option: in
ignore 7.0.5 an escaped `?` never matches a literal `?`, an escaped
backslash before a `.` turns the dot into a wildcard, and every match is
case-insensitive. git prints these paths relative to its cwd, so for a
project in a subfolder of a repo they are relative to that subfolder, the
root the matcher is asked about.

The CLI hint for gitignored nested repos wrote their paths into
codegraph.json `includeIgnored` the same unanchored way, so an opt-in for
`child/` also took in any later `archive/child/`. It now writes a
root-anchored pattern with `*` and `[` escaped.

Validation:
- New git-ignored-dirs-scope tests (9): all fail on main and pass with the
  fix. They cover the issue's layout (scan, index, scoped sync), the lib/,
  build/ and dist/ variants, a `[draft]` folder, the watcher with a
  `scratch[1` folder, a project in a subfolder of its repo with `#tmp` and
  `!keep` folders, and the CLI hint end to end.
- A/B with the kernel in both arms, on real repos after writing the root
  folders a local build writes, against git's own view (ls-files --cached
  --others --exclude-standard, less codegraph's defaults). Files indexed of
  those in scope, main then fix: stencil 1015 and 1707 of 1707, Sentinel
  with the reporter's jib layout 104 and 1423 of 1423, ant-design 2851 and
  3138 of 3138, wger 767 and 770 of 770. The fix misses no file in scope
  and adds none outside it.
- For Sentinel, ant-design and wger, the fix's graph on the built tree is
  byte-identical to main's on a clean checkout. ant-design's 53 changed
  call sites move `useLocale` from the docs site's hook to
  components/locale/useLocale.ts. In stencil, every edge the fix adds on
  the built tree is also in main's graph on a clean checkout, and the 62
  name matches it no longer makes are unresolved there as well.
- Controls gin and flask: byte-identical graphs.
- Index time, median of 3 interleaved: gin 2572 and 2545 ms, flask 2400
  and 2378 ms; the repos above take longer only by their restored files.
  The scope step with 486 and 599 reported entries (a `.DS_Store` in
  every folder): main 206 and 416 ms, fix 105 and 155 ms.

Fixes #2849

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-10-23 20:06:41 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 44adef821b
commit a278c2ee12
4 changed files with 288 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/bin/codegraph.ts
+5 -3
View File
@@ -550,9 +550,11 @@ async function offerIndexIgnoredRepos(
opts: { interactive: boolean },
): Promise<IndexResult | undefined> {
let repos: string[];
let patterns: string[];
try {
const { findUnindexedIgnoredRepos } = await import('../extraction');
const { findUnindexedIgnoredRepos, includeIgnoredPatternFor } = await import('../extraction');
repos = findUnindexedIgnoredRepos(projectPath);
patterns = repos.map(includeIgnoredPatternFor);
} catch {
return; // detection is advisory — never let it break the command
}
@@ -563,7 +565,7 @@ async function offerIndexIgnoredRepos(
const SHOWN = 6;
const names = repos.slice(0, SHOWN).map((r) => r.replace(/\/$/, ''));
const extra = repos.length > SHOWN ? ` (+${formatNumber(repos.length - SHOWN)} more)` : '';
const snippet = `{ "includeIgnored": [${repos.map((p) => JSON.stringify(p)).join(', ')}] }`;
const snippet = `{ "includeIgnored": [${patterns.map((p) => JSON.stringify(p)).join(', ')}] }`;
clack.log.warn(
`Your .gitignore excludes ${isOne ? 'a nested git repository' : `${formatNumber(repos.length)} nested git repositories`} here, ` +
@@ -594,7 +596,7 @@ async function offerIndexIgnoredRepos(
let added: number;
try {
const { addIncludeIgnoredPatterns } = await import('../project-config');
added = addIncludeIgnoredPatterns(projectPath, repos);
added = addIncludeIgnoredPatterns(projectPath, patterns);
} catch (err) {
clack.log.error(`Could not update ${PROJECT_CONFIG_FILENAME}: ${err instanceof Error ? err.message : String(err)}`);
manualHint();
src/extraction/index.ts
+48 -14
View File
@@ -517,9 +517,12 @@ function readGitExcludeExtraPatterns(rootDir: string): string {
/**
* Directories `git ls-files -o -i --exclude-standard --directory` reports as
* ignored-untracked. Seeded into {@link ScopeIgnore} so nested `.gitignore`
* effects (and any exclude-standard rule the flat matcher might miss) prune the
* watcher the same way the indexer skips them (#1728).
* ignored-untracked, relative to `rootDir` (git prints paths relative to its
* cwd, also when `rootDir` is a subfolder of the repo) and trailing-slashed.
* Seeded into {@link ScopeIgnore} so nested `.gitignore` effects (and any
* exclude-standard rule the flat matcher might miss) prune the watcher the
* same way the indexer skips them (#1728). Each one is a path, not a pattern:
* see {@link ScopeIgnore}.
*/
function listGitIgnoredDirectories(rootDir: string): string[] {
try {
@@ -535,8 +538,11 @@ function listGitIgnoredDirectories(rootDir: string): string[] {
},
);
const dirs: string[] = [];
for (const entry of out.split('\0')) {
if (!entry) continue;
for (const raw of out.split('\0')) {
// `./` is git's "this whole directory" for a root that is itself ignored
// (#936); the root's own files stay in scope.
if (isWholeCwdEntry(raw)) continue;
const entry = normalizePath(raw);
dirs.push(entry.endsWith('/') ? entry : `${entry}/`);
}
return dirs;
@@ -958,11 +964,29 @@ export class ScopeIgnore {
*/
private include: Ignore | null = null,
private includeRoots: string[] = [],
/**
* The directories `git ls-files --exclude-standard` reports as ignored
* (see listGitIgnoredDirectories): root-relative, trailing-slashed paths.
* Each names one directory and is matched as a literal path prefix, never
* as a gitignore pattern. As a pattern, `src/` matched every `src/` below
* the root too, hiding a whole Maven monorepo's sources (#2849), and a
* folder named `[draft]` or `a[b` read as a glob (or did not compile).
*/
private gitIgnoredDirs: ReadonlySet<string> = new Set(),
) {
// Longest root first so paths in nested embedded repos hit the innermost matcher.
this.embedded = [...embedded].sort((a, b) => b.root.length - a.root.length);
}
/** True when `rel` is one of {@link gitIgnoredDirs} or lies inside one. */
private inGitIgnoredDir(rel: string): boolean {
if (this.gitIgnoredDirs.size === 0) return false;
for (let slash = rel.indexOf('/'); slash !== -1; slash = rel.indexOf('/', slash + 1)) {
if (this.gitIgnoredDirs.has(rel.slice(0, slash + 1))) return true;
}
return false;
}
ignores(rel: string): boolean {
// User `exclude` (#999) is checked first and against the full root-relative
// path: it must drop git-TRACKED paths (which `.gitignore` can't) and apply
@@ -994,7 +1018,7 @@ export class ScopeIgnore {
if (rel.endsWith('/') && this.embedded.some(({ root }) => root.startsWith(rel))) {
return false;
}
return this.rootMatcher.ignores(rel);
return this.rootMatcher.ignores(rel) || this.inGitIgnoredDir(rel);
}
}
@@ -1006,19 +1030,17 @@ export class ScopeIgnore {
export function buildScopeIgnore(rootDir: string, embeddedRoots?: Iterable<string>): ScopeIgnore {
const roots = embeddedRoots ? [...embeddedRoots] : discoverEmbeddedRepoRoots(rootDir);
const include = loadIncludeMatcher(rootDir);
// Root matcher already has defaults + root `.gitignore` + info/exclude +
// core.excludesFile. Seed ignored-untracked directories from git so nested
// `.gitignore` effects prune the watcher identically to the indexer (#1728).
const rootMatcher = buildDefaultIgnore(rootDir);
for (const dir of listGitIgnoredDirectories(rootDir)) {
rootMatcher.add(dir);
}
// The root matcher has defaults + root `.gitignore` + info/exclude +
// core.excludesFile. The ignored-untracked directories git reports join
// them, so nested `.gitignore` effects prune the watcher identically to the
// indexer (#1728) — as literal paths, not patterns (#2849).
return new ScopeIgnore(
rootMatcher,
buildDefaultIgnore(rootDir),
roots.map((root) => ({ root, matcher: buildDefaultIgnore(path.join(rootDir, root)) })),
loadExcludeMatcher(rootDir),
include,
include ? includeStaticRoots(loadIncludePatterns(rootDir)) : [],
new Set(listGitIgnoredDirectories(rootDir)),
);
}
@@ -1167,6 +1189,18 @@ export function findUnindexedIgnoredRepos(rootDir: string): string[] {
return repos;
}
/**
* The `codegraph.json` `includeIgnored` pattern that opts in exactly the repo
* at `repoDir`, a root-relative, trailing-slashed path from
* {@link findUnindexedIgnoredRepos}. Patterns are gitignore syntax, so the
* path is anchored to the root (a bare `child/` also matches every `child/`
* below it) and its `*` and `[` are escaped (#2849). The `ignore` matcher has
* no working escape for `?`, so a `?` stays a one-character wildcard.
*/
export function includeIgnoredPatternFor(repoDir: string): string {
return '/' + repoDir.replace(/[*[]/g, '\\$&');
}
/**
* Discover embedded repos hidden by `repoDir`'s OWN gitignore rules: for each
* gitignored directory, search for nested `.git` roots. Returns repo paths