diff --git a/docs/implementation/cli-commands.md b/docs/implementation/cli-commands.md index 36618e25..e94b5d22 100644 --- a/docs/implementation/cli-commands.md +++ b/docs/implementation/cli-commands.md @@ -20,7 +20,7 @@ The CLI exposes `example`, `languages`, `feedback`, top-level indexed `search` / | `pkg changelog [spec]` | package spec OR `--repo-url` | `--from`, `--to`, `--limit`, `--git-ref`, `--no-body`, `--verbose`, `--json` | Release notes / changelog entries for a package or GitHub repo (GitHub Releases, CHANGELOG.md, or HexDocs). Default shows each entry with a 10-line body preview; `--verbose` uncaps, `--no-body` drops. | | `docs list ` | package spec (optional `@version`) | `--limit`, `--after`, `--verbose`, `--json` | List hosted/crawled and repository-backed documentation pages for a package. Entries include page IDs for `docs read`; JSON includes exact repo-file follow-up metadata when available. | | `docs read ` | page ID from `docs list` or search results | `--lines`, `--verbose`, `--json` | Read a documentation page by page ID. Default output is content-only; `--lines` fetches a bounded range for long pages. | -| `code files [spec] [path-prefix]` | package spec OR `--repo-url` + `--git-ref`; optional `[path-prefix]` | `--limit`, `--wait`, `--verbose`, `--json` | List files in an indexed dependency. `[path-prefix]` is a literal directory prefix (not a glob). Plain output is one path per line; `--verbose` adds language / type / size annotations. Indexing-retry via `--wait` or the `availableVersions` hint in the error envelope. | +| `code files [spec] [path-prefix]` | package spec OR `--repo-url` + `--git-ref`; optional `[path-prefix]` | `--path`, repeatable `--glob`, repeatable `--ext`, repeatable `--file-type`, repeatable `--language`, repeatable `--file-intent`, repeatable `--exclude-intent`, `--exclude-docs`, `--exclude-tests`, `--hidden`, `--limit`, `--wait`, `--verbose`, `--json` | List files in an indexed dependency. Selectors (`[path-prefix]`, `--path`, `--glob`) are OR-ed; the other flags filter that scope down further. Plain output is one path per line; `--verbose` adds language / type / size annotations. Indexing-retry via `--wait` or the `availableVersions` hint in the error envelope. | | `code read ` | package spec OR `--repo-url` + `--git-ref`; plus `` | `--lines`, `--start`, `--end`, `--wait`, `--verbose`, `--json` | Read a file's contents. Plain output is the raw file bytes (pipe-friendly); `--verbose` adds a header and a line-number gutter. `--lines 10-40` concise form; `--start`/`--end` equivalent. Binary files show a sentinel line. | | `code grep [spec] [path-prefix]` | package spec OR `--repo-url` + `--git-ref`; plus `` and optional `[path-prefix]` | `--path`, repeatable `--glob`, repeatable `--ext`, `--regex`, `--case-sensitive`, `-C/-A/-B`, `--exclude-docs`, `--exclude-tests`, `--limit`, `--per-file-limit`, `--cursor`, `--symbol-field`, `--wait`, `--verbose`, `--json` | Deterministic text grep over indexed dependency or repository source. Defaults to whole-target, literal, ASCII case-insensitive matching; non-ASCII letters match case-sensitively. Narrow with `[path-prefix]`, `--path`, `--glob`, or `--ext`. Plain output is `file:line:text`; `--verbose` groups matches by file. | diff --git a/docs/implementation/tools.md b/docs/implementation/tools.md index a63b498d..b15f0e85 100644 --- a/docs/implementation/tools.md +++ b/docs/implementation/tools.md @@ -28,7 +28,7 @@ The CLI mirrors the production MCP tool contract where equivalent tools exist. C | `pkg_vulns` | `registry`, `package_name`, `version?`, `min_severity?`, `include_withdrawn?` | Known vulnerabilities for a package on npm, PyPI, Hex, or Crates. Count summary, per-advisory OSV ID + severity + affected/fix ranges, and upgrade paths. Malware is surfaced in a disjoint bucket. | | `pkg_deps` | `registry`, `package_name`, `version?`, `lifecycle?`, `include_transitive?`, `include_importers?`, `max_depth?` | Direct runtime dependency list (each `{name, version, constraint}` — the backend resolves each constraint to a concrete version) plus, when the backend has them, structured groups for dev / peer / build / optional with registry-specific condition metadata (PyPI extras, Crates features). Optional transitive block with aggregate edge counts, the preprocessed install footprint as `{name, version}`, typed conflicts and circular-dependency cycles; opt into per-package importer provenance with `include_importers`. | | `pkg_changelog` | `registry?`, `package_name?`, `repo_url?`, `from_version?`, `to_version?`, `limit?`, `git_ref?`, `include_bodies?` | Release notes or changelog entries for a package or GitHub repo. Default latest mode returns the ten most recent entries; `from_version` switches to range mode (no count cap). Dual addressing (spec vs repo URL) mutually exclusive. Response includes optional `source` (`"releases"` / `"changelog_file"` / `"hexdocs"`) when a concrete changelog source exists, `mode` (`"latest"` / `"range"`), and `entries: { count, items }` with full markdown bodies by default; set `include_bodies: false` for a lean version / date / URL timeline. Package-version entries without changelog text succeed with `source` omitted. | -| `code_files` | `target`, `path_prefix?`, `limit?`, `wait_timeout_ms?`, `format?` | List files in an indexed dependency. Returns `{total, hasMore, files: [{path, name, language, fileType, byteSize}], resolution, indexedVersion}` in JSON mode. Dual addressing via `target.registry + target.package_name` (spec) or `target.repo_url + target.git_ref` (repo). `path_prefix` is a literal directory prefix — NOT a glob (`*.ts` won't match); glob / pattern filtering is an upstream enhancement. Emits an `INDEXING` error envelope when the dependency is being indexed on-demand — retry with a longer `wait_timeout_ms` or pick a version from `details.availableVersions`. `format` defaults to `text-v1` (paths-only listing); pass `format: "json"` for the structured envelope. | +| `code_files` | `target`, `path?`, `path_prefix?`, `globs?`, `extensions?`, `file_types?`, `languages?`, `file_intent?`, `file_intents?`, `exclude_file_intents?`, `exclude_doc_files?`, `exclude_test_files?`, `include_hidden?`, `limit?`, `wait_timeout_ms?`, `format?` | List files in an indexed dependency. Returns `{total, hasMore, files: [{path, name, language, fileType, byteSize}], resolution, indexedVersion}` in JSON mode. Dual addressing via `target.registry + target.package_name` (spec) or `target.repo_url + target.git_ref` (repo). Selectors (`path`, `path_prefix`, `globs`) are OR-ed; the other filters intersect on top. Emits an `INDEXING` error envelope when the dependency is being indexed on-demand — retry with a longer `wait_timeout_ms` or pick a version from `details.availableVersions`. `format` defaults to `text-v1` (paths-only listing); pass `format: "json"` for the structured envelope. | | `code_read` | `target`, `path`, `start_line?`, `end_line?`, `wait_timeout_ms?` | Read a file from an indexed dependency. **MCP per-call span cap: 150 lines** — broader requests (or no range) are silently truncated to the first 150 lines from the caller's start, with a `hint` field explaining the cap and the original request. Use `start_line` / `end_line` to pick a focused window — typical 80-150 lines around a known symbol from `search` / `code_grep`. Binary files set `isBinary: true` and omit `content`. On `NOT_FOUND` / `FILE_NOT_FOUND` call `code_files` to discover the actual path. The cap is MCP-only; the CLI command `githits code read` honors arbitrary ranges. | | `code_grep` | `target`, `pattern`, `path?`, `path_prefix?`, `globs?`, `extensions?`, `pattern_type?`, `case_sensitive?`, `exclude_doc_files?`, `exclude_test_files?`, `context_lines?`, `context_lines_before?`, `context_lines_after?`, `max_matches?`, `max_matches_per_file?`, `cursor?`, `symbol_fields?`, `wait_timeout_ms?`, `format?` | Deterministic text grep over indexed dependency or repository source. Defaults to literal, ASCII case-insensitive matching across the whole target; non-ASCII letters match case-sensitively. Narrow with `path`, `path_prefix`, `globs`, or `extensions`. `pattern_type: "regex"` uses RE2 syntax; whole-target regexes must include at least one literal substring for index pre-filtering. Returns matches plus pagination and scan counters; `symbol_fields` hydrates enclosing symbol metadata on each match. `format` defaults to `text-v1` (matches grouped by file, grep -A/-B notation for context); pass `format: "json"` for the structured envelope. | @@ -114,7 +114,7 @@ The CLI mirrors the production MCP tool contract where equivalent tools exist. C These three indexed tools share an addressing and lifecycle contract (documented below) and then each projects its own data-first envelope. All three reuse the shipped `codeTargetSchema` + `resolveCodeTarget` from `src/tools/code-navigation-shared.ts` — no parallel addressing module. -**`code_files` envelope**: `{registry?|repoUrl?+gitRef?, total, hasMore, indexedVersion?, resolution?, files: [{path, name?, language?, fileType?, byteSize?}], hint?, filter?}`. `fileType` values preserve the service vocabulary (`CONFIG`, `SOURCE`, `DOC`, `TEST`). `total` is capped at returned count when `hasMore: true` — the terminal formatter renders `N+ files` in that case to avoid misleading users. `filter.pathPrefix` / `filter.limit` echo only when the caller supplied them explicitly; default limit (200) never round-trips. +**`code_files` envelope**: `{registry?|repoUrl?+gitRef?, total, hasMore, indexedVersion?, resolution?, files: [{path, name?, language?, fileType?, byteSize?}], hint?, filter?}`. `fileType` values preserve the service vocabulary (`CONFIG`, `SOURCE`, `DOC`, `TEST`). `total` is capped at returned count when `hasMore: true` — the terminal formatter renders `N+ files` in that case to avoid misleading users. `filter` echoes only explicit caller filters (`path`, `pathPrefix`, `globs`, `extensions`, `fileTypes`, `languages`, file-intent filters, booleans, and `limit`); default limit (200) never round-trips. **`code_read` envelope**: `{registry?|repoUrl?+gitRef?, path, language?, totalLines?, startLine?, endLine?, content?, isBinary?, hint?}`. `path` (not `filePath`) so the key matches `code_files.files[].path` and `code_grep.filter.path` when exact-file grep is used. Binary files set `isBinary: true` and **omit** `content` (not `null`); agents branch on the flag. `hint` is emitted only when the MCP span cap actually truncated the response — see "code_read span cap" below. @@ -178,13 +178,13 @@ More hits available. Pass offset=N for the next page or limit=N to widen. **Listing anatomy** (`code_files` text-v1): ``` -code_files | [+] paths | [path_prefix="..."] [limit=N] +code_files | [+] paths | [path="..."] [path_prefix="..."] [globs=...] [exts=...] [...] [blank] ... [blank] -More files available. Pass limit=N to widen or refine path_prefix. +More files available. Pass limit=N or refine the filter. ``` `` is `:@` for spec addressing or `@` for repo addressing. Filter echoes appear in the header only when the caller supplied them explicitly (defaults never echo). diff --git a/src/commands/code/files.test.ts b/src/commands/code/files.test.ts index 96d0b5c2..542f9023 100644 --- a/src/commands/code/files.test.ts +++ b/src/commands/code/files.test.ts @@ -136,6 +136,63 @@ describe("pkgFilesAction", () => { writeSpy.mockRestore(); }); + it("forwards advanced file filters to the service", async () => { + const listFiles = mock(() => Promise.resolve(defaultListFilesResult)); + const service = createMockCodeNavigationService({ listFiles }); + const writeSpy = spyOn(process.stdout, "write").mockImplementation( + (() => true) as typeof process.stdout.write, + ); + await pkgFilesAction( + "npm:express", + "src/", + { + path: "README.md", + glob: ["test/**/*.js"], + ext: ["js"], + fileType: ["source"], + language: ["JavaScript"], + fileIntent: ["production", "test"], + excludeIntent: ["generated"], + excludeDocs: true, + excludeTests: false, + hidden: true, + }, + createDeps({ codeNavigationService: service }), + ); + const calls = listFiles.mock.calls as unknown as Array< + [ + { + pathSelectors?: Array<{ kind: string; value: string }>; + pathPrefix?: string; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntents?: string[]; + excludeFileIntents?: string[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + includeHidden?: boolean; + }, + ] + >; + expect(calls[0]?.[0]).toMatchObject({ + pathSelectors: [ + { kind: "EXACT", value: "README.md" }, + { kind: "GLOB", value: "test/**/*.js" }, + ], + pathPrefix: "src/", + extensions: ["js"], + fileTypes: ["source"], + languages: ["JavaScript"], + fileIntents: ["PRODUCTION", "TEST"], + excludeFileIntents: ["GENERATED"], + excludeDocFiles: true, + excludeTestFiles: false, + includeHidden: true, + }); + writeSpy.mockRestore(); + }); + it("rejects a second positional in --repo-url mode", async () => { const errorSpy = spyOn(console, "error").mockImplementation(() => {}); const exitSpy = spyOn(process, "exit").mockImplementation(() => { @@ -172,6 +229,41 @@ describe("pkgFilesAction", () => { logSpy.mockRestore(); }); + it("echoes advanced filters in the JSON envelope", async () => { + const logSpy = spyOn(console, "log").mockImplementation(() => {}); + await pkgFilesAction( + "npm:express", + "src/", + { + path: "README.md", + glob: ["test/**/*.js"], + ext: ["js"], + fileType: ["source"], + language: ["JavaScript"], + fileIntent: ["production", "test"], + excludeIntent: ["generated"], + excludeDocs: true, + hidden: true, + json: true, + }, + createDeps(), + ); + const payload = JSON.parse(logSpy.mock.calls[0]?.[0] as string); + expect(payload.filter).toEqual({ + path: "README.md", + pathPrefix: "src/", + globs: ["test/**/*.js"], + extensions: ["js"], + fileTypes: ["source"], + languages: ["JavaScript"], + fileIntents: ["production", "test"], + excludeFileIntents: ["generated"], + excludeDocFiles: true, + includeHidden: true, + }); + logSpy.mockRestore(); + }); + it("sends waitTimeoutMs defaulting to DEFAULT_WAIT_TIMEOUT_MS (20000)", async () => { const listFiles = mock(() => Promise.resolve(defaultListFilesResult)); const service = createMockCodeNavigationService({ listFiles }); diff --git a/src/commands/code/files.ts b/src/commands/code/files.ts index 7560a038..49c8c0f6 100644 --- a/src/commands/code/files.ts +++ b/src/commands/code/files.ts @@ -7,7 +7,11 @@ import { } from "../../shared/code-navigation-defaults.js"; import { shouldUseColors } from "../../shared/colors.js"; import { InvalidPackageSpecError, requireAuth } from "../../shared/index.js"; -import { buildListFilesParams } from "../../shared/list-files-request.js"; +import { + buildListFilesParams, + type ListFilesRequestBuildResult, + type ListFilesRequestInput, +} from "../../shared/list-files-request.js"; import { buildListFilesSuccessPayload, formatListFilesTerminal, @@ -23,6 +27,16 @@ import { export interface PkgFilesCommandOptions { repoUrl?: string; gitRef?: string; + path?: string; + glob?: string[]; + ext?: string[]; + fileType?: string[]; + language?: string[]; + fileIntent?: string[]; + excludeIntent?: string[]; + excludeDocs?: boolean; + excludeTests?: boolean; + hidden?: boolean; limit?: string; wait?: string; verbose?: boolean; @@ -76,9 +90,19 @@ export async function pkgFilesAction( MAX_WAIT_TIMEOUT_MS, ); - const build = buildListFilesParams({ + const build = buildCliListFilesParams({ target, + path: options.path, pathPrefix, + globs: options.glob, + extensions: options.ext, + fileTypes: options.fileType, + languages: options.language, + fileIntents: options.fileIntent, + excludeFileIntents: options.excludeIntent, + excludeDocFiles: options.excludeDocs, + excludeTestFiles: options.excludeTests, + includeHidden: options.hidden, limit, waitTimeoutMs: wait, }); @@ -91,10 +115,20 @@ export async function pkgFilesAction( name: target.packageName, repoUrl: target.repoUrl, gitRef: target.gitRef, - limitExplicit: build.limitExplicit, - pathPrefixExplicit: build.pathPrefixExplicit, - pathPrefix: build.params.pathPrefix, - limit: build.params.limit, + path: build.filterEcho.path, + pathPrefix: build.filterEcho.pathPrefix, + globs: build.filterEcho.globs, + extensions: build.filterEcho.extensions, + fileTypes: build.filterEcho.fileTypes, + languages: build.filterEcho.languages, + fileIntent: build.filterEcho.fileIntent, + fileIntents: build.filterEcho.fileIntents, + excludeFileIntents: build.filterEcho.excludeFileIntents, + excludeDocFiles: build.filterEcho.excludeDocFiles, + excludeTestFiles: build.filterEcho.excludeTestFiles, + includeHidden: build.filterEcho.includeHidden, + limit: build.filterEcho.limit, + explicit: build.explicit, }); if (options.json) { @@ -117,6 +151,35 @@ export async function pkgFilesAction( } } +function collectRepeatable( + value: string, + previous: string[] | undefined, +): string[] { + return [...(previous ?? []), value]; +} + +function buildCliListFilesParams( + input: ListFilesRequestInput, +): ListFilesRequestBuildResult { + try { + return buildListFilesParams(input); + } catch (error) { + if (!(error instanceof InvalidPackageSpecError)) throw error; + const rewritten = error.message + .replace(/^`path`/, "`--path`") + .replace(/`globs`/g, "`--glob`") + .replace(/`extensions`/g, "`--ext`") + .replace(/`file_types`/g, "`--file-type`") + .replace(/`languages`/g, "`--language`") + .replace(/`file_intent`/g, "`--file-intent`") + .replace(/`file_intents`/g, "`--file-intent`") + .replace(/`exclude_file_intents`/g, "`--exclude-intent`") + .replace(/`path_prefix`/g, "`[path-prefix]`"); + if (rewritten === error.message) throw error; + throw new InvalidPackageSpecError(rewritten); + } +} + // Detects strings that look like `:` — used to flag // a common user mistake where a package spec is passed together with // --repo-url. We'd otherwise silently treat it as a (meaningless) @@ -154,8 +217,11 @@ fetch more. Returned paths feed directly into \`githits code read\` and \`githits code grep\`. [path-prefix] is a literal directory prefix (e.g. \`src/\` or -\`lib/parser\`), NOT a glob — \`*.ts\` and similar patterns won't -match. File-type / extension filtering is not supported server-side. +\`lib/parser\`). Use --path for exact-file selectors, repeatable +--glob for glob selectors, and --ext / --file-type / --language / +--file-intent to intersect further. Selectors ([path-prefix], --path, +--glob) are OR-ed — a file matches if any selector matches. The other +filters intersect on top. Addressing: (registry:name[@version]) OR --repo-url --git-ref . Supported registries: npm, pypi, hex, crates, @@ -189,6 +255,36 @@ export function registerCodeFilesCommand(pkgCommand: Command): Command { "--git-ref ", "Tag, commit, branch, or HEAD. Required with --repo-url.", ) + .option("--path ", "Exact file selector") + .option("--glob ", "Glob selector (repeatable)", collectRepeatable) + .option( + "--ext ", + "Extension filter without leading dot (repeatable)", + collectRepeatable, + ) + .option( + "--file-type ", + "File type filter such as source or doc (repeatable)", + collectRepeatable, + ) + .option( + "--language ", + "Language filter matching aigrep language names (repeatable)", + collectRepeatable, + ) + .option( + "--file-intent ", + "Inclusive file-intent filter. Repeat to include multiple intents: production, test, benchmark, example, generated, fixture, build, vendor", + collectRepeatable, + ) + .option( + "--exclude-intent ", + "Exclude these file intents after inclusive filtering (repeatable)", + collectRepeatable, + ) + .option("--exclude-docs", "Skip files classified as documentation") + .option("--exclude-tests", "Skip files classified as tests") + .option("--hidden", "Include dotfiles and dot-prefixed paths") .option("--limit ", "Max entries (1-1000, default 200)") .option( "--wait ", diff --git a/src/commands/mcp-instructions.ts b/src/commands/mcp-instructions.ts index 700f310c..4a26baac 100644 --- a/src/commands/mcp-instructions.ts +++ b/src/commands/mcp-instructions.ts @@ -46,7 +46,7 @@ const SEARCH_STATUS_BULLET = "- `search_status` — follow up a prior `search` by `searchRef` to check progress, fetch partial hits, or fetch final results."; const CODE_FILES_BULLET = - '- `code_files` — list files in an indexed dependency. Use `path_prefix` to scope to a directory. Returned paths feed into `code_read` and help scope `code_grep`; pass `format: "json"` for language/type/size metadata.'; + '- `code_files` — list files in an indexed dependency. `path`, `path_prefix`, and `globs` are OR-ed selectors; extensions, language, file type, and file-intent filters intersect on top. Returned paths feed into `code_read` and help scope `code_grep`; pass `format: "json"` for language/type/size metadata.'; const CODE_READ_BULLET = "- `code_read` — read a dependency file by `path`. **MCP cap: 150 lines per call**; choose focused `start_line` / `end_line` windows from `search` or `code_grep`. Binary files set `isBinary: true` and omit `content`. On `FILE_NOT_FOUND` / `NOT_FOUND`, call `code_files` for the actual path."; diff --git a/src/services/code-navigation-service.test.ts b/src/services/code-navigation-service.test.ts index 38096ed4..314c1b52 100644 --- a/src/services/code-navigation-service.test.ts +++ b/src/services/code-navigation-service.test.ts @@ -99,6 +99,79 @@ describe("CodeNavigationServiceImpl", () => { expect(result.resolution?.resolvedRef).toBe("v5.2.1"); }); + it("forwards listRepoFiles v7 filters to GraphQL variables", async () => { + let capturedBody = ""; + globalThis.fetch = mock((_, init?: RequestInit) => { + capturedBody = String(init?.body ?? ""); + return Promise.resolve( + new Response( + JSON.stringify({ + data: { + listRepoFiles: { + files: [], + total: 0, + hasMore: false, + indexedVersion: "v5.2.1", + resolution: null, + diagnostics: null, + codeIndexState: "CURRENT", + }, + }, + }), + { status: 200 }, + ), + ); + }) as unknown as typeof fetch; + + const service = new CodeNavigationServiceImpl( + BASE_URL, + createMockTokenProvider(), + ); + + await service.listFiles({ + target: { registry: "NPM", packageName: "express", version: "4.18.2" }, + pathPrefix: "src/", + pathSelectors: [ + { kind: "EXACT", value: "README.md" }, + { kind: "GLOB", value: "test/**/*.js" }, + ], + extensions: ["js", "mjs"], + fileTypes: ["source", "doc"], + languages: ["JavaScript"], + fileIntent: "PRODUCTION", + excludeFileIntents: ["TEST", "BENCHMARK"], + excludeDocFiles: true, + excludeTestFiles: false, + includeHidden: true, + limit: 25, + waitTimeoutMs: 1234, + }); + + const payload = JSON.parse(capturedBody) as { + variables: Record; + }; + expect(payload.variables).toMatchObject({ + registry: "NPM", + packageName: "express", + version: "4.18.2", + pathPrefix: "src/", + pathSelectors: [ + { kind: "EXACT", value: "README.md" }, + { kind: "GLOB", value: "test/**/*.js" }, + ], + extensions: ["js", "mjs"], + fileTypes: ["source", "doc"], + languages: ["JavaScript"], + fileIntent: "PRODUCTION", + excludeFileIntents: ["TEST", "BENCHMARK"], + excludeDocFiles: true, + excludeTestFiles: false, + includeHidden: true, + limit: 25, + waitTimeoutMs: 1234, + }); + }); + it("throws CodeNavigationIndexingError for data-path INDEXING sentinel on listFiles", async () => { mockFetch(() => Promise.resolve( diff --git a/src/services/code-navigation-service.ts b/src/services/code-navigation-service.ts index e0996ad4..11b0c45f 100644 --- a/src/services/code-navigation-service.ts +++ b/src/services/code-navigation-service.ts @@ -244,7 +244,17 @@ export interface AvailableVersion { */ export interface ListFilesParams { target: CodeNavigationTarget; + pathSelectors?: GrepRepoPathSelector[]; pathPrefix?: string; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntent?: FileIntent; + fileIntents?: FileIntent[]; + excludeFileIntents?: FileIntent[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + includeHidden?: boolean; limit?: number; waitTimeoutMs?: number; } @@ -946,6 +956,16 @@ query ListRepoFiles( $gitRef: String $version: String $pathPrefix: String + $pathSelectors: [FilePathSelectorInput!] + $extensions: [String!] + $fileTypes: [String!] + $languages: [String!] + $fileIntent: FileIntent + $fileIntents: [FileIntent!] + $excludeFileIntents: [FileIntent!] + $excludeDocFiles: Boolean + $excludeTestFiles: Boolean + $includeHidden: Boolean $limit: Int $waitTimeoutMs: Int ) { @@ -956,6 +976,16 @@ query ListRepoFiles( gitRef: $gitRef version: $version pathPrefix: $pathPrefix + pathSelectors: $pathSelectors + extensions: $extensions + fileTypes: $fileTypes + languages: $languages + fileIntent: $fileIntent + fileIntents: $fileIntents + excludeFileIntents: $excludeFileIntents + excludeDocFiles: $excludeDocFiles + excludeTestFiles: $excludeTestFiles + includeHidden: $includeHidden limit: $limit waitTimeoutMs: $waitTimeoutMs ) { @@ -1787,6 +1817,19 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { gitRef: params.target.gitRef, version: params.target.version, pathPrefix: params.pathPrefix, + pathSelectors: params.pathSelectors?.map((entry) => ({ + kind: entry.kind, + value: entry.value, + })), + extensions: params.extensions, + fileTypes: params.fileTypes, + languages: params.languages, + fileIntent: params.fileIntent, + fileIntents: params.fileIntents, + excludeFileIntents: params.excludeFileIntents, + excludeDocFiles: params.excludeDocFiles, + excludeTestFiles: params.excludeTestFiles, + includeHidden: params.includeHidden, limit: params.limit, waitTimeoutMs: params.waitTimeoutMs, }, diff --git a/src/shared/code-navigation.ts b/src/shared/code-navigation.ts index 7a4bf694..9b94edf4 100644 --- a/src/shared/code-navigation.ts +++ b/src/shared/code-navigation.ts @@ -68,6 +68,8 @@ const fileIntentMap = { vendor: "VENDOR", } as const satisfies Record; +export type FileIntentArg = keyof typeof fileIntentMap; + /** * Back-compat alias for {@link PkgseerRegistryArg}. The registry map * now lives in `pkgseer-registry.ts` as single source of truth; this @@ -115,3 +117,11 @@ export function toFileIntent( ? fileIntentMap[intent as keyof typeof fileIntentMap] : undefined; } + +export function isKnownFileIntent(value: string): value is FileIntentArg { + return value in fileIntentMap; +} + +export function knownFileIntentList(): ReadonlyArray { + return Object.keys(fileIntentMap) as FileIntentArg[]; +} diff --git a/src/shared/index.ts b/src/shared/index.ts index 8da15639..15f96aa6 100644 --- a/src/shared/index.ts +++ b/src/shared/index.ts @@ -1,6 +1,9 @@ export { type CodeNavigationRegistryArg, + type FileIntentArg, + isKnownFileIntent, isKnownSymbolKind, + knownFileIntentList, knownSymbolCategoryList, knownSymbolKindList, type SymbolCategoryArg, diff --git a/src/shared/list-files-request.test.ts b/src/shared/list-files-request.test.ts index 1e65b88f..f69e0c83 100644 --- a/src/shared/list-files-request.test.ts +++ b/src/shared/list-files-request.test.ts @@ -41,21 +41,73 @@ describe("buildListFilesParams — defaults + passthrough", () => { }); it("passes a trimmed pathPrefix through and marks it explicit", () => { - const { params, pathPrefixExplicit } = buildListFilesParams({ + const { params, explicit } = buildListFilesParams({ target: packageTarget, pathPrefix: " src/ ", }); expect(params.pathPrefix).toBe("src/"); - expect(pathPrefixExplicit).toBe(true); + expect(explicit.pathPrefix).toBe(true); }); it("treats whitespace-only pathPrefix as absent", () => { - const { params, pathPrefixExplicit } = buildListFilesParams({ + const { params, explicit } = buildListFilesParams({ target: packageTarget, pathPrefix: " ", }); expect(params.pathPrefix).toBeUndefined(); - expect(pathPrefixExplicit).toBe(false); + expect(explicit.pathPrefix).toBe(false); + }); + + it("builds exact and glob path selectors", () => { + const { params, explicit, filterEcho } = buildListFilesParams({ + target: packageTarget, + path: " src/index.ts ", + globs: ["src/**/*.ts", "test/**/*.ts"], + }); + expect(params.pathSelectors).toEqual([ + { kind: "EXACT", value: "src/index.ts" }, + { kind: "GLOB", value: "src/**/*.ts" }, + { kind: "GLOB", value: "test/**/*.ts" }, + ]); + expect(explicit.path).toBe(true); + expect(explicit.globs).toBe(true); + expect(filterEcho.path).toBe("src/index.ts"); + expect(filterEcho.globs).toEqual(["src/**/*.ts", "test/**/*.ts"]); + }); + + it("normalises file filters and file intents", () => { + const { params, filterEcho, explicit } = buildListFilesParams({ + target: packageTarget, + extensions: ["ts", "tsx"], + fileTypes: ["source", "doc"], + languages: ["TypeScript", "TSX"], + fileIntents: ["production", "test"], + excludeFileIntents: ["generated", "fixture"], + excludeDocFiles: true, + excludeTestFiles: false, + includeHidden: true, + }); + expect(params.extensions).toEqual(["ts", "tsx"]); + expect(params.fileTypes).toEqual(["source", "doc"]); + expect(params.languages).toEqual(["TypeScript", "TSX"]); + expect(params.fileIntents).toEqual(["PRODUCTION", "TEST"]); + expect(params.excludeFileIntents).toEqual(["GENERATED", "FIXTURE"]); + expect(params.excludeDocFiles).toBe(true); + expect(params.excludeTestFiles).toBe(false); + expect(params.includeHidden).toBe(true); + expect(filterEcho.fileIntents).toEqual(["production", "test"]); + expect(filterEcho.excludeFileIntents).toEqual(["generated", "fixture"]); + expect(explicit.excludeTestFiles).toBe(true); + }); + + it("normalises a singular file intent and echoes it canonically", () => { + const { params, filterEcho, explicit } = buildListFilesParams({ + target: packageTarget, + fileIntent: " Production ", + }); + expect(params.fileIntent).toBe("PRODUCTION"); + expect(filterEcho.fileIntent).toBe("production"); + expect(explicit.fileIntent).toBe(true); }); }); @@ -99,3 +151,42 @@ describe("buildListFilesParams — waitTimeoutMs bounds", () => { ).toBe(60000); }); }); + +describe("buildListFilesParams — filter validation", () => { + it("rejects an empty exact path", () => { + expect(() => + buildListFilesParams({ target: packageTarget, path: " " }), + ).toThrow(/`path` cannot be empty/); + }); + + it("rejects empty list entries", () => { + expect(() => + buildListFilesParams({ target: packageTarget, globs: ["src/**", " "] }), + ).toThrow(/`globs` entries cannot be empty/); + }); + + it("rejects leading dots in extensions", () => { + expect(() => + buildListFilesParams({ target: packageTarget, extensions: [".ts"] }), + ).toThrow(/must not include a leading dot/); + }); + + it("rejects unknown file intents", () => { + expect(() => + buildListFilesParams({ + target: packageTarget, + fileIntents: ["production", "weird"], + }), + ).toThrow(/file_intents/); + }); + + it("rejects file_intent together with file_intents", () => { + expect(() => + buildListFilesParams({ + target: packageTarget, + fileIntent: "production", + fileIntents: ["test"], + }), + ).toThrow(/cannot be combined/); + }); +}); diff --git a/src/shared/list-files-request.ts b/src/shared/list-files-request.ts index 7a20b6b1..f053242e 100644 --- a/src/shared/list-files-request.ts +++ b/src/shared/list-files-request.ts @@ -1,15 +1,22 @@ /** * Shared request builder for the `list_files` tool. CLI and MCP * normalise inputs here so the two surfaces cannot diverge on - * addressing or limit validation. Addressing XOR is delegated to - * the shipped `resolveCodeTarget` helper; this module owns the - * tool-specific bounds. + * selector/filter parsing, file-intent coercion, or limit validation. + * Addressing XOR is delegated to the shipped `resolveCodeTarget` + * helper; this module owns the tool-specific bounds and filter rules. */ import type { CodeNavigationTarget, + FileIntent, + GrepRepoPathSelector, ListFilesParams, } from "../services/index.js"; +import { + isKnownFileIntent, + knownFileIntentList, + toFileIntent, +} from "./code-navigation.js"; import { DEFAULT_WAIT_TIMEOUT_MS } from "./code-navigation-defaults.js"; import { InvalidPackageSpecError } from "./package-spec.js"; @@ -23,7 +30,18 @@ const WAIT_MAX = 60_000; export interface ListFilesRequestInput { target: CodeNavigationTarget; + path?: string; pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntent?: string; + fileIntents?: string[]; + excludeFileIntents?: string[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + includeHidden?: boolean; limit?: number; waitTimeoutMs?: number; } @@ -42,7 +60,36 @@ export interface ListFilesRequestBuildResult { * envelope knows to echo it under `filter.limit`). */ limitExplicit: boolean; - pathPrefixExplicit: boolean; + explicit: { + path: boolean; + pathPrefix: boolean; + globs: boolean; + extensions: boolean; + fileTypes: boolean; + languages: boolean; + fileIntent: boolean; + fileIntents: boolean; + excludeFileIntents: boolean; + excludeDocFiles: boolean; + excludeTestFiles: boolean; + includeHidden: boolean; + limit: boolean; + }; + filterEcho: { + path?: string; + pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntent?: string; + fileIntents?: string[]; + excludeFileIntents?: string[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + includeHidden?: boolean; + limit?: number; + }; } export function buildListFilesParams( @@ -50,25 +97,192 @@ export function buildListFilesParams( ): ListFilesRequestBuildResult { const limitExplicit = input.limit !== undefined; const limit = normaliseLimit(input.limit); - const waitTimeoutMs = normaliseWaitTimeoutMs(input.waitTimeoutMs); - + const path = normalizeOptionalNonEmpty(input.path, "path"); const pathPrefix = normalisePathPrefix(input.pathPrefix); + const globs = normalizeStringList(input.globs, "globs"); + const extensions = normalizeExtensions(input.extensions); + const fileTypes = normalizeStringList(input.fileTypes, "file_types"); + const languages = normalizeStringList(input.languages, "languages"); + const { fileIntent, fileIntentEcho } = normalizeOptionalFileIntent( + input.fileIntent, + "file_intent", + ); + const fileIntents = normalizeFileIntentList( + input.fileIntents, + "file_intents", + ); + const excludeFileIntents = normalizeFileIntentList( + input.excludeFileIntents, + "exclude_file_intents", + ); + + if (fileIntent && fileIntents.length > 0) { + throw new InvalidPackageSpecError( + "`file_intent` cannot be combined with `file_intents`.", + ); + } + + const pathSelectors = buildPathSelectors({ path, globs }); + const pathExplicit = path !== undefined; const pathPrefixExplicit = pathPrefix !== undefined; + const globsExplicit = globs.length > 0; return { params: { target: input.target, + pathSelectors, pathPrefix, + extensions: extensions.length > 0 ? extensions : undefined, + fileTypes: fileTypes.length > 0 ? fileTypes : undefined, + languages: languages.length > 0 ? languages : undefined, + fileIntent, + fileIntents: fileIntents.length > 0 ? fileIntents : undefined, + excludeFileIntents: + excludeFileIntents.length > 0 ? excludeFileIntents : undefined, + excludeDocFiles: input.excludeDocFiles, + excludeTestFiles: input.excludeTestFiles, + includeHidden: input.includeHidden, limit, waitTimeoutMs, }, effectiveLimit: limit, limitExplicit, - pathPrefixExplicit, + explicit: { + path: pathExplicit, + pathPrefix: pathPrefixExplicit, + globs: globsExplicit, + extensions: extensions.length > 0, + fileTypes: fileTypes.length > 0, + languages: languages.length > 0, + fileIntent: fileIntent !== undefined, + fileIntents: fileIntents.length > 0, + excludeFileIntents: excludeFileIntents.length > 0, + excludeDocFiles: input.excludeDocFiles !== undefined, + excludeTestFiles: input.excludeTestFiles !== undefined, + includeHidden: input.includeHidden !== undefined, + limit: limitExplicit, + }, + filterEcho: { + path, + pathPrefix, + globs: globsExplicit ? globs : undefined, + extensions: extensions.length > 0 ? extensions : undefined, + fileTypes: fileTypes.length > 0 ? fileTypes : undefined, + languages: languages.length > 0 ? languages : undefined, + fileIntent: fileIntentEcho, + fileIntents: + fileIntents.length > 0 + ? fileIntents.map((intent) => intent.toLowerCase()) + : undefined, + excludeFileIntents: + excludeFileIntents.length > 0 + ? excludeFileIntents.map((intent) => intent.toLowerCase()) + : undefined, + excludeDocFiles: input.excludeDocFiles, + excludeTestFiles: input.excludeTestFiles, + includeHidden: input.includeHidden, + limit: limitExplicit ? limit : undefined, + }, }; } +function buildPathSelectors(input: { + path?: string; + globs: string[]; +}): GrepRepoPathSelector[] | undefined { + const selectors: GrepRepoPathSelector[] = []; + if (input.path) selectors.push({ kind: "EXACT", value: input.path }); + for (const glob of input.globs) { + selectors.push({ kind: "GLOB", value: glob }); + } + return selectors.length > 0 ? selectors : undefined; +} + +function normalizeOptionalNonEmpty( + raw: string | undefined, + field: string, +): string | undefined { + if (raw === undefined) return undefined; + const trimmed = raw.trim(); + if (trimmed.length === 0) { + throw new InvalidPackageSpecError( + `\`${field}\` cannot be empty when provided.`, + ); + } + return trimmed; +} + +function normalizeStringList( + raw: string[] | undefined, + field: string, +): string[] { + if (!raw) return []; + const values: string[] = []; + for (const entry of raw) { + const trimmed = entry.trim(); + if (trimmed.length === 0) { + throw new InvalidPackageSpecError( + `\`${field}\` entries cannot be empty.`, + ); + } + values.push(trimmed); + } + return values; +} + +function normalizeExtensions(raw: string[] | undefined): string[] { + const values = normalizeStringList(raw, "extensions"); + for (const value of values) { + if (value.startsWith(".")) { + throw new InvalidPackageSpecError( + "`extensions` values must not include a leading dot.", + ); + } + } + return values; +} + +function normalizeOptionalFileIntent( + raw: string | undefined, + field: string, +): { fileIntent?: FileIntent; fileIntentEcho?: string } { + if (raw === undefined) return {}; + const trimmed = raw.trim().toLowerCase(); + if (trimmed.length === 0) { + throw new InvalidPackageSpecError( + `\`${field}\` cannot be empty when provided.`, + ); + } + if (!isKnownFileIntent(trimmed)) { + throw new InvalidPackageSpecError( + `\`${field}\` must be one of: ${knownFileIntentList().join(", ")}. Got ${raw}.`, + ); + } + return { + fileIntent: toFileIntent(trimmed), + fileIntentEcho: trimmed, + }; +} + +function normalizeFileIntentList( + raw: string[] | undefined, + field: string, +): FileIntent[] { + const values = normalizeStringList(raw, field); + const intents: FileIntent[] = []; + for (const value of values) { + const lower = value.toLowerCase(); + if (!isKnownFileIntent(lower)) { + throw new InvalidPackageSpecError( + `\`${field}\` values must be one of: ${knownFileIntentList().join(", ")}. Got ${value}.`, + ); + } + intents.push(toFileIntent(lower) as FileIntent); + } + return intents; +} + function normaliseLimit(raw: number | undefined): number { if (raw === undefined) return LIMIT_DEFAULT; if (!Number.isInteger(raw) || raw < LIMIT_MIN || raw > LIMIT_MAX) { diff --git a/src/shared/list-files-response.test.ts b/src/shared/list-files-response.test.ts index 23af52d5..fe49254f 100644 --- a/src/shared/list-files-response.test.ts +++ b/src/shared/list-files-response.test.ts @@ -37,8 +37,21 @@ const baseResult: ListFilesResult = { const baseOptions = { registry: "npm", name: "express", - limitExplicit: false, - pathPrefixExplicit: false, + explicit: { + path: false, + pathPrefix: false, + globs: false, + extensions: false, + fileTypes: false, + languages: false, + fileIntent: false, + fileIntents: false, + excludeFileIntents: false, + excludeDocFiles: false, + excludeTestFiles: false, + includeHidden: false, + limit: false, + }, }; describe("buildListFilesSuccessPayload", () => { @@ -73,7 +86,7 @@ describe("buildListFilesSuccessPayload", () => { const withFilter = buildListFilesSuccessPayload(baseResult, { ...baseOptions, limit: 100, - limitExplicit: true, + explicit: { ...baseOptions.explicit, limit: true }, }); expect(withFilter.filter).toEqual({ limit: 100 }); }); @@ -82,17 +95,54 @@ describe("buildListFilesSuccessPayload", () => { const envelope = buildListFilesSuccessPayload(baseResult, { ...baseOptions, pathPrefix: "src/", - pathPrefixExplicit: true, + explicit: { ...baseOptions.explicit, pathPrefix: true }, }); expect(envelope.filter).toEqual({ pathPrefix: "src/" }); }); + it("echoes advanced filters only when explicit", () => { + const envelope = buildListFilesSuccessPayload(baseResult, { + ...baseOptions, + path: "README.md", + globs: ["test/**/*.js"], + extensions: ["js"], + fileTypes: ["source"], + languages: ["JavaScript"], + fileIntents: ["production", "test"], + excludeFileIntents: ["generated"], + excludeDocFiles: true, + includeHidden: true, + explicit: { + ...baseOptions.explicit, + path: true, + globs: true, + extensions: true, + fileTypes: true, + languages: true, + fileIntents: true, + excludeFileIntents: true, + excludeDocFiles: true, + includeHidden: true, + }, + }); + expect(envelope.filter).toEqual({ + path: "README.md", + globs: ["test/**/*.js"], + extensions: ["js"], + fileTypes: ["source"], + languages: ["JavaScript"], + fileIntents: ["production", "test"], + excludeFileIntents: ["generated"], + excludeDocFiles: true, + includeHidden: true, + }); + }); + it("emits repoUrl + gitRef for repo-URL addressing", () => { const envelope = buildListFilesSuccessPayload(baseResult, { - limitExplicit: false, - pathPrefixExplicit: false, repoUrl: "https://github.com/expressjs/express", gitRef: "main", + explicit: baseOptions.explicit, }); expect(envelope.registry).toBeUndefined(); expect(envelope.name).toBeUndefined(); @@ -183,7 +233,11 @@ describe("formatListFilesTerminal", () => { it("plain mode hasMore: stdout stays clean; warning goes to stderr", () => { const envelope = buildListFilesSuccessPayload( { ...baseResult, total: 200, hasMore: true }, - { ...baseOptions, limit: 200, limitExplicit: true }, + { + ...baseOptions, + limit: 200, + explicit: { ...baseOptions.explicit, limit: true }, + }, ); const { stdout, stderr } = formatListFilesTerminal(envelope, { useColors: false, @@ -199,7 +253,11 @@ describe("formatListFilesTerminal", () => { it("verbose mode hasMore: truncation warning included inline", () => { const envelope = buildListFilesSuccessPayload( { ...baseResult, total: 200, hasMore: true }, - { ...baseOptions, limit: 200, limitExplicit: true }, + { + ...baseOptions, + limit: 200, + explicit: { ...baseOptions.explicit, limit: true }, + }, ); const { stdout } = formatListFilesTerminal(envelope, { verbose: true, diff --git a/src/shared/list-files-response.ts b/src/shared/list-files-response.ts index 6d84938d..7db4ce37 100644 --- a/src/shared/list-files-response.ts +++ b/src/shared/list-files-response.ts @@ -13,7 +13,7 @@ * before the envelope builder runs, so agents never branch on a * data-path indexing flag. * - **`filter.*` echoes only caller-supplied inputs.** The default - * limit (200) is not echoed; an explicit limit is. + * limit (200) is not echoed; explicit selectors / filters are. */ import type { ListFilesResult, RepoFileEntry } from "../services/index.js"; @@ -35,7 +35,18 @@ export interface LeanListFilesResolution { } export interface LeanListFilesFilter { + path?: string; pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntent?: string; + fileIntents?: string[]; + excludeFileIntents?: string[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + includeHidden?: boolean; limit?: number; } @@ -67,13 +78,35 @@ export interface BuildListFilesPayloadOptions { name?: string; repoUrl?: string; gitRef?: string; - /** Whether the caller supplied an explicit `limit`. */ - limitExplicit: boolean; - /** Whether the caller supplied an explicit `path_prefix`. */ - pathPrefixExplicit: boolean; /** Caller's raw inputs, echoed under `filter.*` when explicit. */ + path?: string; pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntent?: string; + fileIntents?: string[]; + excludeFileIntents?: string[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + includeHidden?: boolean; limit?: number; + explicit: { + path: boolean; + pathPrefix: boolean; + globs: boolean; + extensions: boolean; + fileTypes: boolean; + languages: boolean; + fileIntent: boolean; + fileIntents: boolean; + excludeFileIntents: boolean; + excludeDocFiles: boolean; + excludeTestFiles: boolean; + includeHidden: boolean; + limit: boolean; + }; } export function buildListFilesSuccessPayload( @@ -131,10 +164,63 @@ function buildFilterBlock( options: BuildListFilesPayloadOptions, ): LeanListFilesFilter | undefined { const filter: LeanListFilesFilter = {}; - if (options.pathPrefixExplicit && options.pathPrefix) { + if (options.explicit.path && options.path) { + filter.path = options.path; + } + if (options.explicit.pathPrefix && options.pathPrefix) { filter.pathPrefix = options.pathPrefix; } - if (options.limitExplicit && options.limit !== undefined) { + if (options.explicit.globs && options.globs && options.globs.length > 0) { + filter.globs = options.globs; + } + if ( + options.explicit.extensions && + options.extensions && + options.extensions.length > 0 + ) { + filter.extensions = options.extensions; + } + if ( + options.explicit.fileTypes && + options.fileTypes && + options.fileTypes.length > 0 + ) { + filter.fileTypes = options.fileTypes; + } + if ( + options.explicit.languages && + options.languages && + options.languages.length > 0 + ) { + filter.languages = options.languages; + } + if (options.explicit.fileIntent && options.fileIntent) { + filter.fileIntent = options.fileIntent; + } + if ( + options.explicit.fileIntents && + options.fileIntents && + options.fileIntents.length > 0 + ) { + filter.fileIntents = options.fileIntents; + } + if ( + options.explicit.excludeFileIntents && + options.excludeFileIntents && + options.excludeFileIntents.length > 0 + ) { + filter.excludeFileIntents = options.excludeFileIntents; + } + if (options.explicit.excludeDocFiles) { + filter.excludeDocFiles = options.excludeDocFiles; + } + if (options.explicit.excludeTestFiles) { + filter.excludeTestFiles = options.excludeTestFiles; + } + if (options.explicit.includeHidden) { + filter.includeHidden = options.includeHidden; + } + if (options.explicit.limit && options.limit !== undefined) { filter.limit = options.limit; } return Object.keys(filter).length > 0 ? filter : undefined; diff --git a/src/shared/list-files-text.test.ts b/src/shared/list-files-text.test.ts index 70c9b4ad..655380ab 100644 --- a/src/shared/list-files-text.test.ts +++ b/src/shared/list-files-text.test.ts @@ -53,9 +53,24 @@ describe("renderListFilesText", () => { it("echoes explicit filter inputs in the header", () => { const text = renderListFilesText( - envelope({ filter: { pathPrefix: "src/lib", limit: 50 } }), + envelope({ + filter: { + path: "README.md", + pathPrefix: "src/lib", + globs: ["test/**/*.js"], + extensions: ["js"], + fileTypes: ["source"], + languages: ["JavaScript"], + fileIntent: "production", + excludeDocFiles: true, + includeHidden: true, + limit: 50, + }, + }), + ); + expect(text).toContain( + 'path="README.md" path_prefix="src/lib" globs=test/**/*.js exts=js file_types=source languages=JavaScript file_intent=production exclude_doc_files=true include_hidden=true limit=50', ); - expect(text).toContain('path_prefix="src/lib" limit=50'); }); it("renders the empty-result hint when no files match", () => { @@ -74,7 +89,11 @@ describe("renderListFilesText", () => { const text = renderListFilesText( envelope({ hasMore: true, - filter: { pathPrefix: "src/", limit: 50 }, + filter: { + pathPrefix: "src/", + extensions: ["ts"], + limit: 50, + }, }), ); expect(text).not.toMatch(/[·…—–]/); diff --git a/src/shared/list-files-text.ts b/src/shared/list-files-text.ts index 450a2436..973577cc 100644 --- a/src/shared/list-files-text.ts +++ b/src/shared/list-files-text.ts @@ -30,9 +30,7 @@ export function renderListFilesText(envelope: LeanListFilesEnvelope): string { if (envelope.hasMore) { lines.push(""); - lines.push( - "More files available. Pass limit=N to widen or refine path_prefix.", - ); + lines.push("More files available. Pass limit=N or refine the filter."); } if (envelope.hint) { @@ -74,9 +72,46 @@ function buildIdentity(envelope: LeanListFilesEnvelope): string { function buildFilterEcho(envelope: LeanListFilesEnvelope): string { const parts: string[] = []; + if (envelope.filter?.path) { + parts.push(`path=${quote(envelope.filter.path)}`); + } if (envelope.filter?.pathPrefix) { parts.push(`path_prefix=${quote(envelope.filter.pathPrefix)}`); } + if (envelope.filter?.globs?.length) { + parts.push(`globs=${envelope.filter.globs.join(",")}`); + } + if (envelope.filter?.extensions?.length) { + parts.push(`exts=${envelope.filter.extensions.join(",")}`); + } + if (envelope.filter?.fileTypes?.length) { + parts.push(`file_types=${envelope.filter.fileTypes.join(",")}`); + } + if (envelope.filter?.languages?.length) { + parts.push(`languages=${envelope.filter.languages.join(",")}`); + } + if (envelope.filter?.fileIntent) { + parts.push(`file_intent=${envelope.filter.fileIntent}`); + } + if (envelope.filter?.fileIntents?.length) { + parts.push(`file_intents=${envelope.filter.fileIntents.join(",")}`); + } + if (envelope.filter?.excludeFileIntents?.length) { + parts.push( + `exclude_file_intents=${envelope.filter.excludeFileIntents.join(",")}`, + ); + } + if (envelope.filter?.excludeDocFiles !== undefined) { + parts.push(`exclude_doc_files=${String(envelope.filter.excludeDocFiles)}`); + } + if (envelope.filter?.excludeTestFiles !== undefined) { + parts.push( + `exclude_test_files=${String(envelope.filter.excludeTestFiles)}`, + ); + } + if (envelope.filter?.includeHidden !== undefined) { + parts.push(`include_hidden=${String(envelope.filter.includeHidden)}`); + } if (envelope.filter?.limit !== undefined) { parts.push(`limit=${envelope.filter.limit}`); } diff --git a/src/tools/list-files-parity.test.ts b/src/tools/list-files-parity.test.ts index 21d07896..c72bb87e 100644 --- a/src/tools/list-files-parity.test.ts +++ b/src/tools/list-files-parity.test.ts @@ -83,7 +83,18 @@ interface McpArgs { repo_url?: string; git_ref?: string; }; + path?: string; path_prefix?: string; + globs?: string[]; + extensions?: string[]; + file_types?: string[]; + languages?: string[]; + file_intent?: string; + file_intents?: string[]; + exclude_file_intents?: string[]; + exclude_doc_files?: boolean; + exclude_test_files?: boolean; + include_hidden?: boolean; limit?: number; wait_timeout_ms?: number; format?: "json" | "text" | "text-v1"; @@ -204,6 +215,49 @@ describe("list_files parity", () => { expect((cli as { filter?: { limit?: number } }).filter?.limit).toBe(50); }); + it("PARITY-JSON-KEYS: advanced filters echo identically on both surfaces", async () => { + const fn = mock(() => Promise.resolve(defaultListFilesResult)); + const cli = await cliJson( + "npm:express", + "src/", + { + path: "README.md", + glob: ["test/**/*.js"], + ext: ["js"], + fileType: ["source"], + language: ["JavaScript"], + fileIntent: ["production", "test"], + excludeIntent: ["generated"], + excludeDocs: true, + excludeTests: false, + hidden: true, + }, + cliDeps({ + codeNavigationService: createMockCodeNavigationService({ + listFiles: fn as never, + }), + }), + ); + const mcp = await mcpJson( + { + target: { registry: "npm", package_name: "express" }, + path: "README.md", + path_prefix: "src/", + globs: ["test/**/*.js"], + extensions: ["js"], + file_types: ["source"], + languages: ["JavaScript"], + file_intents: ["production", "test"], + exclude_file_intents: ["generated"], + exclude_doc_files: true, + exclude_test_files: false, + include_hidden: true, + }, + fn as never, + ); + expect(cli).toEqual(mcp); + }); + it("PARITY-ERROR-ENVELOPE: INDEXING identical on both surfaces", async () => { const fn = mock(() => Promise.reject( diff --git a/src/tools/list-files.test.ts b/src/tools/list-files.test.ts index f5db2ab4..d18cec5d 100644 --- a/src/tools/list-files.test.ts +++ b/src/tools/list-files.test.ts @@ -19,8 +19,19 @@ describe("createListFilesTool — metadata", () => { expect(tool.name).toBe("code_files"); expect(tool.description).toContain("List files in an indexed dependency"); expect(Object.keys(tool.schema).sort()).toEqual([ + "exclude_doc_files", + "exclude_file_intents", + "exclude_test_files", + "extensions", + "file_intent", + "file_intents", + "file_types", "format", + "globs", + "include_hidden", + "languages", "limit", + "path", "path_prefix", "target", "wait_timeout_ms", @@ -49,6 +60,52 @@ describe("createListFilesTool — happy path", () => { expect(calls[0]?.[0]?.target?.packageName).toBe("express"); }); + it("forwards advanced list-files filters to the service", async () => { + const listFiles = mock(() => Promise.resolve(defaultListFilesResult)); + const service = createMockCodeNavigationService({ listFiles }); + const tool = createListFilesTool(service); + + await tool.handler( + { + target: { registry: "npm", package_name: "express" }, + path: "README.md", + path_prefix: "src/", + globs: ["test/**/*.js"], + extensions: ["js"], + file_types: ["source"], + languages: ["JavaScript"], + file_intents: ["production", "test"], + exclude_file_intents: ["generated"], + exclude_doc_files: true, + exclude_test_files: false, + include_hidden: true, + }, + {}, + ); + + const calls = listFiles.mock.calls as unknown as Array< + [ + { + pathSelectors?: Array<{ kind: string; value: string }>; + pathPrefix?: string; + fileIntents?: string[]; + excludeFileIntents?: string[]; + includeHidden?: boolean; + }, + ] + >; + expect(calls[0]?.[0]).toMatchObject({ + pathSelectors: [ + { kind: "EXACT", value: "README.md" }, + { kind: "GLOB", value: "test/**/*.js" }, + ], + pathPrefix: "src/", + fileIntents: ["PRODUCTION", "TEST"], + excludeFileIntents: ["GENERATED"], + includeHidden: true, + }); + }); + it("emits the envelope with files, total, hasMore, resolution, indexedVersion", async () => { const tool = createListFilesTool(createMockCodeNavigationService()); const result = await tool.handler( @@ -114,6 +171,50 @@ describe("createListFilesTool — happy path", () => { expect(payload.filter?.pathPrefix).toBe("src/"); }); + it("echoes advanced filters when caller set them", async () => { + const tool = createListFilesTool(createMockCodeNavigationService()); + const result = await tool.handler( + { + target: { registry: "npm", package_name: "express" }, + path: "README.md", + globs: ["test/**/*.js"], + extensions: ["js"], + file_types: ["source"], + languages: ["JavaScript"], + file_intent: "production", + exclude_file_intents: ["generated"], + exclude_doc_files: true, + include_hidden: true, + format: "json", + }, + {}, + ); + const payload = parseText(result) as { + filter?: { + path?: string; + globs?: string[]; + extensions?: string[]; + fileTypes?: string[]; + languages?: string[]; + fileIntent?: string; + excludeFileIntents?: string[]; + excludeDocFiles?: boolean; + includeHidden?: boolean; + }; + }; + expect(payload.filter).toEqual({ + path: "README.md", + globs: ["test/**/*.js"], + extensions: ["js"], + fileTypes: ["source"], + languages: ["JavaScript"], + fileIntent: "production", + excludeFileIntents: ["generated"], + excludeDocFiles: true, + includeHidden: true, + }); + }); + it("omits filter when caller only used defaults", async () => { const tool = createListFilesTool(createMockCodeNavigationService()); const result = await tool.handler( diff --git a/src/tools/list-files.ts b/src/tools/list-files.ts index 722bed71..6d7fb3fb 100644 --- a/src/tools/list-files.ts +++ b/src/tools/list-files.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import type { CodeNavigationService } from "../services/index.js"; +import { knownFileIntentList } from "../shared/code-navigation.js"; import { mapCodeNavigationError } from "../shared/code-navigation-error-map.js"; import { buildListFilesParams } from "../shared/list-files-request.js"; import { buildListFilesSuccessPayload } from "../shared/list-files-response.js"; @@ -14,7 +15,18 @@ import { errorResult, type ToolDefinition, textResult } from "./types.js"; export interface ListFilesArgs { target: CodeTargetArg; + path?: string; path_prefix?: string; + globs?: string[]; + extensions?: string[]; + file_types?: string[]; + languages?: string[]; + file_intent?: string; + file_intents?: string[]; + exclude_file_intents?: string[]; + exclude_doc_files?: boolean; + exclude_test_files?: boolean; + include_hidden?: boolean; limit?: number; wait_timeout_ms?: number; format?: "json" | "text" | "text-v1"; @@ -22,12 +34,59 @@ export interface ListFilesArgs { const schema = { target: codeTargetSchema, + path: z + .string() + .optional() + .describe( + "Exact target-relative file path to include. When combined with `path_prefix` or `globs`, files matching any selector are returned.", + ), path_prefix: z .string() .optional() .describe( - "Literal directory prefix to filter by (e.g. `src/` or `lib/parser`). NOT a glob — `*.ts` and similar patterns won't match. Omit to list from the repository root.", + "Literal directory prefix to filter by (e.g. `src/` or `lib/parser`). NOT a glob. OR-ed with `path` and `globs` when combined.", + ), + globs: z + .array(z.string()) + .optional() + .describe( + "Repeatable glob selectors with real glob semantics (e.g. `src/**/*.ts`). OR-ed with `path` and `path_prefix`.", + ), + extensions: z + .array(z.string()) + .optional() + .describe("File extensions to include, without a leading dot."), + file_types: z + .array(z.string()) + .optional() + .describe( + "File type filters to include, matching aigrep file_type values such as `source` or `doc`.", + ), + languages: z + .array(z.string()) + .optional() + .describe("Language filters to include, matching aigrep language names."), + file_intent: z + .string() + .optional() + .describe( + `Single inclusive file-intent filter. Cannot be combined with \`file_intents\`. Valid values: ${knownFileIntentList().join(", ")}.`, + ), + file_intents: z + .array(z.string()) + .optional() + .describe( + `Inclusive file-intent filters. Cannot be combined with \`file_intent\`. Valid values: ${knownFileIntentList().join(", ")}.`, + ), + exclude_file_intents: z + .array(z.string()) + .optional() + .describe( + `Exclude these file intents after inclusive intent filtering. Valid values: ${knownFileIntentList().join(", ")}.`, ), + exclude_doc_files: z.boolean().optional(), + exclude_test_files: z.boolean().optional(), + include_hidden: z.boolean().optional(), limit: z .number() .optional() @@ -55,9 +114,9 @@ const DESCRIPTION = "language, fileType, byteSize}], resolution, indexedVersion}`. " + "Address via `target.registry` + `target.package_name` (package " + "scope) or `target.repo_url` + `target.git_ref` (repo scope), " + - "mutually exclusive. `path_prefix` is a literal directory prefix — " + - "it does NOT accept globs (`*.ts`) or extension filters. The " + - "returned paths feed directly into `code_read` and help scope " + + "mutually exclusive. Narrow with `path`, `path_prefix`, `globs`, " + + "`extensions`, `file_types`, `languages`, or file-intent filters. " + + "The returned paths feed directly into `code_read` and help scope " + "`code_grep`. Returns an `INDEXING` error envelope when the " + "dependency is being indexed on-demand — retry with a longer " + "`wait_timeout_ms` or use a version from `details.availableVersions`."; @@ -77,7 +136,18 @@ export function createListFilesTool( try { const build = buildListFilesParams({ target, + path: args.path, pathPrefix: args.path_prefix, + globs: args.globs, + extensions: args.extensions, + fileTypes: args.file_types, + languages: args.languages, + fileIntent: args.file_intent, + fileIntents: args.file_intents, + excludeFileIntents: args.exclude_file_intents, + excludeDocFiles: args.exclude_doc_files, + excludeTestFiles: args.exclude_test_files, + includeHidden: args.include_hidden, limit: args.limit, waitTimeoutMs: args.wait_timeout_ms, }); @@ -89,10 +159,20 @@ export function createListFilesTool( name: target.packageName, repoUrl: target.repoUrl, gitRef: target.gitRef, - limitExplicit: build.limitExplicit, - pathPrefixExplicit: build.pathPrefixExplicit, - pathPrefix: build.params.pathPrefix, - limit: build.params.limit, + path: build.filterEcho.path, + pathPrefix: build.filterEcho.pathPrefix, + globs: build.filterEcho.globs, + extensions: build.filterEcho.extensions, + fileTypes: build.filterEcho.fileTypes, + languages: build.filterEcho.languages, + fileIntent: build.filterEcho.fileIntent, + fileIntents: build.filterEcho.fileIntents, + excludeFileIntents: build.filterEcho.excludeFileIntents, + excludeDocFiles: build.filterEcho.excludeDocFiles, + excludeTestFiles: build.filterEcho.excludeTestFiles, + includeHidden: build.filterEcho.includeHidden, + limit: build.filterEcho.limit, + explicit: build.explicit, }); if (isTextFormat(args.format)) { return textResult(renderListFilesText(payload));