diff --git a/README.md b/README.md index a65e68ed..6d40f795 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ When package/source access is enabled for the current token, GitHits also expose | `search_symbols` | Exact-token search inside indexed dependency source | | `list_files` | Discover what files a dependency or repo contains | | `read_file` | Read a dependency file by path | -| `grep_file` | Search for a case-insensitive substring within one file | +| `grep_repo` | Deterministic text grep across indexed dependency or repo files | These advanced tools remain feature-gated. The MCP server advertises them only when the authenticated token is entitled to package/source access. diff --git a/docs/implementation/cli-commands.md b/docs/implementation/cli-commands.md index 552e923c..dd7d0ee8 100644 --- a/docs/implementation/cli-commands.md +++ b/docs/implementation/cli-commands.md @@ -21,7 +21,7 @@ The CLI exposes four primary top-level commands: `example`, `languages`, and `fe | `pkg changelog [spec]` | package spec OR `--repo-url` | `--from`, `--to`, `--limit`, `--git-ref`, `--no-body`, `--verbose`, `--json` | Release notes / changelog entries for a package or GitHub repo (GitHub Releases, CHANGELOG.md, or HexDocs). Default shows each entry with a 10-line body preview; `--verbose` uncaps, `--no-body` drops. | | `code files [spec] [path-prefix]` | package spec OR `--repo-url` + `--git-ref`; optional `[path-prefix]` | `--limit`, `--wait`, `--verbose`, `--json` | List files in an indexed dependency. `[path-prefix]` is a literal directory prefix (not a glob). Plain output is one path per line; `--verbose` adds language / type / size annotations. Indexing-retry via `--wait` or the `availableVersions` hint in the error envelope. | | `code read ` | package spec OR `--repo-url` + `--git-ref`; plus `` | `--lines`, `--start`, `--end`, `--wait`, `--verbose`, `--json` | Read a file's contents. Plain output is the raw file bytes (pipe-friendly); `--verbose` adds a header and a line-number gutter. `--lines 10-40` concise form; `--start`/`--end` equivalent. Binary files show a sentinel line. | -| `code grep ` | package spec OR `--repo-url` + `--git-ref`; plus `` and `` | `--context`, `--limit`, `--wait`, `--verbose`, `--json` | Search within a single file for a case-insensitive substring (not regex). Plain output is matching lines only (pipe-friendly, `grep`-style); `--verbose` adds a header and a line-number gutter with `>` markers on match lines. `--context ` adds surrounding lines (default 0, up to 10); overlapping blocks merge without duplicates. Max 200-char pattern; up to 200 matches. | +| `code grep [spec] [path-prefix]` | package spec OR `--repo-url` + `--git-ref`; plus `` and optional `[path-prefix]` | `--path`, repeatable `--glob`, repeatable `--ext`, `--regex`, `--case-sensitive`, `-C/-A/-B`, `--exclude-docs`, `--exclude-tests`, `--limit`, `--per-file-limit`, `--cursor`, `--wait`, `--verbose`, `--json` | Deterministic text grep over indexed dependency or repository source. Defaults to whole-target, literal, case-insensitive matching; narrow with `[path-prefix]`, `--path`, `--glob`, or `--ext`. Plain output is `file:line:text`; `--verbose` groups matches by file. | ### `githits init` @@ -315,20 +315,20 @@ Reads a file from an indexed dependency. `` is package-relative in spec mo ### `githits code grep` ``` -githits code grep npm:express express lib/express.js -githits code grep npm:express express lib/express.js --context 2 # merged blocks -githits code grep npm:express express lib/express.js --verbose # + header + gutter + `>` marker -githits code grep --repo-url https://github.com/expressjs/express --git-ref main export lib/express.js -githits code grep npm:express express lib/express.js --json +githits code grep npm:express middleware +githits code grep npm:express middleware src/ -C 2 +githits code grep npm:express "router\\.(use|get)" --regex --glob 'lib/**/*.js' +githits code grep --repo-url https://github.com/expressjs/express --git-ref main export lib/ +githits code grep npm:express middleware --path lib/express.js --json ``` -Case-insensitive **substring** search inside a single file — not regex. Max pattern 200 chars; up to 200 matches with up to 10 context lines each. For symbol-shaped searches use `githits code search` (backed by `search_symbols`). +Deterministic text grep over indexed dependency or repository source. Defaults to case-insensitive literal matching across the whole target. Pass `[path-prefix]`, `--path`, `--glob`, or `--ext` to narrow scope. `--regex` switches to RE2 regex mode. Max pattern 200 chars. For discovery and ranking, use top-level `githits search` instead. -**Plain output (default).** Matching lines only, one per line on stdout — mirrors `grep`'s default. `--context ` (0–10, default **0**) adds surrounding lines; nearby matches whose contexts touch or overlap merge into a single block with no duplicated lines. Distinct blocks are separated by `--` on its own line, matching `grep -C` / `rg -C` convention. +**Plain output (default).** One `file:line:text` record per match on stdout, pipe-friendly and deterministic. `-C/--context`, `-A/--after-context`, and `-B/--before-context` add surrounding lines. Distinct match groups are separated by `--`. -**`--verbose`.** Adds header, right-aligned line-number gutter, and a `>` marker on match lines so they're distinguishable from context at a glance. +**`--verbose`.** Adds a summary header and grouped file sections with a `>` marker on match lines. -**`stdout` vs `stderr` routing (plain mode).** "More matches available" truncation warning goes to **stderr**. When a zero-match pattern looks like a regex attempt (`\bfoo\b`, `^start`, character classes, etc.) a one-line nudge — "Note: pattern matched literally — this tool does case-insensitive substring search, not regex." — also goes to stderr. Pipes stay clean; humans still see the hints. +**`stdout` vs `stderr` routing (plain mode).** The pagination hint for `nextCursor` goes to **stderr** so stdout stays machine-friendly. **Exit codes (grep-compatible).** @@ -336,9 +336,9 @@ Case-insensitive **substring** search inside a single file — not regex. Max pa - `1` — zero matches. Fires in both plain and `--json` modes so scripting (`if code grep X file; then …`) behaves consistently across surfaces. - `2` — error (missing file, indexing, invalid arguments, backend failure). Distinguished from "no match" so scripts can branch correctly. -This is the standard `grep(1)` contract; the tool adopts it deliberately because its output shape mirrors grep's. +This is still the standard `grep(1)` contract even though the output includes file paths by default. -**Regex pattern note.** The `GREP_PATTERN_SEMANTICS_NOTE` string ("Case-insensitive substring matching. NOT regex — …") is shared verbatim across the CLI help text, the MCP tool description, and the MCP `pattern` argument's `describe` so the three surfaces never disagree about pattern semantics. +**Pattern note.** The `GREP_REPO_PATTERN_NOTE` string is shared verbatim across the CLI help text, the MCP tool description, and the MCP `pattern` argument's `describe` so the three surfaces never disagree about literal-vs-regex semantics. ## Architecture diff --git a/docs/implementation/mcp-cli-parity.md b/docs/implementation/mcp-cli-parity.md index b116d70a..6748f9d1 100644 --- a/docs/implementation/mcp-cli-parity.md +++ b/docs/implementation/mcp-cli-parity.md @@ -174,11 +174,11 @@ When a new tool lands with both MCP and CLI surfaces: | `src/shared/list-files-response.ts` | JSON envelope builder for `list_files` (shared); terminal formatter (CLI-only). Resolves the `hasMore` → `N+` header behaviour. | | `src/shared/read-file-request.ts` | Shared request builder for `read_file`; trims filePath, validates start/end line positive-integer rules, rejects reversed ranges. | | `src/shared/read-file-response.ts` | JSON envelope builder for `read_file` (shared); terminal formatter (CLI-only). Normalises the envelope key to `path` (not `filePath`) so `list_files` → `read_file` chains without renames. | -| `src/shared/grep-file-request.ts` | Shared request builder for `grep_file`; exports `GREP_PATTERN_SEMANTICS_NOTE` referenced by MCP description, MCP `pattern` describe, and CLI help. Also exports `looksLikeRegexAttempt` heuristic. | -| `src/shared/grep-file-response.ts` | JSON envelope builder for `grep_file` (shared); terminal formatter (CLI-only) owns the regex-char empty-result nudge. | +| `src/shared/grep-repo-request.ts` | Shared request builder for `grep_repo`; exports `GREP_REPO_PATTERN_NOTE` referenced by MCP description, MCP `pattern` describe, and CLI help. Compiles public scope inputs into backend `pathSelectors` and applies internal `allowUnscoped` when no scope filters are given. | +| `src/shared/grep-repo-response.ts` | JSON envelope builder for `grep_repo` (shared); terminal formatter (CLI-only) renders plain `file:line:text` or verbose grouped output and surfaces pagination via stderr. | | `src/shared/code-navigation-error-map.ts` | `mapCodeNavigationError` classifier. Owns the `INDEXING` / `FILE_NOT_FOUND` / `NOT_FOUND` codes shared across all four code-nav tools. | | `src/shared/code-navigation-defaults.ts` | `DEFAULT_WAIT_TIMEOUT_MS = 20_000` + `MAX_WAIT_TIMEOUT_MS = 60_000`. Both CLI and MCP request builders import these so defaults never diverge. | -| `src/tools/code-navigation-shared.ts` | `codeTargetSchema` + `resolveCodeTarget` — the single addressing primitive used by `search_symbols`, `list_files`, `read_file`, `grep_file`. | +| `src/tools/code-navigation-shared.ts` | `codeTargetSchema` + `resolveCodeTarget` — the single addressing primitive used by `search_symbols`, `list_files`, `read_file`, `grep_repo`. | | `src/shared/package-intelligence-error-map.ts` | `mapPackageIntelligenceError` classifier (reuses `MappedError` from the code-nav map). | | `src/services/promote-version-not-found.ts` | Shared helper that promotes generic backend errors with "no matching version" messages into typed `VERSION_NOT_FOUND`. Used by the `packageVulnerabilities`, `packageDependencies`, and `packageChangelog` executors. Handles both `version` (single-version queries) and `fromVersion` / `toVersion` (range queries), and skips `details.package` synthesis when registry/name aren't available (repo-URL mode). | | `src/tools/search-symbols.ts` | MCP tool definition for `search_symbols`. | @@ -190,7 +190,7 @@ When a new tool lands with both MCP and CLI surfaces: | `src/tools/package-changelog.ts` | MCP tool definition for `package_changelog`. | | `src/tools/list-files.ts` | MCP tool definition for `list_files`. | | `src/tools/read-file.ts` | MCP tool definition for `read_file`. | -| `src/tools/grep-file.ts` | MCP tool definition for `grep_file`. | +| `src/tools/grep-repo.ts` | MCP tool definition for `grep_repo`. | | `src/commands/code/search-symbols.ts` | CLI command. | | `src/commands/search.ts` | Top-level CLI commands for unified `search` and `search-status`. | | `src/commands/pkg/info.ts` | CLI command for `pkg info`. | @@ -207,7 +207,7 @@ When a new tool lands with both MCP and CLI surfaces: | `src/tools/package-changelog-parity.test.ts` | Parity tests for `package_changelog` (cite rule IDs). | | `src/tools/list-files-parity.test.ts` | Parity tests for `list_files` (cite rule IDs). | | `src/tools/read-file-parity.test.ts` | Parity tests for `read_file` (cite rule IDs). | -| `src/tools/grep-file-parity.test.ts` | Parity tests for `grep_file` (cite rule IDs). | +| `src/tools/grep-repo-parity.test.ts` | Parity tests for `grep_repo` (cite rule IDs). | ## Per-tool notes @@ -408,7 +408,7 @@ When a new tool lands with both MCP and CLI surfaces: - `toMatchObject` for builder-sourced `INVALID_ARGUMENT` cases: `@` rejection, `--from` + `--limit` mutex. -### `list_files` / `read_file` / `grep_file` (file-exploration bundle) +### `list_files` / `read_file` / `grep_repo` (file-exploration bundle) All three reuse `codeTargetSchema` + `resolveCodeTarget` from `src/tools/code-navigation-shared.ts`. The indexing lifecycle is @@ -431,17 +431,15 @@ so envelope-drift surfaces in the test rather than at an agent. on the backend doesn't return `availableVersions` on INDEXING responses, so its `details` block carries only `indexingRef` — MCP description calls this out explicitly. -- **`grep_file`**: `GREP_PATTERN_SEMANTICS_NOTE` constant - (exported from `grep-file-request.ts`) ensures the - substring-only disclosure is identical in the MCP +- **`grep_repo`**: `GREP_REPO_PATTERN_NOTE` constant + (exported from `grep-repo-request.ts`) ensures the + literal-vs-regex disclosure is identical in the MCP description, MCP `pattern` field describe, and CLI help text. - Regex-char heuristic in the terminal formatter nudges users - who typed clearly-regex patterns; the JSON envelope never - carries this hint. Triggered signals cover `\b\B\w\W\d\D\s\S`, - escaped metacharacters, character classes, non-capturing / - lookaround / named groups / inline flags, and brace - quantifiers. Deliberately excludes bare `.`, `*`, `+`, `?`, - `^`, `$`, `|`, `(`, `)` — too common in ordinary code. + The shared request builder compiles `path`, `path_prefix`, and + `globs` into backend `pathSelectors`, keeps `allowUnscoped` + internal-only, and defaults grep to whole-target, literal, + case-insensitive matching. The shared response builder keeps CLI + `--json` and MCP payloads byte-identical for equivalent inputs. - **Parity assertion policy** (coded in the three parity tests): diff --git a/docs/implementation/tools.md b/docs/implementation/tools.md index 56279a00..eedfe9ed 100644 --- a/docs/implementation/tools.md +++ b/docs/implementation/tools.md @@ -31,9 +31,9 @@ Both expose the same tools with identical names, parameters, and descriptions. T | `package_changelog` | `registry?`, `package_name?`, `repo_url?`, `from_version?`, `to_version?`, `limit?`, `git_ref?`, `include_bodies?` | Release notes or changelog entries for a package or GitHub repo. Default latest mode returns the ten most recent entries; `from_version` switches to range mode (no count cap). Dual addressing (spec vs repo URL) mutually exclusive. Response always includes `source` (`"releases"` / `"changelog_file"` / `"hexdocs"`), `mode` (`"latest"` / `"range"`), and `entries: { count, items }` with full markdown bodies by default; set `include_bodies: false` for a lean version / date / URL timeline. | | `list_files` | `target`, `path_prefix?`, `limit?`, `wait_timeout_ms?` | List files in an indexed dependency. Returns `{total, hasMore, files: [{path, name, language, fileType, byteSize}], resolution, indexedVersion}`. Dual addressing via `target.registry + target.package_name` (spec) or `target.repo_url + target.git_ref` (repo). `path_prefix` is a literal directory prefix — NOT a glob (`*.ts` won't match); glob / pattern filtering is an upstream enhancement. Emits an `INDEXING` error envelope when the dependency is being indexed on-demand — retry with a longer `wait_timeout_ms` or pick a version from `details.availableVersions`. | | `read_file` | `target`, `path`, `start_line?`, `end_line?`, `wait_timeout_ms?` | Read a file from an indexed dependency. Default full file; use `start_line` / `end_line` for a bounded range. Binary files set `isBinary: true` and omit `content` — agents branch on the flag. On `NOT_FOUND` / `FILE_NOT_FOUND` call `list_files` to discover the actual path. | -| `grep_file` | `target`, `path`, `pattern`, `context_lines?`, `max_matches?`, `wait_timeout_ms?` | Search within a single file for a case-insensitive substring (not regex). Returns matches — `context_lines` defaults to 0 (matches only, token-efficient); pass explicitly for surrounding lines (0–10). Max pattern 200 chars; up to 200 matches. For symbol-shaped searches prefer unified `search` with `sources:["symbol"]`. | +| `grep_repo` | `target`, `pattern`, `path?`, `path_prefix?`, `globs?`, `extensions?`, `pattern_type?`, `case_sensitive?`, `exclude_doc_files?`, `exclude_test_files?`, `context_lines?`, `context_lines_before?`, `context_lines_after?`, `max_matches?`, `max_matches_per_file?`, `cursor?`, `wait_timeout_ms?` | Deterministic text grep over indexed dependency or repository source. Defaults to literal, case-insensitive matching across the whole target; narrow with `path`, `path_prefix`, `globs`, or `extensions`. `pattern_type: "regex"` uses RE2 syntax. Returns matches plus pagination and scan counters. | -`search`, `search_status`, `search_symbols`, `package_summary`, `package_vulnerabilities`, `package_dependencies`, `package_changelog`, `list_files`, `read_file`, and `grep_file` are only registered when package/source access is open for the current session. The package/source service URL can be overridden via `GITHITS_CODE_NAV_URL` for local development. +`search`, `search_status`, `search_symbols`, `package_summary`, `package_vulnerabilities`, `package_dependencies`, `package_changelog`, `list_files`, `read_file`, and `grep_repo` are only registered when package/source access is open for the current session. The package/source service URL can be overridden via `GITHITS_CODE_NAV_URL` for local development. `search_symbols` shares request-construction, error classification, and JSON-payload shape with the CLI `githits code search` command via shared helpers under `src/shared/`. The parity rules are codified in [`mcp-cli-parity.md`](./mcp-cli-parity.md); the parity test (`src/tools/search-symbols-parity.test.ts`) asserts that both surfaces emit identical JSON for equivalent inputs. @@ -111,17 +111,17 @@ Both expose the same tools with identical names, parameters, and descriptions. T `package_changelog` shares its envelope builder with the CLI `githits pkg changelog` command via `src/shared/package-changelog-request.ts` and `src/shared/package-changelog-response.ts`. The terminal formatter is CLI-only. The parity test (`src/tools/package-changelog-parity.test.ts`) asserts `toEqual` across every service-sourced success / error fixture (happy latest, range mode, repo-URL addressing, `--no-body` / `include_bodies: false`, default bodies, empty entries, NOT_FOUND, PackageIntelligenceTargetNotFoundError, VERSION_NOT_FOUND, BACKEND_ERROR) and `toMatchObject` for builder-sourced `INVALID_ARGUMENT`. -### `list_files` / `read_file` / `grep_file` response shapes +### `list_files` / `read_file` / `grep_repo` response shapes These three indexing-gated tools share an addressing and lifecycle contract (documented below) and then each projects its own data-first envelope. All three reuse the shipped `codeTargetSchema` + `resolveCodeTarget` from `src/tools/code-navigation-shared.ts` — no parallel addressing module. **`list_files` envelope**: `{registry?|repoUrl?+gitRef?, total, hasMore, indexedVersion?, resolution?, files: [{path, name?, language?, fileType?, byteSize?}], hint?, filter?}`. `fileType` values preserve the service vocabulary (`CONFIG`, `SOURCE`, `DOC`, `TEST`). `total` is capped at returned count when `hasMore: true` — the terminal formatter renders `N+ files` in that case to avoid misleading users. `filter.pathPrefix` / `filter.limit` echo only when the caller supplied them explicitly; default limit (200) never round-trips. -**`read_file` envelope**: `{registry?|repoUrl?+gitRef?, path, language?, totalLines?, startLine?, endLine?, content?, isBinary?}`. `path` (not `filePath`) so the key matches `list_files.files[].path` and `grep_file`'s `path` input — the `list_files` → `read_file` / `grep_file` chain needs no renames. Binary files set `isBinary: true` and **omit** `content` (not `null`); agents branch on the flag. +**`read_file` envelope**: `{registry?|repoUrl?+gitRef?, path, language?, totalLines?, startLine?, endLine?, content?, isBinary?}`. `path` (not `filePath`) so the key matches `list_files.files[].path` and `grep_repo.filter.path` when exact-file grep is used. Binary files set `isBinary: true` and **omit** `content` (not `null`); agents branch on the flag. -**`grep_file` envelope**: `{registry?|repoUrl?+gitRef?, pattern, path, totalMatches, hasMore, matches: [{lineNumber, lineContent, contextBefore?, contextAfter?}], language?, totalLines?, indexedVersion?, resolution?, hint?, filter?}`. Single `path` field (backend echo wins, caller input is the fallback) — no separate `filePath`. Context arrays stripped when empty. +**`grep_repo` envelope**: `{registry?|name?|repoUrl?+gitRef?, pattern, patternType, caseSensitive, matches: [{filePath, line, matchStartByte, matchEndByte, lineContent, contextBefore?, contextAfter?, fileContentHash?, fileIntent?}], nextCursor?, hasMore, truncatedReason, routeTaken?, filesScanned, filesInScope, binaryFilesSkipped, filesTooLargeSkipped, totalMatches, uniqueFilesMatched, indexedVersion?, resolution?, filter?}`. `filter` echoes only explicit caller filters. Match entries carry `filePath` so grep output chains directly into `read_file`. -### Indexing lifecycle (shared across `search_symbols`, `list_files`, `read_file`, `grep_file`) +### Indexing lifecycle (shared across `search_symbols`, `list_files`, `read_file`, `grep_repo`) All four code-navigation tools share the same indexing-retry contract. The state can arrive through either an error response or a success sentinel (`codeIndexState: "INDEXING"`), and the service layer collapses both to the same typed `CodeNavigationIndexingError` before the envelope builder runs. Agents therefore never see a `codeIndexState` field in a success envelope; they branch on the error path instead. @@ -142,7 +142,7 @@ All four code-navigation tools share the same indexing-retry contract. The state **Retry default**: `DEFAULT_WAIT_TIMEOUT_MS = 20_000` (shared, defined in `src/shared/code-navigation-defaults.ts`). Applied inside each request builder so both CLI and MCP surfaces get the same default by construction. CLI's `--wait ` and MCP's `wait_timeout_ms` override. -**`FILE_NOT_FOUND` vs `NOT_FOUND`**: `read_file` / `grep_file` can hit "path doesn't resolve" errors. The classifier is pre-wired to emit `FILE_NOT_FOUND` when the backend sends `extensions.code: "FILE_NOT_FOUND"`, but today the backend emits generic `NOT_FOUND` for both "package missing" and "path missing". The distinction is filed upstream. CLI terminal output for `code read` / `code grep` emits the hint "Use `code files` to list available paths." on both codes so users have an actionable next step regardless of classification. +**`FILE_NOT_FOUND` vs `NOT_FOUND`**: `read_file` / `grep_repo` can hit "path doesn't resolve" errors when an exact path scope is invalid. The classifier is pre-wired to emit `FILE_NOT_FOUND` when the backend sends `extensions.code: "FILE_NOT_FOUND"`, but today the backend emits generic `NOT_FOUND` for both "package missing" and "path missing". The distinction is filed upstream. CLI terminal output for `code read` / `code grep` emits the hint "Use `code files` to list available paths." on both codes so users have an actionable next step regardless of classification. ## Server instructions diff --git a/src/commands/code/code-nav-cli-helpers.ts b/src/commands/code/code-nav-cli-helpers.ts index 71bc9996..cd428ef9 100644 --- a/src/commands/code/code-nav-cli-helpers.ts +++ b/src/commands/code/code-nav-cli-helpers.ts @@ -140,20 +140,51 @@ export function formatIndexingError(mapped: MappedError): string { } /** - * Terminal error renderer for `code read` / `code grep`. Treats - * both `FILE_NOT_FOUND` and the backend's currently-generic - * `NOT_FOUND` (gap #11) the same way — the user experience - * shouldn't depend on which classification the backend picked, - * and the next action is always "call `code files` to discover - * the actual paths". + * Terminal error renderer for `code read` / `code grep`. Adds the + * `code files` recovery hint for concrete missing-path cases, even + * when the backend still collapses them into generic `NOT_FOUND`. + * Leaves unrelated repository / indexing-state `NOT_FOUND` errors + * alone so we don't send users toward path debugging for the wrong + * class of failure. */ export function formatFileErrorWithFilesHint(mapped: MappedError): string { - if (mapped.code === "FILE_NOT_FOUND" || mapped.code === "NOT_FOUND") { + if (mapped.code === "FILE_NOT_FOUND") { return `${mapped.message}\n Use \`code files\` to list available paths.`; } + if ( + mapped.code === "NOT_FOUND" && + looksLikeMissingFileMessage(mapped.message) + ) { + return `${mapped.message}\n Use \`code files\` to list available paths.`; + } + if (looksLikeMissingNavpackMessage(mapped.message)) { + return [ + "Source index for this target is temporarily unavailable.", + " Retry with `--wait 60000`, use an already-indexed version/ref, or try again later.", + ].join("\n"); + } return formatIndexingError(mapped); } +function looksLikeMissingFileMessage(message: string): boolean { + const lower = message.toLowerCase(); + return ( + lower.includes("file not found") || + lower.includes("path not found") || + lower.includes("path doesn't resolve") || + lower.includes("path does not resolve") + ); +} + +function looksLikeMissingNavpackMessage(message: string): boolean { + const lower = message.toLowerCase(); + return ( + lower.includes("has no navpack for this ref") || + lower.includes("navpack was pruned") || + lower.includes("indexedrepository row still claims current state") + ); +} + /** * Shared error-printing + `process.exit` path used by every * indexing-gated `pkg` command. JSON callers get the shared diff --git a/src/commands/code/grep.test.ts b/src/commands/code/grep.test.ts index bca1ef09..626080f1 100644 --- a/src/commands/code/grep.test.ts +++ b/src/commands/code/grep.test.ts @@ -1,11 +1,12 @@ import { describe, expect, it, mock, spyOn } from "bun:test"; import { + CodeNavigationFileNotFoundError, CodeNavigationIndexingError, CodeNavigationTargetNotFoundError, } from "../../services/index.js"; import { createMockCodeNavigationService, - defaultGrepFileResult, + defaultGrepRepoResult, } from "../../services/test-helpers.js"; import { type PkgGrepCommandDependencies, pkgGrepAction } from "./grep.js"; @@ -24,7 +25,7 @@ describe("pkgGrepAction", () => { }; } - it("plain mode: emits matching line(s) only — no header, no gutter", async () => { + it("plain mode emits file:line:text", async () => { const writes: string[] = []; const writeSpy = spyOn(process.stdout, "write").mockImplementation((( chunk: string | Uint8Array, @@ -34,26 +35,31 @@ describe("pkgGrepAction", () => { ); return true; }) as typeof process.stdout.write); + const stdoutDescriptor = Object.getOwnPropertyDescriptor( + process.stdout, + "isTTY", + ); + Object.defineProperty(process.stdout, "isTTY", { + value: false, + configurable: true, + }); await pkgGrepAction( "npm:express", "middleware", - "src/index.js", + undefined, {}, createDeps(), ); - const combined = writes.join(""); - expect(combined).not.toContain("express · npm"); - expect(combined).not.toContain("1 match in src/index.js"); - expect(combined).not.toMatch(/^>/m); - // `defaultGrepFileResult` has one match line — plain mode emits - // its content. - expect(combined.trim().length).toBeGreaterThan(0); + expect(writes.join("")).toContain("src/index.js:4:"); writeSpy.mockRestore(); + if (stdoutDescriptor) { + Object.defineProperty(process.stdout, "isTTY", stdoutDescriptor); + } }); - it("verbose mode: renders the full match block with header and `>` marker", async () => { + it("tty mode emits file heading plus compact line matches", async () => { const writes: string[] = []; const writeSpy = spyOn(process.stdout, "write").mockImplementation((( chunk: string | Uint8Array, @@ -63,109 +69,319 @@ describe("pkgGrepAction", () => { ); return true; }) as typeof process.stdout.write); + const stdoutDescriptor = Object.getOwnPropertyDescriptor( + process.stdout, + "isTTY", + ); + Object.defineProperty(process.stdout, "isTTY", { + value: true, + configurable: true, + }); await pkgGrepAction( "npm:express", "middleware", - "src/index.js", + undefined, + {}, + createDeps(), + ); + + const output = writes.join(""); + expect(output).toContain( + "src/index.js\n4:module.exports = require('./lib/express');", + ); + expect(output).not.toContain( + "src/index.js:4:module.exports = require('./lib/express');", + ); + writeSpy.mockRestore(); + if (stdoutDescriptor) { + Object.defineProperty(process.stdout, "isTTY", stdoutDescriptor); + } + }); + + it("verbose mode renders grouped output", async () => { + const writes: string[] = []; + const writeSpy = spyOn(process.stdout, "write").mockImplementation((( + chunk: string | Uint8Array, + ) => { + writes.push( + typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), + ); + return true; + }) as typeof process.stdout.write); + + await pkgGrepAction( + "npm:express", + "middleware", + undefined, { verbose: true }, createDeps(), ); - const combined = writes.join(""); - expect(combined).toContain("express · npm"); - expect(combined).toContain("1 match in src/index.js"); - expect(combined).toContain(">"); + const output = writes.join(""); + expect(output).toContain("1 match in 1 file"); + expect(output).toContain("src/index.js\n"); + expect(output).toContain("> 4 module.exports = require('./lib/express');"); writeSpy.mockRestore(); }); - it("default contextLines is 0 on the wire (matches-only)", async () => { - const grepFile = mock(() => Promise.resolve(defaultGrepFileResult)); - const writeSpy = spyOn(process.stdout, "write").mockImplementation( + it("prints the actual nextCursor in terminal pagination hints", async () => { + const stderrWrites: string[] = []; + const stdoutSpy = spyOn(process.stdout, "write").mockImplementation( (() => true) as typeof process.stdout.write, ); + const stderrSpy = spyOn(process.stderr, "write").mockImplementation((( + chunk: string | Uint8Array, + ) => { + stderrWrites.push( + typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), + ); + return true; + }) as typeof process.stderr.write); + await pkgGrepAction( "npm:express", "middleware", - "src/index.js", + undefined, {}, createDeps({ - codeNavigationService: createMockCodeNavigationService({ grepFile }), + codeNavigationService: createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.resolve({ + ...defaultGrepRepoResult, + hasMore: true, + nextCursor: "cursor_abc123", + truncatedReason: "MAX_MATCHES" as const, + }), + ), + }), }), ); - const calls = grepFile.mock.calls as unknown as Array< - [{ contextLines?: number }] - >; - expect(calls[0]?.[0]?.contextLines).toBe(0); - writeSpy.mockRestore(); + + const stderr = stderrWrites.join(""); + expect(stderr).toBe( + "More grep results available — rerun with --cursor 'cursor_abc123'\n", + ); + stdoutSpy.mockRestore(); + stderrSpy.mockRestore(); }); - it("emits the JSON envelope with --json", async () => { + it("adds a narrow-scope hint for noisy broad terminal output", async () => { + const stderrWrites: string[] = []; + const stdoutSpy = spyOn(process.stdout, "write").mockImplementation( + (() => true) as typeof process.stdout.write, + ); + const stderrSpy = spyOn(process.stderr, "write").mockImplementation((( + chunk: string | Uint8Array, + ) => { + stderrWrites.push( + typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), + ); + return true; + }) as typeof process.stderr.write); + + await pkgGrepAction( + "npm:express", + "foo", + undefined, + {}, + createDeps({ + codeNavigationService: createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.resolve({ + ...defaultGrepRepoResult, + totalMatches: 6, + uniqueFilesMatched: 6, + matches: [ + { + ...defaultGrepRepoResult.matches[0]!, + filePath: "History.md", + line: 1, + }, + { + ...defaultGrepRepoResult.matches[0]!, + filePath: "Readme.md", + line: 2, + }, + { + ...defaultGrepRepoResult.matches[0]!, + filePath: "benchmarks/run", + line: 3, + }, + { + ...defaultGrepRepoResult.matches[0]!, + filePath: "examples/a.js", + line: 4, + }, + { + ...defaultGrepRepoResult.matches[0]!, + filePath: "test/a.js", + line: 5, + }, + { + ...defaultGrepRepoResult.matches[0]!, + filePath: "lib/app.js", + line: 6, + }, + ], + }), + ), + }), + }), + ); + + expect(stderrWrites.join("")).toContain("Broad results"); + expect(stderrWrites.join("")).toContain("--exclude-tests"); + stdoutSpy.mockRestore(); + stderrSpy.mockRestore(); + }); + + it("JSON mode emits the new envelope", async () => { const logSpy = spyOn(console, "log").mockImplementation(() => {}); await pkgGrepAction( "npm:express", "middleware", - "src/index.js", + "src/", { json: true }, createDeps(), ); const payload = JSON.parse(logSpy.mock.calls[0]?.[0] as string); expect(payload.pattern).toBe("middleware"); - expect(payload.path).toBe("src/index.js"); - expect(payload.matches.length).toBe(1); + expect(payload.filter.pathPrefix).toBe("src/"); + expect(payload.matches[0].filePath).toBe("src/index.js"); logSpy.mockRestore(); }); - it("sends wait default of 20000 on the wire", async () => { - const grepFile = mock(() => Promise.resolve(defaultGrepFileResult)); + it("uses repo grep defaults on the wire", async () => { + const grepRepo = mock(() => Promise.resolve(defaultGrepRepoResult)); const writeSpy = spyOn(process.stdout, "write").mockImplementation( (() => true) as typeof process.stdout.write, ); await pkgGrepAction( "npm:express", "middleware", - "src/index.js", + undefined, {}, createDeps({ - codeNavigationService: createMockCodeNavigationService({ grepFile }), + codeNavigationService: createMockCodeNavigationService({ grepRepo }), }), ); - const calls = grepFile.mock.calls as unknown as Array< - [{ waitTimeoutMs?: number }] + const calls = grepRepo.mock.calls as unknown as Array< + [ + { + allowUnscoped?: boolean; + contextLinesBefore?: number; + contextLinesAfter?: number; + maxMatches?: number; + waitTimeoutMs?: number; + }, + ] >; + expect(calls[0]?.[0]?.allowUnscoped).toBe(true); + expect(calls[0]?.[0]?.contextLinesBefore).toBe(0); + expect(calls[0]?.[0]?.contextLinesAfter).toBe(0); + expect(calls[0]?.[0]?.maxMatches).toBe(50); expect(calls[0]?.[0]?.waitTimeoutMs).toBe(20000); writeSpy.mockRestore(); }); - it("sends repo-URL addressing with two positionals (pattern, path)", async () => { - const grepFile = mock(() => Promise.resolve(defaultGrepFileResult)); + it("maps path prefix, exact path, globs, extensions, regex, and context options", async () => { + const grepRepo = mock(() => Promise.resolve(defaultGrepRepoResult)); const writeSpy = spyOn(process.stdout, "write").mockImplementation( (() => true) as typeof process.stdout.write, ); await pkgGrepAction( + "npm:express", "middleware", - "src/index.js", + "src/", + { + path: "src/index.js", + glob: ["src/**/*.js"], + ext: ["js"], + regex: true, + caseSensitive: true, + beforeContext: "2", + afterContext: "1", + limit: "100", + perFileLimit: "3", + excludeDocs: true, + excludeTests: true, + cursor: "cursor-123", + }, + createDeps({ + codeNavigationService: createMockCodeNavigationService({ grepRepo }), + }), + ); + const calls = grepRepo.mock.calls as unknown as Array< + [ + { + patternType?: string; + caseSensitive?: boolean; + pathSelectors?: Array<{ kind: string; value: string }>; + extensions?: string[]; + contextLinesBefore?: number; + contextLinesAfter?: number; + maxMatches?: number; + maxMatchesPerFile?: number; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + cursor?: string; + }, + ] + >; + expect(calls[0]?.[0]).toMatchObject({ + patternType: "REGEX", + caseSensitive: true, + extensions: ["js"], + contextLinesBefore: 2, + contextLinesAfter: 1, + maxMatches: 100, + maxMatchesPerFile: 3, + excludeDocFiles: true, + excludeTestFiles: true, + cursor: "cursor-123", + }); + expect(calls[0]?.[0]?.pathSelectors).toEqual([ + { kind: "EXACT", value: "src/index.js" }, + { kind: "PREFIX", value: "src/" }, + { kind: "GLOB", value: "src/**/*.js" }, + ]); + writeSpy.mockRestore(); + }); + + it("repo-url mode uses [path-prefix]", async () => { + const grepRepo = mock(() => Promise.resolve(defaultGrepRepoResult)); + const writeSpy = spyOn(process.stdout, "write").mockImplementation( + (() => true) as typeof process.stdout.write, + ); + await pkgGrepAction( + "middleware", + "src/", undefined, { repoUrl: "https://github.com/expressjs/express", gitRef: "main", }, createDeps({ - codeNavigationService: createMockCodeNavigationService({ grepFile }), + codeNavigationService: createMockCodeNavigationService({ grepRepo }), }), ); - const calls = grepFile.mock.calls as unknown as Array< - [{ target: { repoUrl?: string }; pattern: string; path: string }] + const calls = grepRepo.mock.calls as unknown as Array< + [ + { + target: { repoUrl?: string }; + pathSelectors?: Array<{ value: string }>; + }, + ] >; expect(calls[0]?.[0]?.target.repoUrl).toBe( "https://github.com/expressjs/express", ); - expect(calls[0]?.[0]?.pattern).toBe("middleware"); - expect(calls[0]?.[0]?.path).toBe("src/index.js"); + expect(calls[0]?.[0]?.pathSelectors?.[0]?.value).toBe("src/"); writeSpy.mockRestore(); }); - it("rejects extra positional in repo-URL mode", async () => { + it("rejects extra positional in repo-url mode", async () => { const errorSpy = spyOn(console, "error").mockImplementation(() => {}); const exitSpy = spyOn(process, "exit").mockImplementation(() => { throw new Error("process.exit"); @@ -173,8 +389,8 @@ describe("pkgGrepAction", () => { try { await pkgGrepAction( "middleware", - "src/index.js", - "extra-arg", + "src/", + "extra", { repoUrl: "https://github.com/expressjs/express", gitRef: "main", @@ -210,105 +426,79 @@ describe("pkgGrepAction", () => { exitSpy.mockRestore(); }); - it("gives a targeted error when caller passes two positionals without --repo-url", async () => { - const errorSpy = spyOn(console, "error").mockImplementation(() => {}); - const exitSpy = spyOn(process, "exit").mockImplementation(() => { - throw new Error("process.exit"); + it("exits 1 on zero matches", async () => { + const logSpy = spyOn(console, "log").mockImplementation(() => {}); + const exitCalls: number[] = []; + const exitSpy = spyOn(process, "exit").mockImplementation((( + code?: number, + ) => { + exitCalls.push(code ?? 0); + return undefined as never; + }) as typeof process.exit); + const service = createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.resolve({ + ...defaultGrepRepoResult, + matches: [], + totalMatches: 0, + uniqueFilesMatched: 0, + }), + ), }); - // `code grep middleware src/index.js` with no spec + no --repo-url. - // Commander binds these as first=middleware, second=src/index.js. - // The action must recognise this as a missing-spec mistake, not - // a "missing " mistake. - try { - await pkgGrepAction( - "middleware", - "src/index.js", - undefined, - {}, - createDeps(), - ); - } catch { - /* expected */ - } - const msg = errorSpy.mock.calls[0]?.[0] as string; - expect(msg).toContain("all three positionals are required"); - expect(msg).toContain("--repo-url"); - errorSpy.mockRestore(); + await pkgGrepAction( + "npm:express", + "nothing", + undefined, + { json: true }, + createDeps({ codeNavigationService: service }), + ); + expect(logSpy.mock.calls.length).toBe(1); + expect(exitCalls).toEqual([1]); + logSpy.mockRestore(); exitSpy.mockRestore(); }); - it("rejects missing path", async () => { + it("exits 2 on error paths", async () => { const errorSpy = spyOn(console, "error").mockImplementation(() => {}); - const exitSpy = spyOn(process, "exit").mockImplementation(() => { + const exitCalls: number[] = []; + const exitSpy = spyOn(process, "exit").mockImplementation((( + code?: number, + ) => { + exitCalls.push(code ?? 0); throw new Error("process.exit"); + }) as typeof process.exit); + const service = createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.reject( + new CodeNavigationTargetNotFoundError("Package not found"), + ), + ), }); try { await pkgGrepAction( - "npm:express", + "npm:ghost", "middleware", undefined, {}, - createDeps(), - ); - } catch { - /* expected */ - } - expect(errorSpy.mock.calls[0]?.[0]).toMatch(/path/); - errorSpy.mockRestore(); - exitSpy.mockRestore(); - }); - - it("rejects --context out of range", async () => { - const errorSpy = spyOn(console, "error").mockImplementation(() => {}); - const exitSpy = spyOn(process, "exit").mockImplementation(() => { - throw new Error("process.exit"); - }); - try { - await pkgGrepAction( - "npm:express", - "middleware", - "src/index.js", - { context: "11" }, - createDeps(), - ); - } catch { - /* expected */ - } - expect(errorSpy.mock.calls[0]?.[0]).toMatch(/0 and 10/); - errorSpy.mockRestore(); - exitSpy.mockRestore(); - }); - - it("rejects --limit out of range", async () => { - const errorSpy = spyOn(console, "error").mockImplementation(() => {}); - const exitSpy = spyOn(process, "exit").mockImplementation(() => { - throw new Error("process.exit"); - }); - try { - await pkgGrepAction( - "npm:express", - "middleware", - "src/index.js", - { limit: "201" }, - createDeps(), + createDeps({ codeNavigationService: service }), ); } catch { /* expected */ } - expect(errorSpy.mock.calls[0]?.[0]).toMatch(/1 and 200/); + expect(exitCalls).toEqual([2]); errorSpy.mockRestore(); exitSpy.mockRestore(); }); - it("routes NOT_FOUND with a code-files hint", async () => { + it("enriches INDEXING error", async () => { const errorSpy = spyOn(console, "error").mockImplementation(() => {}); const exitSpy = spyOn(process, "exit").mockImplementation(() => { throw new Error("process.exit"); }); const service = createMockCodeNavigationService({ - grepFile: mock(() => + grepRepo: mock(() => Promise.reject( - new CodeNavigationTargetNotFoundError("File not found in repository"), + new CodeNavigationIndexingError("Indexing...", "ref_abc"), ), ), }); @@ -316,283 +506,109 @@ describe("pkgGrepAction", () => { await pkgGrepAction( "npm:express", "middleware", - "nope.js", + undefined, {}, createDeps({ codeNavigationService: service }), ); } catch { /* expected */ } - const output = errorSpy.mock.calls[0]?.[0] as string; - expect(output).toContain("File not found"); - expect(output).toContain("code files"); + expect(errorSpy.mock.calls[0]?.[0]).toContain("indexingRef: ref_abc"); errorSpy.mockRestore(); exitSpy.mockRestore(); }); - it("enriches INDEXING error", async () => { + it("adds file listing hint only for file-path failures", async () => { const errorSpy = spyOn(console, "error").mockImplementation(() => {}); const exitSpy = spyOn(process, "exit").mockImplementation(() => { throw new Error("process.exit"); }); - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.reject( - new CodeNavigationIndexingError("Indexing...", "ref_abc"), - ), - ), - }); + try { await pkgGrepAction( "npm:express", "middleware", - "src/index.js", + undefined, {}, - createDeps({ codeNavigationService: service }), + createDeps({ + codeNavigationService: createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.reject( + new CodeNavigationTargetNotFoundError("Package not found"), + ), + ), + }), + }), ); } catch { /* expected */ } - expect(errorSpy.mock.calls[0]?.[0]).toContain("indexingRef: ref_abc"); - errorSpy.mockRestore(); - exitSpy.mockRestore(); - }); - // ------------------------------------------------------------------ - // Exit-code contract (grep-style: 0 match / 1 no-match / 2 error) - // ------------------------------------------------------------------ - - it("exits 0 when there is at least one match", async () => { - const writeSpy = spyOn(process.stdout, "write").mockImplementation( - (() => true) as typeof process.stdout.write, - ); - const exitSpy = spyOn(process, "exit").mockImplementation((() => { - return undefined as never; - }) as typeof process.exit); - await pkgGrepAction( - "npm:express", - "middleware", - "src/index.js", - {}, - createDeps(), - ); - // `defaultGrepFileResult` has one match — exit 0 means `process.exit` - // was not called at all (happy path returns normally). - expect(exitSpy.mock.calls.length).toBe(0); - writeSpy.mockRestore(); - exitSpy.mockRestore(); - }); + expect(errorSpy.mock.calls[0]?.[0]).not.toContain("Use `code files`"); - it("exits 1 when there are zero matches (plain mode)", async () => { - const writeSpy = spyOn(process.stdout, "write").mockImplementation( - (() => true) as typeof process.stdout.write, - ); - const exitCalls: number[] = []; - // Intentionally non-throwing — `process.exit(1)` on zero matches - // is the last statement in the happy path, so letting the mock - // return keeps the action function's control flow clean. - const exitSpy = spyOn(process, "exit").mockImplementation((( - code?: number, - ) => { - exitCalls.push(code ?? 0); - return undefined as never; - }) as typeof process.exit); - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.resolve({ - matches: [], - totalMatches: 0, - hasMore: false, - filePath: "src/index.js", - language: "javascript", - }), - ), - }); - await pkgGrepAction( - "npm:express", - "nonexistent-pattern", - "src/index.js", - {}, - createDeps({ codeNavigationService: service }), - ); - expect(exitCalls).toEqual([1]); - writeSpy.mockRestore(); - exitSpy.mockRestore(); - }); + errorSpy.mockReset(); - it("exits 1 when there are zero matches (--json mode)", async () => { - const logSpy = spyOn(console, "log").mockImplementation(() => {}); - const exitCalls: number[] = []; - const exitSpy = spyOn(process, "exit").mockImplementation((( - code?: number, - ) => { - exitCalls.push(code ?? 0); - return undefined as never; - }) as typeof process.exit); - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.resolve({ - matches: [], - totalMatches: 0, - hasMore: false, - filePath: "src/index.js", - }), - ), - }); - await pkgGrepAction( - "npm:express", - "nonexistent", - "src/index.js", - { json: true }, - createDeps({ codeNavigationService: service }), - ); - // JSON is logged BEFORE the exit-1 fires, so callers can still - // parse it via `jq` even under `pipefail`. - expect(logSpy.mock.calls.length).toBe(1); - const payload = JSON.parse(logSpy.mock.calls[0]?.[0] as string); - expect(payload.totalMatches).toBe(0); - expect(exitCalls).toEqual([1]); - logSpy.mockRestore(); - exitSpy.mockRestore(); - }); - - it("exits 2 on error paths (distinct from 'no match' = 1)", async () => { - const errorSpy = spyOn(console, "error").mockImplementation(() => {}); - const exitCalls: number[] = []; - const exitSpy = spyOn(process, "exit").mockImplementation((( - code?: number, - ) => { - exitCalls.push(code ?? 0); - throw new Error("process.exit"); - }) as typeof process.exit); - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.reject( - new CodeNavigationTargetNotFoundError("File not found in repository"), - ), - ), - }); try { await pkgGrepAction( "npm:express", "middleware", - "nope.js", + undefined, {}, - createDeps({ codeNavigationService: service }), + createDeps({ + codeNavigationService: createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.reject( + new CodeNavigationFileNotFoundError( + "File not found: nope.js", + "nope.js", + ), + ), + ), + }), + }), ); } catch { /* expected */ } - expect(exitCalls).toEqual([2]); + + expect(errorSpy.mock.calls[0]?.[0]).toContain("Use `code files`"); errorSpy.mockRestore(); exitSpy.mockRestore(); }); - // ------------------------------------------------------------------ - // stdout vs stderr routing (plain mode) - // ------------------------------------------------------------------ - - it("plain mode hasMore: truncation warning goes to stderr, not stdout", async () => { - const stdoutWrites: string[] = []; - const stderrWrites: string[] = []; - const stdoutSpy = spyOn(process.stdout, "write").mockImplementation((( - chunk: string | Uint8Array, - ) => { - stdoutWrites.push( - typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), - ); - return true; - }) as typeof process.stdout.write); - const stderrSpy = spyOn(process.stderr, "write").mockImplementation((( - chunk: string | Uint8Array, - ) => { - stderrWrites.push( - typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), - ); - return true; - }) as typeof process.stderr.write); - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.resolve({ - matches: [ - { - lineNumber: 1, - lineContent: "export const foo = 1;", - }, - ], - totalMatches: 50, - hasMore: true, - filePath: "src/index.js", - }), - ), - }); - await pkgGrepAction( - "npm:express", - "foo", - "src/index.js", - {}, - createDeps({ codeNavigationService: service }), - ); - const stdout = stdoutWrites.join(""); - const stderr = stderrWrites.join(""); - expect(stdout).toContain("export const foo"); - expect(stdout).not.toContain("More matches available"); - expect(stderr).toContain("More matches available"); - stdoutSpy.mockRestore(); - stderrSpy.mockRestore(); - }); - - it("plain mode zero-match with regex-shaped pattern: nudge goes to stderr", async () => { - const stdoutWrites: string[] = []; - const stderrWrites: string[] = []; - const stdoutSpy = spyOn(process.stdout, "write").mockImplementation((( - chunk: string | Uint8Array, - ) => { - stdoutWrites.push( - typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), - ); - return true; - }) as typeof process.stdout.write); - const stderrSpy = spyOn(process.stderr, "write").mockImplementation((( - chunk: string | Uint8Array, - ) => { - stderrWrites.push( - typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk), - ); - return true; - }) as typeof process.stderr.write); + it("rewrites navpack backend failures into actionable CLI guidance", async () => { + const errorSpy = spyOn(console, "error").mockImplementation(() => {}); const exitSpy = spyOn(process, "exit").mockImplementation(() => { throw new Error("process.exit"); }); - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.resolve({ - matches: [], - totalMatches: 0, - hasMore: false, - filePath: "src/index.js", - }), - ), - }); + try { await pkgGrepAction( "npm:express", - "\\bfoo\\b", - "src/index.js", + "middleware", + undefined, {}, - createDeps({ codeNavigationService: service }), + createDeps({ + codeNavigationService: createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.reject( + new CodeNavigationTargetNotFoundError( + "aigrep has no navpack for this ref and the repository is not marked as stale on any known artifact axis.", + ), + ), + ), + }), + }), ); } catch { - /* expected — exit 1 */ + /* expected */ } - const stdout = stdoutWrites.join(""); - const stderr = stderrWrites.join(""); - // Plain mode stdout stays empty on zero matches. - expect(stdout).toBe(""); - // Regex-hint nudge visible on stderr. - expect(stderr).toContain("substring"); - stdoutSpy.mockRestore(); - stderrSpy.mockRestore(); + + expect(errorSpy.mock.calls[0]?.[0]).toContain( + "Source index for this target is temporarily unavailable.", + ); + expect(errorSpy.mock.calls[0]?.[0]).toContain("--wait 60000"); + errorSpy.mockRestore(); exitSpy.mockRestore(); }); }); diff --git a/src/commands/code/grep.ts b/src/commands/code/grep.ts index 2c08c646..419c7ce5 100644 --- a/src/commands/code/grep.ts +++ b/src/commands/code/grep.ts @@ -1,4 +1,4 @@ -import type { Command } from "commander"; +import { type Command, Option } from "commander"; import { createContainer } from "../../container.js"; import type { CodeNavigationService } from "../../services/index.js"; import { @@ -7,14 +7,13 @@ import { } from "../../shared/code-navigation-defaults.js"; import { shouldUseColors } from "../../shared/colors.js"; import { - buildGrepFileParams, - GREP_PATTERN_SEMANTICS_NOTE, -} from "../../shared/grep-file-request.js"; -import { - buildGrepFileSuccessPayload, - formatGrepFileTerminal, -} from "../../shared/grep-file-response.js"; -import { InvalidPackageSpecError, requireAuth } from "../../shared/index.js"; + buildGrepRepoParams, + buildGrepRepoSuccessPayload, + formatGrepRepoTerminal, + GREP_REPO_PATTERN_NOTE, + InvalidPackageSpecError, + requireAuth, +} from "../../shared/index.js"; import { toPkgseerRegistryLowercase } from "../../shared/pkgseer-registry.js"; import { formatFileErrorWithFilesHint, @@ -26,8 +25,19 @@ import { export interface PkgGrepCommandOptions { repoUrl?: string; gitRef?: string; + path?: string; + glob?: string[]; + ext?: string[]; + regex?: boolean; + caseSensitive?: boolean; context?: string; + beforeContext?: string; + afterContext?: string; limit?: string; + perFileLimit?: string; + excludeDocs?: boolean; + excludeTests?: boolean; + cursor?: string; wait?: string; verbose?: boolean; json?: boolean; @@ -40,13 +50,6 @@ export interface PkgGrepCommandDependencies { mcpUrl: string; } -/** - * Core `code grep` action. Addressing: `` or - * `--repo-url --git-ref `. Positional order: - * ` ` in spec mode (after the spec); just - * ` ` in repo-URL mode. Commander binds left-to- - * right so we resolve the three positionals with context. - */ export async function pkgGrepAction( first: string | undefined, second: string | undefined, @@ -64,26 +67,39 @@ export async function pkgGrepAction( } const hasRepoUrl = Boolean(options.repoUrl); - const { spec, pattern, path } = resolvePositionals( + const { spec, pattern, pathPrefix } = resolvePositionals( first, second, third, hasRepoUrl, ); - if (!pattern || pattern.length === 0) { - throw new InvalidPackageSpecError( - "A argument is required — pass the substring to search for.", - ); - } - if (!path || path.trim().length === 0) { + if (pattern === undefined) { throw new InvalidPackageSpecError( - "A argument is required — pass the path to the file within the package or repo.", + "A argument is required — pass the text to search for.", ); } const target = resolveCliCodeNavTarget(spec, options); const contextLines = parseIntCliOption(options.context, "--context", 0, 10); - const maxMatches = parseIntCliOption(options.limit, "--limit", 1, 200); + const beforeContext = parseIntCliOption( + options.beforeContext, + "--before-context", + 0, + 10, + ); + const afterContext = parseIntCliOption( + options.afterContext, + "--after-context", + 0, + 10, + ); + const maxMatches = parseIntCliOption(options.limit, "--limit", 1, 1000); + const maxMatchesPerFile = parseIntCliOption( + options.perFileLimit, + "--per-file-limit", + 0, + 1000, + ); const wait = parseIntCliOption( options.wait, "--wait", @@ -91,17 +107,28 @@ export async function pkgGrepAction( MAX_WAIT_TIMEOUT_MS, ); - const build = buildGrepFileParams({ + const build = buildGrepRepoParams({ target, - path, pattern, + path: options.path, + pathPrefix, + globs: options.glob, + extensions: options.ext, + patternType: options.regex ? "regex" : undefined, + caseSensitive: options.caseSensitive, + excludeDocFiles: options.excludeDocs, + excludeTestFiles: options.excludeTests, contextLines, + contextLinesBefore: beforeContext, + contextLinesAfter: afterContext, maxMatches, + maxMatchesPerFile, + cursor: options.cursor, waitTimeoutMs: wait, }); - const result = await deps.codeNavigationService.grepFile(build.params); + const result = await deps.codeNavigationService.grepRepo(build.params); - const payload = buildGrepFileSuccessPayload(result, { + const payload = buildGrepRepoSuccessPayload(result, { registry: target.registry ? toPkgseerRegistryLowercase(target.registry) : undefined, @@ -109,33 +136,42 @@ export async function pkgGrepAction( repoUrl: target.repoUrl, gitRef: target.gitRef, pattern: build.params.pattern, - path: build.params.path, - contextLinesExplicit: build.contextLinesExplicit, - maxMatchesExplicit: build.maxMatchesExplicit, - contextLines: build.params.contextLines ?? 0, + patternType: build.params.patternType === "REGEX" ? "regex" : "literal", + caseSensitive: build.params.caseSensitive ?? false, + path: options.path, + pathPrefix, + globs: options.glob, + extensions: options.ext, + contextLines, + contextLinesBefore: build.params.contextLinesBefore ?? 0, + contextLinesAfter: build.params.contextLinesAfter ?? 0, maxMatches: build.params.maxMatches ?? 50, + maxMatchesPerFile: build.params.maxMatchesPerFile, + cursor: options.cursor, + excludeDocFiles: build.params.excludeDocFiles, + excludeTestFiles: build.params.excludeTestFiles, + explicit: build.explicit, }); if (options.json) { console.log(JSON.stringify(payload)); - // `grep` convention: exit 1 when no match, 0 when ≥1 match. - // `--json` still honours this so scripting stays consistent - // across `--json` and plain callers. if (payload.totalMatches === 0) process.exit(1); return; } - const rendered = formatGrepFileTerminal(payload, { + const rendered = formatGrepRepoTerminal(payload, { useColors: shouldUseColors(), verbose: options.verbose ?? false, + headingStyle: + (process.stdout.isTTY ?? false) && !(options.verbose ?? false), + withContext: + (build.params.contextLinesBefore ?? 0) > 0 || + (build.params.contextLinesAfter ?? 0) > 0, }); process.stdout.write(rendered.stdout); if (rendered.stderr) process.stderr.write(rendered.stderr); if (payload.totalMatches === 0) process.exit(1); } catch (error) { - // `grep` uses exit 2 for errors (distinct from "no match" = - // exit 1). Keeps `if code grep X file; then …` scripts - // correctly classifying missing-file / indexing errors. handleCodeNavCommandError( error, options.json ?? false, @@ -153,52 +189,46 @@ function resolvePositionals( ): { spec: string | undefined; pattern: string | undefined; - path: string | undefined; + pathPrefix: string | undefined; } { if (hasRepoUrl) { - // In repo-URL mode: ` `. Third positional is - // a user error. if (third !== undefined) { throw new InvalidPackageSpecError( - "In --repo-url mode, pass only — the package spec is replaced by --repo-url.", + "In --repo-url mode, pass only [path-prefix] — the package spec is replaced by --repo-url.", ); } - return { spec: undefined, pattern: first, path: second }; + return { spec: undefined, pattern: first, pathPrefix: second }; } - // Spec mode: ` `. Pre-check the positional - // count so users who forget the spec see a targeted error - // rather than the generic " is required" from later in the - // action. Two args + no --repo-url is almost always "I forgot - // the spec" or "I meant to use --repo-url". - if (first !== undefined && second !== undefined && third === undefined) { + if (first !== undefined && second === undefined) { throw new InvalidPackageSpecError( - "In spec mode, all three positionals are required: . If you meant to target a repository instead, pass --repo-url --git-ref .", + "In spec mode, pass at least . If you meant to target a repository instead, pass --repo-url --git-ref .", ); } - return { spec: first, pattern: second, path: third }; + return { spec: first, pattern: second, pathPrefix: third }; } -const PKG_GREP_DESCRIPTION = `Search within a single file for a substring match. +function collectRepeatable(value: string, previous: string[] = []): string[] { + return [...previous, value]; +} -${GREP_PATTERN_SEMANTICS_NOTE} -For symbol-shaped searches, prefer \`githits search --source symbol\`. +const PKG_GREP_DESCRIPTION = `Deterministic text grep over indexed dependency and repository source files. -Addressing: (registry:name[@version]) OR --repo-url ---git-ref . In spec mode pass ; in -repo-URL mode pass only . +${GREP_REPO_PATTERN_NOTE} +Use \`githits search\` for discovery; use \`githits code grep\` when you know the text or regex to match. -Default output is matching lines only (no line numbers, no -context) — same shape as \`grep\`, pipe-friendly. Use --context - to include surrounding lines (0–10, default 0); nearby -matches with overlapping context merge into a single block. -Pass --verbose for a header, line-number gutter, and a \`>\` -marker on match lines. --limit caps the number of matches -(1–200, default 50).`; +Addressing: (registry:name[@version]) OR --repo-url --git-ref . +In spec mode pass [path-prefix]; in repo-URL mode pass only [path-prefix]. + +[path-prefix] matches the same literal prefix semantics as \`githits code files\`. Use --path for one exact file, +repeatable --glob for glob narrowing, and repeatable --ext for extension filtering. + +Default output is \`file:line:text\`, pipe-friendly like grep. Use -C/ -A / -B for context, --verbose for grouped output, +and --cursor to continue a paginated grep run.`; export function registerCodeGrepCommand(pkgCommand: Command): Command { return pkgCommand .command("grep") - .summary("Search within a file in an indexed dependency") + .summary("Deterministic text grep over indexed dependency source") .description(PKG_GREP_DESCRIPTION) .argument( "[arg1]", @@ -206,9 +236,12 @@ export function registerCodeGrepCommand(pkgCommand: Command): Command { ) .argument( "[arg2]", - "In spec mode: the pattern. In --repo-url mode: the path.", + "In spec mode: the pattern. In --repo-url mode: optional path-prefix.", + ) + .argument( + "[arg3]", + "In spec mode: optional path-prefix. Unused in --repo-url mode.", ) - .argument("[arg3]", "In spec mode: the path. Unused in --repo-url mode.") .option( "--repo-url ", "Repository URL addressing (requires --git-ref)", @@ -217,19 +250,49 @@ export function registerCodeGrepCommand(pkgCommand: Command): Command { "--git-ref ", "Tag, commit, branch, or HEAD. Required with --repo-url.", ) + .option("--path ", "Exact file path to grep") .option( - "--context ", - "Context lines before and after each match (0-10, default 0). Nearby blocks merge — no duplicated lines.", + "--glob ", + "Glob scope (repeatable)", + collectRepeatable, + [] as string[], ) - .option("--limit ", "Max matches to return (1-200, default 50)") .option( - "--wait ", - `Indexing wait timeout (0-${MAX_WAIT_TIMEOUT_MS}, default ${DEFAULT_WAIT_TIMEOUT_MS})`, + "--ext ", + "Extension filter without leading dot (repeatable)", + collectRepeatable, + [] as string[], + ) + .option("--regex", "Interpret the pattern as RE2 regex") + .option("--case-sensitive", "Enable ASCII case-sensitive matching") + .option( + "-C, --context ", + "Context lines before and after each match (0-10)", + ) + .option( + "-B, --before-context ", + "Context lines before each match (0-10)", ) + .option("-A, --after-context ", "Context lines after each match (0-10)") + .option("--exclude-docs", "Skip files classified as documentation") + .option("--exclude-tests", "Skip files classified as tests") .option( - "-v, --verbose", - "Render a header and a line-number gutter alongside the matches", + "--limit ", + "Max matches to return on this page (1-1000, default 50)", + ) + .option( + "--per-file-limit ", + "Cap matches per file within this page (0-1000, 0 = unlimited)", + ) + .option( + "--cursor ", + "Opaque nextCursor from a previous grep result", + ) + .option( + "--wait ", + `Indexing wait timeout (0-${MAX_WAIT_TIMEOUT_MS}, default ${DEFAULT_WAIT_TIMEOUT_MS})`, ) + .option("-v, --verbose", "Render grouped output with file headers") .option("--json", "Emit the JSON envelope") .action( async ( diff --git a/src/commands/mcp-instructions.test.ts b/src/commands/mcp-instructions.test.ts index 2765e2bc..9a49447a 100644 --- a/src/commands/mcp-instructions.test.ts +++ b/src/commands/mcp-instructions.test.ts @@ -50,7 +50,7 @@ const KNOWN_TOOLS = [ "search_status", "list_files", "read_file", - "grep_file", + "grep_repo", "package_summary", "package_vulnerabilities", "package_dependencies", @@ -255,7 +255,7 @@ describe("buildMcpInstructions", () => { "search_status", "list_files", "read_file", - "grep_file", + "grep_repo", "package_summary", "package_vulnerabilities", "package_dependencies", diff --git a/src/commands/mcp-instructions.ts b/src/commands/mcp-instructions.ts index 0d4516b9..77ce823c 100644 --- a/src/commands/mcp-instructions.ts +++ b/src/commands/mcp-instructions.ts @@ -42,16 +42,16 @@ const SEARCH_STATUS_BULLET = "- `search_status` — follow up a prior unified search by `searchRef`. Use it after `search` returns incomplete state to check progress or fetch final results."; const LIST_FILES_BULLET = - "- `list_files` — discover what files a dependency ships. Use `path_prefix` to scope to a subdirectory; the response includes each file's language, type, and byte size. Returned `path` values feed directly into `read_file` and `grep_file`."; + "- `list_files` — discover what files a dependency ships. Use `path_prefix` to scope to a subdirectory; the response includes each file's language, type, and byte size. Returned `path` values feed directly into `read_file` and help scope `grep_repo`."; const READ_FILE_BULLET = "- `read_file` — fetch a file's contents from a dependency. Pass the same `path` emitted by `list_files`. Default returns the full file; pass `start_line` / `end_line` for a bounded range. Binary files set `isBinary: true` and omit `content` — branch on the flag, not the null. A `FILE_NOT_FOUND` (or `NOT_FOUND`) response is the signal to call `list_files` for the actual path."; -const GREP_FILE_BULLET = - "- `grep_file` — find a case-insensitive substring within a single file (not regex). Pass the same `path` emitted by `list_files`. Returns matches with context lines. Max pattern 200 chars, up to 200 matches with up to 10 context lines each. Use unified `search` when you do not yet know the exact file. Same addressing and indexing-retry rules as `list_files`."; +const GREP_REPO_BULLET = + "- `grep_repo` — deterministic text grep over indexed source files. Use it when you know the exact text or regex to match; use `search` for discovery. Whole-target grep is the default; narrow with `path`, `path_prefix`, `globs`, or `extensions`. Returned `matches[].filePath` feeds directly into `read_file`."; const SEARCH_VS_SYMBOLS_TIP = - 'Prefer `get_example` for canonical example retrieval; prefer unified `search` for indexed dependency and repository search; use `sources:["symbol"]` when you want symbol-shaped results. Use `list_files` → `read_file` / `grep_file` once you know the file you need.'; + 'Prefer `get_example` for canonical example retrieval; prefer unified `search` for indexed dependency and repository discovery; use `sources:["symbol"]` when you want symbol-shaped results. Use `grep_repo` for deterministic text matching and `read_file` for full-file inspection.'; /** * Whether the MCP session should register and describe package tools. @@ -101,7 +101,7 @@ export function buildMcpInstructions(deps: Dependencies): string { bullets.push(SEARCH_STATUS_BULLET); bullets.push(LIST_FILES_BULLET); bullets.push(READ_FILE_BULLET); - bullets.push(GREP_FILE_BULLET); + bullets.push(GREP_REPO_BULLET); } if (bullets.length === 0) { diff --git a/src/commands/mcp.test.ts b/src/commands/mcp.test.ts index b7fedb2e..e4a991e7 100644 --- a/src/commands/mcp.test.ts +++ b/src/commands/mcp.test.ts @@ -81,7 +81,7 @@ describe("createMcpServer", () => { "search_status", "list_files", "read_file", - "grep_file", + "grep_repo", ]); }); diff --git a/src/commands/mcp.ts b/src/commands/mcp.ts index 53b42fd9..7d5ac4ac 100644 --- a/src/commands/mcp.ts +++ b/src/commands/mcp.ts @@ -11,7 +11,7 @@ import { import { createFeedbackTool, createGetExampleTool, - createGrepFileTool, + createGrepRepoTool, createListFilesTool, createPackageChangelogTool, createPackageDependenciesTool, @@ -19,8 +19,8 @@ import { createPackageVulnerabilitiesTool, createReadFileTool, createSearchLanguageTool, - createSearchTool, createSearchStatusTool, + createSearchTool, type ToolDefinition, } from "../tools/index.js"; import { @@ -47,7 +47,7 @@ export function getMcpToolDefinitions( tools.push(createSearchStatusTool(deps.codeNavigationService)); tools.push(createListFilesTool(deps.codeNavigationService)); tools.push(createReadFileTool(deps.codeNavigationService)); - tools.push(createGrepFileTool(deps.codeNavigationService)); + tools.push(createGrepRepoTool(deps.codeNavigationService)); } if (gateOpen && deps.packageIntelligenceService) { diff --git a/src/services/code-navigation-service.test.ts b/src/services/code-navigation-service.test.ts index 8f23836c..72b61d24 100644 --- a/src/services/code-navigation-service.test.ts +++ b/src/services/code-navigation-service.test.ts @@ -630,7 +630,7 @@ describe("CodeNavigationServiceImpl", () => { const message = (err as Error).message; expect(message).toContain("Target is still indexing"); expect(message).toContain("usually completes within 30 seconds"); - expect(message).toContain("--wait 60"); + expect(message).toContain("--wait 60000"); expect(message).toContain("wait_timeout_ms: 60000"); expect(message).toContain("idx-123"); } @@ -1015,35 +1015,44 @@ describe("CodeNavigationServiceImpl", () => { }); // ------------------------------------------------------------------ - // grepFile + // grepRepo // ------------------------------------------------------------------ - it("normalises a successful grepRepoFile response", async () => { + it("normalises a successful grepRepo response", async () => { mockFetch(() => Promise.resolve( new Response( JSON.stringify({ data: { - grepRepoFile: { + grepRepo: { matches: [ { - lineNumber: 10, + filePath: "src/index.js", + line: 10, + matchStartByte: 6, + matchEndByte: 13, lineContent: "const app = express();", contextBefore: ["", "// setup"], contextAfter: ["", "app.get();"], + fileContentHash: "abc123", + fileIntent: "production", }, ], + nextCursor: null, totalMatches: 1, hasMore: false, - filePath: "src/index.js", - language: "javascript", - totalLines: 50, + truncatedReason: "NONE", + routeTaken: "CONTENT_INDEX", + filesScanned: 1, + filesInScope: 1, + binaryFilesSkipped: 0, + filesTooLargeSkipped: 0, + uniqueFilesMatched: 1, indexedVersion: "v5.2.1", resolution: { resolvedRef: "v5.2.1", commitSha: "abc", }, - diagnostics: null, codeIndexState: "CURRENT", }, }, @@ -1056,28 +1065,35 @@ describe("CodeNavigationServiceImpl", () => { BASE_URL, createMockTokenProvider(), ); - const result = await service.grepFile({ + const result = await service.grepRepo({ target: { registry: "NPM", packageName: "express" }, - path: "src/index.js", pattern: "middleware", + pathSelectors: [{ kind: "PREFIX", value: "src/" }], }); expect(result.matches.length).toBe(1); - expect(result.matches[0]?.lineNumber).toBe(10); + expect(result.matches[0]?.line).toBe(10); expect(result.totalMatches).toBe(1); - expect(result.filePath).toBe("src/index.js"); + expect(result.routeTaken).toBe("CONTENT_INDEX"); expect(result.resolution?.resolvedRef).toBe("v5.2.1"); }); - it("throws CodeNavigationIndexingError for data-path INDEXING sentinel on grepFile", async () => { + it("throws CodeNavigationIndexingError for data-path INDEXING sentinel on grepRepo", async () => { mockFetch(() => Promise.resolve( new Response( JSON.stringify({ data: { - grepRepoFile: { + grepRepo: { matches: [], + nextCursor: null, totalMatches: 0, hasMore: false, + truncatedReason: "NONE", + filesScanned: 0, + filesInScope: 0, + binaryFilesSkipped: 0, + filesTooLargeSkipped: 0, + uniqueFilesMatched: 0, codeIndexState: "INDEXING", indexingRef: "ref_grep", availableVersions: [{ version: "4.21.0", ref: "v4.21.0" }], @@ -1093,12 +1109,12 @@ describe("CodeNavigationServiceImpl", () => { createMockTokenProvider(), ); try { - await service.grepFile({ + await service.grepRepo({ target: { registry: "NPM", packageName: "express" }, - path: "src/index.js", pattern: "middleware", + pathSelectors: [{ kind: "PREFIX", value: "src/" }], }); - throw new Error("expected grepFile to throw"); + throw new Error("expected grepRepo to throw"); } catch (error) { expect(error).toBeInstanceOf(CodeNavigationIndexingError); expect((error as CodeNavigationIndexingError).indexingRef).toBe( @@ -1107,16 +1123,61 @@ describe("CodeNavigationServiceImpl", () => { } }); - it("sends grepRepoFile variables with the correct shape", async () => { + it("throws CodeNavigationBackendError when backend emits GREP_FILE_TOO_LARGE", async () => { + mockFetch(() => + Promise.resolve( + new Response( + JSON.stringify({ + errors: [ + { + message: "File too large to grep: dist/bundle.js", + extensions: { + code: "GREP_FILE_TOO_LARGE", + file_path: "dist/bundle.js", + }, + }, + ], + }), + { status: 200 }, + ), + ), + ); + const service = new CodeNavigationServiceImpl( + BASE_URL, + createMockTokenProvider(), + ); + try { + await service.grepRepo({ + target: { registry: "NPM", packageName: "express" }, + pattern: "middleware", + pathSelectors: [{ kind: "EXACT", value: "dist/bundle.js" }], + }); + throw new Error("expected grepRepo to throw"); + } catch (error) { + expect(error).toBeInstanceOf(CodeNavigationBackendError); + expect((error as CodeNavigationBackendError).graphqlCode).toBe( + "GREP_FILE_TOO_LARGE", + ); + } + }); + + it("sends grepRepo variables with the correct shape", async () => { const fn = mockFetch(() => Promise.resolve( new Response( JSON.stringify({ data: { - grepRepoFile: { + grepRepo: { matches: [], + nextCursor: null, totalMatches: 0, hasMore: false, + truncatedReason: "NONE", + filesScanned: 0, + filesInScope: 0, + binaryFilesSkipped: 0, + filesTooLargeSkipped: 0, + uniqueFilesMatched: 0, codeIndexState: "CURRENT", }, }, @@ -1129,12 +1190,23 @@ describe("CodeNavigationServiceImpl", () => { BASE_URL, createMockTokenProvider(), ); - await service.grepFile({ + await service.grepRepo({ target: { registry: "NPM", packageName: "express", version: "5.2.1" }, - path: "src/index.js", pattern: "middleware", - contextLines: 5, + patternType: "REGEX", + caseSensitive: true, + pathSelectors: [ + { kind: "EXACT", value: "src/index.js" }, + { kind: "GLOB", value: "src/**/*.js" }, + ], + extensions: ["js"], + excludeDocFiles: true, + excludeTestFiles: true, + contextLinesBefore: 5, + contextLinesAfter: 2, maxMatches: 100, + maxMatchesPerFile: 3, + cursor: "cursor-123", waitTimeoutMs: 5000, }); const [, init] = fn.mock.calls[0] as unknown as [string, RequestInit]; @@ -1143,10 +1215,21 @@ describe("CodeNavigationServiceImpl", () => { registry: "NPM", packageName: "express", version: "5.2.1", - filePath: "src/index.js", pattern: "middleware", - contextLines: 5, + patternType: "REGEX", + caseSensitive: true, + pathSelectors: [ + { kind: "EXACT", value: "src/index.js" }, + { kind: "GLOB", value: "src/**/*.js" }, + ], + extensions: ["js"], + excludeDocFiles: true, + excludeTestFiles: true, + contextLinesBefore: 5, + contextLinesAfter: 2, maxMatches: 100, + maxMatchesPerFile: 3, + cursor: "cursor-123", waitTimeoutMs: 5000, }); }); diff --git a/src/services/code-navigation-service.ts b/src/services/code-navigation-service.ts index 0cb1120b..bd3e3771 100644 --- a/src/services/code-navigation-service.ts +++ b/src/services/code-navigation-service.ts @@ -342,39 +342,67 @@ export interface ReadFileResult { isBinary?: boolean; } -/** - * Input for {@link CodeNavigationService.grepFile}. - * - * `path` today addresses a single file; naming it `path` (rather - * than `filePath`) leaves room for the same slot to accept broader - * shapes later without a rename. - */ -export interface GrepFileParams { +export type GrepRepoPatternType = "LITERAL" | "REGEX"; + +export type GrepPathSelectorKind = "EXACT" | "PREFIX" | "GLOB"; + +export interface GrepRepoPathSelector { + kind: GrepPathSelectorKind; + value: string; +} + +export interface GrepRepoParams { target: CodeNavigationTarget; - path: string; pattern: string; - contextLines?: number; + patternType?: GrepRepoPatternType; + caseSensitive?: boolean; + pathSelectors?: GrepRepoPathSelector[]; + extensions?: string[]; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + allowUnscoped?: boolean; + contextLinesBefore?: number; + contextLinesAfter?: number; maxMatches?: number; + maxMatchesPerFile?: number; + cursor?: string; waitTimeoutMs?: number; } -export interface GrepMatch { - lineNumber: number; +export interface GrepRepoMatch { + filePath: string; + line: number; + matchStartByte: number; + matchEndByte: number; lineContent: string; contextBefore?: string[]; contextAfter?: string[]; + fileContentHash?: string; + fileIntent?: string; } -export interface GrepFileResult { - matches: GrepMatch[]; - totalMatches: number; +export type GrepRouteTaken = "SINGLE_FILE" | "CONTENT_INDEX"; + +export type GrepTruncatedReason = + | "NONE" + | "MAX_MATCHES" + | "MAX_MATCHES_PER_FILE" + | "DEADLINE"; + +export interface GrepRepoResult { + matches: GrepRepoMatch[]; + nextCursor?: string; hasMore: boolean; - filePath?: string; - language?: string; - totalLines?: number; + truncatedReason: GrepTruncatedReason; + routeTaken?: GrepRouteTaken; + filesScanned: number; + filesInScope: number; + binaryFilesSkipped: number; + filesTooLargeSkipped: number; + totalMatches: number; + uniqueFilesMatched: number; indexedVersion?: string; resolution?: SearchSymbolsResolution; - hint?: string; } export interface CodeNavigationService { @@ -383,7 +411,7 @@ export interface CodeNavigationService { searchSymbols(params: SearchSymbolsParams): Promise; listFiles(params: ListFilesParams): Promise; readFile(params: ReadFileParams): Promise; - grepFile(params: GrepFileParams): Promise; + grepRepo(params: GrepRepoParams): Promise; } export class CodeNavigationAccessError extends Error { @@ -920,7 +948,7 @@ const graphQLErrorSchema = z.object({ // -------------------------------------------------------------------- // Zod schemas + queries for the file-exploration bundle. -// `listRepoFiles` / `fetchCodeContext` / `grepRepoFile` share the same +// `listRepoFiles` / `fetchCodeContext` / `grepRepo` share the same // indexing lifecycle (`codeIndexState` + `indexingRef` + // `availableVersions`) but otherwise have distinct result shapes — // normalise per tool rather than under one abstraction. @@ -1089,76 +1117,118 @@ query FetchCodeContext( } }`; -// grepRepoFile ------------------------------------------------------- +// grepRepo ----------------------------------------------------------- -const grepMatchSchema = z.object({ - lineNumber: z.number().int(), +const grepRepoMatchSchema = z.object({ + filePath: z.string(), + line: z.number().int(), + matchStartByte: z.number().int(), + matchEndByte: z.number().int(), lineContent: z.string(), contextBefore: z.array(z.string()).nullable().optional(), contextAfter: z.array(z.string()).nullable().optional(), + fileContentHash: z.string().nullable().optional(), + fileIntent: z.string().nullable().optional(), }); -const grepRepoFileResponseSchema = z.object({ - matches: z.array(grepMatchSchema), - totalMatches: z.number().int(), +const grepRepoResponseSchema = z.object({ + matches: z.array(grepRepoMatchSchema), + nextCursor: z.string().nullable().optional(), hasMore: z.boolean(), - filePath: z.string().nullable().optional(), - language: z.string().nullable().optional(), - totalLines: z.number().int().nullable().optional(), + truncatedReason: z.enum([ + "NONE", + "MAX_MATCHES", + "MAX_MATCHES_PER_FILE", + "DEADLINE", + ]), + routeTaken: z.enum(["SINGLE_FILE", "CONTENT_INDEX"]).nullable().optional(), + filesScanned: z.number().int(), + filesInScope: z.number().int(), + binaryFilesSkipped: z.number().int(), + filesTooLargeSkipped: z.number().int(), + totalMatches: z.number().int(), + uniqueFilesMatched: z.number().int(), indexedVersion: z.string().nullable().optional(), resolution: navigationResolutionSchema, - diagnostics: navigationDiagnosticsSchema, codeIndexState: z.string(), indexingRef: z.string().nullable().optional(), availableVersions: z.array(availableVersionSchema).nullable().optional(), }); -const grepRepoFileGraphQLResponseSchema = z.object({ +const grepRepoGraphQLResponseSchema = z.object({ data: z .object({ - grepRepoFile: grepRepoFileResponseSchema.nullable().optional(), + grepRepo: grepRepoResponseSchema.nullable().optional(), }) .nullable() .optional(), errors: z.array(graphQLErrorSchema).optional(), }); -const GREP_REPO_FILE_QUERY = ` -query GrepRepoFile( +const GREP_REPO_QUERY = ` +query GrepRepo( $registry: Registry $packageName: String $repoUrl: String $gitRef: String - $filePath: String! - $pattern: String! - $contextLines: Int - $maxMatches: Int $version: String $waitTimeoutMs: Int + $pattern: String! + $patternType: GrepPatternType + $caseSensitive: Boolean + $pathSelectors: [GrepPathSelectorInput!] + $extensions: [String!] + $excludeDocFiles: Boolean + $excludeTestFiles: Boolean + $allowUnscoped: Boolean + $contextLinesBefore: Int + $contextLinesAfter: Int + $maxMatches: Int + $maxMatchesPerFile: Int + $cursor: String ) { - grepRepoFile( + grepRepo( registry: $registry packageName: $packageName repoUrl: $repoUrl gitRef: $gitRef - filePath: $filePath - pattern: $pattern - contextLines: $contextLines - maxMatches: $maxMatches version: $version waitTimeoutMs: $waitTimeoutMs + pattern: $pattern + patternType: $patternType + caseSensitive: $caseSensitive + pathSelectors: $pathSelectors + extensions: $extensions + excludeDocFiles: $excludeDocFiles + excludeTestFiles: $excludeTestFiles + allowUnscoped: $allowUnscoped + contextLinesBefore: $contextLinesBefore + contextLinesAfter: $contextLinesAfter + maxMatches: $maxMatches + maxMatchesPerFile: $maxMatchesPerFile + cursor: $cursor ) { matches { - lineNumber + filePath + line + matchStartByte + matchEndByte lineContent contextBefore contextAfter + fileContentHash + fileIntent } + nextCursor totalMatches hasMore - filePath - language - totalLines + truncatedReason + routeTaken + filesScanned + filesInScope + binaryFilesSkipped + filesTooLargeSkipped + uniqueFilesMatched indexedVersion resolution { requestedVersion @@ -1166,9 +1236,6 @@ query GrepRepoFile( resolvedRef commitSha } - diagnostics { - hint - } codeIndexState indexingRef availableVersions { @@ -1204,7 +1271,9 @@ const unifiedSearchGraphQLResponseSchema = z.object({ const unifiedSearchStatusGraphQLResponseSchema = z.object({ data: z .object({ - discoverySearchProgress: unifiedSearchProgressSchema.nullable().optional(), + discoverySearchProgress: unifiedSearchProgressSchema + .nullable() + .optional(), }) .nullable() .optional(), @@ -1569,6 +1638,21 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { parseAvailableVersions(extensions), ); + case "GREP_PATTERN_TOO_SHORT": + case "GREP_PATTERN_TOO_LONG": + case "GREP_PATTERN_INVALID": + case "GREP_INVALID_REGEX": + case "GREP_UNSUPPORTED_PATTERN": + case "GREP_PATTERN_TOO_UNSELECTIVE": + case "GREP_SCOPE_REQUIRED": + case "GREP_SELECTOR_INVALID": + case "GREP_CURSOR_INVALID": + case "GREP_CONTEXT_TOO_LARGE": + case "GREP_CONTEXT_NEGATIVE": + case "GREP_MAX_MATCHES_TOO_LARGE": + case "GREP_MAX_MATCHES_INVALID": + return new CodeNavigationValidationError(message); + case "VERSION_NOT_FOUND": return new CodeNavigationVersionNotFoundError( message, @@ -1619,6 +1703,11 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { case "UPSTREAM_ERROR": case "TIMEOUT": case "RATE_LIMITED": + case "GREP_FILE_TOO_LARGE": + case "GREP_TIMEOUT": + case "GREP_SERVICE_UNAVAILABLE": + case "GREP_FAILED": + case "GREP_INDEX_NOT_AVAILABLE": case "INTERNAL_ERROR": case "UNKNOWN_ERROR": return new CodeNavigationBackendError( @@ -1660,7 +1749,7 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { // s. Give callers both a concrete "retry shortly" expectation and // the option to block until ready via a longer wait timeout. const base = - "Target is still indexing. Indexing usually completes within 30 seconds. Retry this request, or pass a longer wait timeout (CLI: `--wait 60`, MCP: `wait_timeout_ms: 60000`) to block until ready."; + "Target is still indexing. Indexing usually completes within 30 seconds. Retry this request, or pass a longer wait timeout (CLI: `--wait 60000`, MCP: `wait_timeout_ms: 60000`) to block until ready."; if (indexingRef) { return `${base} Indexing reference: ${indexingRef}.`; } @@ -1984,39 +2073,51 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { } // ------------------------------------------------------------------ - // grepFile → grepRepoFile + // grepRepo // ------------------------------------------------------------------ - async grepFile(params: GrepFileParams): Promise { + async grepRepo(params: GrepRepoParams): Promise { return executeWithTokenRefresh({ getToken: () => this.tokenProvider.getToken(), forceRefresh: () => this.tokenProvider.forceRefresh(), shouldRefresh: (error) => error instanceof AuthenticationError, - executeWithToken: (token) => this.executeGrepFile(token, params), + executeWithToken: (token) => this.executeGrepRepo(token, params), }); } - private async executeGrepFile( + private async executeGrepRepo( token: string, - params: GrepFileParams, - ): Promise { + params: GrepRepoParams, + ): Promise { let response: PkgseerGraphqlResponse; try { response = await postPkgseerGraphql({ endpointUrl: this.codeNavigationUrl, token, - query: GREP_REPO_FILE_QUERY, + query: GREP_REPO_QUERY, variables: { registry: params.target.registry, packageName: params.target.packageName, repoUrl: params.target.repoUrl, gitRef: params.target.gitRef, version: params.target.version, - filePath: params.path, + waitTimeoutMs: params.waitTimeoutMs, pattern: params.pattern, - contextLines: params.contextLines, + patternType: params.patternType, + caseSensitive: params.caseSensitive, + pathSelectors: params.pathSelectors?.map((entry) => ({ + kind: entry.kind, + value: entry.value, + })), + extensions: params.extensions, + excludeDocFiles: params.excludeDocFiles, + excludeTestFiles: params.excludeTestFiles, + allowUnscoped: params.allowUnscoped, + contextLinesBefore: params.contextLinesBefore, + contextLinesAfter: params.contextLinesAfter, maxMatches: params.maxMatches, - waitTimeoutMs: params.waitTimeoutMs, + maxMatchesPerFile: params.maxMatchesPerFile, + cursor: params.cursor, }, fetchFn: this.fetchFn, }); @@ -2034,9 +2135,7 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { throw this.createHttpError(response); } - const parsed = grepRepoFileGraphQLResponseSchema.safeParse( - response.parsedBody, - ); + const parsed = grepRepoGraphQLResponseSchema.safeParse(response.parsedBody); if (!parsed.success) { throw new MalformedCodeNavigationResponseError( "Malformed response from code navigation service.", @@ -2047,7 +2146,7 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { throw this.createGraphQLError(parsed.data.errors); } - const data = parsed.data.data?.grepRepoFile; + const data = parsed.data.data?.grepRepo; if (!data) { throw new MalformedCodeNavigationResponseError( "Malformed response from code navigation service.", @@ -2058,16 +2157,26 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { return { matches: data.matches.map((entry) => ({ - lineNumber: entry.lineNumber, + filePath: entry.filePath, + line: entry.line, + matchStartByte: entry.matchStartByte, + matchEndByte: entry.matchEndByte, lineContent: entry.lineContent, contextBefore: entry.contextBefore ?? undefined, contextAfter: entry.contextAfter ?? undefined, + fileContentHash: entry.fileContentHash ?? undefined, + fileIntent: entry.fileIntent ?? undefined, })), - totalMatches: data.totalMatches, + nextCursor: data.nextCursor ?? undefined, hasMore: data.hasMore, - filePath: data.filePath ?? undefined, - language: data.language ?? undefined, - totalLines: data.totalLines ?? undefined, + truncatedReason: data.truncatedReason, + routeTaken: data.routeTaken ?? undefined, + filesScanned: data.filesScanned, + filesInScope: data.filesInScope, + binaryFilesSkipped: data.binaryFilesSkipped, + filesTooLargeSkipped: data.filesTooLargeSkipped, + totalMatches: data.totalMatches, + uniqueFilesMatched: data.uniqueFilesMatched, indexedVersion: data.indexedVersion ?? undefined, resolution: data.resolution ? { @@ -2077,7 +2186,6 @@ export class CodeNavigationServiceImpl implements CodeNavigationService { commitSha: data.resolution.commitSha ?? undefined, } : undefined, - hint: data.diagnostics?.hint ?? undefined, }; } } diff --git a/src/services/index.ts b/src/services/index.ts index e1be937a..b520890a 100644 --- a/src/services/index.ts +++ b/src/services/index.ts @@ -29,14 +29,27 @@ export type { CodeNavigationRegistry, CodeNavigationService, CodeNavigationTarget, - GrepFileParams, - GrepFileResult, - GrepMatch, + GrepPathSelectorKind, + GrepRepoMatch, + GrepRepoParams, + GrepRepoPathSelector, + GrepRepoPatternType, + GrepRepoResult, + GrepRouteTaken, + GrepTruncatedReason, ListFilesParams, ListFilesResult, ReadFileParams, ReadFileResult, RepoFileEntry, + SearchSymbolsFileIntent, + SearchSymbolsKind, + SearchSymbolsMatchMode, + SearchSymbolsParams, + SearchSymbolsResolution, + SearchSymbolsResult, + SearchSymbolsResultEntry, + SymbolCategory, UnifiedSearchCompleted, UnifiedSearchFilters, UnifiedSearchHit, @@ -51,14 +64,6 @@ export type { UnifiedSearchSessionStatus, UnifiedSearchSource, UnifiedSearchSourceStatus, - SearchSymbolsFileIntent, - SearchSymbolsKind, - SearchSymbolsMatchMode, - SearchSymbolsParams, - SearchSymbolsResolution, - SearchSymbolsResult, - SearchSymbolsResultEntry, - SymbolCategory, } from "./code-navigation-service.js"; export { CodeNavigationAccessError, diff --git a/src/services/test-helpers.ts b/src/services/test-helpers.ts index 4f4f92a8..7b5b84e9 100644 --- a/src/services/test-helpers.ts +++ b/src/services/test-helpers.ts @@ -14,8 +14,9 @@ import type { import type { BrowserService } from "./browser-service.js"; import type { CodeNavigationService, - UnifiedSearchOutcome, + GrepRepoResult, SearchSymbolsResult, + UnifiedSearchOutcome, } from "./code-navigation-service.js"; import type { ExecResult, ExecService } from "./exec-service.js"; import type { FileSystemService } from "./filesystem-service.js"; @@ -330,20 +331,30 @@ export const defaultReadFileResult = { isBinary: false, }; -export const defaultGrepFileResult = { +export const defaultGrepRepoResult: GrepRepoResult = { matches: [ { - lineNumber: 4, + filePath: "src/index.js", + line: 4, + matchStartByte: 17, + matchEndByte: 24, lineContent: "module.exports = require('./lib/express');", contextBefore: ["// Express entry point", "'use strict';", ""], contextAfter: [""], + fileContentHash: "abc123", + fileIntent: "production", }, ], - totalMatches: 1, + nextCursor: undefined, hasMore: false, - filePath: "src/index.js", - language: "javascript", - totalLines: 5, + truncatedReason: "NONE", + routeTaken: "CONTENT_INDEX", + filesScanned: 1, + filesInScope: 1, + binaryFilesSkipped: 0, + filesTooLargeSkipped: 0, + totalMatches: 1, + uniqueFilesMatched: 1, indexedVersion: "v5.2.1", resolution: { requestedVersion: undefined, @@ -351,7 +362,6 @@ export const defaultGrepFileResult = { resolvedRef: "v5.2.1", commitSha: "abc123", }, - hint: undefined, }; /** @@ -366,7 +376,7 @@ export function createMockCodeNavigationService( searchSymbols: mock(() => Promise.resolve(defaultSearchSymbolsResult)), listFiles: mock(() => Promise.resolve(defaultListFilesResult)), readFile: mock(() => Promise.resolve(defaultReadFileResult)), - grepFile: mock(() => Promise.resolve(defaultGrepFileResult)), + grepRepo: mock(() => Promise.resolve(defaultGrepRepoResult)), ...impl, }; } diff --git a/src/shared/grep-file-request.test.ts b/src/shared/grep-file-request.test.ts deleted file mode 100644 index d35c8b04..00000000 --- a/src/shared/grep-file-request.test.ts +++ /dev/null @@ -1,162 +0,0 @@ -import { describe, expect, it } from "bun:test"; -import type { CodeNavigationTarget } from "../services/index.js"; -import { - buildGrepFileParams, - GREP_PATTERN_SEMANTICS_NOTE, - looksLikeRegexAttempt, -} from "./grep-file-request.js"; - -const target: CodeNavigationTarget = { - registry: "NPM", - packageName: "express", -}; - -describe("buildGrepFileParams — defaults + happy path", () => { - it("applies defaults for context (0) and max_matches (50) and wait (20000)", () => { - const { params } = buildGrepFileParams({ - target, - path: "src/index.js", - pattern: "middleware", - }); - expect(params.contextLines).toBe(0); - expect(params.maxMatches).toBe(50); - expect(params.waitTimeoutMs).toBe(20000); - }); - - it("passes explicit values through and marks them explicit", () => { - const { params, contextLinesExplicit, maxMatchesExplicit } = - buildGrepFileParams({ - target, - path: "src/index.js", - pattern: "middleware", - contextLines: 5, - maxMatches: 100, - }); - expect(params.contextLines).toBe(5); - expect(params.maxMatches).toBe(100); - expect(contextLinesExplicit).toBe(true); - expect(maxMatchesExplicit).toBe(true); - }); - - it("trims the path", () => { - const { params } = buildGrepFileParams({ - target, - path: " src/index.js ", - pattern: "middleware", - }); - expect(params.path).toBe("src/index.js"); - }); -}); - -describe("buildGrepFileParams — rejection cases", () => { - it("rejects empty path", () => { - expect(() => - buildGrepFileParams({ - target, - path: " ", - pattern: "middleware", - }), - ).toThrow(/`path` is required/); - }); - - it("rejects empty pattern", () => { - expect(() => - buildGrepFileParams({ - target, - path: "src/index.js", - pattern: "", - }), - ).toThrow(/`pattern` is required/); - }); - - it("rejects pattern over 200 characters", () => { - expect(() => - buildGrepFileParams({ - target, - path: "src/index.js", - pattern: "a".repeat(201), - }), - ).toThrow(/≤ 200 characters/); - }); - - it.each([ - -1, 11, 3.5, - ])("rejects out-of-range contextLines %s", (contextLines) => { - expect(() => - buildGrepFileParams({ - target, - path: "src/index.js", - pattern: "middleware", - contextLines, - }), - ).toThrow(/between 0 and 10/); - }); - - it.each([0, 201, 3.5])("rejects out-of-range maxMatches %s", (maxMatches) => { - expect(() => - buildGrepFileParams({ - target, - path: "src/index.js", - pattern: "middleware", - maxMatches, - }), - ).toThrow(/between 1 and 200/); - }); -}); - -describe("GREP_PATTERN_SEMANTICS_NOTE constant", () => { - it("mentions case-insensitive, substring, not-regex, and 200-char cap", () => { - expect(GREP_PATTERN_SEMANTICS_NOTE).toMatch(/case-insensitive/i); - expect(GREP_PATTERN_SEMANTICS_NOTE).toMatch(/substring/i); - expect(GREP_PATTERN_SEMANTICS_NOTE).toMatch(/NOT regex|not regex/i); - expect(GREP_PATTERN_SEMANTICS_NOTE).toMatch(/200/); - }); -}); - -describe("looksLikeRegexAttempt — narrow heuristic", () => { - it.each([ - "\\bfoo\\b", - "\\Bnot-boundary", - "\\w+", - "\\W+", - "\\d{3}", - "\\D", - "\\s", - "\\S", - "\\.foo", - "\\/path", - "\\(captured\\)", - "\\[bracket\\]", - "[abc]", - "[a-z]", - "(?:foo|bar)", - "(?=bar)", - "(?!bar)", - "(?<=foo)bar", - "(?foo)", - "(?i)case", - "\\\\path", - "a{3}", - "b{2,5}", - "c{2,}", - ])("flags '%s' as a regex attempt", (pattern) => { - expect(looksLikeRegexAttempt(pattern)).toBe(true); - }); - - it.each([ - "foo.bar", // dot in filename — common, not a regex signal - "*.js", // glob — not regex - "hello world", - "ENOTFOUND", - "price?", - "a+b", - "^start", // raw ^ — too common in code to flag - "end$", // raw $ — too common as variable / shell / regex end - "foo|bar", // alternation — common as literal OR text - "middleware()", // parens as function call, not regex group - "{count: 5}", // braces in object literals - ])("does not flag '%s' (avoids false positives)", (pattern) => { - expect(looksLikeRegexAttempt(pattern)).toBe(false); - }); -}); diff --git a/src/shared/grep-file-request.ts b/src/shared/grep-file-request.ts deleted file mode 100644 index 0fa621a1..00000000 --- a/src/shared/grep-file-request.ts +++ /dev/null @@ -1,159 +0,0 @@ -/** - * Shared request builder for `grep_file`. CLI and MCP normalise - * inputs here so the two surfaces cannot diverge on pattern / - * context / match-count bounds. - * - * The `path` argument deliberately keeps its generic name (rather - * than `file_path`) to leave room for broader shapes later; today - * it addresses a single file. - */ - -import type { - CodeNavigationTarget, - GrepFileParams, -} from "../services/index.js"; -import { DEFAULT_WAIT_TIMEOUT_MS } from "./code-navigation-defaults.js"; -import { InvalidPackageSpecError } from "./package-spec.js"; - -const PATTERN_MAX = 200; -const CONTEXT_MIN = 0; -const CONTEXT_MAX = 10; -// Default is no context — matches-only output stays pipe-friendly -// (`grep -o`-style) and token-efficient for agents. Callers opt into -// context explicitly. -const CONTEXT_DEFAULT = 0; -const LIMIT_MIN = 1; -const LIMIT_MAX = 200; -const LIMIT_DEFAULT = 50; -const WAIT_MIN = 0; -const WAIT_MAX = 60_000; - -/** - * The pattern-semantics disclosure. Shared verbatim across the MCP - * tool description, the MCP `pattern` arg describe, and the CLI - * help text so the three surfaces never disagree. - */ -export const GREP_PATTERN_SEMANTICS_NOTE = - "Case-insensitive substring matching. NOT regex — `\\b`, `^`, `.`, etc. match literally. Max 200 characters."; - -/** - * Regex-like metacharacters that almost always signal a deliberate - * regex attempt. Bare `.`, `*`, `+`, `?`, `^`, `$`, `|`, `(`, `)` - * are intentionally NOT in this set — they show up in ordinary - * filenames / words / code fragments (`foo.bar`, `*.js`, `$foo`, - * `middleware()`, `a|b`), and firing the hint on those would - * create noise. - * - * Covered (deliberate regex signals only): - * - Escape classes: `\b \B \w \W \d \D \s \S`. - * - Escaped regex metacharacters: `\. \/ \( \) \[ \] \{ \} \+ \* - * \? \^ \$ \|` — these are almost never intentional literal - * searches; the user is escaping because they know regex syntax. - * - Double backslash (raw regex string common form). - * - Character class: `[...]`. - * - Non-capturing / lookaround / named group / inline flags: - * `(?:...)`, `(?=...)`, `(?!...)`, `(?<=...)`, `(?...)`, `(?i)` etc. - * - Brace quantifier: `{N}`, `{N,}`, `{N,M}`. - * - * Best-effort: a pattern like `foo|bar` or `^start` won't fire, - * but the backend's "case-insensitive substring" hint still plays - * on zero-match responses, so the user isn't left in the dark. - */ -const REGEX_SIGNAL_PATTERNS: readonly RegExp[] = [ - /\\[bBwWdDsS]/, // word/digit/whitespace/boundary escapes (both cases) - /\\[./(){}[\]+*?^$|]/, // escaped regex metacharacters - /\\\\/, // double backslash - /\[[^\]]*\]/, // character class - /\(\?[=!: re.test(pattern)); -} - -export interface GrepFileRequestInput { - target: CodeNavigationTarget; - path: string; - pattern: string; - contextLines?: number; - maxMatches?: number; - waitTimeoutMs?: number; -} - -export interface GrepFileRequestBuildResult { - params: GrepFileParams; - contextLinesExplicit: boolean; - maxMatchesExplicit: boolean; -} - -export function buildGrepFileParams( - input: GrepFileRequestInput, -): GrepFileRequestBuildResult { - const path = input.path?.trim() ?? ""; - if (!path) { - throw new InvalidPackageSpecError( - "`path` is required — pass the path to the file within the package or repo.", - ); - } - - const pattern = input.pattern ?? ""; - if (pattern.length === 0) { - throw new InvalidPackageSpecError( - "`pattern` is required — pass the substring to search for.", - ); - } - if (pattern.length > PATTERN_MAX) { - throw new InvalidPackageSpecError( - `\`pattern\` must be ≤ ${PATTERN_MAX} characters. Got ${pattern.length}.`, - ); - } - - const contextLines = normaliseContextLines(input.contextLines); - const maxMatches = normaliseMaxMatches(input.maxMatches); - const waitTimeoutMs = normaliseWaitTimeoutMs(input.waitTimeoutMs); - - return { - params: { - target: input.target, - path, - pattern, - contextLines, - maxMatches, - waitTimeoutMs, - }, - contextLinesExplicit: input.contextLines !== undefined, - maxMatchesExplicit: input.maxMatches !== undefined, - }; -} - -function normaliseContextLines(raw: number | undefined): number { - if (raw === undefined) return CONTEXT_DEFAULT; - if (!Number.isInteger(raw) || raw < CONTEXT_MIN || raw > CONTEXT_MAX) { - throw new InvalidPackageSpecError( - `\`context_lines\` must be an integer between ${CONTEXT_MIN} and ${CONTEXT_MAX}. Got ${raw}.`, - ); - } - return raw; -} - -function normaliseMaxMatches(raw: number | undefined): number { - if (raw === undefined) return LIMIT_DEFAULT; - if (!Number.isInteger(raw) || raw < LIMIT_MIN || raw > LIMIT_MAX) { - throw new InvalidPackageSpecError( - `\`max_matches\` must be an integer between ${LIMIT_MIN} and ${LIMIT_MAX}. Got ${raw}.`, - ); - } - return raw; -} - -function normaliseWaitTimeoutMs(raw: number | undefined): number { - if (raw === undefined) return DEFAULT_WAIT_TIMEOUT_MS; - if (!Number.isInteger(raw) || raw < WAIT_MIN || raw > WAIT_MAX) { - throw new InvalidPackageSpecError( - `\`wait_timeout_ms\` must be an integer between ${WAIT_MIN} and ${WAIT_MAX}. Got ${raw}.`, - ); - } - return raw; -} diff --git a/src/shared/grep-file-response.test.ts b/src/shared/grep-file-response.test.ts deleted file mode 100644 index 074ebaeb..00000000 --- a/src/shared/grep-file-response.test.ts +++ /dev/null @@ -1,445 +0,0 @@ -import { describe, expect, it } from "bun:test"; -import type { GrepFileResult } from "../services/index.js"; -import { - buildGrepFileSuccessPayload, - formatGrepFileTerminal, -} from "./grep-file-response.js"; - -const baseResult: GrepFileResult = { - matches: [ - { - lineNumber: 10, - lineContent: "const app = express();", - contextBefore: ["", "// set up express", ""], - contextAfter: ["", "app.get('/', …);"], - }, - ], - totalMatches: 1, - hasMore: false, - filePath: "src/index.js", - language: "javascript", - totalLines: 50, - indexedVersion: "v5.2.1", - resolution: { - resolvedRef: "v5.2.1", - commitSha: "abc123def", - }, - hint: undefined, -}; - -const baseOptions = { - registry: "npm", - name: "express", - pattern: "express()", - path: "src/index.js", - contextLinesExplicit: false, - maxMatchesExplicit: false, - contextLines: 2, - maxMatches: 50, -}; - -describe("buildGrepFileSuccessPayload", () => { - it("projects the envelope shape", () => { - const envelope = buildGrepFileSuccessPayload(baseResult, baseOptions); - expect(envelope.registry).toBe("npm"); - expect(envelope.name).toBe("express"); - expect(envelope.pattern).toBe("express()"); - expect(envelope.path).toBe("src/index.js"); - expect(envelope.totalMatches).toBe(1); - expect(envelope.hasMore).toBe(false); - expect(envelope.matches.length).toBe(1); - expect(envelope.matches[0]?.lineNumber).toBe(10); - expect(envelope.indexedVersion).toBe("v5.2.1"); - expect(envelope.filter).toBeUndefined(); - }); - - it("echoes filter.contextLines and filter.maxMatches when explicit", () => { - const envelope = buildGrepFileSuccessPayload(baseResult, { - ...baseOptions, - contextLines: 5, - maxMatches: 100, - contextLinesExplicit: true, - maxMatchesExplicit: true, - }); - expect(envelope.filter).toEqual({ contextLines: 5, maxMatches: 100 }); - }); - - it("does not echo filter when defaults are used", () => { - const envelope = buildGrepFileSuccessPayload(baseResult, baseOptions); - expect(envelope.filter).toBeUndefined(); - }); - - it("strips empty context arrays", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - matches: [ - { - lineNumber: 10, - lineContent: "const app = express();", - contextBefore: [], - contextAfter: [], - }, - ], - }, - baseOptions, - ); - expect(envelope.matches[0]?.contextBefore).toBeUndefined(); - expect(envelope.matches[0]?.contextAfter).toBeUndefined(); - }); - - it("surfaces hint on empty results", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - matches: [], - totalMatches: 0, - hint: "Pattern not found in file.", - }, - baseOptions, - ); - expect(envelope.matches).toEqual([]); - expect(envelope.hint).toBe("Pattern not found in file."); - }); - - it("surfaces repo-URL addressing", () => { - const envelope = buildGrepFileSuccessPayload(baseResult, { - ...baseOptions, - registry: undefined, - name: undefined, - repoUrl: "https://github.com/expressjs/express", - gitRef: "main", - }); - expect(envelope.repoUrl).toBe("https://github.com/expressjs/express"); - expect(envelope.gitRef).toBe("main"); - }); -}); - -describe("formatGrepFileTerminal", () => { - it("plain mode: emits matching lines only — no header, no gutter", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - matches: [ - { - lineNumber: 10, - lineContent: "const app = express();", - contextBefore: [], - contextAfter: [], - }, - ], - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - }); - expect(output).toContain("const app = express();"); - expect(output).not.toContain("express · npm"); - expect(output).not.toMatch(/^>/m); - // No line number prefix in plain mode. - expect(output).not.toMatch(/^\s*10\s+const/m); - }); - - it("plain mode: zero matches → completely silent (matches grep's exit-1-with-no-output convention)", () => { - const envelope = buildGrepFileSuccessPayload( - { ...baseResult, matches: [], totalMatches: 0 }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - }); - expect(output).toBe(""); - }); - - it("plain mode with context: merges overlapping blocks into a single block", () => { - // Two matches at lines 10 and 12 with contextBefore/After=2 each. - // The contexts for match-1 [8..12] and match-2 [10..14] overlap. - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 2, - matches: [ - { - lineNumber: 10, - lineContent: "match one", - contextBefore: ["line 8", "line 9"], - contextAfter: ["line 11", "match two"], - }, - { - lineNumber: 12, - lineContent: "match two", - contextBefore: ["line 10 (dup)", "line 11 (dup)"], - contextAfter: ["line 13", "line 14"], - }, - ], - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - }); - // Every context line must appear exactly once. - expect(output.match(/line 8/g)?.length).toBe(1); - expect(output.match(/line 9/g)?.length).toBe(1); - expect(output.match(/line 11/g)?.length).toBe(1); - expect(output.match(/line 13/g)?.length).toBe(1); - expect(output.match(/line 14/g)?.length).toBe(1); - // Match-1 content (line 10) and match-2 content (line 12) both - // appear once — the context duplicate at line 10 in match-2's - // contextBefore must not overwrite the match line content. - expect(output.match(/match one/g)?.length).toBe(1); - expect(output.match(/match two/g)?.length).toBe(1); - // No `--` separator because both matches merged into a single - // block. - expect(output).not.toContain("--"); - }); - - it("plain mode with context: inserts `--` separator between distinct blocks", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 2, - matches: [ - { - lineNumber: 5, - lineContent: "match near top", - contextBefore: ["line 4"], - contextAfter: ["line 6"], - }, - { - lineNumber: 50, - lineContent: "match far below", - contextBefore: ["line 49"], - contextAfter: ["line 51"], - }, - ], - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - }); - expect(output).toContain("--"); - expect(output).toContain("line 4"); - expect(output).toContain("line 51"); - }); - - it("verbose mode: renders header + gutter + `>` marker on match lines", () => { - const envelope = buildGrepFileSuccessPayload(baseResult, baseOptions); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - expect(output).toContain("express · npm · 1 match in src/index.js"); - expect(output).toContain("> 10 const app = express();"); - expect(output).toContain("// set up express"); - expect(output).toContain("app.get('/', …);"); - }); - - it("verbose mode: renders plural 'matches' for counts ≠ 1", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 3, - matches: Array.from({ length: 3 }, (_, i) => ({ - lineNumber: 10 + i, - lineContent: `line ${10 + i}`, - })), - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - expect(output).toContain("3 matches"); - }); - - it("verbose mode: uses N+ header when hasMore is true", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 50, - hasMore: true, - matches: Array.from({ length: 50 }, (_, i) => ({ - lineNumber: i + 1, - lineContent: `line ${i + 1}`, - })), - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - expect(output).toContain("50+ matches"); - expect(output).toContain("More matches available"); - }); - - it("verbose mode: empty-result + regex-char hint when pattern looks like regex", () => { - const envelope = buildGrepFileSuccessPayload( - { ...baseResult, matches: [], totalMatches: 0 }, - { ...baseOptions, pattern: "\\bfoo\\b" }, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - expect(output).toContain("No matches for '\\bfoo\\b'"); - expect(output).toContain("literal substring matching"); - }); - - it("verbose mode: does not add the regex hint when pattern doesn't look like regex", () => { - const envelope = buildGrepFileSuccessPayload( - { ...baseResult, matches: [], totalMatches: 0 }, - { ...baseOptions, pattern: "foo.bar" }, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - expect(output).toContain("No matches for 'foo.bar'"); - expect(output).not.toContain("literal substring matching"); - }); - - it("verbose mode: uses server-supplied hint when present", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - matches: [], - totalMatches: 0, - hint: "Check the file path — we indexed this repo but the path didn't match.", - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - expect(output).toContain("Check the file path"); - }); - - it("plain mode: handles unsorted matches from the backend — final output is line-number sorted", () => { - // Defensive: the backend should return matches sorted by line - // number, but the merger has no dependency on arrival order. - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 2, - matches: [ - { - lineNumber: 80, - lineContent: "match two", - contextBefore: [], - contextAfter: [], - }, - { - lineNumber: 10, - lineContent: "match one", - contextBefore: [], - contextAfter: [], - }, - ], - }, - baseOptions, - ); - const { stdout } = formatGrepFileTerminal(envelope, { useColors: false }); - const oneIndex = stdout.indexOf("match one"); - const twoIndex = stdout.indexOf("match two"); - expect(oneIndex).toBeGreaterThan(-1); - expect(twoIndex).toBeGreaterThan(-1); - expect(oneIndex).toBeLessThan(twoIndex); - }); - - it("plain mode with context: match line wins over another match's context entry at the same line", () => { - // Match A at line 10; match B at line 12 with contextBefore that - // reaches back to line 10. The match-line entry at 10 must stay - // flagged as a match, not get overwritten by B's context copy. - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 2, - matches: [ - { - lineNumber: 10, - lineContent: "TRUE_MATCH_CONTENT", - contextBefore: [], - contextAfter: ["ctx 11"], - }, - { - lineNumber: 12, - lineContent: "match two", - contextBefore: ["stale context version", "ctx 11 dup"], - contextAfter: [], - }, - ], - }, - baseOptions, - ); - const { stdout } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - // The match line's actual content must appear with the `>` marker. - expect(stdout).toMatch(/>\s+10\s+TRUE_MATCH_CONTENT/); - // The stale context version must not leak in. - expect(stdout).not.toContain("stale context version"); - }); - - it("plain mode: handles matches with null contextBefore / contextAfter (builder strips empties to undefined)", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - matches: [ - { - lineNumber: 42, - lineContent: "lonely match", - contextBefore: [], - contextAfter: [], - }, - ], - }, - baseOptions, - ); - // Confirm the envelope stripped empty arrays. - expect(envelope.matches[0]?.contextBefore).toBeUndefined(); - expect(envelope.matches[0]?.contextAfter).toBeUndefined(); - const { stdout } = formatGrepFileTerminal(envelope, { useColors: false }); - expect(stdout).toContain("lonely match"); - }); - - it("verbose mode with context: merges overlapping blocks and deduplicates lines", () => { - const envelope = buildGrepFileSuccessPayload( - { - ...baseResult, - totalMatches: 2, - matches: [ - { - lineNumber: 10, - lineContent: "match one", - contextBefore: ["ctx 9"], - contextAfter: ["ctx 11"], - }, - { - lineNumber: 12, - lineContent: "match two", - contextBefore: ["ctx 11 (dup)"], - contextAfter: ["ctx 13"], - }, - ], - }, - baseOptions, - ); - const { stdout: output } = formatGrepFileTerminal(envelope, { - useColors: false, - verbose: true, - }); - // `ctx 11` appears once (match-1's contextAfter wins; match-2's - // contextBefore version is dropped as a dup). - expect(output.match(/ctx 11/g)?.length).toBe(1); - expect(output.match(/ctx 11 \(dup\)/g)?.length).toBeFalsy(); - // Both matches are marked with `>`. - expect(output.match(/^> /gm)?.length).toBe(2); - }); -}); diff --git a/src/shared/grep-file-response.ts b/src/shared/grep-file-response.ts deleted file mode 100644 index 7a31ae16..00000000 --- a/src/shared/grep-file-response.ts +++ /dev/null @@ -1,488 +0,0 @@ -/** - * Response envelope for `grep_file`. Shared across CLI `--json` - * output and MCP `content[0].text`; terminal formatter is CLI-only. - * - * Key rules: - * - **Data-first.** `matches` always present (possibly empty); - * `resolution` when backend returned one; `hint` when empty - * results carry a backend diagnostic. - * - **No indexing metadata in the success envelope.** Service - * promotes `codeIndexState: INDEXING` to a typed error first. - * - **`filter.*` echoes only caller-supplied inputs.** - * - **Regex-char heuristic on empty results (terminal-only).** If - * the pattern looks like unambiguous regex AND zero matches, - * the terminal appends a nudge. JSON never carries this hint. - */ - -import type { GrepFileResult, GrepMatch } from "../services/index.js"; -import { colorize, dim } from "./colors.js"; -import { looksLikeRegexAttempt } from "./grep-file-request.js"; - -export interface LeanGrepMatch { - lineNumber: number; - lineContent: string; - contextBefore?: string[]; - contextAfter?: string[]; -} - -export interface LeanGrepResolution { - requestedVersion?: string; - requestedRef?: string; - resolvedRef?: string; - commitSha?: string; -} - -export interface LeanGrepFilter { - contextLines?: number; - maxMatches?: number; -} - -export interface LeanGrepFileEnvelope { - registry?: string; - name?: string; - repoUrl?: string; - gitRef?: string; - pattern: string; - /** - * Resolved file path. Uses the backend's echoed path when - * available, falling back to the caller's input. Single `path` - * field (no `filePath` alongside) so the `list_files.files[].path` - * → `grep_file({path})` / `read_file({path})` chain stays - * mechanically consistent. - */ - path: string; - totalMatches: number; - hasMore: boolean; - language?: string; - totalLines?: number; - indexedVersion?: string; - resolution?: LeanGrepResolution; - matches: LeanGrepMatch[]; - hint?: string; - filter?: LeanGrepFilter; -} - -export interface BuildGrepFilePayloadOptions { - registry?: string; - name?: string; - repoUrl?: string; - gitRef?: string; - pattern: string; - path: string; - contextLinesExplicit: boolean; - maxMatchesExplicit: boolean; - contextLines: number; - maxMatches: number; -} - -export function buildGrepFileSuccessPayload( - result: GrepFileResult, - options: BuildGrepFilePayloadOptions, -): LeanGrepFileEnvelope { - const matches: LeanGrepMatch[] = result.matches.map((m) => projectMatch(m)); - - const envelope: LeanGrepFileEnvelope = { - pattern: options.pattern, - // Prefer the backend's echoed path (may be normalised); fall - // back to the caller's input so `path` is always present. - path: result.filePath ?? options.path, - totalMatches: result.totalMatches, - hasMore: result.hasMore, - matches, - }; - - if (options.registry) envelope.registry = options.registry; - if (options.name) envelope.name = options.name; - if (options.repoUrl) envelope.repoUrl = options.repoUrl; - if (options.gitRef) envelope.gitRef = options.gitRef; - if (result.language) envelope.language = result.language; - if (result.totalLines != null) envelope.totalLines = result.totalLines; - if (result.indexedVersion) envelope.indexedVersion = result.indexedVersion; - if (result.resolution) - envelope.resolution = projectResolution(result.resolution); - if (result.hint) envelope.hint = result.hint; - - const filter = buildFilterBlock(options); - if (filter) envelope.filter = filter; - - return envelope; -} - -function projectMatch(match: GrepMatch): LeanGrepMatch { - const lean: LeanGrepMatch = { - lineNumber: match.lineNumber, - lineContent: match.lineContent, - }; - if (match.contextBefore && match.contextBefore.length > 0) { - lean.contextBefore = match.contextBefore; - } - if (match.contextAfter && match.contextAfter.length > 0) { - lean.contextAfter = match.contextAfter; - } - return lean; -} - -function projectResolution( - resolution: GrepFileResult["resolution"], -): LeanGrepResolution | undefined { - if (!resolution) return undefined; - const lean: LeanGrepResolution = {}; - if (resolution.requestedVersion) - lean.requestedVersion = resolution.requestedVersion; - if (resolution.requestedRef) lean.requestedRef = resolution.requestedRef; - if (resolution.resolvedRef) lean.resolvedRef = resolution.resolvedRef; - if (resolution.commitSha) lean.commitSha = resolution.commitSha; - return Object.keys(lean).length > 0 ? lean : undefined; -} - -function buildFilterBlock( - options: BuildGrepFilePayloadOptions, -): LeanGrepFilter | undefined { - const filter: LeanGrepFilter = {}; - if (options.contextLinesExplicit) filter.contextLines = options.contextLines; - if (options.maxMatchesExplicit) filter.maxMatches = options.maxMatches; - return Object.keys(filter).length > 0 ? filter : undefined; -} - -// -------------------------------------------------------------------- -// Terminal formatter (CLI-only). -// -------------------------------------------------------------------- - -export interface FormatGrepFileTerminalOptions { - useColors: boolean; - /** - * When `true`, render a contextual header plus a line-number gutter - * (`>` marker on match lines). When `false` (default), emit matching - * lines only, no header, no line numbers — pipe-friendly like - * `grep` default output. - */ - verbose?: boolean; -} - -/** - * Render result split into stdout (clean, pipeable payload) and - * stderr (human-facing hints — truncation warnings, diagnostics). - * Callers write each stream independently so plain-mode pipes - * stay uncorrupted by informational text. - */ -export interface FormattedGrepFileTerminal { - stdout: string; - stderr?: string; -} - -/** - * Terminal rendering for `code grep`. - * - * Plain (default) mode mirrors `grep`'s default output: matching - * lines printed as raw content, one per line, no header, no line - * numbers. When `--context` is non-zero, context lines are included - * in-line (still no line numbers) and distinct blocks are separated - * by `--` in grep's convention. - * - * Verbose mode adds a contextual header, a right-aligned - * line-number gutter, and a `>` marker on match lines to - * distinguish them from context at a glance. - * - * Overlapping context blocks — two nearby matches whose contexts - * touch or overlap — are merged into a single block so no line is - * printed twice. This matches `grep -C` / `rg -C` behaviour. - */ -export function formatGrepFileTerminal( - envelope: LeanGrepFileEnvelope, - options: FormatGrepFileTerminalOptions, -): FormattedGrepFileTerminal { - const verbose = options.verbose ?? false; - - if (envelope.matches.length === 0) { - return formatNoMatches(envelope, options, verbose); - } - - const blocks = mergeMatchBlocks(envelope.matches); - - if (!verbose) { - return formatPlain(envelope, blocks, options); - } - return formatVerbose(envelope, blocks, options); -} - -// -------------------------------------------------------------------- -// Block merging. -// -------------------------------------------------------------------- - -interface RenderLine { - lineNumber: number; - content: string; - isMatch: boolean; -} - -/** - * Flatten the per-match `{lineContent, contextBefore, contextAfter}` - * shape into ordered, deduplicated blocks of lines. When two matches - * sit close enough that their contexts overlap or touch, the - * resulting lines merge into one block without duplicates — what - * `grep -C` calls "merging adjacent context lines". - */ -export function mergeMatchBlocks(matches: LeanGrepMatch[]): RenderLine[][] { - if (matches.length === 0) return []; - - const lineMap = new Map(); - for (const match of matches) { - const contextBefore = match.contextBefore ?? []; - const beforeStart = match.lineNumber - contextBefore.length; - for (let i = 0; i < contextBefore.length; i++) { - const ln = beforeStart + i; - if (!lineMap.has(ln)) { - lineMap.set(ln, { - lineNumber: ln, - content: contextBefore[i] ?? "", - isMatch: false, - }); - } - } - // The match line always wins over a context entry at the same - // line number (another match's contextBefore/contextAfter). - lineMap.set(match.lineNumber, { - lineNumber: match.lineNumber, - content: match.lineContent, - isMatch: true, - }); - const contextAfter = match.contextAfter ?? []; - for (let i = 0; i < contextAfter.length; i++) { - const ln = match.lineNumber + 1 + i; - if (!lineMap.has(ln)) { - lineMap.set(ln, { - lineNumber: ln, - content: contextAfter[i] ?? "", - isMatch: false, - }); - } - } - } - - const sorted = [...lineMap.values()].sort( - (a, b) => a.lineNumber - b.lineNumber, - ); - - const blocks: RenderLine[][] = []; - let current: RenderLine[] = []; - for (const line of sorted) { - const last = current[current.length - 1]; - if (!last || line.lineNumber === last.lineNumber + 1) { - current.push(line); - } else { - blocks.push(current); - current = [line]; - } - } - if (current.length > 0) blocks.push(current); - return blocks; -} - -// -------------------------------------------------------------------- -// Plain mode: matches only, optional merged context, no line numbers. -// -------------------------------------------------------------------- - -function formatPlain( - envelope: LeanGrepFileEnvelope, - blocks: RenderLine[][], - options: FormatGrepFileTerminalOptions, -): FormattedGrepFileTerminal { - const hasContext = blocks.some((block) => - block.some((line) => !line.isMatch), - ); - - const out: string[] = []; - blocks.forEach((block, i) => { - if (i > 0 && hasContext) { - // `grep -C` separator between distinct blocks. Without context, - // consecutive matches are printed as-is with no separator. - out.push(dim("--", options.useColors)); - } - for (const line of block) { - out.push(line.content); - } - }); - - // Trailing newline so output composes nicely in pipes / files. - out.push(""); - - const stderr: string[] = []; - if (envelope.hasMore) { - stderr.push( - dim( - "More matches available — pass --limit higher to fetch more.", - options.useColors, - ), - ); - } - - return { - stdout: out.join("\n"), - stderr: stderr.length > 0 ? `${stderr.join("\n")}\n` : undefined, - }; -} - -// -------------------------------------------------------------------- -// Verbose mode: header + gutter + block separators. -// -------------------------------------------------------------------- - -function formatVerbose( - envelope: LeanGrepFileEnvelope, - blocks: RenderLine[][], - options: FormatGrepFileTerminalOptions, -): FormattedGrepFileTerminal { - const lines: string[] = []; - lines.push(buildHeader(envelope, options)); - if (envelope.resolution || envelope.indexedVersion) { - lines.push(buildResolutionLine(envelope, options)); - } - lines.push(""); - - const gutterWidth = widestLineNumberInBlocks(blocks); - blocks.forEach((block, i) => { - if (i > 0) lines.push(dim("--", options.useColors)); - for (const line of block) { - lines.push(renderVerboseLine(line, gutterWidth, options)); - } - }); - - if (envelope.hasMore) { - lines.push(""); - lines.push( - dim( - "More matches available — pass --limit higher to fetch more.", - options.useColors, - ), - ); - } - - lines.push(""); - return { stdout: lines.join("\n") }; -} - -function renderVerboseLine( - line: RenderLine, - gutterWidth: number, - options: FormatGrepFileTerminalOptions, -): string { - const gutter = padLeft(String(line.lineNumber), gutterWidth); - if (line.isMatch) { - const marker = colorize(">", "bold", options.useColors); - return `${marker} ${gutter} ${colorize(line.content, "bold", options.useColors)}`; - } - return ` ${dim(gutter, options.useColors)} ${dim(line.content, options.useColors)}`; -} - -// -------------------------------------------------------------------- -// Zero-match path. -// -------------------------------------------------------------------- - -function formatNoMatches( - envelope: LeanGrepFileEnvelope, - options: FormatGrepFileTerminalOptions, - verbose: boolean, -): FormattedGrepFileTerminal { - // Plain mode: match `grep`'s behaviour — silent on stdout, exit - // code carries the "no match" signal. If the pattern looks like - // a regex, write a single-line nudge to stderr so humans piping - // see it without polluting the pipe. - if (!verbose) { - if (looksLikeRegexAttempt(envelope.pattern)) { - return { - stdout: "", - stderr: `${dim("Note: pattern matched literally — this tool does case-insensitive substring search, not regex.", options.useColors)}\n`, - }; - } - return { stdout: "" }; - } - - const lines: string[] = []; - lines.push(buildHeader(envelope, options)); - if (envelope.resolution || envelope.indexedVersion) { - lines.push(buildResolutionLine(envelope, options)); - } - lines.push(""); - - if (envelope.hint) { - lines.push(dim(envelope.hint, options.useColors)); - } else { - lines.push( - dim( - `No matches for '${envelope.pattern}' in ${envelope.path}.`, - options.useColors, - ), - ); - } - if (looksLikeRegexAttempt(envelope.pattern)) { - lines.push( - dim( - "If you intended regex syntax, note: this tool does literal substring matching.", - options.useColors, - ), - ); - } - lines.push(""); - return { stdout: lines.join("\n") }; -} - -// -------------------------------------------------------------------- -// Header / identity helpers. -// -------------------------------------------------------------------- - -function buildHeader( - envelope: LeanGrepFileEnvelope, - options: FormatGrepFileTerminalOptions, -): string { - const identity = buildIdentityLabel(envelope); - const countLabel = envelope.hasMore - ? `${envelope.matches.length}+ matches` - : `${envelope.totalMatches} ${plural("match", "matches", envelope.totalMatches)}`; - return colorize( - `${identity} · ${countLabel} in ${envelope.path}`, - "bold", - options.useColors, - ); -} - -function buildIdentityLabel(envelope: LeanGrepFileEnvelope): string { - if (envelope.registry && envelope.name) { - return `${envelope.name} · ${envelope.registry}`; - } - if (envelope.repoUrl) { - return envelope.gitRef - ? `${envelope.repoUrl} @ ${envelope.gitRef}` - : envelope.repoUrl; - } - return "(unknown)"; -} - -function buildResolutionLine( - envelope: LeanGrepFileEnvelope, - options: FormatGrepFileTerminalOptions, -): string { - const parts: string[] = []; - const ref = envelope.resolution?.resolvedRef ?? envelope.indexedVersion; - if (ref) parts.push(`indexed at ${ref}`); - const commit = envelope.resolution?.commitSha; - if (commit) parts.push(`commit ${commit.slice(0, 7)}`); - return dim(parts.join(" · "), options.useColors); -} - -function widestLineNumberInBlocks(blocks: RenderLine[][]): number { - let max = 0; - for (const block of blocks) { - for (const line of block) { - const w = String(line.lineNumber).length; - if (w > max) max = w; - } - } - return max; -} - -function plural(singular: string, pluralForm: string, count: number): string { - return count === 1 ? singular : pluralForm; -} - -function padLeft(text: string, width: number): string { - return text.length >= width ? text : " ".repeat(width - text.length) + text; -} diff --git a/src/shared/grep-repo-request.test.ts b/src/shared/grep-repo-request.test.ts new file mode 100644 index 00000000..cff5739a --- /dev/null +++ b/src/shared/grep-repo-request.test.ts @@ -0,0 +1,96 @@ +import { describe, expect, it } from "bun:test"; +import type { CodeNavigationTarget } from "../services/index.js"; +import { + buildGrepRepoParams, + GREP_REPO_PATTERN_NOTE, +} from "./grep-repo-request.js"; + +const target: CodeNavigationTarget = { + registry: "NPM", + packageName: "express", +}; + +describe("buildGrepRepoParams", () => { + it("defaults to whole-target literal grep", () => { + const { params } = buildGrepRepoParams({ + target, + pattern: "middleware", + }); + expect(params.patternType).toBeUndefined(); + expect(params.allowUnscoped).toBe(true); + expect(params.contextLinesBefore).toBe(0); + expect(params.contextLinesAfter).toBe(0); + expect(params.maxMatches).toBe(50); + expect(params.waitTimeoutMs).toBe(20000); + }); + + it("compiles path, pathPrefix, and globs into pathSelectors", () => { + const { params } = buildGrepRepoParams({ + target, + pattern: "middleware", + path: "src/index.js", + pathPrefix: "src/", + globs: ["src/**/*.js"], + }); + expect(params.allowUnscoped).toBeUndefined(); + expect(params.pathSelectors).toEqual([ + { kind: "EXACT", value: "src/index.js" }, + { kind: "PREFIX", value: "src/" }, + { kind: "GLOB", value: "src/**/*.js" }, + ]); + }); + + it("supports symmetric and asymmetric context", () => { + const symmetric = buildGrepRepoParams({ + target, + pattern: "middleware", + contextLines: 3, + }); + expect(symmetric.params.contextLinesBefore).toBe(3); + expect(symmetric.params.contextLinesAfter).toBe(3); + + const asymmetric = buildGrepRepoParams({ + target, + pattern: "middleware", + contextLines: 3, + contextLinesBefore: 1, + contextLinesAfter: 5, + }); + expect(asymmetric.params.contextLinesBefore).toBe(1); + expect(asymmetric.params.contextLinesAfter).toBe(5); + }); + + it("rejects leading dots in extensions", () => { + expect(() => + buildGrepRepoParams({ + target, + pattern: "middleware", + extensions: [".js"], + }), + ).toThrow(/leading dot/); + }); + + it("rejects whitespace-only patterns without trimming real content", () => { + expect(() => + buildGrepRepoParams({ + target, + pattern: " ", + }), + ).toThrow(/pattern/); + + const { params } = buildGrepRepoParams({ + target, + pattern: " middleware ", + }); + expect(params.pattern).toBe(" middleware "); + }); +}); + +describe("GREP_REPO_PATTERN_NOTE", () => { + it("mentions literal, regex, RE2, and byte limit", () => { + expect(GREP_REPO_PATTERN_NOTE).toMatch(/literal/i); + expect(GREP_REPO_PATTERN_NOTE).toMatch(/regex/i); + expect(GREP_REPO_PATTERN_NOTE).toMatch(/RE2/i); + expect(GREP_REPO_PATTERN_NOTE).toMatch(/200/i); + }); +}); diff --git a/src/shared/grep-repo-request.ts b/src/shared/grep-repo-request.ts new file mode 100644 index 00000000..568dbbcd --- /dev/null +++ b/src/shared/grep-repo-request.ts @@ -0,0 +1,263 @@ +import type { + CodeNavigationTarget, + GrepPathSelectorKind, + GrepRepoParams, +} from "../services/index.js"; +import { + DEFAULT_WAIT_TIMEOUT_MS, + MAX_WAIT_TIMEOUT_MS, +} from "./code-navigation-defaults.js"; +import { InvalidPackageSpecError } from "./package-spec.js"; + +const PATTERN_MAX = 200; +const CONTEXT_MIN = 0; +const CONTEXT_MAX = 10; +const LIMIT_MIN = 1; +const LIMIT_MAX = 1000; +const LIMIT_DEFAULT = 50; +const WAIT_MIN = 0; + +export const GREP_REPO_PATTERN_NOTE = + "Text grep over indexed source files. `literal` (default) does substring matching; `regex` uses RE2 syntax (no lookaround, no backreferences). Pattern max 200 UTF-8 bytes."; + +export interface GrepRepoRequestPathSelectorInput { + kind: "exact" | "prefix" | "glob"; + value: string; +} + +export interface GrepRepoRequestInput { + target: CodeNavigationTarget; + pattern: string; + path?: string; + pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + patternType?: "literal" | "regex"; + caseSensitive?: boolean; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + contextLines?: number; + contextLinesBefore?: number; + contextLinesAfter?: number; + maxMatches?: number; + maxMatchesPerFile?: number; + cursor?: string; + waitTimeoutMs?: number; +} + +export interface GrepRepoRequestBuildResult { + params: GrepRepoParams; + explicit: { + path: boolean; + pathPrefix: boolean; + globs: boolean; + extensions: boolean; + patternType: boolean; + caseSensitive: boolean; + excludeDocFiles: boolean; + excludeTestFiles: boolean; + contextLines: boolean; + contextLinesBefore: boolean; + contextLinesAfter: boolean; + maxMatches: boolean; + maxMatchesPerFile: boolean; + cursor: boolean; + }; +} + +export function buildGrepRepoParams( + input: GrepRepoRequestInput, +): GrepRepoRequestBuildResult { + const pattern = input.pattern ?? ""; + if (pattern.length === 0 || pattern.trim().length === 0) { + throw new InvalidPackageSpecError( + "`pattern` is required — pass the text to search for.", + ); + } + if (Buffer.byteLength(pattern, "utf8") > PATTERN_MAX) { + throw new InvalidPackageSpecError( + `\`pattern\` must be ≤ ${PATTERN_MAX} UTF-8 bytes.`, + ); + } + + const path = normalizeOptionalNonEmpty(input.path, "path"); + const pathPrefix = normalizeOptionalNonEmpty(input.pathPrefix, "path_prefix"); + const globs = normalizeStringList(input.globs, "globs"); + const extensions = normalizeExtensions(input.extensions); + + const contextLines = normalizeOptionalContext( + input.contextLines, + "context_lines", + ); + const contextLinesBefore = normalizeOptionalContext( + input.contextLinesBefore, + "context_lines_before", + ); + const contextLinesAfter = normalizeOptionalContext( + input.contextLinesAfter, + "context_lines_after", + ); + + const resolvedBefore = + contextLinesBefore ?? (contextLines !== undefined ? contextLines : 0); + const resolvedAfter = + contextLinesAfter ?? (contextLines !== undefined ? contextLines : 0); + + const maxMatches = normalizeMaxMatches(input.maxMatches); + const maxMatchesPerFile = normalizeMaxMatchesPerFile(input.maxMatchesPerFile); + const waitTimeoutMs = normalizeWaitTimeoutMs(input.waitTimeoutMs); + const cursor = normalizeOptionalNonEmpty(input.cursor, "cursor"); + + const pathSelectors = buildPathSelectors({ path, pathPrefix, globs }); + const hasPathSelectors = (pathSelectors?.length ?? 0) > 0; + + return { + params: { + target: input.target, + pattern, + patternType: + input.patternType === "regex" + ? "REGEX" + : input.patternType === "literal" + ? "LITERAL" + : undefined, + caseSensitive: input.caseSensitive, + pathSelectors, + extensions, + excludeDocFiles: input.excludeDocFiles, + excludeTestFiles: input.excludeTestFiles, + allowUnscoped: hasPathSelectors ? undefined : true, + contextLinesBefore: resolvedBefore, + contextLinesAfter: resolvedAfter, + maxMatches, + maxMatchesPerFile, + cursor, + waitTimeoutMs, + }, + explicit: { + path: path !== undefined, + pathPrefix: pathPrefix !== undefined, + globs: globs.length > 0, + extensions: extensions.length > 0, + patternType: input.patternType !== undefined, + caseSensitive: input.caseSensitive !== undefined, + excludeDocFiles: input.excludeDocFiles !== undefined, + excludeTestFiles: input.excludeTestFiles !== undefined, + contextLines: input.contextLines !== undefined, + contextLinesBefore: input.contextLinesBefore !== undefined, + contextLinesAfter: input.contextLinesAfter !== undefined, + maxMatches: input.maxMatches !== undefined, + maxMatchesPerFile: input.maxMatchesPerFile !== undefined, + cursor: cursor !== undefined, + }, + }; +} + +function buildPathSelectors(input: { + path?: string; + pathPrefix?: string; + globs: string[]; +}): GrepRepoParams["pathSelectors"] { + const selectors: Array<{ kind: GrepPathSelectorKind; value: string }> = []; + if (input.path) selectors.push({ kind: "EXACT", value: input.path }); + if (input.pathPrefix) { + selectors.push({ kind: "PREFIX", value: input.pathPrefix }); + } + for (const glob of input.globs) { + selectors.push({ kind: "GLOB", value: glob }); + } + return selectors.length > 0 ? selectors : undefined; +} + +function normalizeOptionalNonEmpty( + value: string | undefined, + field: string, +): string | undefined { + if (value === undefined) return undefined; + const trimmed = value.trim(); + if (trimmed.length === 0) { + throw new InvalidPackageSpecError( + `\`${field}\` cannot be empty when provided.`, + ); + } + return trimmed; +} + +function normalizeStringList( + values: string[] | undefined, + field: string, +): string[] { + if (!values) return []; + const out: string[] = []; + for (const value of values) { + const trimmed = value.trim(); + if (trimmed.length === 0) { + throw new InvalidPackageSpecError( + `\`${field}\` entries cannot be empty.`, + ); + } + out.push(trimmed); + } + return out; +} + +function normalizeExtensions(values: string[] | undefined): string[] { + const out = normalizeStringList(values, "extensions"); + for (const value of out) { + if (value.startsWith(".")) { + throw new InvalidPackageSpecError( + "`extensions` values must not include a leading dot.", + ); + } + } + return out; +} + +function normalizeOptionalContext( + value: number | undefined, + field: string, +): number | undefined { + if (value === undefined) return undefined; + if (!Number.isInteger(value) || value < CONTEXT_MIN || value > CONTEXT_MAX) { + throw new InvalidPackageSpecError( + `\`${field}\` must be an integer between ${CONTEXT_MIN} and ${CONTEXT_MAX}. Got ${value}.`, + ); + } + return value; +} + +function normalizeMaxMatches(value: number | undefined): number { + if (value === undefined) return LIMIT_DEFAULT; + if (!Number.isInteger(value) || value < LIMIT_MIN || value > LIMIT_MAX) { + throw new InvalidPackageSpecError( + `\`max_matches\` must be an integer between ${LIMIT_MIN} and ${LIMIT_MAX}. Got ${value}.`, + ); + } + return value; +} + +function normalizeMaxMatchesPerFile( + value: number | undefined, +): number | undefined { + if (value === undefined) return undefined; + if (!Number.isInteger(value) || value < 0 || value > LIMIT_MAX) { + throw new InvalidPackageSpecError( + `\`max_matches_per_file\` must be an integer between 0 and ${LIMIT_MAX}. Got ${value}.`, + ); + } + return value; +} + +function normalizeWaitTimeoutMs(value: number | undefined): number { + if (value === undefined) return DEFAULT_WAIT_TIMEOUT_MS; + if ( + !Number.isInteger(value) || + value < WAIT_MIN || + value > MAX_WAIT_TIMEOUT_MS + ) { + throw new InvalidPackageSpecError( + `\`wait_timeout_ms\` must be an integer between ${WAIT_MIN} and ${MAX_WAIT_TIMEOUT_MS}. Got ${value}.`, + ); + } + return value; +} diff --git a/src/shared/grep-repo-response.test.ts b/src/shared/grep-repo-response.test.ts new file mode 100644 index 00000000..6c1f19b8 --- /dev/null +++ b/src/shared/grep-repo-response.test.ts @@ -0,0 +1,306 @@ +import { describe, expect, it } from "bun:test"; +import type { GrepRepoResult } from "../services/index.js"; +import { + buildGrepRepoSuccessPayload, + formatGrepRepoTerminal, +} from "./grep-repo-response.js"; + +const baseResult: GrepRepoResult = { + matches: [ + { + filePath: "src/index.js", + line: 10, + matchStartByte: 6, + matchEndByte: 13, + lineContent: "const app = express();", + contextBefore: ["", "// set up express", ""], + contextAfter: ["", "app.get('/', …);"], + fileContentHash: "abc123", + fileIntent: "production", + }, + ], + nextCursor: undefined, + hasMore: false, + truncatedReason: "NONE", + routeTaken: "CONTENT_INDEX", + filesScanned: 1, + filesInScope: 1, + binaryFilesSkipped: 0, + filesTooLargeSkipped: 0, + totalMatches: 1, + uniqueFilesMatched: 1, + indexedVersion: "v5.2.1", + resolution: { + resolvedRef: "v5.2.1", + commitSha: "abc123def", + }, +}; + +const baseOptions = { + registry: "npm", + name: "express", + pattern: "express()", + patternType: "literal" as const, + caseSensitive: false, + path: undefined, + pathPrefix: "src/", + globs: ["src/**/*.js"], + extensions: ["js"], + contextLines: 2, + contextLinesBefore: 2, + contextLinesAfter: 2, + maxMatches: 50, + maxMatchesPerFile: 3, + cursor: undefined, + excludeDocFiles: true, + excludeTestFiles: true, + explicit: { + path: false, + pathPrefix: true, + globs: true, + extensions: true, + patternType: false, + caseSensitive: false, + excludeDocFiles: true, + excludeTestFiles: true, + contextLines: true, + contextLinesBefore: false, + contextLinesAfter: false, + maxMatches: true, + maxMatchesPerFile: true, + cursor: false, + }, +}; + +describe("buildGrepRepoSuccessPayload", () => { + it("projects the new envelope shape", () => { + const envelope = buildGrepRepoSuccessPayload(baseResult, baseOptions); + expect(envelope.pattern).toBe("express()"); + expect(envelope.patternType).toBe("literal"); + expect(envelope.totalMatches).toBe(1); + expect(envelope.matches[0]?.filePath).toBe("src/index.js"); + expect(envelope.routeTaken).toBe("content_index"); + expect(envelope.filter).toEqual({ + pathPrefix: "src/", + globs: ["src/**/*.js"], + extensions: ["js"], + excludeDocFiles: true, + excludeTestFiles: true, + contextLines: 2, + maxMatches: 50, + maxMatchesPerFile: 3, + }); + }); +}); + +describe("formatGrepRepoTerminal", () => { + it("plain mode emits file:line:text", () => { + const envelope = buildGrepRepoSuccessPayload(baseResult, baseOptions); + const { stdout } = formatGrepRepoTerminal(envelope, { + useColors: false, + }); + expect(stdout).toContain("src/index.js:10:const app = express();"); + }); + + it("heading mode emits a file heading with compact line rows", () => { + const envelope = buildGrepRepoSuccessPayload(baseResult, baseOptions); + const { stdout } = formatGrepRepoTerminal(envelope, { + useColors: false, + headingStyle: true, + }); + expect(stdout).toContain("src/index.js\n10:const app = express();"); + expect(stdout).not.toContain("src/index.js:10:const app = express();"); + }); + + it("verbose mode emits a grouped header", () => { + const envelope = buildGrepRepoSuccessPayload(baseResult, baseOptions); + const { stdout } = formatGrepRepoTerminal(envelope, { + useColors: false, + verbose: true, + }); + expect(stdout).toContain("1 match in 1 file"); + expect(stdout).toContain("src/index.js\n"); + expect(stdout).toContain("> 10 const app = express();"); + }); + + it("renders context in chronological order", () => { + const envelope = buildGrepRepoSuccessPayload( + { + ...baseResult, + matches: [ + { + ...baseResult.matches[0]!, + contextBefore: ["before"], + contextAfter: ["after"], + }, + ], + }, + baseOptions, + ); + + const plain = formatGrepRepoTerminal(envelope, { + useColors: false, + withContext: true, + }).stdout; + expect(plain.indexOf("src/index.js\n")).toBeLessThan( + plain.indexOf("9-before"), + ); + expect(plain.indexOf("9-before")).toBeLessThan( + plain.indexOf("10:const app = express();"), + ); + expect(plain.indexOf("10:const app = express();")).toBeLessThan( + plain.indexOf("11-after"), + ); + + const verbose = formatGrepRepoTerminal(envelope, { + useColors: false, + verbose: true, + }).stdout; + expect(verbose.indexOf(" 9 before")).toBeLessThan( + verbose.indexOf("> 10 const app = express();"), + ); + expect(verbose.indexOf("> 10 const app = express();")).toBeLessThan( + verbose.indexOf(" 11 after"), + ); + }); + + it("dedupes multiple matches on the same line in terminal output", () => { + const envelope = buildGrepRepoSuccessPayload( + { + ...baseResult, + totalMatches: 2, + matches: [ + baseResult.matches[0]!, + { + ...baseResult.matches[0]!, + matchStartByte: 20, + matchEndByte: 27, + }, + ], + }, + baseOptions, + ); + + const plain = formatGrepRepoTerminal(envelope, { + useColors: false, + }).stdout; + expect( + plain.match(/src\/index\.js:10:const app = express\(\);/g)?.length, + ).toBe(1); + + const verbose = formatGrepRepoTerminal(envelope, { + useColors: false, + verbose: true, + }).stdout; + expect(verbose.match(/^src\/index\.js$/gm)?.length).toBe(1); + expect(verbose.match(/^>\s+10\s+const app = express\(\);$/gm)?.length).toBe( + 1, + ); + }); + + it("merges overlapping context blocks with grep-style line prefixes", () => { + const envelope = buildGrepRepoSuccessPayload( + { + ...baseResult, + totalMatches: 2, + uniqueFilesMatched: 1, + matches: [ + { + ...baseResult.matches[0]!, + filePath: "lib/app.js", + line: 10, + lineContent: "match one", + contextBefore: ["line 8", "line 9"], + contextAfter: ["line 11", "match two"], + }, + { + ...baseResult.matches[0]!, + filePath: "lib/app.js", + line: 12, + lineContent: "match two", + contextBefore: ["line 10 stale", "line 11 stale"], + contextAfter: ["line 13", "line 14"], + }, + ], + }, + baseOptions, + ); + + const plain = formatGrepRepoTerminal(envelope, { + useColors: false, + withContext: true, + }).stdout; + + expect(plain).toContain("lib/app.js\n8-line 8"); + expect(plain).toContain("9-line 9"); + expect(plain).toContain("10:match one"); + expect(plain).toContain("11-line 11"); + expect(plain).toContain("12:match two"); + expect(plain).toContain("13-line 13"); + expect(plain).toContain("14-line 14"); + expect(plain).not.toContain("line 10 stale"); + expect(plain).not.toContain("line 11 stale"); + expect(plain).not.toContain("--\n--"); + }); + + it("prints nextCursor continuation instructions with the real cursor value", () => { + const envelope = buildGrepRepoSuccessPayload( + { + ...baseResult, + hasMore: true, + nextCursor: "cursor_abc123", + truncatedReason: "MAX_MATCHES", + }, + baseOptions, + ); + + const rendered = formatGrepRepoTerminal(envelope, { + useColors: false, + }); + + expect(rendered.stderr).toBe( + "More grep results available — rerun with --cursor 'cursor_abc123'\n", + ); + }); + + it("adds a narrow-scope hint for noisy broad results", () => { + const envelope = buildGrepRepoSuccessPayload( + { + ...baseResult, + totalMatches: 6, + uniqueFilesMatched: 6, + matches: [ + { ...baseResult.matches[0]!, filePath: "History.md", line: 1 }, + { ...baseResult.matches[0]!, filePath: "Readme.md", line: 2 }, + { ...baseResult.matches[0]!, filePath: "benchmarks/run", line: 3 }, + { ...baseResult.matches[0]!, filePath: "examples/a.js", line: 4 }, + { ...baseResult.matches[0]!, filePath: "test/a.js", line: 5 }, + { ...baseResult.matches[0]!, filePath: "lib/app.js", line: 6 }, + ], + }, + { + ...baseOptions, + pathPrefix: undefined, + globs: undefined, + extensions: undefined, + excludeDocFiles: undefined, + excludeTestFiles: undefined, + explicit: { + ...baseOptions.explicit, + pathPrefix: false, + globs: false, + extensions: false, + excludeDocFiles: false, + excludeTestFiles: false, + }, + }, + ); + + const rendered = formatGrepRepoTerminal(envelope, { + useColors: false, + }); + + expect(rendered.stderr).toContain("Broad results"); + expect(rendered.stderr).toContain("--exclude-docs"); + }); +}); diff --git a/src/shared/grep-repo-response.ts b/src/shared/grep-repo-response.ts new file mode 100644 index 00000000..905cc375 --- /dev/null +++ b/src/shared/grep-repo-response.ts @@ -0,0 +1,528 @@ +import type { GrepRepoMatch, GrepRepoResult } from "../services/index.js"; +import { colorize, dim } from "./colors.js"; + +export interface LeanGrepRepoMatch { + filePath: string; + line: number; + matchStartByte: number; + matchEndByte: number; + lineContent: string; + contextBefore?: string[]; + contextAfter?: string[]; + fileContentHash?: string; + fileIntent?: string; +} + +export interface LeanGrepRepoResolution { + requestedVersion?: string; + requestedRef?: string; + resolvedRef?: string; + commitSha?: string; +} + +export interface LeanGrepRepoFilter { + path?: string; + pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + patternType?: "literal" | "regex"; + caseSensitive?: boolean; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + contextLines?: number; + contextLinesBefore?: number; + contextLinesAfter?: number; + maxMatches?: number; + maxMatchesPerFile?: number; + cursor?: string; +} + +export interface LeanGrepRepoEnvelope { + registry?: string; + name?: string; + repoUrl?: string; + gitRef?: string; + pattern: string; + patternType: "literal" | "regex"; + caseSensitive: boolean; + matches: LeanGrepRepoMatch[]; + nextCursor?: string; + hasMore: boolean; + truncatedReason: string; + routeTaken?: string; + filesScanned: number; + filesInScope: number; + binaryFilesSkipped: number; + filesTooLargeSkipped: number; + totalMatches: number; + uniqueFilesMatched: number; + indexedVersion?: string; + resolution?: LeanGrepRepoResolution; + filter?: LeanGrepRepoFilter; +} + +export interface BuildGrepRepoPayloadOptions { + registry?: string; + name?: string; + repoUrl?: string; + gitRef?: string; + pattern: string; + patternType: "literal" | "regex"; + caseSensitive: boolean; + path?: string; + pathPrefix?: string; + globs?: string[]; + extensions?: string[]; + contextLines?: number; + contextLinesBefore: number; + contextLinesAfter: number; + maxMatches: number; + maxMatchesPerFile?: number; + cursor?: string; + excludeDocFiles?: boolean; + excludeTestFiles?: boolean; + explicit: { + path: boolean; + pathPrefix: boolean; + globs: boolean; + extensions: boolean; + patternType: boolean; + caseSensitive: boolean; + excludeDocFiles: boolean; + excludeTestFiles: boolean; + contextLines: boolean; + contextLinesBefore: boolean; + contextLinesAfter: boolean; + maxMatches: boolean; + maxMatchesPerFile: boolean; + cursor: boolean; + }; +} + +export function buildGrepRepoSuccessPayload( + result: GrepRepoResult, + options: BuildGrepRepoPayloadOptions, +): LeanGrepRepoEnvelope { + const envelope: LeanGrepRepoEnvelope = { + pattern: options.pattern, + patternType: options.patternType, + caseSensitive: options.caseSensitive, + matches: result.matches.map(projectMatch), + hasMore: result.hasMore, + truncatedReason: result.truncatedReason.toLowerCase(), + routeTaken: result.routeTaken?.toLowerCase(), + filesScanned: result.filesScanned, + filesInScope: result.filesInScope, + binaryFilesSkipped: result.binaryFilesSkipped, + filesTooLargeSkipped: result.filesTooLargeSkipped, + totalMatches: result.totalMatches, + uniqueFilesMatched: result.uniqueFilesMatched, + }; + + if (options.registry) envelope.registry = options.registry; + if (options.name) envelope.name = options.name; + if (options.repoUrl) envelope.repoUrl = options.repoUrl; + if (options.gitRef) envelope.gitRef = options.gitRef; + if (result.nextCursor) envelope.nextCursor = result.nextCursor; + if (result.indexedVersion) envelope.indexedVersion = result.indexedVersion; + if (result.resolution) { + envelope.resolution = projectResolution(result.resolution); + } + + const filter = buildFilterBlock(options); + if (filter) envelope.filter = filter; + return envelope; +} + +function projectMatch(match: GrepRepoMatch): LeanGrepRepoMatch { + return { + filePath: match.filePath, + line: match.line, + matchStartByte: match.matchStartByte, + matchEndByte: match.matchEndByte, + lineContent: match.lineContent, + contextBefore: match.contextBefore, + contextAfter: match.contextAfter, + fileContentHash: match.fileContentHash, + fileIntent: match.fileIntent, + }; +} + +function projectResolution( + resolution: GrepRepoResult["resolution"], +): LeanGrepRepoResolution | undefined { + if (!resolution) return undefined; + const out: LeanGrepRepoResolution = {}; + if (resolution.requestedVersion) + out.requestedVersion = resolution.requestedVersion; + if (resolution.requestedRef) out.requestedRef = resolution.requestedRef; + if (resolution.resolvedRef) out.resolvedRef = resolution.resolvedRef; + if (resolution.commitSha) out.commitSha = resolution.commitSha; + return Object.keys(out).length > 0 ? out : undefined; +} + +function buildFilterBlock( + options: BuildGrepRepoPayloadOptions, +): LeanGrepRepoFilter | undefined { + const filter: LeanGrepRepoFilter = {}; + if (options.explicit.path && options.path) filter.path = options.path; + if (options.explicit.pathPrefix && options.pathPrefix) { + filter.pathPrefix = options.pathPrefix; + } + if (options.explicit.globs && options.globs && options.globs.length > 0) { + filter.globs = options.globs; + } + if ( + options.explicit.extensions && + options.extensions && + options.extensions.length > 0 + ) { + filter.extensions = options.extensions; + } + if (options.explicit.patternType) filter.patternType = options.patternType; + if (options.explicit.caseSensitive) { + filter.caseSensitive = options.caseSensitive; + } + if (options.explicit.excludeDocFiles) { + filter.excludeDocFiles = options.excludeDocFiles; + } + if (options.explicit.excludeTestFiles) { + filter.excludeTestFiles = options.excludeTestFiles; + } + if (options.explicit.contextLines && options.contextLines !== undefined) { + filter.contextLines = options.contextLines; + } + if (options.explicit.contextLinesBefore) { + filter.contextLinesBefore = options.contextLinesBefore; + } + if (options.explicit.contextLinesAfter) { + filter.contextLinesAfter = options.contextLinesAfter; + } + if (options.explicit.maxMatches) filter.maxMatches = options.maxMatches; + if ( + options.explicit.maxMatchesPerFile && + options.maxMatchesPerFile !== undefined + ) { + filter.maxMatchesPerFile = options.maxMatchesPerFile; + } + if (options.explicit.cursor && options.cursor) filter.cursor = options.cursor; + return Object.keys(filter).length > 0 ? filter : undefined; +} + +export interface FormatGrepRepoTerminalOptions { + useColors: boolean; + verbose?: boolean; + withContext?: boolean; + headingStyle?: boolean; +} + +export interface FormattedGrepRepoTerminal { + stdout: string; + stderr?: string; +} + +interface RenderLine { + lineNumber: number; + content: string; + isMatch: boolean; +} + +interface RenderBlock { + filePath: string; + lines: RenderLine[]; +} + +export function formatGrepRepoTerminal( + envelope: LeanGrepRepoEnvelope, + options: FormatGrepRepoTerminalOptions, +): FormattedGrepRepoTerminal { + if (envelope.matches.length === 0) { + return { stdout: "" }; + } + + const blocks = buildRenderBlocks(envelope.matches); + return options.verbose + ? formatVerbose(envelope, blocks, options) + : formatPlain(envelope, blocks, options); +} + +function formatPlain( + envelope: LeanGrepRepoEnvelope, + blocks: RenderBlock[], + options: FormatGrepRepoTerminalOptions, +): FormattedGrepRepoTerminal { + if (options.headingStyle || options.withContext) { + return formatHeadingPlain(envelope, blocks, options); + } + + const stdoutLines: string[] = []; + blocks.forEach((block) => { + for (const line of block.lines) { + if (!line.isMatch) continue; + stdoutLines.push(renderPlainLine(block.filePath, line, false)); + } + }); + stdoutLines.push(""); + + return { + stdout: stdoutLines.join("\n"), + stderr: formatTerminalNotes(envelope, options.useColors), + }; +} + +function formatHeadingPlain( + envelope: LeanGrepRepoEnvelope, + blocks: RenderBlock[], + options: FormatGrepRepoTerminalOptions, +): FormattedGrepRepoTerminal { + const lines: string[] = []; + const blocksByFile = groupBlocksByFile(blocks); + const withContext = options.withContext ?? false; + + for (const [filePath, fileBlocks] of blocksByFile) { + if (lines.length > 0) lines.push(""); + lines.push(filePath); + fileBlocks.forEach((block, index) => { + if (withContext && index > 0) lines.push("--"); + for (const line of block.lines) { + if (!withContext && !line.isMatch) continue; + lines.push(renderHeadingLine(line, withContext)); + } + }); + } + lines.push(""); + + return { + stdout: `${lines.join("\n")}`, + stderr: formatTerminalNotes(envelope, options.useColors), + }; +} + +function formatVerbose( + envelope: LeanGrepRepoEnvelope, + blocks: RenderBlock[], + options: FormatGrepRepoTerminalOptions, +): FormattedGrepRepoTerminal { + const lines: string[] = []; + lines.push( + colorize( + `${formatCount(envelope.totalMatches, "match", "matches")} in ${formatCount(envelope.uniqueFilesMatched, "file")}`, + "bold", + options.useColors, + ), + ); + if (envelope.indexedVersion) { + lines.push(dim(`Indexed ${envelope.indexedVersion}`, options.useColors)); + } + lines.push(""); + + const blocksByFile = groupBlocksByFile(blocks); + for (const [filePath, fileBlocks] of blocksByFile) { + lines.push(colorize(filePath, "bold", options.useColors)); + const gutterWidth = widestLineNumberInBlocks(fileBlocks); + fileBlocks.forEach((block, index) => { + if (index > 0) lines.push(dim("--", options.useColors)); + for (const line of block.lines) { + lines.push(renderVerboseLine(line, gutterWidth, options.useColors)); + } + }); + lines.push(""); + } + + return { + stdout: `${lines.join("\n").trimEnd()}\n`, + stderr: formatTerminalNotes(envelope, options.useColors), + }; +} + +function buildRenderBlocks(matches: LeanGrepRepoMatch[]): RenderBlock[] { + if (matches.length === 0) return []; + + const linesByFile = new Map>(); + for (const match of matches) { + let lineMap = linesByFile.get(match.filePath); + if (!lineMap) { + lineMap = new Map(); + linesByFile.set(match.filePath, lineMap); + } + + const contextBefore = match.contextBefore ?? []; + const beforeStart = match.line - contextBefore.length; + for (let index = 0; index < contextBefore.length; index += 1) { + const lineNumber = beforeStart + index; + if (!lineMap.has(lineNumber)) { + lineMap.set(lineNumber, { + lineNumber, + content: contextBefore[index] ?? "", + isMatch: false, + }); + } + } + + lineMap.set(match.line, { + lineNumber: match.line, + content: match.lineContent, + isMatch: true, + }); + + const contextAfter = match.contextAfter ?? []; + for (let index = 0; index < contextAfter.length; index += 1) { + const lineNumber = match.line + index + 1; + if (!lineMap.has(lineNumber)) { + lineMap.set(lineNumber, { + lineNumber, + content: contextAfter[index] ?? "", + isMatch: false, + }); + } + } + } + + const blocks: RenderBlock[] = []; + for (const [filePath, lineMap] of linesByFile) { + const sortedLines = [...lineMap.values()].sort( + (left, right) => left.lineNumber - right.lineNumber, + ); + + let current: RenderLine[] = []; + for (const line of sortedLines) { + const previous = current[current.length - 1]; + if (!previous || line.lineNumber === previous.lineNumber + 1) { + current.push(line); + continue; + } + + blocks.push({ filePath, lines: current }); + current = [line]; + } + + if (current.length > 0) { + blocks.push({ filePath, lines: current }); + } + } + + return blocks; +} + +function renderPlainLine( + filePath: string, + line: RenderLine, + withContext = false, +): string { + if (!withContext || line.isMatch) { + return withContext + ? `${line.lineNumber}:${line.content}` + : `${filePath}:${line.lineNumber}:${line.content}`; + } + + return `${line.lineNumber}-${line.content}`; +} + +function renderHeadingLine(line: RenderLine, withContext: boolean): string { + if (!withContext || line.isMatch) { + return `${line.lineNumber}:${line.content}`; + } + + return `${line.lineNumber}-${line.content}`; +} + +function groupBlocksByFile(blocks: RenderBlock[]): Map { + const grouped = new Map(); + for (const block of blocks) { + const existing = grouped.get(block.filePath); + if (existing) { + existing.push(block); + continue; + } + grouped.set(block.filePath, [block]); + } + return grouped; +} + +function renderVerboseLine( + line: RenderLine, + gutterWidth: number, + useColors: boolean, +): string { + const gutter = padLeft(String(line.lineNumber), gutterWidth); + if (line.isMatch) { + return `${colorize(">", "bold", useColors)} ${gutter} ${colorize(line.content, "bold", useColors)}`; + } + + return ` ${dim(gutter, useColors)} ${dim(line.content, useColors)}`; +} + +function widestLineNumberInBlocks(blocks: RenderBlock[]): number { + let maxWidth = 1; + for (const block of blocks) { + for (const line of block.lines) { + maxWidth = Math.max(maxWidth, String(line.lineNumber).length); + } + } + return maxWidth; +} + +function formatTerminalNotes( + envelope: LeanGrepRepoEnvelope, + useColors: boolean, +): string | undefined { + const lines: string[] = []; + + if (shouldSuggestNarrowingScope(envelope)) { + lines.push( + dim( + "Broad results — consider narrowing with a path prefix, --path, --glob, --ext, --exclude-docs, or --exclude-tests.", + useColors, + ), + ); + } + + if (envelope.hasMore && envelope.nextCursor) { + lines.push( + dim( + `More grep results available — rerun with --cursor ${shellQuote(envelope.nextCursor)}`, + useColors, + ), + ); + } + + if (lines.length === 0) return undefined; + + return `${lines.join("\n")}\n`; +} + +function shouldSuggestNarrowingScope(envelope: LeanGrepRepoEnvelope): boolean { + const filter = envelope.filter; + const hasScopeFilter = Boolean( + filter?.path || + filter?.pathPrefix || + (filter?.globs && filter.globs.length > 0) || + (filter?.extensions && filter.extensions.length > 0) || + filter?.excludeDocFiles || + filter?.excludeTestFiles, + ); + + return ( + !hasScopeFilter && + envelope.uniqueFilesMatched >= 5 && + envelope.matches.length >= 5 + ); +} + +function formatCount( + count: number, + singular: string, + plural = `${singular}s`, +): string { + return `${count} ${count === 1 ? singular : plural}`; +} + +function shellQuote(value: string): string { + return `'${value.replaceAll("'", `'"'"'`)}'`; +} + +function padLeft(text: string, width: number): string { + return text.length >= width + ? text + : `${" ".repeat(width - text.length)}${text}`; +} diff --git a/src/shared/index.ts b/src/shared/index.ts index 8861ef1c..ae5c4e3e 100644 --- a/src/shared/index.ts +++ b/src/shared/index.ts @@ -33,6 +33,22 @@ export { warning, } from "./colors.js"; export { debugLog } from "./debug-log.js"; +export { + buildGrepRepoParams, + GREP_REPO_PATTERN_NOTE, + type GrepRepoRequestBuildResult, + type GrepRepoRequestInput, +} from "./grep-repo-request.js"; +export { + type BuildGrepRepoPayloadOptions, + buildGrepRepoSuccessPayload, + type FormatGrepRepoTerminalOptions, + type FormattedGrepRepoTerminal, + formatGrepRepoTerminal, + type LeanGrepRepoEnvelope, + type LeanGrepRepoFilter, + type LeanGrepRepoMatch, +} from "./grep-repo-response.js"; export { filterLanguages, type LanguageMatch, @@ -124,12 +140,23 @@ export { type SearchSymbolsQueryEcho, type SearchSymbolsSuccessPayload, } from "./search-symbols-response.js"; +export { + endTelemetrySpan, + flushTelemetry, + isTelemetryEnabled, + resetTelemetryCollectorForTests, + startTelemetrySpan, + type TelemetryAttributes, + TelemetryCollector, + type TelemetrySpanHandle, + withTelemetrySpan, + withTelemetrySpanSync, +} from "./telemetry.js"; export { buildUnifiedSearchParams, type UnifiedSearchRequestBuildResult, type UnifiedSearchRequestInput, } from "./unified-search-request.js"; -export { parseUnifiedSearchTargetSpec } from "./unified-search-target.js"; export { buildUnifiedSearchErrorPayload, buildUnifiedSearchStatusPayload, @@ -143,15 +170,4 @@ export { type UnifiedSearchStatusIncompletePayload, type UnifiedSearchStatusResultPayload, } from "./unified-search-response.js"; -export { - endTelemetrySpan, - flushTelemetry, - isTelemetryEnabled, - resetTelemetryCollectorForTests, - startTelemetrySpan, - type TelemetryAttributes, - TelemetryCollector, - type TelemetrySpanHandle, - withTelemetrySpan, - withTelemetrySpanSync, -} from "./telemetry.js"; +export { parseUnifiedSearchTargetSpec } from "./unified-search-target.js"; diff --git a/src/shared/read-file-response.ts b/src/shared/read-file-response.ts index 1aa369fd..1d4248ed 100644 --- a/src/shared/read-file-response.ts +++ b/src/shared/read-file-response.ts @@ -18,8 +18,8 @@ export interface LeanReadFileEnvelope { gitRef?: string; /** * File path. Named `path` (not `filePath`) so the envelope key - * matches `list_files.files[].path` and `grep_file`'s `path` - * input — keeps the `list_files` → `read_file` / `grep_file` + * matches `list_files.files[].path` and `grep_repo`'s exact-file + * `path` input — keeps the `list_files` → `read_file` / `grep_repo` * chain free of rename friction. */ path: string; diff --git a/src/tools/grep-file.test.ts b/src/tools/grep-file.test.ts deleted file mode 100644 index a6588570..00000000 --- a/src/tools/grep-file.test.ts +++ /dev/null @@ -1,209 +0,0 @@ -import { describe, expect, it, mock } from "bun:test"; -import { - CodeNavigationIndexingError, - CodeNavigationTargetNotFoundError, -} from "../services/index.js"; -import { - createMockCodeNavigationService, - defaultGrepFileResult, -} from "../services/test-helpers.js"; -import { createGrepFileTool } from "./grep-file.js"; - -function parseText(result: { content: Array<{ text: string }> }): unknown { - return JSON.parse(result.content[0]?.text ?? ""); -} - -describe("createGrepFileTool — metadata", () => { - it("registers the correct tool name, description, and schema keys", () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - expect(tool.name).toBe("grep_file"); - expect(tool.description).toContain("case-insensitive substring"); - expect(tool.description).toContain("not regex"); - expect(Object.keys(tool.schema).sort()).toEqual([ - "context_lines", - "max_matches", - "path", - "pattern", - "target", - "wait_timeout_ms", - ]); - expect(tool.annotations?.readOnlyHint).toBe(true); - }); -}); - -describe("createGrepFileTool — happy path", () => { - it("calls grepFile with the resolved target, path, pattern", async () => { - const grepFile = mock(() => Promise.resolve(defaultGrepFileResult)); - const service = createMockCodeNavigationService({ grepFile }); - const tool = createGrepFileTool(service); - - await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "middleware", - }, - {}, - ); - - const calls = grepFile.mock.calls as unknown as Array< - [{ target: { registry?: string }; path: string; pattern: string }] - >; - expect(calls[0]?.[0]?.target?.registry).toBe("NPM"); - expect(calls[0]?.[0]?.path).toBe("src/index.js"); - expect(calls[0]?.[0]?.pattern).toBe("middleware"); - }); - - it("emits envelope with matches + metadata", async () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "express();", - }, - {}, - ); - const payload = parseText(result) as { - registry: string; - pattern: string; - path: string; - totalMatches: number; - matches: Array<{ lineNumber: number }>; - }; - expect(payload.registry).toBe("npm"); - expect(payload.pattern).toBe("express();"); - expect(payload.path).toBe("src/index.js"); - expect(payload.totalMatches).toBe(1); - expect(payload.matches[0]?.lineNumber).toBe(4); - }); - - it("emits filter.contextLines + filter.maxMatches when explicit", async () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "middleware", - context_lines: 5, - max_matches: 100, - }, - {}, - ); - const payload = parseText(result) as { - filter?: { contextLines?: number; maxMatches?: number }; - }; - expect(payload.filter).toEqual({ contextLines: 5, maxMatches: 100 }); - }); -}); - -describe("createGrepFileTool — validation errors", () => { - it("returns INVALID_ARGUMENT for empty pattern", async () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "", - }, - {}, - ); - expect(result.isError).toBe(true); - expect((parseText(result) as { code: string }).code).toBe( - "INVALID_ARGUMENT", - ); - }); - - it("returns INVALID_ARGUMENT for empty path", async () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: " ", - pattern: "middleware", - }, - {}, - ); - expect(result.isError).toBe(true); - expect((parseText(result) as { code: string }).code).toBe( - "INVALID_ARGUMENT", - ); - }); - - it("returns INVALID_ARGUMENT for pattern > 200 chars", async () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "a".repeat(201), - }, - {}, - ); - expect(result.isError).toBe(true); - const payload = parseText(result) as { code: string; error: string }; - expect(payload.code).toBe("INVALID_ARGUMENT"); - expect(payload.error).toContain("200"); - }); - - it("returns INVALID_ARGUMENT for context_lines out of range", async () => { - const tool = createGrepFileTool(createMockCodeNavigationService()); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "middleware", - context_lines: 11, - }, - {}, - ); - expect(result.isError).toBe(true); - expect((parseText(result) as { code: string }).code).toBe( - "INVALID_ARGUMENT", - ); - }); -}); - -describe("createGrepFileTool — service errors", () => { - it("classifies CodeNavigationIndexingError as INDEXING", async () => { - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.reject( - new CodeNavigationIndexingError("Indexing...", "ref_abc"), - ), - ), - }); - const tool = createGrepFileTool(service); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "express" }, - path: "src/index.js", - pattern: "middleware", - }, - {}, - ); - expect(result.isError).toBe(true); - expect((parseText(result) as { code: string }).code).toBe("INDEXING"); - }); - - it("classifies CodeNavigationTargetNotFoundError as NOT_FOUND", async () => { - const service = createMockCodeNavigationService({ - grepFile: mock(() => - Promise.reject( - new CodeNavigationTargetNotFoundError("Package not found"), - ), - ), - }); - const tool = createGrepFileTool(service); - const result = await tool.handler( - { - target: { registry: "npm", package_name: "ghost" }, - path: "src/index.js", - pattern: "middleware", - }, - {}, - ); - expect(result.isError).toBe(true); - expect((parseText(result) as { code: string }).code).toBe("NOT_FOUND"); - }); -}); diff --git a/src/tools/grep-file.ts b/src/tools/grep-file.ts deleted file mode 100644 index 81147999..00000000 --- a/src/tools/grep-file.ts +++ /dev/null @@ -1,123 +0,0 @@ -import { z } from "zod"; -import type { CodeNavigationService } from "../services/index.js"; -import { mapCodeNavigationError } from "../shared/code-navigation-error-map.js"; -import { - buildGrepFileParams, - GREP_PATTERN_SEMANTICS_NOTE, -} from "../shared/grep-file-request.js"; -import { buildGrepFileSuccessPayload } from "../shared/grep-file-response.js"; -import { toPkgseerRegistryLowercase } from "../shared/pkgseer-registry.js"; -import { - type CodeTargetArg, - codeTargetSchema, - resolveCodeTarget, -} from "./code-navigation-shared.js"; -import { errorResult, type ToolDefinition, textResult } from "./types.js"; - -export interface GrepFileArgs { - target: CodeTargetArg; - path: string; - pattern: string; - context_lines?: number; - max_matches?: number; - wait_timeout_ms?: number; -} - -const schema = { - target: codeTargetSchema, - path: z - .string() - .describe( - "Path to the file to search. Package addressing: package-relative. Repo addressing: repo-relative.", - ), - pattern: z - .string() - .describe( - `${GREP_PATTERN_SEMANTICS_NOTE} For symbol-shaped searches prefer unified \`search\` with \`sources:["symbol"]\`.`, - ), - context_lines: z - .number() - .optional() - .describe( - "Lines of context before and after each match (0–10, default 0 — matches only). Set explicitly when you need surrounding lines; nearby matches with overlapping context are returned unmerged (each match carries its own `contextBefore` / `contextAfter`).", - ), - max_matches: z - .number() - .optional() - .describe("Max matches to return (1–200, default 50)."), - wait_timeout_ms: z - .number() - .optional() - .describe( - "Max milliseconds to wait for indexing (0–60000, default 20000). On an `INDEXING` error envelope, retry with a longer timeout or pass a version from `details.availableVersions`.", - ), -}; - -const DESCRIPTION = - "Search within a single file for a case-insensitive substring " + - "(not regex). Returns matches only by default — pass " + - "`context_lines` for surrounding lines (0–10, default 0). " + - `${GREP_PATTERN_SEMANTICS_NOTE} ` + - "Response: `{pattern, path, totalMatches, hasMore, matches: " + - "[{lineNumber, lineContent, contextBefore, contextAfter}], " + - "language, totalLines}`. The `path` field matches `list_files`' " + - "entry `path` and `read_file`'s `path` input, so chaining tools " + - "needs no renames. Address via `target.registry` + " + - "`target.package_name` (package scope) or `target.repo_url` + " + - "`target.git_ref` (repo scope), mutually exclusive. For " + - 'symbol-shaped searches prefer unified `search` with `sources:["symbol"]`. When the path ' + - "doesn't resolve the response is a `NOT_FOUND` (or " + - "`FILE_NOT_FOUND`) error — call `list_files` to check the " + - "actual paths."; - -export function createGrepFileTool( - service: CodeNavigationService, -): ToolDefinition { - return { - name: "grep_file", - description: DESCRIPTION, - schema, - annotations: { readOnlyHint: true }, - handler: async (args) => { - const target = resolveCodeTarget(args.target); - if ("content" in target) return target; - - try { - const build = buildGrepFileParams({ - target, - path: args.path, - pattern: args.pattern, - contextLines: args.context_lines, - maxMatches: args.max_matches, - waitTimeoutMs: args.wait_timeout_ms, - }); - const result = await service.grepFile(build.params); - const payload = buildGrepFileSuccessPayload(result, { - registry: target.registry - ? toPkgseerRegistryLowercase(target.registry) - : undefined, - name: target.packageName, - repoUrl: target.repoUrl, - gitRef: target.gitRef, - pattern: build.params.pattern, - path: build.params.path, - contextLinesExplicit: build.contextLinesExplicit, - maxMatchesExplicit: build.maxMatchesExplicit, - contextLines: build.params.contextLines ?? 0, - maxMatches: build.params.maxMatches ?? 50, - }); - return textResult(JSON.stringify(payload)); - } catch (error) { - const mapped = mapCodeNavigationError(error); - return errorResult( - JSON.stringify({ - error: mapped.message, - code: mapped.code, - retryable: mapped.retryable ?? false, - ...(mapped.details ? { details: mapped.details } : {}), - }), - ); - } - }, - }; -} diff --git a/src/tools/grep-file-parity.test.ts b/src/tools/grep-repo-parity.test.ts similarity index 57% rename from src/tools/grep-file-parity.test.ts rename to src/tools/grep-repo-parity.test.ts index 311edaba..58cc1d9b 100644 --- a/src/tools/grep-file-parity.test.ts +++ b/src/tools/grep-repo-parity.test.ts @@ -1,8 +1,3 @@ -// PARITY TEST — enforces: -// PARITY-JSON-KEYS CLI --json output and MCP text payload parse to -// deepEqual JSON objects for equivalent inputs. -// PARITY-ERROR-ENVELOPE Both surfaces emit { error, code, retryable, details? }. - import { describe, expect, it, mock, spyOn } from "bun:test"; import { type PkgGrepCommandDependencies, @@ -11,13 +6,13 @@ import { import { CodeNavigationIndexingError, CodeNavigationTargetNotFoundError, - type GrepFileResult, + type GrepRepoResult, } from "../services/index.js"; import { createMockCodeNavigationService, - defaultGrepFileResult, + defaultGrepRepoResult, } from "../services/test-helpers.js"; -import { createGrepFileTool } from "./grep-file.js"; +import { createGrepRepoTool } from "./grep-repo.js"; function cliDeps( overrides: Partial = {}, @@ -83,60 +78,34 @@ interface McpArgs { repo_url?: string; git_ref?: string; }; - path: string; pattern: string; - context_lines?: number; - max_matches?: number; + path_prefix?: string; wait_timeout_ms?: number; } async function mcpJson( args: McpArgs, - grepFileMock?: () => Promise, + grepRepoMock?: () => Promise, ): Promise { const service = createMockCodeNavigationService( - grepFileMock ? { grepFile: grepFileMock as never } : {}, + grepRepoMock ? { grepRepo: grepRepoMock as never } : {}, ); - const tool = createGrepFileTool(service); + const tool = createGrepRepoTool(service); const result = await tool.handler(args, {}); return JSON.parse(result.content[0]?.text ?? ""); } -describe("grep_file parity", () => { +describe("grep_repo parity", () => { it("PARITY-JSON-KEYS: happy package grep CLI === MCP", async () => { - const fn = mock(() => Promise.resolve(defaultGrepFileResult)); + const fn = mock(() => Promise.resolve(defaultGrepRepoResult)); const cli = await cliJson( "npm:express", "middleware", - "src/index.js", + "src/", {}, cliDeps({ codeNavigationService: createMockCodeNavigationService({ - grepFile: fn as never, - }), - }), - ); - const mcp = await mcpJson( - { - target: { registry: "npm", package_name: "express" }, - pattern: "middleware", - path: "src/index.js", - }, - fn as never, - ); - expect(cli).toEqual(mcp); - }); - - it("PARITY-JSON-KEYS: filter echoes context + max_matches on both surfaces", async () => { - const fn = mock(() => Promise.resolve(defaultGrepFileResult)); - const cli = await cliJson( - "npm:express", - "middleware", - "src/index.js", - { context: "5", limit: "100" }, - cliDeps({ - codeNavigationService: createMockCodeNavigationService({ - grepFile: fn as never, + grepRepo: fn as never, }), }), ); @@ -144,46 +113,7 @@ describe("grep_file parity", () => { { target: { registry: "npm", package_name: "express" }, pattern: "middleware", - path: "src/index.js", - context_lines: 5, - max_matches: 100, - }, - fn as never, - ); - expect(cli).toEqual(mcp); - expect( - (cli as { filter?: { contextLines?: number; maxMatches?: number } }) - .filter, - ).toEqual({ - contextLines: 5, - maxMatches: 100, - }); - }); - - it("PARITY-JSON-KEYS: repo-URL addressing CLI === MCP", async () => { - const fn = mock(() => Promise.resolve(defaultGrepFileResult)); - const cli = await cliJson( - "middleware", - "src/index.js", - undefined, - { - repoUrl: "https://github.com/expressjs/express", - gitRef: "main", - }, - cliDeps({ - codeNavigationService: createMockCodeNavigationService({ - grepFile: fn as never, - }), - }), - ); - const mcp = await mcpJson( - { - target: { - repo_url: "https://github.com/expressjs/express", - git_ref: "main", - }, - pattern: "middleware", - path: "src/index.js", + path_prefix: "src/", }, fn as never, ); @@ -201,11 +131,11 @@ describe("grep_file parity", () => { const cli = await cliJson( "npm:express", "middleware", - "src/index.js", + undefined, {}, cliDeps({ codeNavigationService: createMockCodeNavigationService({ - grepFile: fn as never, + grepRepo: fn as never, }), }), ); @@ -213,7 +143,6 @@ describe("grep_file parity", () => { { target: { registry: "npm", package_name: "express" }, pattern: "middleware", - path: "src/index.js", }, fn as never, ); @@ -224,25 +153,24 @@ describe("grep_file parity", () => { it("PARITY-ERROR-ENVELOPE: NOT_FOUND identical on both surfaces", async () => { const fn = mock(() => Promise.reject( - new CodeNavigationTargetNotFoundError("File not found in repository"), + new CodeNavigationTargetNotFoundError("Package not found"), ), ); const cli = await cliJson( - "npm:express", + "npm:ghost", "middleware", - "nope.js", + undefined, {}, cliDeps({ codeNavigationService: createMockCodeNavigationService({ - grepFile: fn as never, + grepRepo: fn as never, }), }), ); const mcp = await mcpJson( { - target: { registry: "npm", package_name: "express" }, + target: { registry: "npm", package_name: "ghost" }, pattern: "middleware", - path: "nope.js", }, fn as never, ); @@ -250,14 +178,13 @@ describe("grep_file parity", () => { expect((cli as { code: string }).code).toBe("NOT_FOUND"); }); - it("PARITY-ERROR-ENVELOPE: INVALID_ARGUMENT for empty pattern on both surfaces", async () => { - const cli = await cliJson("npm:express", "", "src/index.js", {}); + it("PARITY-ERROR-ENVELOPE: whitespace-only pattern is INVALID_ARGUMENT on both surfaces", async () => { + const cli = await cliJson("npm:express", " ", undefined); const mcp = await mcpJson({ target: { registry: "npm", package_name: "express" }, - pattern: "", - path: "src/index.js", + pattern: " ", }); - expect(cli).toMatchObject({ code: "INVALID_ARGUMENT" }); - expect(mcp).toMatchObject({ code: "INVALID_ARGUMENT" }); + expect(cli).toEqual(mcp); + expect((cli as { code: string }).code).toBe("INVALID_ARGUMENT"); }); }); diff --git a/src/tools/grep-repo.test.ts b/src/tools/grep-repo.test.ts new file mode 100644 index 00000000..b1915007 --- /dev/null +++ b/src/tools/grep-repo.test.ts @@ -0,0 +1,174 @@ +import { describe, expect, it, mock } from "bun:test"; +import { + CodeNavigationIndexingError, + CodeNavigationTargetNotFoundError, +} from "../services/index.js"; +import { + createMockCodeNavigationService, + defaultGrepRepoResult, +} from "../services/test-helpers.js"; +import { createGrepRepoTool } from "./grep-repo.js"; + +function parseText(result: { content: Array<{ text: string }> }): unknown { + return JSON.parse(result.content[0]?.text ?? ""); +} + +describe("createGrepRepoTool — metadata", () => { + it("registers the correct tool name and schema keys", () => { + const tool = createGrepRepoTool(createMockCodeNavigationService()); + expect(tool.name).toBe("grep_repo"); + expect(tool.description).toContain("Deterministic text grep"); + expect(Object.keys(tool.schema).sort()).toEqual([ + "case_sensitive", + "context_lines", + "context_lines_after", + "context_lines_before", + "cursor", + "exclude_doc_files", + "exclude_test_files", + "extensions", + "globs", + "max_matches", + "max_matches_per_file", + "path", + "path_prefix", + "pattern", + "pattern_type", + "target", + "wait_timeout_ms", + ]); + expect(tool.annotations?.readOnlyHint).toBe(true); + }); +}); + +describe("createGrepRepoTool — happy path", () => { + it("calls grepRepo with resolved target and grep params", async () => { + const grepRepo = mock(() => Promise.resolve(defaultGrepRepoResult)); + const service = createMockCodeNavigationService({ grepRepo }); + const tool = createGrepRepoTool(service); + + await tool.handler( + { + target: { registry: "npm", package_name: "express" }, + pattern: "middleware", + path_prefix: "src/", + globs: ["src/**/*.js"], + extensions: ["js"], + pattern_type: "regex", + case_sensitive: true, + context_lines_before: 2, + context_lines_after: 1, + }, + {}, + ); + + const calls = grepRepo.mock.calls as unknown as Array< + [ + { + target: { registry?: string }; + pattern: string; + patternType?: string; + pathSelectors?: Array<{ kind: string; value: string }>; + extensions?: string[]; + caseSensitive?: boolean; + contextLinesBefore?: number; + contextLinesAfter?: number; + }, + ] + >; + expect(calls[0]?.[0]?.target.registry).toBe("NPM"); + expect(calls[0]?.[0]?.pattern).toBe("middleware"); + expect(calls[0]?.[0]?.patternType).toBe("REGEX"); + expect(calls[0]?.[0]?.caseSensitive).toBe(true); + expect(calls[0]?.[0]?.extensions).toEqual(["js"]); + expect(calls[0]?.[0]?.pathSelectors).toEqual([ + { kind: "PREFIX", value: "src/" }, + { kind: "GLOB", value: "src/**/*.js" }, + ]); + }); + + it("emits the new envelope shape", async () => { + const tool = createGrepRepoTool(createMockCodeNavigationService()); + const result = await tool.handler( + { + target: { registry: "npm", package_name: "express" }, + pattern: "middleware", + }, + {}, + ); + const payload = parseText(result) as { + pattern: string; + patternType: string; + caseSensitive: boolean; + totalMatches: number; + matches: Array<{ filePath: string; line: number }>; + }; + expect(payload.pattern).toBe("middleware"); + expect(payload.patternType).toBe("literal"); + expect(payload.caseSensitive).toBe(false); + expect(payload.totalMatches).toBe(1); + expect(payload.matches[0]).toMatchObject({ + filePath: "src/index.js", + line: 4, + }); + }); +}); + +describe("createGrepRepoTool — validation errors", () => { + it("returns INVALID_ARGUMENT for empty pattern", async () => { + const tool = createGrepRepoTool(createMockCodeNavigationService()); + const result = await tool.handler( + { + target: { registry: "npm", package_name: "express" }, + pattern: "", + }, + {}, + ); + expect(result.isError).toBe(true); + expect((parseText(result) as { code: string }).code).toBe( + "INVALID_ARGUMENT", + ); + }); +}); + +describe("createGrepRepoTool — service errors", () => { + it("classifies CodeNavigationIndexingError as INDEXING", async () => { + const service = createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.reject( + new CodeNavigationIndexingError("Indexing...", "ref_abc"), + ), + ), + }); + const tool = createGrepRepoTool(service); + const result = await tool.handler( + { + target: { registry: "npm", package_name: "express" }, + pattern: "middleware", + }, + {}, + ); + expect(result.isError).toBe(true); + expect((parseText(result) as { code: string }).code).toBe("INDEXING"); + }); + + it("classifies CodeNavigationTargetNotFoundError as NOT_FOUND", async () => { + const service = createMockCodeNavigationService({ + grepRepo: mock(() => + Promise.reject( + new CodeNavigationTargetNotFoundError("Package not found"), + ), + ), + }); + const tool = createGrepRepoTool(service); + const result = await tool.handler( + { + target: { registry: "npm", package_name: "ghost" }, + pattern: "middleware", + }, + {}, + ); + expect(result.isError).toBe(true); + expect((parseText(result) as { code: string }).code).toBe("NOT_FOUND"); + }); +}); diff --git a/src/tools/grep-repo.ts b/src/tools/grep-repo.ts new file mode 100644 index 00000000..f2e67cc0 --- /dev/null +++ b/src/tools/grep-repo.ts @@ -0,0 +1,158 @@ +import { z } from "zod"; +import type { CodeNavigationService } from "../services/index.js"; +import { mapCodeNavigationError } from "../shared/code-navigation-error-map.js"; +import { + buildGrepRepoParams, + buildGrepRepoSuccessPayload, + GREP_REPO_PATTERN_NOTE, +} from "../shared/index.js"; +import { toPkgseerRegistryLowercase } from "../shared/pkgseer-registry.js"; +import { + type CodeTargetArg, + codeTargetSchema, + resolveCodeTarget, +} from "./code-navigation-shared.js"; +import { errorResult, type ToolDefinition, textResult } from "./types.js"; + +export interface GrepRepoArgs { + target: CodeTargetArg; + pattern: string; + path?: string; + path_prefix?: string; + globs?: string[]; + extensions?: string[]; + pattern_type?: "literal" | "regex"; + case_sensitive?: boolean; + exclude_doc_files?: boolean; + exclude_test_files?: boolean; + context_lines?: number; + context_lines_before?: number; + context_lines_after?: number; + max_matches?: number; + max_matches_per_file?: number; + cursor?: string; + wait_timeout_ms?: number; +} + +const pathSelectorSchema = z.object({ + kind: z.enum(["exact", "prefix", "glob"]), + value: z.string(), +}); + +const schema = { + target: codeTargetSchema, + pattern: z.string().describe(GREP_REPO_PATTERN_NOTE), + path: z + .string() + .optional() + .describe( + "Exact file path to grep. Shares the same path vocabulary as `read_file`.", + ), + path_prefix: z + .string() + .optional() + .describe( + "Literal directory prefix to scope grep, matching `list_files` / `search` naming.", + ), + globs: z + .array(z.string()) + .optional() + .describe( + "Repeatable glob scopes with real glob semantics (e.g. `src/**/*.ts`).", + ), + extensions: z + .array(z.string()) + .optional() + .describe("Extensions to include, without a leading dot."), + pattern_type: z.enum(["literal", "regex"]).optional(), + case_sensitive: z.boolean().optional(), + exclude_doc_files: z.boolean().optional(), + exclude_test_files: z.boolean().optional(), + context_lines: z.number().optional(), + context_lines_before: z.number().optional(), + context_lines_after: z.number().optional(), + max_matches: z.number().optional(), + max_matches_per_file: z.number().optional(), + cursor: z.string().optional(), + wait_timeout_ms: z.number().optional(), +}; + +const DESCRIPTION = + "Deterministic text grep over indexed dependency and repository source files. " + + "Use this when you know the text pattern you want; use `search` for discovery. " + + "Whole-target grep is the default. Narrow with `path`, `path_prefix`, `globs`, or `extensions`. " + + "Matches chain directly into `read_file` via `matches[].filePath`."; + +export function createGrepRepoTool( + service: CodeNavigationService, +): ToolDefinition { + return { + name: "grep_repo", + description: DESCRIPTION, + schema, + annotations: { readOnlyHint: true }, + handler: async (args) => { + const target = resolveCodeTarget(args.target); + if ("content" in target) return target; + + try { + const build = buildGrepRepoParams({ + target, + pattern: args.pattern, + path: args.path, + pathPrefix: args.path_prefix, + globs: args.globs, + extensions: args.extensions, + patternType: args.pattern_type, + caseSensitive: args.case_sensitive, + excludeDocFiles: args.exclude_doc_files, + excludeTestFiles: args.exclude_test_files, + contextLines: args.context_lines, + contextLinesBefore: args.context_lines_before, + contextLinesAfter: args.context_lines_after, + maxMatches: args.max_matches, + maxMatchesPerFile: args.max_matches_per_file, + cursor: args.cursor, + waitTimeoutMs: args.wait_timeout_ms, + }); + const result = await service.grepRepo(build.params); + const payload = buildGrepRepoSuccessPayload(result, { + registry: target.registry + ? toPkgseerRegistryLowercase(target.registry) + : undefined, + name: target.packageName, + repoUrl: target.repoUrl, + gitRef: target.gitRef, + pattern: build.params.pattern, + patternType: + build.params.patternType === "REGEX" ? "regex" : "literal", + caseSensitive: build.params.caseSensitive ?? false, + path: args.path, + pathPrefix: args.path_prefix, + globs: args.globs, + extensions: args.extensions, + contextLines: args.context_lines, + contextLinesBefore: build.params.contextLinesBefore ?? 0, + contextLinesAfter: build.params.contextLinesAfter ?? 0, + maxMatches: build.params.maxMatches ?? 50, + maxMatchesPerFile: build.params.maxMatchesPerFile, + cursor: args.cursor, + excludeDocFiles: build.params.excludeDocFiles, + excludeTestFiles: build.params.excludeTestFiles, + explicit: build.explicit, + }); + return textResult(JSON.stringify(payload)); + } catch (error) { + const mapped = mapCodeNavigationError(error); + return errorResult( + JSON.stringify({ + error: mapped.message, + code: mapped.code, + retryable: mapped.retryable ?? false, + ...(mapped.details ? { details: mapped.details } : {}), + }), + ); + } + }, + }; +} diff --git a/src/tools/index.ts b/src/tools/index.ts index a52d616f..ed2419c5 100644 --- a/src/tools/index.ts +++ b/src/tools/index.ts @@ -1,6 +1,6 @@ export { createFeedbackTool } from "./feedback.js"; export { createGetExampleTool } from "./get-example.js"; -export { createGrepFileTool } from "./grep-file.js"; +export { createGrepRepoTool } from "./grep-repo.js"; export { createListFilesTool } from "./list-files.js"; export { createPackageChangelogTool } from "./package-changelog.js"; export { createPackageDependenciesTool } from "./package-dependencies.js"; diff --git a/src/tools/list-files.ts b/src/tools/list-files.ts index 9342a7e1..c570ff42 100644 --- a/src/tools/list-files.ts +++ b/src/tools/list-files.ts @@ -48,7 +48,7 @@ const DESCRIPTION = "`target.git_ref` (repo scope), mutually exclusive. `path_prefix` " + "is a literal directory prefix — it does NOT accept globs " + "(`*.ts`) or extension filters. The returned `path` values feed " + - "directly into `read_file` and `grep_file`. Returns an `INDEXING` " + + "directly into `read_file` and help scope `grep_repo`. Returns an `INDEXING` " + "error envelope when the dependency is being indexed on-demand — " + "retry with a longer `wait_timeout_ms` or use a version from " + "`details.availableVersions`."; diff --git a/tsconfig.json b/tsconfig.json index 03669c7e..8a95d5d2 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -26,5 +26,6 @@ "noUnusedLocals": false, "noUnusedParameters": false, "noPropertyAccessFromIndexSignature": false - } + }, + "exclude": ["dist"] }