Skip to content

Speed up OR search, fd filename search, and preview#30

Merged
oahsiao merged 1 commit into
mainfrom
perf/parallel-search-and-preview
Jun 24, 2026
Merged

Speed up OR search, fd filename search, and preview#30
oahsiao merged 1 commit into
mainfrom
perf/parallel-search-and-preview

Conversation

@oahsiao

@oahsiao oahsiao commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Performance improvements to the search backend (P0 + P1 from the optimization review).

P0 — search core

  • OR content search → single ripgrep process. Multi-keyword OR previously spawned one ripgrep per keyword and traversed the filesystem N times. It now emits multiple -e patterns to a single process, scanning the tree once. rgSearchArgs accepts a keyword array; using -e also protects patterns starting with -.
    • Behavior note: a line matching several keywords now counts as 1 (matching line) instead of summing per keyword — arguably more correct.

P1 — filename search + preview

  • fd filename search runs keywords concurrently (Promise.all) to overlap filesystem I/O instead of one scan per keyword.
  • Preview reads files asynchronously (fs.promises.readFile) so the main process is no longer blocked on large files.
  • Preview lowercases each line once (not once per keyword) and collects match line numbers as a pre-sorted, de-duplicated array, removing the Set + sort.

Testing

  • npm test — all 5 self-tests pass.
  • End-to-end smoke tests with real rg/fd against the repo:
    • OR content search (single process): 30 files
    • AND intersection: 19 files
    • fd OR (parallel): rg.js, fd.js
    • async preview: builds correctly

- OR content search now runs as a single ripgrep process using multiple -e patterns, traversing the filesystem once instead of once per keyword. rgSearchArgs accepts a keyword array; -e also guards patterns starting with '-'.

- fd filename search runs all keywords concurrently (Promise.all) to overlap filesystem I/O.

- Preview build reads files asynchronously (fs.promises) so the main process is not blocked; lowercases each line once instead of per keyword and collects match line numbers as a pre-sorted array (no Set/sort).
@oahsiao oahsiao merged commit b43de98 into main Jun 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant