feat(update): add --watch mode for periodic re-indexing#646
Open
mavaali wants to merge 1 commit into
Open
Conversation
Adds 'qmd update --watch [--interval <dur>] [--embed]' so users can keep
the QMD index fresh without running an external scheduler. This is the
piece most often paired with 'qmd mcp' in mcp.json-style setups where
the agent expects the index to stay current as notes change.
Design:
- Polling-based (setInterval), not fs.watch. Cheap because the existing
reindexCollection() hashes files and skips unchanged content; idle
ticks are near-zero cost. Avoids a new dependency (chokidar) and
keeps cross-platform behavior identical on Linux, macOS, and Windows
where fs.watch semantics differ.
- Default interval: 5 minutes. Configurable via --interval with units
ms/s/m/h (e.g. '30s', '5m', '1.5h').
- Quiet by default: ticks only emit a line when documents actually
change. Routine no-op ticks are silent so logs stay useful.
- Optional --embed flag runs vectorIndex() after any tick that left
pending hashes, so semantic search stays current without a second
cron job.
- 3-strike circuit breaker: after N consecutive tick failures (default
3), the loop exits non-zero so a supervisor (launchd/systemd) can
restart it instead of failing silently.
- Clean shutdown on SIGINT/SIGTERM between ticks. Removes the top-level
cursor-restoring signal handlers (same pattern as 'qmd mcp --http')
so the watch loop can install its own.
Refactor:
- updateCollections() now accepts { quiet, keepDbOpen } and returns
an UpdateSummary describing what changed. The non-watch behavior is
identical to before.
- New module src/watch.ts exposes parseDurationMs() and runWatchLoop()
with full test coverage (17 unit tests).
Verified:
- 843/843 tests pass.
- E2E smoke test: 'qmd update --watch --interval 1s' detects a
newly-created markdown file on the next tick, logs a single line,
and exits cleanly on SIGINT.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
qmd update --watch [--interval <dur>] [--embed]so users can keep the QMD index fresh without an external scheduler. This is the piece most often paired withqmd mcpinmcp.json-style setups where the agent expects the index to stay current as notes change.Why
A friend of mine asked whether he could drop QMD into
.vscode/mcp.jsonthe same way you'd drop in@playwright/mcpand forget about it. Today the missing piece is re-indexing:qmd updateworks great manually, but there's no built-in way to keep it running. People end up wiring cron jobs, launchd agents, or just forgetting and getting stale results.This PR adds watch mode so the common case ("re-index every few minutes, embed if anything new shows up") is one flag.
Design choices
fs.watch.reindexCollection()already hashes files and skips unchanged content, so idle ticks are near-zero cost. Polling avoids a new dependency (chokidar) and keeps cross-platform behavior identical on Linux, macOS, and Windows wherefs.watchsemantics differ.--intervalwith unitsms/s/m/h(e.g.30s,5m,1.5h). Bare numbers are treated as seconds.[2026-05-15T20:27:31.844Z] smoke: 1 new, 0 updated, 0 removed.--embed. RunsvectorIndex()after any tick that left pending hashes, so semantic search stays current without a second cron job.process.removeAllListenerspattern used byqmd mcp --httpto bypass the top-level cursor-restoring handlers.Usage
Paired with the MCP server:
What I didn't do
qmd mcp. Keeping watch as a sibling process avoids contention concerns and keeps the PR small. Happy to land that as a follow-up if you'd like.fs.watch. Open to revisiting if anyone wants true real-time, but polling covers the stated use case.Tests
test/watch.test.tscoveringparseDurationMs(units, fractions, edge cases, error paths) andrunWatchLoop(tick counting, failure tolerance, circuit breaker, consecutive-failure reset, invalid interval).qmd update --watch --interval 1s, added a new.mdfile mid-run, observed exactly one timestamped log line, sent SIGINT, observed clean exit.Refactor notes
updateCollections()now takes{ quiet?, keepDbOpen? }and returns anUpdateSummary. Non-watch behavior is byte-identical to before. The only externally visible change in the legacy path: the existingconsole.logfor custom-update-command stderr now routes throughconsole.error(correct destination for stderr content; previously it went to stdout).Checklist
npm run buildpassesnpx vitest run)## [Unreleased]