Skip to content

Retire search --fast: cold-start model load makes the speed trade illusory #84

Description

@dmwyatt

Cold-invocation timings on main @ a55bec8 (release build with --features directml, RTX 3080, fully embedded 872-doc database, 4 runs per mode):

invocation wall time
grep "budget" ~0.09s
search "budget" --fast ~3.7-4.0s
search "budget" (full rerank) ~4.7-5.9s

In a warm process the rerank stage costs ~334ms per query, but the CLI never sees a warm process: every invocation pays ~3.7s of ONNX and embedding-model init first. --fast skips only the reranker's ~1.2s share, so the choice it offers is 4 seconds versus 5, while giving up the measured quality gain (hybrid 0.860 / 0.759 / 0.722 vs rerank 0.935 / 0.805 / 0.813 hit-rate/recall/MRR@10 on the golden set). Both variants are the same wait-a-few-seconds experience. The genuinely fast path is grep, which loads no models.

Proposal: remove --fast (plain removal, the same treatment --keyword, --semantic, and --hybrid received in #59 and #73). search then means exactly one thing: the highest-quality ranked answer. The --fast/--min-score interaction disappears with it.

Touches cli/args, commands/search.rs, README, help text, and the CLAUDE.md sanity-check line that uses --fast (CLAUDE.md edits need owner sign-off). Sequence after #75 / PR #81, which is in flight in the same modules; debug-build test ergonomics also deserve a look since CLAUDE.md currently recommends --fast there precisely because debug rerank is slow.

The real latency work is #82 (hide reranker init behind retrieval, which erases most of the delta this flag exploits) and #83 (persistent process to remove the init floor entirely). Neither needs --fast to exist.

Metadata

Metadata

Assignees

No one assigned

    Labels

    prio: p3Low priority, backlogsize: sSmall, quick fixstatus: readyReady to be worked ontype: choreMaintenance, refactoring, dependencies

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions