perf(transliterate): fuse the compose pre-scan into the engine pass

[raeq]

Recovers most of the form-invariance transliterate regression the perf-results branch flagged.

Root cause

The compose-at-lookup boundary (#475/#477/#481) added a needs_composition(text) pre-scan that walked every non-ASCII input a second time (UTF-8 decode + is_combining_mark trie lookup per character) before the actual transliteration. So the hot path went from one pass to two. On the perf-results bucket (EPYC 7763 / CPython 3.12.13 / corpus 803c316e) the latin/unidecode speedup ratio fell ~18.3 → ~11.4 across PR #474 → #480, all version-pinned comparators dropping proportionally (so it's disarm, not the comparators). This is the 2026-06-20 review's M-3 finding.

Fix

Detect composition inside the engine's existing decode loop instead of in a separate pass:

• transliterate_run takes detect_compose and returns Option. On the fast (borrowed-input) attempt it bails the instant it decodes a combining mark / conjoining jamo — off the scalar it already decoded — and the caller redoes the work over the composed buffer (with detect_compose = false, so a leftover lone mark there doesn't re-trigger). Mark-free input (the common case) now completes in one pass.
• needs_composition (still used by the confusables fold) also drops the trie lookup for a could_compose range fast-path over U+0000–058F, where marks live only in two small sub-blocks.

Correctness

Behaviour is identical — the bail fires on exactly the chars needs_composition did. Verified:

• Gated exhaustive (tier 3): 16/16 — all 11,172 Hangul syllables, full BMP, all CJK, 15 Indic blocks
• Formal invariants: 14, tier-1: 21 binaries, 949 Python transliterate/confusables/form-invariance/surrogate tests
• Confirmed pre/post-fusion output identical on ≇/≅ etc.

Measurements (Rust-level micro-bench, no PyO3 dilution; pre → post)

corpus	before	after	gain
latin	6.7	4.5	−33%
cyrillic	13.7	11.0	−20%
mixed	10.0	7.9	−21%
greek	12.9	11.4	−12%

The CI perf-gate will quantify the recovery on the canonical bucket via the comparator ratio (which cancels per-call overhead).

Separate finding (not this PR): the #492 adversarial oracle (hypothesis-marked, dev-tier, excluded from CI) flags that ml_normalize('≇') is non-idempotent (≇ → ≅ → "approximately equal"). Confirmed pre-existing on main (identical pre/post fusion) — a symbol-naming/NFKC fixed-point issue, orthogonal to this change.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR improves transliterate hot-path performance by eliminating the extra needs_composition(text) pre-scan and instead detecting “compose-required” characters inside the transliteration engine’s existing UTF-8 decode loop, preserving the compose-at-lookup form-invariance behavior while recovering most of the regression introduced by the boundary compose work.

Changes:

• Fuse compose detection into the transliteration engine pass: attempt a borrowed-input run first and bail to a composed-buffer rerun only when a compose-triggering scalar is encountered.
• Thread a detect_compose flag through transliterate_dispatch/transliterate_run, switching them to return Option<Cow<str>> so the fast pass can signal “needs compose”.
• Speed up needs_composition with a new could_compose predicate that adds a cheap range fast-path over U+0000–058F; document the perf recovery in the changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
src/transliterate.rs	Implements the fused fast-path attempt, adds detect_compose plumbing, and makes the engine bail to compose only when needed.
src/compose.rs	Adds could_compose and updates needs_composition to use it, reducing per-scalar overhead in common ranges.
CHANGELOG.md	Records the performance regression root cause and the fused-pass recovery.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[raeq]

Review comments addressed in abdad76 (comment-only): the two fusion comments now read "combining mark or conjoining Hangul L jamo (the could_compose predicate)" instead of the broader "conjoining jamo", and transliterate_dispatch's doc now documents that None is returned only as the compose-bail control-flow signal (with detect_compose = true), never as a normal "no output" case.