procedural: over-fits one-off single-project sequences (recurrence needs cross-project + identifier abstraction + generalizability gate)

[itlackey]

Problem

The procedural compilation pass (#615) over-fits: it compiles one-off, project-specific session sequences into "reusable workflows" that are not reusable. Examples (all rejected): an openpalm ctx.tokenstore.delete auth flow, a 13-step openpalm feature build, and a transcription of a single one-off config edit.

minRecurrence is NOT the issue — the 5 over-fit proposals had memberCount 3–6, all clearing the threshold. The failure is what recurs: an agent worked the same feature across 6 real sessions, extract transcribed it 6× near-identically, and the compiler counted that as a recurring procedure. Three structural gaps:

1. normalizeStep (src/commands/improve/procedural.ts:99-106) only lowercases/trims/strips punctuation — it keeps ctx.tokenstore.delete, file paths, CamelCase identifiers, quoted literals verbatim. The groupKey (procedural.ts:164) therefore bakes in project specifics → guaranteed over-fit output.
2. No project scoping. getAllEntries (src/indexer/db/db.ts:1657-1679) selects across the whole index with no project filter. projectHint exists at extract time (extract-prompt.ts:174) but is never preserved into asset frontmatter (extract.ts:290-305), so procedural can't distinguish projects and one project's repeated work pools as "recurrence."
3. Recurrence unit is the full ordered step list (procedural.ts:164), so a single sustained task masquerades as a recurring recipe. No requirement that members span different tasks/projects/time windows.

The LLM seam (procedural-system.md) is told the recurrence is already established and forbidden from dropping/reordering steps — so it transcribes rather than abstracts and is structurally incapable of rejecting over-fit. The only gate (procedural.ts:413) checks description presence, not generalizability.

Proposed fix (keep default-OFF until B1+B2+B3 land)

• B1 — cross-project/task recurrence gate (highest value). Preserve projectHint (+ ideally task/issue id) into asset frontmatter at extract; require a cluster to span ≥2 distinct projects (or ≥2 distinct tasks) before it qualifies.
• B2 — identifier-stripping normalization. Extend normalizeStep to abstract/parameterize call chains, file paths, identifiers, quoted literals. Enables real cross-project generalization and de-noises output.
• B3 — generalizability gate. Reject compiled workflows that reference concrete identifiers/paths/single-project nouns; reframe procedural-system.md to abstract identifiers and return null more aggressively for single-project implementations.
• B4 (complementary) — require members dispersed in time / across ≥N runs.
• B5 (complementary) — cap step count (>8 ⇒ likely a one-off feature build, not a recipe).
• A3 (safety) — maxProposalsPerRun: 1 during the trial.

Verdict

Conditionally valuable but currently premature. The safety design (deterministic ordered-step list as source of truth, local re-parse validation) is sound; the signal (surface step recurrence) is wrong. Stays default-off (improve-profiles.ts:76) until B1+B2+B3.

Effort: M–L.