diff --git a/CLAUDE.md b/CLAUDE.md index be001a7..5af3b55 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -32,4 +32,4 @@ The oracle cases (vignette §9, `O-1…O-4`) are hand-computed and must never be ## Current phase (update this line when it changes) -Stage 0 complete; PR #8 merged to `main`. Register batch 3 ratified (ASSAY-DEC-22…27, closing concept §6.6–6.11) — walkthrough and findings ledger now canonical, G6 a standing invariant. Stage 1 built (issue #10): research note `01-knowledge.md` (DEC-16 floor); SPEC-05 implemented (`src/{seam,deltas,encoding,lint,knowledge}.ts` + `components/{refusalBanner,s1Table}.ts`); all four Stage-1 exits demonstrated on Meridian. Stage 2 built: research note `docs/research/02-compile.md` (DEC-11 gate — decides the **sparse channel representation** from the Stage-0 dense-world numbers + ATP 2-01.3 MCOO doctrine, retiring dense `Channel.cells`); SPEC-06 specified via spec-kit (`specs/006-compile/`) and **implemented** (`src/compile.ts` + `components/channelTrace.ts`; `fixtures/vignette-config.json`; sparse-channel schema regen; 107 tests, typecheck clean; compile demo moment renders in the gallery). All three Stage-2 exits demonstrated on Meridian (K12 contested ⇒ compile refuses `contested_knowledge`, resolve⇒recompile; byte-identical stamp; every channel traces back to named knowledge via `compiled_into` + `source`). The sparse-channel schema change is recorded as a flagged register candidate (concept §6, item 12), built under seam open item 2's delegated authority pending batch ratification. **Next: Stage 3 — SPEC-07 score, then SPEC-08 plan** (research note `03-score-plan.md` first, DEC-11 — must define the four-stop verdict mapping and reproduce vignette §9 oracle cases O-1…O-4 / G6). +Stage 0 complete; PR #8 merged to `main`. Register batch 3 ratified (ASSAY-DEC-22…27, closing concept §6.6–6.11) — walkthrough and findings ledger now canonical, G6 a standing invariant. Stage 1 built (issue #10): research note `01-knowledge.md` (DEC-16 floor); SPEC-05 implemented (`src/{seam,deltas,encoding,lint,knowledge}.ts` + `components/{refusalBanner,s1Table}.ts`); all four Stage-1 exits demonstrated on Meridian. Stage 2 built: research note `docs/research/02-compile.md` (DEC-11 gate — decides the **sparse channel representation** from the Stage-0 dense-world numbers + ATP 2-01.3 MCOO doctrine, retiring dense `Channel.cells`); SPEC-06 specified via spec-kit (`specs/006-compile/`) and **implemented** (`src/compile.ts` + `components/channelTrace.ts`; `fixtures/vignette-config.json`; sparse-channel schema regen; 107 tests, typecheck clean; compile demo moment renders in the gallery). All three Stage-2 exits demonstrated on Meridian (K12 contested ⇒ compile refuses `contested_knowledge`, resolve⇒recompile; byte-identical stamp; every channel traces back to named knowledge via `compiled_into` + `source`). The sparse-channel schema change is recorded as a flagged register candidate (concept §6, item 12), built under seam open item 2's delegated authority pending batch ratification. Stage 3 scorer built: research note `docs/research/03-score-plan.md` (DEC-11 gate — **decides the four-stop verdict mapping as signed margin bands, signs-only**: robust iff m_lo>0, marginal iff m_lo=0, tight iff m_lo<0≤m_hi, violated iff m_hi<0 — the unique O-3-satisfying rule with no interior cut; also settles `(cell,time)` route evaluation over a time-expanded graph, closing note 02 §3's forward flag, and reserves ε-dominance for SPEC-08 plan comparison); SPEC-07 specified via spec-kit (`specs/007-score/`) and **implemented** (`src/{interval,materialise,metrics,score}.ts` + `components/s2Matrix.ts`; `fixtures/plans.json` canned handful; no schema change — verdict/score/plan shapes already exist; 139 tests, typecheck clean; the honest-matrix demo moment renders in the gallery). The correctness leg holds on Meridian: oracle O-1 (`[9,13]`), O-2 (robust, margin `[15,19]`), O-3 (four-stop sweep changes only at band edges 9/13) reproduced exactly, and O-4 propagation-honesty (G6) under property-based testing (`fast-check`). SPEC-07 asserts no new register decision (the mapping is decided in the research note under the latitude vignette §9/O-3 delegated to Stage-3). **Next: SPEC-08 — handful generator + banded non-dominated organisation (sacrificial scope, delivery plan §3), over the same `03-score-plan.md` note; then Stage 4 SPEC-09 relaxation.** diff --git a/docs/assets/gallery/index.html b/docs/assets/gallery/index.html index b9b3cb6..03a4c35 100644 --- a/docs/assets/gallery/index.html +++ b/docs/assets/gallery/index.html @@ -526,7 +526,40 @@

Compile — knowledge → CompiledWor K12a assessed · moderateassessment, not factowner: J-2 red cell -

Generated from fixtures/ by npm run gallery · identifiers frozen per assay-vignette.md §8 · compile per research note docs/research/02-compile.md

+

Score — the honest matrix (SPEC-07)

+

The Stage-3 demo moment, run by the actual scorer over the canned Meridian handful against the base world. Verdicts are a four-stop scale — robust / marginal / tight / violated, one colour language, no decimals anywhere; the banded margin rides on hover, never as a headline number (G2). Each commitment's verdict propagates its metric by interval arithmetic — worst-and-best-case on pure bands, no midpoint (DEC-15) — and is validated by the vignette §9 oracle cases (O-1–O-3 exact, O-4 containment). Pick a verdict, walk it to the assessment and owner it rests on: every cell was written with a scored_from edge back to the world, whose channels trace to named knowledge above. score stamp c5e11b4a027f1053… · every verdict carries a scored_from edge to the world (G3) · margins on hover, no decimals (G2)

+
+ + + + + +
plan \ commitmentC1C2C3C4C5C6
P1 · Strait-early / battery-suppression + robust + + robust + + marginal + + tight + + marginal + + robust +
P2 · Sweep-first / fires-forward + robust + + robust + + violated + + robust + + marginal + + robust +
+

Generated from fixtures/ by npm run gallery · identifiers frozen per assay-vignette.md §8 · compile per research note docs/research/02-compile.md · score per docs/research/03-score-plan.md

diff --git a/docs/research/03-score-plan.md b/docs/research/03-score-plan.md new file mode 100644 index 0000000..1bb00e5 --- /dev/null +++ b/docs/research/03-score-plan.md @@ -0,0 +1,90 @@ +# Research note 03 — Score, then plan + +Stage 3 · per ASSAY-DEC-11 · 2026-07-13 · bounded to hours, one page preferred +Prompts (build plan §Stage 3): banded/epsilon-dominance and the NSGA-II family — what "distinct in banded space" means formally; JP 5-0 COA comparison criteria (suitability, feasibility, acceptability) as a cross-check that verdict semantics match staff practice; **define the four-stop verdict mapping against banded margins** so the vignette §9 oracle outcomes (O-2/O-3) are decided by the note, not discovered in code. Also owed here: the routing-search structure flagged forward from note 02 §3, and the handful strategy axes for this domain (concept §6.2). + +## 1. Metric propagation is interval arithmetic, and nothing more + +The scorer's inner loop turns a plan and a compiled world into a **metric band** for each commitment's `metric`, then compares that band to the commitment's threshold. The propagation is classical interval arithmetic (Moore, *Interval Analysis*, 1966): a metric is built from banded inputs by `+`, `−`, `×` scalar, and `max`/`min`, and each operator has the textbook rule on closed intervals — `[a,b] + [c,d] = [a+c, b+d]`, `[a,b] − [c,d] = [a−d, b−c]`, `max([a,b],[c,d]) = [max(a,c), max(b,d)]`. Metric bands carry a `unit`; combining mismatched units is a fixture/config defect surfaced, never silently coerced (the compile already units its channels). This is the whole of oracle O-1: `strait_open_step = start + duration(A) + duration(B) = 2 + [4,6] + [3,5] = [9,13]`, exactly, by interval addition — **any other band is a propagation defect, not a design choice** (vignette §9). The scorer implements interval arithmetic as a small, separately-tested module (`src/interval.ts`) precisely so O-1 tests the arithmetic, not the plumbing around it. + +The honesty content of DEC-15 lives here: bands are pure closed intervals with **no midpoint**, so the scorer may never take a mean, a mid, or a "most likely" — every operation is worst-and-best-case on the interval. A scorer that collapsed `[4,6]` to `5` to "simplify" would pass no oracle and violate G6 (below). Interval arithmetic is not a nicety; it is the representation of DEC-15 in the compute layer. + +## 2. Route metrics: `(cell, time)` resolution, not a time-expanded graph + +Note 02 §3 flagged forward *"whether the scorer walks routes over a `(cell, time)` state search or a locally time-expanded subgraph is a Stage-3 decision."* **Decided: `(cell, time)` resolution over the sparse world, no graph expansion.** A `Plan` (DEC-20) is already timed geometry — `RouteLeg[]` with `enter_step`/`exit_step` and `TaskWindow[]` — so the scorer does not *search* for routes; it *evaluates* the routes the plan states. For each leg the scorer resolves `channel(x, y, t)` by the lazy materialisation function note 02 §2.2 defined: pick, for that cell and timestep, the innermost active `RegionOverride` (later `from_step` wins on overlap — note 02 §3's documented tie order), else the channel default. A metric is then a reduction over the leg's `(cell, time)` samples: + +- **reach-step metrics** (`strait_open_step`, `port_open_step`, `extraction_step`) — `start + Σ leg durations`, where a leg's duration band is its distance divided by the leg's banded mobility factor (interval division on positive factors; a zeroed mobility region — R3's demolished causeway — makes the band unbounded, i.e. the route is severed, surfaced as a `violated` reach rather than an infinity in a cell). +- **exposure metrics** (`threat_exposure(FE)`, `civil_harm_exposure`) — `Σ over the element's occupied (cell, time) samples of the threat/civil band × dwell`, scoped to the element or district the commitment names (`scope`). Exposure accumulates as a band because the channel it reads is banded. +- **state metrics** (`causeway_intact`) — a banded boolean read of the relevant channel/region at the relevant step. + +No time-expanded graph is ever built: expansion re-materialises the 60×60×56 dense world note 02 retired, times the horizon, for no gain when the plan already fixes the route. The scorer touches only the cells a plan actually occupies — tens to hundreds of `(cell,time)` samples per plan, not 200k nodes. This keeps DEC-4's "scorer honestly real even in the mock" affordable: sensitivity, discrimination, and robustness are re-scoring loops over this cheap evaluator, not searches. + +## 3. The four-stop verdict mapping — signed margin bands (the note's binding decision) + +A commitment is `metric comparator threshold` (DEC-19; the threshold is scalar fact-of-intent, DEC-14). Reduce every comparator to a **signed margin band** `M = [m_lo, m_hi]`, oriented so that **satisfied ⟺ margin ≥ 0**: + +| comparator | satisfied when | margin(v) | margin band M | +|---|---|---|---| +| `at_most T` | v ≤ T | T − v | `[T − hi, T − lo]` | +| `by_step T` | v ≤ T (completion step) | T − v | `[T − hi, T − lo]` | +| `at_least T` | v ≥ T | v − T | `[lo − T, hi − T]` | +| `never` | metric stays ≤ 0 (no occurrence) | −v | `[−hi, −lo]` | + +(`v = [lo, hi]` is the metric band; the worst realisation for `at_most`/`by_step`/`never` is `hi`, for `at_least` is `lo` — the table orients each so `m_lo` is always the worst case.) Oracle O-2 checks the arithmetic: `strait_open_step [9,13]` vs C2-style `at_most 28` gives `M = [28−13, 28−9] = [15,19]` — the required margin band, exactly. + +**The verdict is a pure function of the signs of the margin-band endpoints — no interior cut, no invented ε:** + +| condition | meaning | verdict | +|---|---|---| +| `m_lo > 0` | worst realisation satisfies with room | **robust** | +| `m_lo = 0 ≤ m_hi` | worst realisation exactly meets the line; none violate | **marginal** | +| `m_lo < 0 ≤ m_hi` | the band straddles the line — violation is possible | **tight** | +| `m_hi < 0` | best realisation already fails | **violated** | + +This is the honest scale. The *only* boundaries at which the verdict changes are where a margin-band endpoint crosses zero — and an endpoint of `M` crosses zero exactly when the threshold crosses a **band edge** of the metric. It is therefore impossible to draw a marginal/tight line *inside* the band, because the band gives no honest interior point to draw it at (DEC-15: no midpoint, ever). A "closeness" rule — marginal if the margin is within one band-width of zero, say — would put a transition at a threshold value that is *not* a band edge, and O-3 forbids exactly that. The signs-only rule is the unique mapping that (a) is monotone from robust to violated, (b) uses the four stops the register froze (DEC-9), and (c) satisfies O-3. + +**O-3, decided here and re-verified.** Metric band `[9,13]`, `at_most T`, `M = [T−13, T−9]`. Sweep the threshold 8 → 14: + +| T | M | signs | verdict | +|---|---|---|---| +| 8 | [−5,−1] | m_hi < 0 | violated | +| 9 | [−4, 0] | m_lo<0, m_hi=0 | tight | +| 10–12 | [−3,1]…[−1,3] | m_lo<00 | robust | + +The verdict transitions only at **T = 9** (`m_hi` crosses 0 — the lower band edge) and **T = 13** (`m_lo` crosses 0 — the upper band edge), and is constant between and outside them — *"the verdict changes only at the band edges (9, 13), never inside the band"* (O-3), satisfied. At the note's cited threshold `at_most 12` the verdict is **tight** — "neither robust nor violated (both outcomes lie inside the band)", as O-3 requires. The sweep also happens to exercise all four stops on one metric, which is a good sign the scale is not over-fitted to the marginal case. + +**Why not epsilon-dominance for the *verdict*.** Laumanns et al.'s ε-dominance (2002) is the right tool for *comparing plans* (§4), where an ε makes "distinct" decidable on a continuum. It is the wrong tool for a *verdict against a commander's threshold*: an ε there is a false-precision knob — it would let a plan that violates a `must` read as "close enough" by a tunable margin the commander never set. The threshold is fact-of-intent (DEC-14); the honest verdict asks only *does the band clear it, straddle it, or fail it*, and the signs of `M` answer exactly that. ε belongs in plan comparison, not in commitment verdicts. + +## 4. Banded honesty as a theorem — the containment property (G6, oracle O-4) + +The mapping above is only honest if the propagation under it is monotone. State it as the property the scorer must satisfy, which is candidate G6 / oracle O-4: + +> **Containment / monotonicity.** Widening any input band never narrows any output metric band, and every point-realisation of the inputs scores inside the output band. + +Interval arithmetic gives this by construction: each operator is inclusion-monotone (`A ⊆ A′ ⇒ A ∘ B ⊆ A′ ∘ B`), so a wider input yields a superset output; and each operator is *sound* (contains every real combination), so every point-realisation lands inside. O-4 instances it: re-running O-1 with `A` widened `[4,6] → [3,7]` gives `strait_open_step = 2 + [3,7] + [3,5] = [8,14] ⊇ [9,13]`, exactly. The verdict mapping preserves the honest direction too: widening an input can only move `m_lo` down and `m_hi` up, so a verdict can only move *toward* uncertainty (robust → marginal → tight) or stay — **more input uncertainty never buys a more confident verdict**. That is banded honesty stated as a theorem rather than typographically (seam §G, G6): a scorer could satisfy every other invariant while propagating intervals wrongly, and would be exactly the false-precision machine ASSAY refuses. SPEC-07 asserts O-1–O-3 as hand-computed instances and O-4 as a **property-based test** over random bands and thresholds (fast-check style: generate `[lo,hi]`, a widening, a comparator, a threshold; assert superset and point-membership and verdict-monotonicity), not just the one widening in the vignette. + +## 5. Distinctness in banded space, and the JP 5-0 cross-check (folds into SPEC-08) + +*What "genuinely distinct" means (concept §6.2; gates SPEC-08).* Two plans are **distinct in banded space** when neither ε-dominates the other across the criteria vector of banded scores — banded/ε-non-domination in the sense of Laumanns et al. (2002), the NSGA-II family's diversity mechanism (Deb et al., 2002) adapted to intervals: `A` dominates `B` if `A` is no worse than `B` on every criterion band and strictly better (by more than ε) on at least one, where "no worse" on bands uses the conservative order (`A ⪯ B` iff `A.hi ≤ B.lo`, i.e. the bands do not overlap in the wrong direction). The handful is the ε-non-dominated set, capped at 3–5; ε is the *organiser's* distinctness knob and lives here, never in a verdict (§3). This is recorded for SPEC-08; SPEC-07 owes only the scorer whose banded criteria vectors SPEC-08 will organise. + +*Handful strategy axes for this domain (concept §6.2 — REMIT's time/exposure/robustness/completeness axes "do not transfer unexamined").* The Meridian vignette's four genuine axes, read off C1–C6 and the force elements, are: +1. **approach** — strait-early vs sweep-first-enter-late (the thesis-C robustness trap: PACKHORSE timing); +2. **suppression posture** — fires-forward (risks C3 civil harm) vs stand-off (risks C4 exposure / C2 tempo); +3. **causeway** — contest intact (C5) vs bypass by sea; +4. **extraction coupling** — KINGFISHER pull-out early vs mission-tail (C6 vs tempo). +These are the strategy-biased fan-out axes for the seeded generator; they are the domain's own, not REMIT's imported. Recorded for SPEC-08. + +*JP 5-0 cross-check.* JP 5-0 (*Joint Planning*, ch. on COA comparison) evaluates COAs on **suitability, feasibility, acceptability**. The four-stop scale must read as staff language, and it does: a `violated` **must** is *infeasible* (the COA cannot accomplish the task within the constraint); a `tight`/`marginal` verdict is the *acceptability* question (risk the commander must weigh — the band straddles the line); `robust` is *suitable and feasible with margin*. Crucially JP 5-0 keeps comparison a **commander's judgement**, not an arithmetic total — which is why ASSAY refuses a weighted scalar score (DEC-19: ordinal tiers, no weights) and surfaces banded verdicts for a human to weigh, rather than a single ranked number. The scale matches practice; the honesty is that ASSAY declines to do the commander's weighing for them. + +## What we will do differently + +1. **Implement interval arithmetic as its own module** (`src/interval.ts`: `add`, `sub`, `scaleBy`, `max`, `min`, `contains`, `width`, unit-checked), separately tested, so oracle O-1 tests the arithmetic and the scorer composes it. No mean/mid/most-likely operation exists anywhere (DEC-15). +2. **Evaluate stated routes by `(cell, time)` resolution over the sparse world** — the lazy materialisation function of note 02 §2.2 — and build no time-expanded graph (note 02 §3's forward-flagged decision, now made). The scorer touches only occupied cells. +3. **Map verdicts by signed margin bands, signs-only** (§3): `robust` iff `m_lo>0`, `marginal` iff `m_lo=0`, `tight` iff `m_lo<0≤m_hi`, `violated` iff `m_hi<0`. This is the mapping O-2/O-3 are decided against; it is the unique O-3-satisfying, no-interior-cut, four-stop rule. `margin` returns as a `Band` shown only on demand (knowledge model §108); verdicts cross the seam only as the four-stop scale (G2). +4. **Assert G6 as a property-based test** (O-4): superset-under-widening, point-realisation membership, and verdict-monotonicity across random bands/thresholds — not only the vignette's single widening. Oracle expected values stay hand-computed and are never regenerated from the scorer (vignette §9 rule). +5. **Keep ε out of verdicts and reserve it for plan comparison** (§3, §5): the `at_least/at_most` line is the commander's, not a tunable margin; distinctness in banded space is ε-non-domination and belongs to SPEC-08's organiser, recorded here with the domain's four generation axes. +6. **Honour DEC-10's scenario-blindness and the perturbation hook**: `/score` takes an already-excursioned world and a `knowledge_overrides?` list that substitutes answers *for the call only* (seam §5), writing nothing; and refuses `stamp_mismatch` when a plan's assumptions and the world disagree on stamp (the comparability guard), a first-class `Refusal` like every other seam decline. + +Sources: R. E. Moore, *Interval Analysis* (1966) — closed-interval arithmetic and inclusion monotonicity; K. Deb et al., "A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II," *IEEE TEC* 6(2), 2002 — non-dominated sorting and diversity; M. Laumanns et al., "Combining Convergence and Diversity in Evolutionary Multiobjective Optimization," *Evol. Comput.* 10(3), 2002 — ε-dominance; JP 5-0 *Joint Planning* (2020) ch. on COA comparison — suitability/feasibility/acceptability and comparison-as-commander's-judgement; JP 2-01.3 (2014) — most-likely/most-dangerous (forward to note 05); ASSAY register DEC-4/9/10/14/15/19/20, knowledge model §4/§6 (materialisation)/§108, seam contract §5/§6/§G (G6), vignette §6 (C1–C6)/§9 (oracle O-1–O-4), research note 02-compile.md §2–3, concept §6.2. diff --git a/fixtures/plans.json b/fixtures/plans.json new file mode 100644 index 0000000..c10f3e1 --- /dev/null +++ b/fixtures/plans.json @@ -0,0 +1,91 @@ +[ + { + "logical_id": "P1", + "version": 1, + "name": "Strait-early / battery-suppression", + "seed": 1, + "generator": "canned-handful (delivery plan §3 fallback; SPEC-08 replaces)", + "elements": [ + { + "force_element": "FE-BROOM", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 4 }, + { "x": 26, "y": 25, "enter_step": 4, "exit_step": 22 } + ] + }, + { + "force_element": "FE-PACKHORSE", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 4 }, + { "x": 26, "y": 25, "enter_step": 4, "exit_step": 24 }, + { "x": 32, "y": 38, "enter_step": 24, "exit_step": 32 } + ] + }, + { + "force_element": "FE-ANVIL", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 8 }, + { "x": 32, "y": 22, "enter_step": 8, "exit_step": 9 } + ] + }, + { + "force_element": "FE-FALCON", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 6 }, + { "x": 48, "y": 15, "enter_step": 6, "exit_step": 10 } + ] + }, + { + "force_element": "FE-KINGFISHER", + "route": [ + { "x": 48, "y": 15, "enter_step": 0, "exit_step": 4 }, + { "x": 30, "y": 50, "enter_step": 4, "exit_step": 40 } + ] + } + ] + }, + { + "logical_id": "P2", + "version": 1, + "name": "Sweep-first / fires-forward", + "seed": 1, + "generator": "canned-handful (delivery plan §3 fallback; SPEC-08 replaces)", + "elements": [ + { + "force_element": "FE-BROOM", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 2 }, + { "x": 26, "y": 25, "enter_step": 2, "exit_step": 24 } + ] + }, + { + "force_element": "FE-PACKHORSE", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 10 }, + { "x": 26, "y": 25, "enter_step": 10, "exit_step": 30 }, + { "x": 32, "y": 38, "enter_step": 30, "exit_step": 36 } + ] + }, + { + "force_element": "FE-ANVIL", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 12 } + ] + }, + { + "force_element": "FE-FALCON", + "route": [ + { "x": 30, "y": 50, "enter_step": 0, "exit_step": 4 }, + { "x": 32, "y": 38, "enter_step": 4, "exit_step": 8 } + ] + }, + { + "force_element": "FE-KINGFISHER", + "route": [ + { "x": 48, "y": 15, "enter_step": 0, "exit_step": 4 }, + { "x": 30, "y": 50, "enter_step": 4, "exit_step": 42 } + ] + } + ] + } +] diff --git a/package-lock.json b/package-lock.json index 1e7f9fd..b0c22cb 100644 --- a/package-lock.json +++ b/package-lock.json @@ -13,6 +13,7 @@ }, "devDependencies": { "@types/node": "^22.0.0", + "fast-check": "^4.9.0", "tsx": "^4.19.0", "typescript": "^5.6.0", "vitest": "^2.1.0" @@ -1091,6 +1092,29 @@ "node": ">=12.0.0" } }, + "node_modules/fast-check": { + "version": "4.9.0", + "resolved": "https://registry.npmjs.org/fast-check/-/fast-check-4.9.0.tgz", + "integrity": "sha512-7ms6T7SybUev/PQITciI0yLM2pOSFy5zpG8Ty7tQofcVaQUvrMXp6CBwqF6fThLCLOrfBtuHAtwq6Yu4XPCllg==", + "dev": true, + "funding": [ + { + "type": "individual", + "url": "https://github.com/sponsors/dubzzz" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fast-check" + } + ], + "license": "MIT", + "dependencies": { + "pure-rand": "^8.0.0" + }, + "engines": { + "node": ">=12.17.0" + } + }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", @@ -1202,6 +1226,23 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/pure-rand": { + "version": "8.4.2", + "resolved": "https://registry.npmjs.org/pure-rand/-/pure-rand-8.4.2.tgz", + "integrity": "sha512-vvuOGgcuPJAirlHvuQw1TrOiw7ptaIXXmIbNuiNOY6lNGJJH49PQ1Kj4nd783nPdQhQdicgOjVI2yI/9BD6/Ng==", + "dev": true, + "funding": [ + { + "type": "individual", + "url": "https://github.com/sponsors/dubzzz" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fast-check" + } + ], + "license": "MIT" + }, "node_modules/rollup": { "version": "4.62.2", "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.62.2.tgz", diff --git a/package.json b/package.json index 6fd35d8..61be6f2 100644 --- a/package.json +++ b/package.json @@ -18,6 +18,7 @@ }, "devDependencies": { "@types/node": "^22.0.0", + "fast-check": "^4.9.0", "tsx": "^4.19.0", "typescript": "^5.6.0", "vitest": "^2.1.0" diff --git a/scripts/build-gallery.ts b/scripts/build-gallery.ts index 43d1e02..1062baa 100644 --- a/scripts/build-gallery.ts +++ b/scripts/build-gallery.ts @@ -16,14 +16,18 @@ import { provenanceChip } from '../src/components/provenanceChip.js'; import { s1Table, type S1Row } from '../src/components/s1Table.js'; import { channelTrace } from '../src/components/channelTrace.js'; import { refusalBanner } from '../src/components/refusalBanner.js'; +import { s2Matrix, type S2Cell } from '../src/components/s2Matrix.js'; import { checkEncoding } from '../src/encoding.js'; import { confidenceLint } from '../src/lint.js'; import { KnowledgeService } from '../src/knowledge.js'; import { CompileService } from '../src/compile.js'; +import { ScoreService } from '../src/score.js'; import { isRefusal } from '../src/seam.js'; import type { + Commitment, CompiledWorld, KnowledgeObject, + Plan, ScenarioCOA, VignetteConfig, } from '../src/generated/types.js'; @@ -34,6 +38,12 @@ const knowledge = JSON.parse( const coas = JSON.parse( readFileSync(new URL('../fixtures/coas.json', import.meta.url), 'utf8'), ) as ScenarioCOA[]; +const commitments = JSON.parse( + readFileSync(new URL('../fixtures/commitments.json', import.meta.url), 'utf8'), +) as Commitment[]; +const plans = JSON.parse( + readFileSync(new URL('../fixtures/plans.json', import.meta.url), 'utf8'), +) as Plan[]; const vignetteConfig = JSON.parse( readFileSync(new URL('../fixtures/vignette-config.json', import.meta.url), 'utf8'), ) as VignetteConfig; @@ -109,6 +119,35 @@ const stampLine = ? `stamp ${compiled.stamp.slice(0, 16)}… · ${world.consumed.length} knowledge objects consumed · sparse channels (no dense cell stored)` : ''; +// SPEC-07 — the Stage-3 demo moment, "the honest matrix", run by the ACTUAL +// scorer over the canned Meridian handful (P1/P2) against the resolved base +// world. Four-stop chips, margin bands on hover, no decimals; every verdict was +// written with a `scored_from` edge back to the world it rests on (G3). +for (const c of commitments) await svc.store.put(c as unknown as Record); +const planRefs = new Map(); +for (const p of plans) planRefs.set(p.logical_id, await svc.store.put(p as unknown as Record)); +const scorer = new ScoreService({ store: svc.store, trace: svc.trace, config: vignetteConfig, commitments }); +const matrixRows: S2Cell[] = []; +let scoreStamp = ''; +if (!isRefusal(compiled)) { + for (const p of plans) { + const scored = await scorer.score({ + plan: planRefs.get(p.logical_id)!, + world: compiled.world, + scenario: 'BASE', + engine_version: '0.1.0', + }); + if (!isRefusal(scored)) { + matrixRows.push({ plan: `${p.logical_id} · ${p.name}`, verdicts: scored.verdicts }); + scoreStamp = scored.stamp; + } + } +} +const s2 = matrixRows.length > 0 ? s2Matrix(['C1', 'C2', 'C3', 'C4', 'C5', 'C6'], matrixRows) : ''; +const scoreStampLine = scoreStamp + ? `score stamp ${scoreStamp.slice(0, 16)}… · every verdict carries a scored_from edge to the world (G3) · margins on hover, no decimals (G2)` + : ''; + const html = ` @@ -131,7 +170,10 @@ ${rows}
${compileRefusalHtml}

Resolve the contest, and the recompile builds the world. Channels are stored sparse — a per-channel default plus named region overrides (the MCOO idiom; no dense 1.2M-cell surface is ever stored or hashed). Every channel value walks back to the named knowledge it came from, with its owner (G3). ${stampLine}

${channelHtml}
-

Generated from fixtures/ by npm run gallery · identifiers frozen per assay-vignette.md §8 · compile per research note docs/research/02-compile.md

+

Score — the honest matrix (SPEC-07)

+

The Stage-3 demo moment, run by the actual scorer over the canned Meridian handful against the base world. Verdicts are a four-stop scale — robust / marginal / tight / violated, one colour language, no decimals anywhere; the banded margin rides on hover, never as a headline number (G2). Each commitment's verdict propagates its metric by interval arithmetic — worst-and-best-case on pure bands, no midpoint (DEC-15) — and is validated by the vignette §9 oracle cases (O-1–O-3 exact, O-4 containment). Pick a verdict, walk it to the assessment and owner it rests on: every cell was written with a scored_from edge back to the world, whose channels trace to named knowledge above. ${scoreStampLine}

+
${s2}
+

Generated from fixtures/ by npm run gallery · identifiers frozen per assay-vignette.md §8 · compile per research note docs/research/02-compile.md · score per docs/research/03-score-plan.md

diff --git a/specs/007-score/checklists/requirements.md b/specs/007-score/checklists/requirements.md new file mode 100644 index 0000000..cd228ee --- /dev/null +++ b/specs/007-score/checklists/requirements.md @@ -0,0 +1,37 @@ +# Requirements checklist — SPEC-07 (Score) + +Traceability from spec requirements to acceptance. Every box is a committed test or a rendered surface. + +## Correctness leg (the reason SPEC-07 exists) + +- [ ] O-1 — `start 2 + A[4,6] + B[3,5] = [9,13]` exactly (FR-002, SC-001) +- [ ] O-2 — `[9,13]` vs `at_most 28` ⇒ **robust**, margin `[15,19]` (FR-003, SC-002) +- [ ] O-3 — `[9,13]` vs `at_most 12` ⇒ **tight** (neither robust nor violated) (FR-003, SC-003) +- [ ] O-3 sweep — threshold 8→14 changes verdict only at edges 9 and 13; sweep yields violated/tight/marginal/robust (FR-003, SC-003) +- [ ] O-4 — `A` widened `[4,6]→[3,7]` ⇒ `[8,14] ⊇ [9,13]` (FR-005, SC-004) +- [ ] O-4 property — superset-under-widening, point-realisation membership, verdict-monotonicity over random inputs (fast-check) (FR-005, SC-004) +- [ ] Oracle constants are committed, never regenerated from the scorer (SC-009) + +## Honest matrix (demo moment) + +- [ ] A Meridian plan scores to one four-stop verdict per C1–C6, each with a margin band (FR-001/003/004, SC-005) +- [ ] Every verdict opens a complete backward trace chain to named owners via `scored_from` (FR-007, SC-005) +- [ ] S2 matrix renders four-stop chips, margin-on-hover, no decimals in verdict cells (FR-004, SC-008) +- [ ] The gallery shows the honest-matrix moment run by the actual scorer (SC-008) +- [ ] R1-early-strait plan scores healthier under R1 than R2 (thesis-C direction; correct scoring, not the strip) (US3.4) + +## Discipline & determinism + +- [ ] Same inputs ⇒ byte-identical stamp and identical verdicts/scores; commitment order irrelevant (FR-008, SC-006) +- [ ] `knowledge_overrides` re-scores without persisting the override or writing a knowledge edge (FR-009, SC-007) +- [ ] `stamp_mismatch` refuses across incompatible stamps, persists nothing (FR-010, SC-007) +- [ ] Severed route ⇒ `violated` reach, never a non-finite band; unit mismatch ⇒ score-time error (FR-011) +- [ ] Scorer is scenario-blind — reads the excursioned world, does not re-derive it (FR-012) +- [ ] No mean/mid/most-likely operation exists anywhere in the scorer (FR-002, DEC-15) + +## Gates + +- [ ] Research note `03-score-plan.md` present before implementation (DEC-11) +- [ ] Oracle/contract tests written and confirmed failing before scorer behaviour (constitution quality gate 2) +- [ ] `npm run typecheck` clean; `npm test` green (SPEC-06 baseline + SPEC-07) (SC-009) +- [ ] Batch propagation: CLAUDE.md phase line, build/delivery peers, gallery swept in one change diff --git a/specs/007-score/contracts/score-service.md b/specs/007-score/contracts/score-service.md new file mode 100644 index 0000000..7940939 --- /dev/null +++ b/specs/007-score/contracts/score-service.md @@ -0,0 +1,44 @@ +# Contract — Score service (`POST /score`, seam §5) + +Independently callable (DEC-10), scenario-blind, honestly real even in the mock (DEC-4). Propagates bands (DEC-15); verdicts return on the four-stop scale with `margin` bands. All declines are first-class `Refusal`s (seam §1). + +## Request + +``` +POST /score +{ + plan: Ref, // the plan to score (stated routes; not searched) + world: Ref, // an already-excursioned CompiledWorld (DEC-10) + scenario: LogicalId, // recorded on verdicts for provenance only + knowledge_overrides?: {ref: Ref, answer: Band}[], // perturbation hook — for this call only + engine_version: string +} +``` + +## Response + +``` +→ { verdicts: CommitmentVerdict[], scores: PlanScore[], stamp } // success +| Refusal { refused: true, reason, offending: Ref[], explanation } // decline +``` + +## Behavioural contract + +1. **Verdicts** — one `CommitmentVerdict` per commitment, `verdict ∈ {robust, marginal, tight, violated}` via the signed-margin-band mapping (research note §3), `margin` a `Band` shown on demand. Verdicts crossing the seam are the four-stop scale only (G2). Returned canonically ordered by `commitment` id. +2. **Scores** — banded `PlanScore[]` (criteria vector); no scalar, no weighted total (DEC-19). +3. **Metric propagation** — interval arithmetic on pure bands; reproduces O-1 exactly. Route metrics by `(cell,time)` resolution over the sparse world (research note §2). +4. **Trace** — one `scored_from` edge from each returned verdict/score to the `world` (G3); a backward walk from any verdict reaches named knowledge and owners. A verdict with no backward chain is a defect. +5. **Determinism** — `stamp = hash(inputs)` (plan ref, world stamp, sorted overrides, engine version); same inputs ⇒ byte-identical stamp and identical verdicts/scores (G1). Never hashes materialised cells. +6. **Perturbation hook** — `knowledge_overrides` substitutes answers for the call only; **nothing is stored** and no knowledge edge is written; only the returned verdict/score artefacts join the store/trace on their own lineage. +7. **Comparability guard** — mixed stamps between the plan's assumptions and the `world` refuse `stamp_mismatch`, naming the disagreeing refs; persists nothing. +8. **Honesty (G6)** — widening any input band never narrows any output band; every point-realisation scores inside the output band; a widening never yields a more confident verdict. Binding via oracle O-1–O-4 (vignette §9). +9. **No delta** — scoring is a read-of-world producing verdicts, not a knowledge write; it publishes no delta (seam §10 is for cross-surface *writes*). + +## Refusals (all first-class, persist nothing) + +| reason | when | +|---|---| +| `stamp_mismatch` | plan's assumption lineage disagrees with the world's stamp (comparability guard) | +| `unknown_ref` | `plan` or `world` names no live object | + +Everything else (a valid plan scoring `violated` on a `must`) is a **success** carrying honest verdicts — a failing plan is not a refusal. diff --git a/specs/007-score/data-model.md b/specs/007-score/data-model.md new file mode 100644 index 0000000..e839a64 --- /dev/null +++ b/specs/007-score/data-model.md @@ -0,0 +1,41 @@ +# Phase 1 — Data Model (SPEC-07) + +Domain entities are generated from LinkML (`src/generated/types.ts`, slice D2) and must not be hand-edited. **SPEC-07 changes no schema** — every stored shape it needs already exists (added by D2 and SPEC-06). This file maps the spec's entities to the existing LinkML shapes and defines the scorer's service types (not stored objects, so not LinkML shapes). + +## Existing LinkML types reused (source of truth — do not hand-edit) + +| Entity | Shape | Notes | +|---|---|---| +| `Band` | `{lo, hi, unit}` | pure closed interval, no midpoint (DEC-15); the atom of every propagation | +| `Commitment` | `{statement, tier, metric, comparator, threshold, unit, owner, scope?}` | DEC-19; threshold is scalar fact-of-intent (DEC-14) | +| `Comparator` | `at_most \| at_least \| by_step \| never` | reduced to a signed margin band (research note §3) | +| `VerdictBand` | `robust \| marginal \| tight \| violated` | four stops, no decimals (DEC-9) | +| `CommitmentVerdict` | `{plan, commitment, scenario, world_stamp, verdict, margin?, engine_version}` | one per commitment; `margin` on demand | +| `PlanScore` | `{plan, scenario, world_stamp, criterion, score, engine_version}` | banded criteria vector (SPEC-08 organises) | +| `Plan` / `ElementPlan` / `RouteLeg` / `TaskWindow` | timed routes + task windows (DEC-20) | the scorer evaluates stated geometry, never searches | +| `CompiledWorld` / `Channel` / `RegionOverride` | sparse channels (SPEC-06) | materialised lazily at score time; never stored dense | +| `VignetteConfig` / `RegionGeometry` | grid + region geometry | the `RegionName → cell rect` map read by materialisation | +| `TraceEdge` (`scored_from`) | `{from_hash, to_hash, edge_type, stamp?, written_by}` | one per verdict/score → world (G3) | + +`scored_from` is already in `TraceEdgeType` (D2). No enum or class edit is required; `npm run gen` is **not** run in this slice. + +## Service types added (`src/seam.ts` — movement, not stored objects) + +| Type | Shape | Notes | +|---|---|---| +| `KnowledgeOverride` | `{ref: Ref, answer: Band}` | perturbation hook; substitutes for the call only (seam §5) | +| `ScoreRequest` | `{plan: Ref, world: Ref, scenario: LogicalId, knowledge_overrides?: KnowledgeOverride[], engine_version: string}` | scenario is recorded for provenance; the world is already excursioned (DEC-10) | +| `ScoreSuccess` | `{verdicts: CommitmentVerdict[], scores: PlanScore[], stamp: string}` | verdicts canonically ordered by commitment id | +| `ScoreResult` | `ScoreSuccess \| Refusal` | `stamp_mismatch` is the only scorer-specific refusal reason (already in `RefusalReason`) | + +## Internal shapes (not exported at the seam) + +| Type | Shape | Notes | +|---|---|---| +| `Interval` | operates on `Band` directly | `src/interval.ts`: `add/sub/scaleBy/max/min/contains/width`; unit-checked, rejects non-finite | +| `MetricFn` | `(plan, world, config, commitment) => Band` | `src/metrics.ts`: reach-step / exposure / state families keyed by `Commitment.metric` | +| margin band | `Band` oriented so satisfied ⟺ `lo ≥ 0` | `src/score.ts`: `verdictFor(marginBand) => VerdictBand` is the signs-only mapping | + +## Determinism + +`stamp = SHA-256(canonicalJson({plan: ref, world: world_stamp, overrides: sortedOverrides, engine_version}))` — over inputs, never over materialised cells (research note §2; seam §1). Overrides sorted by `ref.logical_id`; verdicts returned sorted by `commitment` id. Same inputs ⇒ byte-identical stamp and identical verdicts/scores (G1). diff --git a/specs/007-score/plan.md b/specs/007-score/plan.md new file mode 100644 index 0000000..39aca17 --- /dev/null +++ b/specs/007-score/plan.md @@ -0,0 +1,59 @@ +# Implementation Plan: Score — plan × world → verdicts + banded scores (SPEC-07) + +**Branch**: `claude/next-spec-bbbs94` · **Spec dir**: `specs/007-score/` · **Date**: 2026-07-13 +**Spec**: `specs/007-score/spec.md` · **Research note (DEC-11 gate)**: `docs/research/03-score-plan.md` (present) + +## Summary + +Build the scorer unit (DEC-10, the delivery linchpin): plan × world × scenario → `CommitmentVerdict[]` + `PlanScore[]`. Propagate banded metrics by interval arithmetic (its own tested module), evaluate stated routes by `(cell, time)` resolution over the sparse world, map each commitment to a four-stop verdict against a signed margin band (research note §3), write `scored_from` edges, and be deterministic. The acceptance leg is the vignette §9 oracle cases (O-1–O-3 exact, O-4 as a property). No schema change: `CommitmentVerdict`, `PlanScore`, `Plan`, `Band` all exist (D2). The demo moment ("the honest matrix") renders via a new `s2Matrix` component over a canned Meridian handful (sanctioned generator fallback, delivery plan §3). + +## Technical Context + +**Language**: TypeScript (ESM, `type: module`), Node ≥ 22, in-browser-capable (no Node-only APIs in `src/`). +**Testing**: vitest; `fast-check` added as a dev dependency for the O-4 property tests (property-based, per research note §4). +**Reuse (do not re-implement)**: `src/{store,trace,canonical,validate,seam}.ts`, `src/generated/types.ts`, `src/knowledge.ts` (for override resolution), `src/compile.ts` (world shape), `src/components/{bandPill,provenanceChip}.ts`. The sparse-channel materialisation is new (score-time) but reads the same `Channel`/`RegionOverride`/`VignetteConfig` shapes SPEC-06 produced. +**New source**: `src/interval.ts` (interval arithmetic), `src/materialise.ts` (score-time `(cell,time)` channel resolution), `src/metrics.ts` (the C1–C6 + abstract-oracle metric registry), `src/score.ts` (the service), `src/components/s2Matrix.ts` (S2 chips). New fixtures: `fixtures/plans.json` (canned Meridian handful). +**Constraints**: banded honesty at the seam (G2); no midpoint anywhere (DEC-15); determinism over inputs, never over cells (G1); scenario-blind scorer (DEC-10); first-class refusals (seam §1). + +## Constitution Check + +- **Register-first (DEC-2)**: SPEC-07 introduces no new decision — the four-stop mapping is *decided in the research note* under the latitude vignette §9/O-3 explicitly delegated to Stage-3 research; no peer document asserts a decision. No new register candidate is required by the scorer itself (the sparse-channel candidate from Stage 2, concept §6 item 12, remains the only open one and is unaffected). +- **Research-first (DEC-11)**: `docs/research/03-score-plan.md` present before implementation — gate satisfied. +- **Banded honesty (G2)**: verdicts cross as the four-stop scale, margins as bands; the scorer has no scalar-emitting path. Interval arithmetic *is* DEC-15 in the compute layer. +- **Frozen identifiers**: C1–C6, R1/R2/R3/R3m, FE-*, O-1–O-4 rendered as-is; canned plans use P* identifiers (vignette §8 frozen family), clearly marked demonstrator scaffolding. +- **Batch propagation**: the current-phase line (CLAUDE.md), build/delivery peers, and the gallery sweep in the same change. + +## Project Structure (this slice) + +``` +docs/research/03-score-plan.md # DEC-11 gate (done) +specs/007-score/{spec,plan,tasks,data-model,research,quickstart}.md +specs/007-score/contracts/score-service.md +specs/007-score/checklists/requirements.md +src/interval.ts # +,-,scaleBy,max,min,contains,width,unit-checked +src/materialise.ts # channelAt(world, config, kind, x, y, t) -> Band +src/metrics.ts # metric registry: reach-step / exposure / state + abstract oracle +src/score.ts # ScoreService: /score, verdict mapping, guards, edges, stamp +src/components/s2Matrix.ts +fixtures/plans.json # canned Meridian handful (P1/P2) +tests/{interval,score,oracle,s2Matrix}.test.ts +``` + +## Design decisions (from the research note) + +1. **Interval arithmetic is its own module** so O-1 tests the arithmetic, not the plumbing. Operators are the textbook closed-interval rules; every op checks units and rejects non-finite results (a severed route is caught before it produces `Infinity`). +2. **`(cell, time)` resolution, no graph expansion** (note §2): `channelAt` picks the innermost active `RegionOverride` (later `from_step` wins) or the default; metrics reduce over a plan's stated leg samples. +3. **Signed margin bands, signs-only verdict** (note §3): reduce comparator to `M = [m_lo, m_hi]` with satisfied ⟺ margin ≥ 0, then map by the signs of the endpoints. Unique O-3-satisfying four-stop rule with no interior cut. +4. **G6 as a property** (note §4): fast-check over random bands/widenings/thresholds for superset, membership, and verdict-monotonicity. +5. **ε stays out of verdicts** (note §3/§5): distinctness/ε-non-domination is SPEC-08's organiser, recorded in the note, not built here. + +## Complexity Tracking + +Metric library scope is bounded to exactly what C1–C6 and the abstract oracle need (reach-step, exposure, state families). No general optimiser, no generator (SPEC-08), no scenario strip (SPEC-10). `fast-check` is the only new dependency, justified by FR-005/O-4 (a hand-rolled random-property loop would be weaker and unaudited). + +## Phase Gates + +- Research complete (DEC-11): ✅ `03-score-plan.md`. +- Spec approved: this slice. +- Tests-first: oracle + contract tests written and confirmed failing before `src/score.ts` behaviour (constitution quality gate 2). +- Green: typecheck + full suite; oracle constants never regenerated. diff --git a/specs/007-score/quickstart.md b/specs/007-score/quickstart.md new file mode 100644 index 0000000..cf29691 --- /dev/null +++ b/specs/007-score/quickstart.md @@ -0,0 +1,50 @@ +# Quickstart — SPEC-07 (Score) + +## Run it + +``` +npm install # picks up fast-check (dev dep for O-4 property tests) +npm run typecheck +npm test # SPEC-06 baseline + SPEC-07 (interval, oracle, score, s2Matrix) +npm run gallery # regenerates the gallery incl. the honest-matrix demo moment +``` + +## The oracle, by hand (never regenerate these) + +``` +O-1 strait_open_step = 2 + [4,6] + [3,5] = [9,13] +O-2 [9,13] vs at_most 28 -> M = [28-13, 28-9] = [15,19] -> robust +O-3 [9,13] vs at_most 12 -> M = [12-13, 12-9] = [-1, 3] -> tight (neither robust nor violated) + sweep at_most T, T=8..14: 8→violated 9..12→tight 13→marginal 14→robust (changes only at 9,13) +O-4 A widened [4,6]→[3,7]: 2 + [3,7] + [3,5] = [8,14] ⊇ [9,13] +``` + +## The verdict mapping (research note §3) + +Reduce a commitment to a signed margin band `M = [m_lo, m_hi]`, satisfied ⟺ margin ≥ 0: + +``` +at_most/by_step T: M = [T - hi, T - lo] +at_least T: M = [lo - T, hi - T] +never: M = [-hi, -lo] + +verdict: m_lo > 0 -> robust + m_lo = 0 ≤ m_hi -> marginal + m_lo < 0 ≤ m_hi -> tight + m_hi < 0 -> violated +``` + +## Score a Meridian plan + +```ts +import { ScoreService } from '../src/score.js'; +const scorer = new ScoreService({ store, trace }); +const result = await scorer.score({ + plan: planRef, world: worldRef, scenario: 'R1', engine_version: '0.1.0', +}); +// result.verdicts: one per C1..C6, four-stop + margin band +// result.scores: banded criteria vector +// backward trace from any verdict reaches the named knowledge + owners (G3) +``` + +`knowledge_overrides` re-scores for the call only (persists nothing). A plan whose stamp disagrees with the world refuses `stamp_mismatch`. A plan that fails a `must` is a **success** with a `violated` verdict — a failing plan is honest output, not a refusal. diff --git a/specs/007-score/research.md b/specs/007-score/research.md new file mode 100644 index 0000000..2f61ad7 --- /dev/null +++ b/specs/007-score/research.md @@ -0,0 +1,15 @@ +# Phase 0 — Research (SPEC-07) + +The DEC-11 research gate for this slice is the canonical note **`docs/research/03-score-plan.md`** (Stage 3), present before implementation. This file does not duplicate it; it records the decisions that bind the build, all sourced there. + +| Decision | Resolution | Source | +|---|---|---| +| Metric propagation | Classical interval arithmetic on pure closed bands (`+`, `−`, `× scalar`, `max`, `min`); no mean/mid/most-likely ever (DEC-15). Its own tested module. | note §1 | +| Route evaluation | `(cell, time)` resolution over the sparse world (innermost active override, later `from_step` wins); **no** time-expanded graph. The scorer evaluates stated routes, never searches. | note §2 (closes note 02 §3's forward flag) | +| Four-stop verdict | Signed margin band `M`, satisfied ⟺ margin ≥ 0; verdict by **signs only**: `robust` iff `m_lo>0`, `marginal` iff `m_lo=0`, `tight` iff `m_lo<0≤m_hi`, `violated` iff `m_hi<0`. Unique O-3-satisfying, no-interior-cut rule. | note §3 | +| O-2 / O-3 outcomes | O-2 ⇒ robust, margin `[15,19]`; O-3 straddle ⇒ tight, transitions only at band edges 9 & 13. Decided here, not in code. | note §3, vignette §9 | +| G6 / O-4 | Inclusion-monotone + sound (containment) + verdict-monotone; asserted by property-based tests, not only the vignette widening. | note §4 | +| ε-dominance | Reserved for *plan comparison* (SPEC-08 organiser), never for verdicts — a threshold is the commander's, not a tunable margin. | note §3, §5 | +| Handful axes / JP 5-0 | Domain's four generation axes recorded for SPEC-08; four-stop scale cross-checked against suitability/feasibility/acceptability. | note §5 | + +**Unknowns resolved**: none outstanding. The generator (SPEC-08), the scenario strip (SPEC-10), and ε-non-domination organisation are explicitly out of this slice; the canned Meridian handful (`fixtures/plans.json`) is the sanctioned fallback (delivery plan §3). diff --git a/specs/007-score/spec.md b/specs/007-score/spec.md new file mode 100644 index 0000000..de0aa6b --- /dev/null +++ b/specs/007-score/spec.md @@ -0,0 +1,141 @@ +# Feature Specification: Score — plan × world → verdicts + banded scores (SPEC-07) + +**Feature Branch**: `claude/next-spec-bbbs94` (spec dir `007-score`) + +**Created**: 2026-07-13 + +**Status**: Draft + +**Stage**: Build-plan Stage 3 · **Depends on**: SPEC-01 (store), SPEC-02 (trace store), SPEC-06 (compile — CompiledWorld, sparse channels, VignetteConfig geometry), D3 (seam contract §5) · **Research note**: `docs/research/03-score-plan.md` (DEC-11 gate — present) + +**Input**: The scorer unit (`POST /score`: plan × world × scenario → `CommitmentVerdict[]` + `PlanScore[]`, with the `knowledge_overrides` perturbation hook) — the delivery linchpin (DEC-10). It propagates banded metrics by interval arithmetic, maps each commitment to a four-stop verdict against a signed margin band, and writes `scored_from` edges so every verdict opens a complete trace chain. Its correctness leg is the vignette §9 oracle cases (O-1–O-3 reproduced exactly, O-4 as a property) — a deterministic, traced scorer that propagated intervals wrongly would pass every other exit while being exactly the false-precision machine ASSAY refuses. + +## User Scenarios & Testing *(mandatory)* + +Thesis A (an honest JIPOE pipeline) reaches the scorer here. SPEC-05 made the *knowledge* honest; SPEC-06 made the *world* honest; SPEC-07 makes the *judgement of a plan against that world* honest — banded, four-stop, backward-traceable to named owners, and monotone under uncertainty (G6). Every scenario is played on the Meridian fixtures (vignette §4–§6) except the oracle cases, which are the deliberately-abstract independent correctness leg (vignette §9). No assessed scalar crosses the seam unbanded (G2); verdicts cross only as the four-stop scale with `margin` shown on demand. + +### User Story 1 — The scorer reproduces the oracle cases exactly (Priority: P1) 🎯 correctness leg + +The scorer's inner loop is interval arithmetic and threshold comparison. The vignette §9 oracle cases pin exactly those parts, hand-computed on paper and binding on acceptance: O-1 (`start 2 + A[4,6] + B[3,5] = [9,13]`), O-2 (`[9,13]` vs `at_most 28` ⇒ **robust**, margin **[15,19]**), O-3 (`[9,13]` vs `at_most 12` ⇒ **neither robust nor violated**, and as the threshold sweeps 8→14 the verdict changes only at the band edges 9 and 13). The four-stop mapping is decided by research note `03-score-plan.md` §3 (signed margin bands, signs-only); this story is where that decision is proven against the oracle. + +**Why this priority**: This is the whole reason SPEC-07 exists ahead of the generator (DEC-10, build plan §Stage 3): a scorer that is deterministic and fully traced but arithmetically wrong is the exact failure ASSAY is built to refuse, and it would pass every other exit criterion. The oracle is the independent leg that no other test provides. + +**Independent Test**: Feed the abstract two-segment metric (`strait_open_step = start + dur(A) + dur(B)`) and the C2-style thresholds directly to the interval module and the verdict mapping; assert O-1's `[9,13]`, O-2's **robust** + `[15,19]`, and O-3's straddle verdict plus the band-edge-only transitions across the sweep. Expected values are the hand-computed constants, never regenerated from the scorer. + +**Acceptance Scenarios**: + +1. **Given** `start = 2`, `A = [4,6]`, `B = [3,5]` (assessed, banded — G2), **When** the metric is propagated, **Then** `strait_open_step = [9,13]` exactly — pure interval addition (O-1). +2. **Given** metric band `[9,13]` and commitment `at_most 28`, **When** it is scored, **Then** the verdict is **robust** and the `margin` band is `[15,19]` (O-2). +3. **Given** metric band `[9,13]` and commitment `at_most 12`, **When** it is scored, **Then** the verdict is **tight** — neither robust nor violated, both outcomes inside the band (O-3). +4. **Given** metric band `[9,13]` and `at_most T` for `T` swept 8→14, **When** each is scored, **Then** the verdict changes only at `T = 9` and `T = 13` and is constant between and outside them (O-3); the sweep yields violated (T<9), tight (9≤T<13), marginal (T=13), robust (T>13). + +--- + +### User Story 2 — Propagation honesty holds under widening (Priority: P1) + +Widening any input band never narrows any output metric band, and every point-realisation of the inputs scores inside the output band (candidate G6 / oracle O-4). This is banded honesty stated as a theorem: the scorer must be inclusion-monotone and sound, or its bands are decoration. + +**Why this priority**: G6 is the machine-checkable form of the constitution's banded-honesty invariant (seam §G). It is P1 alongside US1 because the two together are SPEC-07's correctness leg — O-1–O-3 pin specific values, O-4 pins the property across all values. + +**Independent Test**: Re-run O-1 with `A` widened `[4,6] → [3,7]`; assert `strait_open_step = [8,14] ⊇ [9,13]`. Then property-based (fast-check): for random bands, widenings, comparators and thresholds, assert output-superset-under-widening, point-realisation membership, and verdict-monotonicity (widening never yields a *more* confident verdict). + +**Acceptance Scenarios**: + +1. **Given** O-1 with `A` widened to `[3,7]`, **When** propagated, **Then** `strait_open_step = [8,14]` and `[8,14] ⊇ [9,13]` (O-4). +2. **Given** any input bands and any widening of one of them, **When** the metric is propagated, **Then** the output band contains the un-widened output band (inclusion monotonicity). +3. **Given** any point-realisation of the input bands (a scalar in each), **When** the metric is evaluated at those points, **Then** the scalar result lies within the banded output (soundness). +4. **Given** any commitment and a widening of the metric band, **When** re-scored, **Then** the verdict does not move toward *more* confidence (robust ← marginal ← tight ← violated is the only permitted direction of movement). + +--- + +### User Story 3 — The honest matrix over real Meridian output (Priority: P2) + +A commander looks at the S2 planner matrix: a handful of plans × the commitments C1–C6, each cell a four-stop chip, margin bands on hover, no decimals anywhere. Clicking any verdict opens a complete why-chain — verdict → the plan and the world it was scored against → the compiled channels → the named knowledge and its owners. This is the demo moment: **"the honest matrix — pick a verdict, walk it to the assessment and owner it rests on."** + +**Why this priority**: It is the Stage-3 user-observable and the surface every depth stage (4–6) re-scores into. P2 because it composes US1/US2's correctness onto real fixture plans; the canned handful (2–3 hand-authored Meridian plans) is the sanctioned fallback for the generator (delivery plan §3) — a real scorer over a canned handful keeps every downstream consumer honest. + +**Independent Test**: Score a hand-authored Meridian plan against the compiled base world; assert one `CommitmentVerdict` per commitment on the four-stop scale, each carrying a `margin` Band; assert a `scored_from` edge from each verdict to the world (and to the plan), so a backward trace walk reaches named knowledge; render the S2 matrix component and assert four-stop chips with margin-on-hover and no bare decimals in verdict cells. + +**Acceptance Scenarios**: + +1. **Given** a Meridian plan, the compiled base world, and the commitments C1–C6, **When** `/score` is called, **Then** it returns one `CommitmentVerdict` per commitment (four-stop `verdict`, `margin` Band) and the `PlanScore[]` banded criteria, with a stamp (seam §5). +2. **Given** a returned verdict, **When** its trace is walked backward, **Then** it reaches the CompiledWorld, the consumed knowledge, and the named owners — a complete chain (G3); a verdict with no backward chain is a defect. +3. **Given** the S2 matrix render, **When** it is inspected, **Then** every cell is a four-stop chip, `margin` appears only on hover as a band, and no verdict cell shows a decimal (ui-design §2, G2). +4. **Given** the robustness-trap plan under R1 vs R2 (thesis C), **When** both are scored, **Then** the R1-early-strait plan's C1/C2 verdicts are healthier under R1 than under R2 — the honest matrix shows the collapse the later Stage-5 exit dramatises (demonstrated here only as correct scoring, not yet as the scenario strip). + +--- + +### User Story 4 — Determinism: same inputs, identical verdicts (Priority: P2) + +Scoring the same plan against the same world with the same overrides yields a byte-identical stamp and identical verdicts and scores. The scorer is a pure function of its stamped inputs. + +**Why this priority**: Every depth demo (sensitivity, discrimination, robustness) is a re-scoring loop; if the same inputs can produce two verdicts, the loops are theatre (DEC-4). P2 as it underwrites US3's reproducibility. + +**Independent Test**: Score a plan twice against the same world; assert identical stamp, identical verdict list, identical margin bands. Present the commitment set in a different order; assert the returned verdicts are canonicalised to a fixed order and the stamp is unchanged. + +**Acceptance Scenarios**: + +1. **Given** a fixed plan, world, and (absent or fixed) overrides, **When** `/score` is called twice, **Then** both return the same stamp and the same verdicts/scores (G1; seam §5). +2. **Given** the commitments presented in a different order, **When** scored, **Then** the returned `CommitmentVerdict[]` is in a fixed canonical order and the stamp is unchanged. + +--- + +### User Story 5 — The perturbation hook and the comparability guard (Priority: P3) + +Sensitivity (Stage 6) is a re-scoring loop, not a fork: `knowledge_overrides` substitutes an answer *for this call only* — nothing is stored, no edge is written except on the returned artefacts' own lineage. And the scorer refuses to compare across incompatible stamps: if a plan's assumptions and the world disagree on stamp, `/score` refuses `stamp_mismatch` (the comparability guard), a first-class `Refusal` like every other seam decline. + +**Why this priority**: The hook is what makes Stages 4–6 cheap re-scoring loops over one honest evaluator (delivery plan §critical-path); the guard is what stops the matrix silently comparing worlds that cannot be compared. P3 because the base scorer (US1–US4) stands without them, but they are in the seam §5 contract and cheap to land now. + +**Independent Test**: Score with a `knowledge_overrides` entry substituting a wider band for one answer; assert the verdict moves toward uncertainty (US2 direction) and that the store and trace graph are unchanged apart from the returned verdict/score lineage. Score a plan carrying a stamp that disagrees with the world's; assert `Refusal{stamp_mismatch}` naming the disagreeing refs. + +**Acceptance Scenarios**: + +1. **Given** a `knowledge_overrides` entry widening one answer band, **When** `/score` is called, **Then** the affected verdict moves no more confident (G6) and **nothing is stored** beyond the returned verdict/score artefacts — the override is not persisted and writes no knowledge edge (seam §5). +2. **Given** a plan whose stamp lineage disagrees with the world's stamp, **When** `/score` is called, **Then** it refuses `stamp_mismatch`, `offending` naming the disagreeing refs, and persists nothing. + +--- + +### Edge Cases + +- **Severed route** (R3 zeroes `mobility.causeway` from its demolition step): a reach-step metric over a leg crossing a zero-mobility region has an unbounded duration — surfaced as a `violated` reach (the objective is unreachable), never an `Infinity` leaking into a band or a canonical-JSON throw. +- **Unit mismatch** in a metric composition (a fixture defect) is surfaced as an error at score time, never silently coerced (mirrors compile's config-defect surfacing). +- **Answer-absent knowledge** referenced by a metric: the metric reads the channel default (the world already dropped answer-absent objects at compile), so an open question does not crash scoring — it scores against the quiet baseline, which is itself banded. +- **Point band** (`lo = hi`) is a legal degenerate interval; a threshold exactly on it yields `marginal` (worst = best = on the line), consistent with the signs-only mapping. +- **`at_least`/`never`/`by_step`** comparators reduce to the signed margin band per research note §3; each is exercised (C5 `at_least`, C3 `at_most 0` behaving as a never-fire bound). + +## Requirements *(mandatory)* + +### Functional Requirements + +- **FR-001**: `/score` MUST accept `{plan: Ref, world: Ref, scenario: Ref, knowledge_overrides?: {ref, answer: Band}[]}` and return `{verdicts: CommitmentVerdict[], scores: PlanScore[], stamp}` or a first-class `Refusal` (seam §5). +- **FR-002**: Metric propagation MUST be interval arithmetic on pure closed bands (DEC-15) — `+`, `−`, `× scalar`, `max`, `min`, unit-checked — with **no** mean/mid/most-likely operation anywhere. It MUST reproduce O-1 (`[9,13]`) exactly. +- **FR-003**: Each commitment MUST be scored to a four-stop `verdict` via the signed margin band mapping (research note §3): `robust` iff `m_lo > 0`, `marginal` iff `m_lo = 0 ≤ m_hi`, `tight` iff `m_lo < 0 ≤ m_hi`, `violated` iff `m_hi < 0`. It MUST reproduce O-2 and O-3. +- **FR-004**: Every `CommitmentVerdict` MUST carry a `margin` Band; verdicts MUST cross the seam only as the four-stop scale (G2); no decimal appears in a verdict cell on any surface (ui-design §2). +- **FR-005**: The scorer MUST satisfy G6 (oracle O-4): inclusion-monotone under widening, sound under point-realisation, verdict-monotone toward uncertainty — asserted by property-based tests, not only the vignette instance. +- **FR-006**: Route metrics MUST be evaluated by `(cell, time)` resolution over the sparse world (research note §2) — lazy materialisation of the innermost active `RegionOverride` (later `from_step` wins) or the channel default — building no time-expanded graph and touching only cells a plan occupies. +- **FR-007**: On success the scorer MUST write a `scored_from` edge from each verdict/score to the world it was scored against, so every verdict opens a complete backward trace chain to named knowledge (G3). +- **FR-008**: The scorer MUST be deterministic: same plan + world + overrides ⇒ byte-identical stamp and identical, canonically-ordered verdicts/scores (G1). The stamp is a hash over inputs (plan ref, world stamp, overrides, engine version), never over materialised cells. +- **FR-009**: `knowledge_overrides` MUST substitute answers for the call only — persisting nothing and writing no knowledge edge; only the returned verdict/score artefacts join the store/trace on their own lineage (seam §5). +- **FR-010**: The scorer MUST refuse `stamp_mismatch` when a plan's assumption lineage and the world disagree on stamp (the comparability guard), naming the disagreeing refs and persisting nothing. +- **FR-011**: A severed route (zero-mobility region on a leg) MUST surface as a `violated` reach, never a non-finite band; unit mismatches MUST surface as a score-time error (config/fixture defect), never a silent coercion. +- **FR-012**: The scorer MUST be scenario-blind (DEC-10): it reads the already-excursioned world and does not re-derive the excursion; the `scenario` field is recorded on the verdicts for provenance only. + +### Key Entities + +- **CommitmentVerdict** (existing LinkML) — `{plan, commitment, scenario, world_stamp, verdict: VerdictBand, margin?: Band, engine_version}`. `verdict` is the four-stop scale; `margin` is on-demand only. +- **PlanScore** (existing LinkML) — `{plan, scenario, world_stamp, criterion, score: Band, engine_version}`. The banded criteria vector SPEC-08 will organise by non-domination. +- **Plan / ElementPlan / RouteLeg / TaskWindow** (existing LinkML, DEC-20) — timed routes + task windows; the scorer evaluates, never searches. +- **ScoreRequest / ScoreSuccess / ScoreResult** (service types, `src/seam.ts` — not stored) — the seam §5 movement shapes. +- **Band** (existing, DEC-15) — pure closed interval `{lo, hi, unit}`, no midpoint; the atom of every propagation. + +## Success Criteria *(mandatory)* + +- **SC-001**: O-1 reproduced exactly — `start 2 + A[4,6] + B[3,5] = [9,13]`. +- **SC-002**: O-2 reproduced exactly — verdict **robust**, margin **[15,19]**. +- **SC-003**: O-3 reproduced exactly — straddle verdict is neither robust nor violated, and the swept-threshold verdict changes only at band edges 9 and 13. +- **SC-004**: O-4 holds — `[8,14] ⊇ [9,13]` on the vignette widening, and the containment/monotonicity/soundness property holds under property-based testing. +- **SC-005**: A Meridian plan scores to one four-stop verdict per commitment C1–C6, each with a margin band; every verdict opens a complete backward trace chain to named owners. +- **SC-006**: Scoring the same inputs twice yields a byte-identical stamp and identical verdicts/scores; commitment order does not affect the stamp. +- **SC-007**: `knowledge_overrides` re-scores without persisting the override; `stamp_mismatch` refuses across incompatible stamps and persists nothing. +- **SC-008**: The S2 matrix renders four-stop chips with margin-on-hover and no decimals in verdict cells; the gallery shows the honest-matrix demo moment run by the actual scorer. +- **SC-009**: `npm run typecheck` clean; `npm test` green (SPEC-06 baseline plus the SPEC-07 suite); oracle expected values are committed constants, never regenerated. diff --git a/specs/007-score/tasks.md b/specs/007-score/tasks.md new file mode 100644 index 0000000..c331ebb --- /dev/null +++ b/specs/007-score/tasks.md @@ -0,0 +1,91 @@ +--- +description: "Task list for SPEC-07 — Score (plan × world → verdicts + banded scores)" +--- + +# Tasks: Score — plan × world → verdicts + banded scores (SPEC-07) + +**Input**: Design documents from `specs/007-score/` +**Prerequisites**: plan.md, spec.md, research.md, data-model.md, contracts/score-service.md, quickstart.md; research note `docs/research/03-score-plan.md` (DEC-11 gate — present) +**Tests**: REQUIRED — the oracle cases (vignette §9) and the contract invariants are written against the seam §5 shape and **confirmed failing before implementation** (TDD; constitution quality gate 2). Oracle expected values are committed constants, never regenerated from the scorer. +**Organization**: shared substrate (interval arithmetic + service skeleton), then by user story in priority order. Stage-3 exits are the acceptance scenarios. + +## Format: `[ID] [P?] [Story] Description` +- **[P]**: parallelizable (different file, no dependency on an incomplete task) +- **[Story]**: US1 (oracle O-1..O-3, P1) · US2 (G6/O-4, P1) · US3 (honest matrix, P2) · US4 (determinism, P2) · US5 (hook + guard, P3) + +## Path conventions +Single project: `src/`, `tests/`, `fixtures/`. Reuse `src/{store,trace,canonical,validate,seam,knowledge,compile}.ts`, `src/generated/types.ts`, `src/components/{bandPill,provenanceChip}.ts`. **No schema change** — `CommitmentVerdict`/`PlanScore`/`Plan`/`Band` already exist; do **not** run `npm run gen`. + +--- + +## Phase 1: Setup + +- [ ] T001 Confirm the SPEC-06 baseline is green: `npm install`, `npm run typecheck`, `npm test` (107 tests). Record any failing baseline before Stage-3 code. +- [ ] T002 Add `fast-check` as a dev dependency (`package.json`), for the O-4 property tests (research note §4). `npm install`. +- [ ] T003 Confirm the pieces SPEC-07 reuses exist: `CommitmentVerdict`/`PlanScore`/`Plan`/`Band` in `src/generated/types.ts`; `scored_from` in `TraceEdgeType`; `stamp_mismatch` in `RefusalReason` (`src/seam.ts`); the sparse `Channel`/`RegionOverride`/`VignetteConfig` shapes SPEC-06 produced. + +## Phase 2: Foundational (interval arithmetic + service skeleton — blocks all stories) + +**Purpose**: the tested arithmetic module, the score-time materialiser, and the service shell every story composes. + +- [ ] T004 Write `tests/interval.test.ts` FIRST and confirm failing: add/sub/scaleBy/max/min on closed bands; unit-checked; rejects non-finite; O-1 (`2 + [4,6] + [3,5] = [9,13]`) and O-4 widening (`[8,14] ⊇ [9,13]`); `contains`/`width`. +- [ ] T005 Implement `src/interval.ts`: `add`, `sub`, `scaleBy`, `max`, `min`, `contains`, `width`, `scalar(n, unit)` — textbook closed-interval rules on `Band`; unit-checked; reject non-finite (a severed route is caught here). Make T004 pass. **No mean/mid operation** (DEC-15). +- [ ] T006 [P] Implement `src/materialise.ts`: `channelAt(world, config, kind, x, y, t) => Band` — innermost active `RegionOverride` whose geometry contains `(x,y)` and whose window contains `t` (later `from_step` wins), else the channel default. Pure; reads config geometry; stores nothing (research note §2). +- [ ] T007 [P] Add scorer service types to `src/seam.ts`: `KnowledgeOverride`, `ScoreRequest`, `ScoreSuccess`, `ScoreResult` (per data-model.md). Types only; reuse `Refusal`/`Ref`. +- [ ] T008 Implement `src/metrics.ts`: a registry keyed by `Commitment.metric` — reach-step (`strait_open_step`/`port_open_step`/`extraction_step` = start + Σ leg durations via interval division on mobility), exposure (`threat_exposure`/`civil_harm_exposure` = Σ over occupied (cell,time) of channel band × dwell, scoped), state (`causeway_intact` = banded read). Plus the abstract oracle metric (start + dur(A) + dur(B)) for O-1. Each returns a `Band`; unknown metric surfaces a defect. +- [ ] T009 Scaffold `ScoreService` in `src/score.ts`: constructor composes `ObjectStore` + `TraceStore`; private helpers for override resolution, the verdict mapping, and edge writes. No behaviour yet. + +**Checkpoint**: interval tests green; materialiser + metric registry compile; the scorer shell compiles. + +--- + +## Phase 3: User Story 1 — the scorer reproduces the oracle (P1) 🎯 correctness leg + +**Goal**: interval arithmetic reproduces O-1; the signed-margin-band mapping reproduces O-2 and O-3 including the swept-threshold band-edge-only transitions. +**Independent test**: abstract two-segment metric + C2-style thresholds → O-1/O-2/O-3 constants. + +- [ ] T010 [US1] Write `tests/oracle.test.ts` FIRST and confirm failing: O-1 `[9,13]`; O-2 robust + margin `[15,19]`; O-3 tight; O-3 sweep `at_most T` 8→14 → transitions only at 9,13 and the violated/tight/marginal/robust sequence. Constants hand-typed from vignette §9. +- [ ] T011 [US1] Implement `marginBand(comparator, threshold, valueBand)` and `verdictFor(marginBand)` in `src/score.ts` (research note §3 mapping). Make O-1/O-2/O-3 pass. Verdicts cross only as the four-stop scale; `margin` returned as a Band (G2). + +## Phase 4: User Story 2 — propagation honesty (G6/O-4) (P1) + +**Goal**: containment under widening, soundness, verdict-monotonicity — the vignette instance and the property. +**Independent test**: O-4 widening + fast-check property. + +- [ ] T012 [US2] Extend `tests/oracle.test.ts` FIRST and confirm failing: O-4 `[8,14] ⊇ [9,13]`; fast-check — for random `[lo,hi]`, widening, comparator, threshold: output-superset-under-widening, point-realisation membership, verdict never more confident under widening. +- [ ] T013 [US2] Verify/adjust `src/interval.ts` + verdict mapping so the property holds by construction (inclusion-monotone operators; signs-only mapping is monotone). Make T012 pass. No code should special-case the property — it must fall out of the design. + +## Phase 5: User Story 3 — the honest matrix (P2) + +**Goal**: score a canned Meridian plan against the base world → four-stop verdicts + margins + `scored_from` edges; render the S2 matrix. +**Independent test**: score P1 vs base world; one verdict per C1–C6; backward trace reaches named owners; matrix chips render. + +- [ ] T014 [US3] Author `fixtures/plans.json`: a canned Meridian handful (P1 "strait-early", P2 "sweep-first" — the thesis-C pair), each a `Plan` with `elements` of timed `RouteLeg`s over BROOM/PACKHORSE/ANVIL/KINGFISHER, using region-consistent cells. Validate against generated types in `tests/fixtures.test.ts`. +- [ ] T015 [US3] Write `tests/score.test.ts` FIRST and confirm failing: `/score(P1, baseWorld, R1)` → one `CommitmentVerdict` per C1–C6 on the four-stop scale, each with a `margin` Band; a `scored_from` edge per verdict/score to the world; backward trace from a verdict reaches K-owners (G3); P1 healthier under R1 than R2 on C1/C2 (thesis-C direction). +- [ ] T016 [US3] Implement the full `score()` in `src/score.ts`: resolve plan+world (`unknown_ref`), apply `knowledge_overrides` (call-only), evaluate each commitment's metric via `src/metrics.ts` + `src/materialise.ts`, map to verdict + margin, build `PlanScore[]`, stamp (over inputs), store verdicts/scores, write `scored_from` edges, return canonically-ordered results. Make T015 pass; no delta (contract §9). +- [ ] T017 [US3] Implement `src/components/s2Matrix.ts`: plans × commitments grid of four-stop chips (one colour language, ui-design §2), `margin` band on hover via `bandPill`, no decimals in cells; click-through affordance to the trace. Write `tests/s2Matrix.test.ts`: chip per cell, margin-on-hover markup present, assert no bare decimal in any verdict cell. + +## Phase 6: User Story 4 — determinism (P2) + +- [ ] T018 [US4] Extend `tests/score.test.ts` FIRST and confirm failing: score P1 twice → identical stamp + identical verdicts/scores; commitments in shuffled order → identical stamp and canonically-ordered verdicts. +- [ ] T019 [US4] Ensure `src/score.ts` stamps over sorted inputs and returns verdicts sorted by `commitment` id. Make T018 pass. + +## Phase 7: User Story 5 — perturbation hook + comparability guard (P3) + +- [ ] T020 [US5] Extend `tests/score.test.ts` FIRST and confirm failing: `knowledge_overrides` widening one answer → affected verdict no more confident (G6) and store/trace unchanged apart from the returned artefacts; a stamp-disagreeing plan → `Refusal{stamp_mismatch}` naming the refs, nothing persisted. +- [ ] T021 [US5] Implement the hook (substitute answer for the call only; persist nothing) and the `stamp_mismatch` guard in `src/score.ts`. Make T020 pass. + +## Phase 8: Gallery + docs sweep (batch propagation) + +- [ ] T022 Wire the honest-matrix demo moment into `scripts/build-gallery.ts`: run the actual scorer over P1/P2 against the resolved base world, render `s2Matrix`, add the "pick a verdict, walk it" caption. `npm run gallery`. +- [ ] T023 Batch-propagate canonical docs in this change: CLAUDE.md current-phase line (Stage 3 SPEC-07 built); build/delivery peers where they name SPEC-07 status; note that SPEC-08 (generator) remains sacrificial next. No register candidate (SPEC-07 asserts no new decision). +- [ ] T024 Green gate: `npm run typecheck`, `npm test` (SPEC-06 baseline + SPEC-07), `npm run gallery`. Confirm oracle constants unchanged and the gallery renders the matrix. + +--- + +## Dependencies + +- Phase 2 (interval + materialiser + metrics + shell) blocks all stories. +- US1 (verdict mapping) and US2 (property) are the correctness leg — land before US3 consumes them. +- US3 (matrix) depends on the full `score()` and the plan fixtures; US4/US5 extend `score()`. +- Gallery (T022) depends on US3; docs sweep (T023) and green gate (T024) last. diff --git a/src/components/s2Matrix.ts b/src/components/s2Matrix.ts new file mode 100644 index 0000000..ff125a9 --- /dev/null +++ b/src/components/s2Matrix.ts @@ -0,0 +1,72 @@ +/** + * SPEC-07 — S2 planner verdict matrix, "the honest matrix" (ui-design §2, §S2). + * + * Renders plans × commitments as a grid of four-stop chips — robust / marginal / + * tight / violated, one colour language, no decimals in any cell (DEC-9). The + * `margin` Band appears only on demand (a hover title), never as a headline + * number (knowledge model §108; G2). Framework-free HTML string; depends only on + * generated types and the scorer's verdict objects; calls no service and holds + * no state (constitution I — it arranges projections only). A verdict cell links + * conceptually to its trace chain (the `scored_from` edge the scorer wrote); the + * click-through surface is the S3 why-chain (a later slice). + */ +import type { Band, CommitmentVerdict, VerdictBand } from '../generated/types.js'; + +const esc = (s: string): string => + s.replace(/&/g, '&').replace(//g, '>').replace(/"/g, '"'); + +/** One colour language across every surface (ui-design §2). */ +const STOP: Record = { + robust: { bg: '#1E6B3A', fg: '#FFFFFF', label: 'robust' }, + marginal: { bg: '#7A6A12', fg: '#FFFFFF', label: 'marginal' }, + tight: { bg: '#9A5212', fg: '#FFFFFF', label: 'tight' }, + violated: { bg: '#8A2020', fg: '#FFFFFF', label: 'violated' }, +}; + +const marginText = (m?: Band): string => + m === undefined + ? 'no margin (objective unreachable)' + : m.lo === m.hi + ? `margin ${m.lo} ${m.unit}` + : `margin ${m.lo}–${m.hi} ${m.unit}`; + +function chip(v: CommitmentVerdict): string { + const s = STOP[v.verdict]; + // The margin band rides on the hover title — shown only on demand, never a + // decimal in the cell face (G2). The cell face is the four-stop word alone. + return ` + ${s.label} + `; +} + +export interface S2Cell { + plan: string; + verdicts: CommitmentVerdict[]; +} + +/** + * @param commitmentIds column order (e.g. ['C1','C2','C3','C4','C5','C6']) + * @param rows one entry per plan, its verdict objects (any order) + */ +export function s2Matrix(commitmentIds: string[], rows: S2Cell[]): string { + const head = ` + plan \\ commitment + ${commitmentIds.map((c) => `${esc(c)}`).join('')} + `; + const body = rows + .map((row) => { + const byCommitment = new Map(row.verdicts.map((v) => [v.commitment, v])); + const cells = commitmentIds + .map((cid) => { + const v = byCommitment.get(cid); + return v ? chip(v) : `—`; + }) + .join(''); + return `${esc(row.plan)}${cells}`; + }) + .join(''); + return ` + ${head} + ${body} +
`; +} diff --git a/src/interval.ts b/src/interval.ts new file mode 100644 index 0000000..53a32b5 --- /dev/null +++ b/src/interval.ts @@ -0,0 +1,88 @@ +/** + * SPEC-07 — interval arithmetic (research note `03-score-plan.md` §1). + * + * Metric propagation is classical interval arithmetic on pure closed bands + * (Moore, *Interval Analysis*, 1966), and nothing more. Each operator is the + * textbook closed-interval rule, is inclusion-monotone (a wider input yields a + * superset output) and sound (contains every real combination) — which is + * exactly the propagation-honesty property G6/oracle O-4 by construction. + * + * The honesty content of DEC-15 lives here: a Band is a pure interval with NO + * midpoint, so this module offers no mean/mid/most-likely operation. A scorer + * that collapsed [4,6] to 5 to "simplify" would pass no oracle. Units are + * checked on every combining operator; a non-finite result (a severed route's + * 1/0) is rejected here rather than leaked into a band or a canonical-JSON throw. + */ +import type { Band } from './generated/types.js'; + +const finite = (n: number, where: string): number => { + if (!Number.isFinite(n)) { + throw new Error(`interval: non-finite value from ${where} — a severed route or bad fixture, surface it (research note §2)`); + } + return n; +}; + +const sameUnit = (a: Band, b: Band, op: string): string => { + if (a.unit !== b.unit) { + throw new Error(`interval.${op}: unit mismatch '${a.unit}' vs '${b.unit}' — config/fixture defect, never coerced`); + } + return a.unit; +}; + +/** A degenerate point interval — a scalar as a band (a plan/threshold fact). */ +export function scalar(n: number, unit: string): Band { + return { lo: finite(n, 'scalar'), hi: finite(n, 'scalar'), unit }; +} + +/** [a.lo+b.lo, a.hi+b.hi]. */ +export function add(a: Band, b: Band): Band { + const unit = sameUnit(a, b, 'add'); + return { lo: finite(a.lo + b.lo, 'add'), hi: finite(a.hi + b.hi, 'add'), unit }; +} + +/** [a.lo-b.hi, a.hi-b.lo] — the worst case subtracts the largest of b. */ +export function sub(a: Band, b: Band): Band { + const unit = sameUnit(a, b, 'sub'); + return { lo: finite(a.lo - b.hi, 'sub'), hi: finite(a.hi - b.lo, 'sub'), unit }; +} + +/** Multiply by a dimensionless scalar; unit unchanged. Negative k flips the ends. */ +export function scaleBy(a: Band, k: number): Band { + finite(k, 'scaleBy'); + const p = a.lo * k; + const q = a.hi * k; + return { lo: finite(Math.min(p, q), 'scaleBy'), hi: finite(Math.max(p, q), 'scaleBy'), unit: a.unit }; +} + +/** Elementwise interval max: [max(lo), max(hi)]. */ +export function max(a: Band, b: Band): Band { + const unit = sameUnit(a, b, 'max'); + return { lo: Math.max(a.lo, b.lo), hi: Math.max(a.hi, b.hi), unit }; +} + +/** Elementwise interval min: [min(lo), min(hi)]. */ +export function min(a: Band, b: Band): Band { + const unit = sameUnit(a, b, 'min'); + return { lo: Math.min(a.lo, b.lo), hi: Math.min(a.hi, b.hi), unit }; +} + +/** Sum of a list; the empty sum is the zero band in `unit`. */ +export function sum(bands: Band[], unit: string): Band { + return bands.reduce((acc, b) => add(acc, b), scalar(0, unit)); +} + +/** Does `outer` contain `inner`? (O-4 containment.) Units must match. */ +export function contains(outer: Band, inner: Band): boolean { + sameUnit(outer, inner, 'contains'); + return outer.lo <= inner.lo && inner.hi <= outer.hi; +} + +/** Interval width hi-lo (≥ 0 for a well-formed band). */ +export function width(a: Band): number { + return a.hi - a.lo; +} + +/** Is a scalar point inside the band? (O-4 point-realisation soundness.) */ +export function member(a: Band, point: number): boolean { + return a.lo <= point && point <= a.hi; +} diff --git a/src/materialise.ts b/src/materialise.ts new file mode 100644 index 0000000..b562f21 --- /dev/null +++ b/src/materialise.ts @@ -0,0 +1,54 @@ +/** + * SPEC-07 — score-time channel materialisation (research note `03-score-plan.md` §2). + * + * The compile stores channels SPARSE — a default plus named, optionally + * time-boxed region overrides (research note `02-compile.md`). When the scorer + * needs `channel(x, y, t)` it resolves, for that cell and timestep, the innermost + * active `RegionOverride` (later `from_step` wins on overlap — note 02 §3's + * documented tie order), else the channel default. This is a pure function of + * `(sparse channels, config geometry)`: it produces no stored object, writes no + * edge, and participates in no content addressing. Nothing dense is ever + * persisted or hashed — the 60×60×56 world note 02 retired is never built. + */ +import type { Band, ChannelKind, CompiledWorld, RegionOverride, VignetteConfig } from './generated/types.js'; + +const area = (config: VignetteConfig, region: string): number => { + const g = config.regions.find((r) => r.name === region); + if (!g) throw new Error(`materialise: region '${region}' has no geometry in the config (fixture defect)`); + return (g.x1 - g.x0 + 1) * (g.y1 - g.y0 + 1); +}; + +const containsCell = (config: VignetteConfig, region: string, x: number, y: number): boolean => { + const g = config.regions.find((r) => r.name === region); + if (!g) throw new Error(`materialise: region '${region}' has no geometry in the config (fixture defect)`); + return x >= g.x0 && x <= g.x1 && y >= g.y0 && y <= g.y1; +}; + +const activeAt = (o: RegionOverride, t: number): boolean => + (o.from_step === undefined || t >= o.from_step) && (o.until_step === undefined || t <= o.until_step); + +/** + * The banded value of `kind` at cell (x,y) and timestep t. Overlap resolves + * innermost-wins with a documented tie order: later `from_step` first, then the + * geometrically smaller (innermost) region. Absent any active override, the + * channel's quiet default (itself banded — the source is assessed). + */ +export function channelAt( + world: CompiledWorld, + config: VignetteConfig, + kind: ChannelKind, + x: number, + y: number, + t: number, +): Band { + const channel = world.channels.find((c) => c.kind === kind); + if (!channel) throw new Error(`materialise: world has no '${kind}' channel`); + const candidates = (channel.regions ?? []).filter( + (o) => activeAt(o, t) && containsCell(config, o.region, x, y), + ); + if (candidates.length === 0) return channel.default; + candidates.sort( + (a, b) => (b.from_step ?? 0) - (a.from_step ?? 0) || area(config, a.region) - area(config, b.region), + ); + return candidates[0]!.value; +} diff --git a/src/metrics.ts b/src/metrics.ts new file mode 100644 index 0000000..87e1fa7 --- /dev/null +++ b/src/metrics.ts @@ -0,0 +1,138 @@ +/** + * SPEC-07 — the metric registry (research note `03-score-plan.md` §2). + * + * A metric turns a plan + compiled world into a banded quantity, by evaluating + * the plan's STATED routes (DEC-20) over the sparse world by `(cell, time)` + * resolution — never by searching, never by building a time-expanded graph. The + * families are exactly what the Meridian commitments C1–C6 need: + * + * - reach-step (strait_open_step / port_open_step / extraction_step) + * = start + Σ leg durations, a leg's duration = nominal steps ÷ its banded + * mobility factor. A zero-mobility leg (R3's demolished causeway) severs + * the route — surfaced as a `violated` reach, never an Infinity. + * - exposure (threat_exposure) + * = Σ over the element's legs of channel band × dwell, scoped to the element. + * - fires (civil_harm_exposure) + * = count of the fires element's legs inside the scoped district (bands stay + * bands, but a route decision is a count). + * - state (causeway_intact) + * = a banded read of a channel/region at the horizon. + * + * Every propagation is interval arithmetic on pure bands (`src/interval.ts`); no + * mean/mid/most-likely operation appears (DEC-15). The abstract oracle metric + * (O-1) is pure `src/interval.ts` and lives in the oracle test, not here. + */ +import type { Band, ChannelKind, CompiledWorld, Plan, VignetteConfig } from './generated/types.js'; +import { channelAt } from './materialise.js'; +import * as I from './interval.js'; + +/** A metric is either a band or a severed route (an unreachable objective). */ +export type MetricResult = { band: Band } | { severed: true }; + +export function isSevered(r: MetricResult): r is { severed: true } { + return (r as { severed?: true }).severed === true; +} + +interface MetricSpec { + kind: 'reach' | 'exposure' | 'fires' | 'state'; + /** Fixed responsible element; when absent the commitment `scope` names it. */ + element?: string; + channel?: ChannelKind; + /** Fixed region; when absent the commitment `scope` names it. */ + region?: string; +} + +const METRICS: Record = { + strait_open_step: { kind: 'reach', element: 'FE-BROOM' }, + port_open_step: { kind: 'reach', element: 'FE-PACKHORSE' }, + extraction_step: { kind: 'reach', element: 'FE-KINGFISHER' }, + threat_exposure: { kind: 'exposure', channel: 'threat' }, // element from scope (FE-ANVIL) + civil_harm_exposure: { kind: 'fires', element: 'FE-FALCON' }, // region from scope (port_district) + causeway_intact: { kind: 'state', channel: 'mobility', region: 'causeway' }, +}; + +interface Commitmentish { + metric: string; + unit: string; + scope?: string; +} + +/** Evaluate a commitment's metric over a plan and world. Unit = the commitment's. */ +export function evaluateMetric( + commitment: Commitmentish, + plan: Plan, + world: CompiledWorld, + config: VignetteConfig, +): MetricResult { + const spec = METRICS[commitment.metric]; + if (!spec) { + throw new Error(`metrics: no evaluator for metric '${commitment.metric}' (fixture/spec defect)`); + } + const unit = commitment.unit; + switch (spec.kind) { + case 'reach': + return reach(plan, world, config, spec.element!, unit); + case 'exposure': + return { band: exposure(plan, world, config, commitment.scope ?? spec.element!, spec.channel!, unit) }; + case 'fires': + return { band: fires(plan, spec.element!, commitment.scope ?? spec.region!, config, unit) }; + case 'state': + return { band: state(world, config, spec.channel!, spec.region!, unit) }; + } +} + +/** start + Σ leg durations; a zero-mobility leg severs the route. */ +function reach(plan: Plan, world: CompiledWorld, config: VignetteConfig, element: string, unit: string): MetricResult { + const ep = plan.elements.find((e) => e.force_element === element); + const legs = ep?.route ?? []; + if (legs.length === 0) { + throw new Error(`metrics: reach metric needs a route for ${element} (fixture defect)`); + } + const durations: Band[] = []; + for (const leg of legs) { + const nominal = leg.exit_step - leg.enter_step; // plan's intended steps (fact-of-intent) + const factor = channelAt(world, config, 'mobility', leg.x, leg.y, leg.enter_step); + if (factor.lo <= 0) return { severed: true }; // 1/0 — the route is cut + // duration = nominal ÷ factor; factor [flo,fhi] (flo>0) ⇒ inverse [1/fhi, 1/flo] + durations.push({ lo: nominal / factor.hi, hi: nominal / factor.lo, unit }); + } + const start = I.scalar(legs[0]!.enter_step, unit); + return { band: I.add(start, I.sum(durations, unit)) }; +} + +/** Σ over the element's legs of channel band × dwell-hours. */ +function exposure(plan: Plan, world: CompiledWorld, config: VignetteConfig, element: string, channel: ChannelKind, unit: string): Band { + const ep = plan.elements.find((e) => e.force_element === element); + const legs = ep?.route ?? []; + const dwellHours = config.grid.timestep_hours; + let acc: Band = { lo: 0, hi: 0, unit }; + for (const leg of legs) { + const dwell = (leg.exit_step - leg.enter_step) * dwellHours; + const val = channelAt(world, config, channel, leg.x, leg.y, leg.enter_step); + acc = { lo: acc.lo + val.lo * dwell, hi: acc.hi + val.hi * dwell, unit }; + } + return acc; +} + +/** Count of the fires element's legs inside the scoped region (a route decision). */ +function fires(plan: Plan, element: string, region: string, config: VignetteConfig, unit: string): Band { + const ep = plan.elements.find((e) => e.force_element === element); + const legs = ep?.route ?? []; + const g = config.regions.find((r) => r.name === region); + if (!g) throw new Error(`metrics: fires region '${region}' has no geometry (fixture defect)`); + let count = 0; + for (const leg of legs) { + if (leg.x >= g.x0 && leg.x <= g.x1 && leg.y >= g.y0 && leg.y <= g.y1) count += 1; + } + return { lo: count, hi: count, unit }; +} + +/** Banded boolean read of a channel/region at the horizon (intact ⟺ value > 0). */ +function state(world: CompiledWorld, config: VignetteConfig, channel: ChannelKind, region: string, unit: string): Band { + const g = config.regions.find((r) => r.name === region); + if (!g) throw new Error(`metrics: state region '${region}' has no geometry (fixture defect)`); + const cx = Math.floor((g.x0 + g.x1) / 2); + const cy = Math.floor((g.y0 + g.y1) / 2); + const val = channelAt(world, config, channel, cx, cy, config.grid.horizon_steps); + return { lo: val.lo > 0 ? 1 : 0, hi: val.hi > 0 ? 1 : 0, unit }; +} diff --git a/src/score.ts b/src/score.ts new file mode 100644 index 0000000..603dd1a --- /dev/null +++ b/src/score.ts @@ -0,0 +1,226 @@ +/** + * SPEC-07 — the scorer (seam contract §5; DEC-10, the unit everything orbits). + * + * plan × world × scenario → CommitmentVerdict[] + PlanScore[]. It propagates + * banded metrics by interval arithmetic (`src/metrics.ts` over `src/interval.ts` + * and the score-time materialiser), maps each commitment to a four-stop verdict + * against a SIGNED MARGIN BAND (research note `03-score-plan.md` §3), and writes + * one `scored_from` edge per verdict/score so every verdict opens a complete + * backward trace to named knowledge (G3). + * + * The verdict mapping is signs-only — no interior cut, no invented ε — which is + * the unique four-stop rule satisfying oracle O-3 (the verdict changes only at + * band edges). Verdicts cross the seam as the four-stop scale only; `margin` is + * a Band shown on demand (G2). Scoring is deterministic over its inputs (the + * stamp is a hash over inputs, never over materialised cells — G1), scenario- + * blind (the world is already excursioned — DEC-10), and honestly real even in + * the mock (DEC-4). `knowledge_overrides` re-scores for the call only, storing + * nothing; a scenario/stamp disagreement refuses `stamp_mismatch` (comparability + * guard). A plan that fails a `must` is a SUCCESS carrying a `violated` verdict — + * a failing plan is honest output, not a refusal. + */ +import type { + Band, + Comparator, + Commitment, + CommitmentVerdict, + CompiledWorld, + KnowledgeObject, + Plan, + PlanScore, + RegionOverride, + TraceEdge, + VerdictBand, + VignetteConfig, +} from './generated/types.js'; +import { ObjectStore, type Ref } from './store.js'; +import { TraceStore } from './trace.js'; +import { contentHash } from './canonical.js'; +import { validateInstance } from './validate.js'; +import { evaluateMetric, isSevered } from './metrics.js'; +import type { KnowledgeOverride, Refusal, ScoreRequest, ScoreResult } from './seam.js'; + +export class ScoreService { + #store: ObjectStore; + #trace: TraceStore; + #config: VignetteConfig; + #commitmentSet: Commitment[]; + #writtenBy: string; + + constructor(opts: { + store: ObjectStore; + trace: TraceStore; + config: VignetteConfig; + /** The commander's commitments (C1–C6) — the intent the plan is scored against. */ + commitments: Commitment[]; + writtenBy?: string; + }) { + this.#store = opts.store; + this.#trace = opts.trace; + this.#config = opts.config; + // Canonically ordered by logical_id so the returned verdict list is + // order-independent of how the caller presents the set (determinism, G1). + this.#commitmentSet = [...opts.commitments].sort((a, b) => cmp(a.logical_id, b.logical_id)); + this.#writtenBy = opts.writtenBy ?? 'score-service'; + } + + /** POST /score — pure over its stamped inputs; refuses first-class, persists nothing on refusal. */ + async score(req: ScoreRequest): Promise { + // 1. Resolve plan and world. + const plan = this.#resolve(req.plan); + if (!plan) return refusal('unknown_ref', [req.plan], `${req.plan.logical_id}: no such plan to score.`); + const world = this.#resolve(req.world); + if (!world) return refusal('unknown_ref', [req.world], `${req.world.logical_id}: no such world to score against.`); + + // 2. Comparability guard: a world compiled for a specific COA is incomparable + // under a different scenario label (the scenario is folded into the world's + // stamp — SPEC-06). A base world (no excursion) is scenario-agnostic. + if (world.scenario !== undefined && world.scenario !== req.scenario) { + return refusal('stamp_mismatch', [req.world], + `world ${req.world.logical_id} was compiled for ${world.scenario}, not ${req.scenario} — incomparable stamps (comparability guard).`); + } + + // 3. Apply knowledge_overrides as a TRANSIENT channel layer over a cloned world + // — for this call only; nothing is stored, no edge written (seam §5). + const scored = this.#applyOverrides(world, req.knowledge_overrides ?? []); + + // 4. Score each commitment (the commander's set), canonically ordered. + const commitments = this.#commitmentSet; + const verdicts: CommitmentVerdict[] = []; + const scores: PlanScore[] = []; + for (const c of commitments) { + const result = evaluateMetric(c, plan, scored, this.#config); + let verdict: VerdictBand; + let margin: Band | undefined; + if (isSevered(result)) { + verdict = 'violated'; // the objective is unreachable — an honest worst verdict + } else { + margin = marginBand(c.comparator, c.threshold, result.band); + verdict = verdictFor(margin); + scores.push({ + logical_id: `PS-${plan.logical_id}-${c.logical_id}-${req.scenario}`, + version: 1, + plan: plan.logical_id, + scenario: req.scenario, + world_stamp: world.stamp, + criterion: c.metric, + score: result.band, + engine_version: req.engine_version, + }); + } + const v: CommitmentVerdict = { + logical_id: `V-${plan.logical_id}-${c.logical_id}-${req.scenario}`, + version: 1, + plan: plan.logical_id, + commitment: c.logical_id, + scenario: req.scenario, + world_stamp: world.stamp, + verdict, + engine_version: req.engine_version, + }; + if (margin) v.margin = margin; + verdicts.push(v); + } + + // 5. Deterministic stamp over inputs (never over materialised cells — G1). + const overridesForStamp = (req.knowledge_overrides ?? []) + .map((o) => ({ ref: o.ref.logical_id, answer: o.answer })) + .sort((a, b) => cmp(a.ref, b.ref)); + const stamp = await contentHash({ + plan: { logical_id: plan.logical_id, content_hash: req.plan.content_hash }, + world: world.stamp, + scenario: req.scenario, + overrides: overridesForStamp, + engine_version: req.engine_version, + }); + + // 6. Store verdicts/scores and write one scored_from edge each → the world (G3). + const worldHash = req.world.content_hash || (await contentHash(world as unknown as Record)); + for (const v of verdicts) { + const errors = validateInstance('CommitmentVerdict', v); + if (errors.length > 0) throw new Error(`score produced an invalid CommitmentVerdict: ${errors.join('; ')}`); + const ref = await this.#store.put(v as unknown as Record); + this.#trace.add(this.#edge(ref.content_hash, worldHash, stamp)); + } + for (const s of scores) { + const errors = validateInstance('PlanScore', s); + if (errors.length > 0) throw new Error(`score produced an invalid PlanScore: ${errors.join('; ')}`); + const ref = await this.#store.put(s as unknown as Record); + this.#trace.add(this.#edge(ref.content_hash, worldHash, stamp)); + } + + // 7. No delta (contract §9): scoring is a read-of-world, not a knowledge write. + return { verdicts, scores, stamp }; + } + + #resolve(ref: Ref): T | undefined { + if (ref.content_hash) return this.#store.get(ref.content_hash) as T | undefined; + const versions = this.#store.versions(ref.logical_id); + const hash = versions.length > 0 ? versions[versions.length - 1]!.content_hash : undefined; + return hash ? (this.#store.get(hash) as T) : undefined; + } + + /** + * Clone the world and SUBSTITUTE override answers into their channel region — + * for this call only; nothing is stored, no edge written (seam §5). An override + * replaces any existing overrides on that region (it is the substituted answer + * the compile would have laid down), so it is unambiguously what the scorer reads. + */ + #applyOverrides(world: CompiledWorld, overrides: KnowledgeOverride[]): CompiledWorld { + if (overrides.length === 0) return world; + const clone = structuredClone(world); + for (const o of overrides) { + const ko = this.#resolve(o.ref); + if (!ko) continue; // an override for a ref we cannot see is a no-op, not a store write + const entry = this.#config.subject_map.find((e) => e.subject === ko.subject); + if (!entry) continue; + const channel = clone.channels.find((c) => c.kind === entry.channel); + if (!channel) continue; + const kept = (channel.regions ?? []).filter((r) => r.region !== entry.region); + const layer: RegionOverride = { region: entry.region, value: o.answer, source: o.ref.logical_id }; + channel.regions = [...kept, layer]; + } + return clone; + } + + #edge(from: string, to: string, stamp: string): TraceEdge { + return { from_hash: from, to_hash: to, edge_type: 'scored_from', stamp, written_by: this.#writtenBy }; + } +} + +/** + * Reduce a commitment to a signed margin band `M`, oriented so satisfied ⟺ + * margin ≥ 0 (research note §3). `M.lo` is always the worst-case margin. + */ +export function marginBand(comparator: Comparator, threshold: number, value: Band): Band { + switch (comparator) { + case 'at_most': + case 'by_step': + return { lo: threshold - value.hi, hi: threshold - value.lo, unit: value.unit }; + case 'at_least': + return { lo: value.lo - threshold, hi: value.hi - threshold, unit: value.unit }; + case 'never': + return { lo: -value.hi, hi: -value.lo, unit: value.unit }; + } +} + +/** + * The four-stop verdict, by the SIGNS of the margin-band endpoints only — the + * unique O-3-satisfying rule (research note §3). No interior cut exists, because + * the band gives no honest interior point to cut at (DEC-15: no midpoint). + */ +export function verdictFor(margin: Band): VerdictBand { + if (margin.hi < 0) return 'violated'; // best realisation already fails + if (margin.lo > 0) return 'robust'; // worst realisation satisfies with room + if (margin.lo === 0) return 'marginal'; // worst realisation exactly on the line + return 'tight'; // m_lo < 0 ≤ m_hi — the band straddles the line +} + +const cmp = (a: string, b: string): number => (a < b ? -1 : a > b ? 1 : 0); + +const refusal = (reason: Refusal['reason'], offending: Ref[], explanation: string): Refusal => ({ + refused: true, + reason, + offending, + explanation, +}); diff --git a/src/seam.ts b/src/seam.ts index 64190d5..1013686 100644 --- a/src/seam.ts +++ b/src/seam.ts @@ -8,6 +8,7 @@ * rather than in the generated schema. */ import type { Ref } from './store.js'; +import type { Band, CommitmentVerdict, PlanScore } from './generated/types.js'; export type RefusalReason = | 'contested_knowledge' @@ -52,3 +53,29 @@ export interface Delta { export function isRefusal(r: unknown): r is Refusal { return typeof r === 'object' && r !== null && (r as Refusal).refused === true; } + +/** + * SPEC-07 — scorer movement types (seam §5). The perturbation hook substitutes + * an answer for this call only (nothing stored); `scenario` is recorded on the + * verdicts for provenance — the world is already excursioned (DEC-10). + */ +export interface KnowledgeOverride { + ref: Ref; + answer: Band; +} + +export interface ScoreRequest { + plan: Ref; + world: Ref; + scenario: string; + knowledge_overrides?: KnowledgeOverride[]; + engine_version: string; +} + +export interface ScoreSuccess { + verdicts: CommitmentVerdict[]; + scores: PlanScore[]; + stamp: string; +} + +export type ScoreResult = ScoreSuccess | Refusal; diff --git a/tests/fixtures.test.ts b/tests/fixtures.test.ts index 9319fbf..7c65175 100644 --- a/tests/fixtures.test.ts +++ b/tests/fixtures.test.ts @@ -1,7 +1,7 @@ import { readFileSync } from 'node:fs'; import { describe, expect, it } from 'vitest'; import { validateInstance } from '../src/validate.js'; -import type { Commitment, KnowledgeObject, ScenarioCOA, VignetteConfig } from '../src/generated/types.js'; +import type { Commitment, KnowledgeObject, Plan, ScenarioCOA, VignetteConfig } from '../src/generated/types.js'; const load = (name: string): T[] => JSON.parse(readFileSync(new URL(`../fixtures/${name}.json`, import.meta.url), 'utf8')) as T[]; @@ -10,6 +10,7 @@ const knowledge = load('knowledge'); const commitments = load('commitments'); const coas = load('coas'); const forceElements = load>('force-elements'); +const plans = load('plans'); const vignetteConfig = JSON.parse( readFileSync(new URL('../fixtures/vignette-config.json', import.meta.url), 'utf8'), ) as VignetteConfig; @@ -27,6 +28,46 @@ describe('Meridian fixtures validate against generated types (SPEC-04; Stage-0 e it.each(forceElements.map((f) => [f.logical_id, f] as const))('%s validates', (_id, f) => { expect(validateInstance('ForceElement', f)).toEqual([]); }); + it.each(plans.map((p) => [p.logical_id, p] as const))('%s validates (SPEC-07 canned handful)', (_id, p) => { + expect(validateInstance('Plan', p)).toEqual([]); + }); +}); + +describe('SPEC-07 canned handful (delivery plan §3 fallback)', () => { + const feIds = new Set(forceElements.map((f) => f.logical_id as string)); + const regions = JSON.parse( + readFileSync(new URL('../fixtures/vignette-config.json', import.meta.url), 'utf8'), + ).regions as { name: string; x0: number; y0: number; x1: number; y1: number }[]; + const horizon = vignetteConfig.grid.horizon_steps; + + it('is P1/P2 (thesis-C pair), every leg in-grid, every element a known FE-*', () => { + expect(plans.map((p) => p.logical_id)).toEqual(['P1', 'P2']); + for (const p of plans) { + for (const ep of p.elements) { + expect(feIds.has(ep.force_element), `${p.logical_id} ${ep.force_element}`).toBe(true); + for (const leg of ep.route ?? []) { + expect(leg.x).toBeGreaterThanOrEqual(0); + expect(leg.x).toBeLessThan(vignetteConfig.grid.cols); + expect(leg.y).toBeLessThan(vignetteConfig.grid.rows); + expect(leg.enter_step).toBeLessThanOrEqual(leg.exit_step); + expect(leg.exit_step).toBeLessThanOrEqual(horizon); + } + } + } + }); + + it('routes reach the regions their metrics read (BROOM/PACKHORSE through the strait)', () => { + const inRegion = (name: string, x: number, y: number): boolean => { + const g = regions.find((r) => r.name === name)!; + return x >= g.x0 && x <= g.x1 && y >= g.y0 && y <= g.y1; + }; + for (const p of plans) { + const broom = p.elements.find((e) => e.force_element === 'FE-BROOM')!; + expect(broom.route!.some((l) => inRegion('halcyon_strait', l.x, l.y)), `${p.logical_id} BROOM sweeps the strait`).toBe(true); + const packhorse = p.elements.find((e) => e.force_element === 'FE-PACKHORSE')!; + expect(packhorse.route!.some((l) => inRegion('port_district', l.x, l.y)), `${p.logical_id} PACKHORSE reaches port`).toBe(true); + } + }); }); describe('fixture set is complete against vignette identifiers (§5, §6, §3)', () => { diff --git a/tests/interval.test.ts b/tests/interval.test.ts new file mode 100644 index 0000000..a6af162 --- /dev/null +++ b/tests/interval.test.ts @@ -0,0 +1,51 @@ +import { describe, expect, it } from 'vitest'; +import type { Band } from '../src/generated/types.js'; +import * as I from '../src/interval.js'; + +const B = (lo: number, hi: number, unit = 'step'): Band => ({ lo, hi, unit }); + +describe('interval arithmetic (research note 03 §1)', () => { + it('adds closed intervals endpoint-wise', () => { + expect(I.add(B(4, 6), B(3, 5))).toEqual(B(7, 11)); + }); + + it('subtracts with the worst-case orientation [a.lo-b.hi, a.hi-b.lo]', () => { + expect(I.sub(B(9, 13), B(3, 5))).toEqual(B(4, 10)); + }); + + it('scales by a dimensionless scalar, flipping ends on a negative factor', () => { + expect(I.scaleBy(B(2, 6), 3)).toEqual(B(6, 18)); + expect(I.scaleBy(B(2, 6), -1)).toEqual(B(-6, -2)); + }); + + it('takes elementwise max/min', () => { + expect(I.max(B(2, 6), B(3, 4))).toEqual(B(3, 6)); + expect(I.min(B(2, 6), B(3, 4))).toEqual(B(2, 4)); + }); + + it('reproduces oracle O-1 exactly — start 2 + A[4,6] + B[3,5] = [9,13]', () => { + const strait_open_step = I.add(I.scalar(2, 'step'), I.add(B(4, 6), B(3, 5))); + expect(strait_open_step).toEqual(B(9, 13)); + }); + + it('reproduces oracle O-4 widening — A[4,6]→[3,7] gives [8,14] ⊇ [9,13]', () => { + const widened = I.add(I.scalar(2, 'step'), I.add(B(3, 7), B(3, 5))); + expect(widened).toEqual(B(8, 14)); + expect(I.contains(widened, B(9, 13))).toBe(true); + }); + + it('rejects unit mismatch rather than coercing', () => { + expect(() => I.add(B(1, 2, 'step'), B(1, 2, 'band-hours'))).toThrow(/unit mismatch/); + }); + + it('rejects a non-finite result (a severed route) rather than leaking Infinity', () => { + expect(() => I.scalar(1 / 0, 'step')).toThrow(/non-finite/); + }); + + it('has no mean/mid/most-likely operation (DEC-15)', () => { + const ops = Object.keys(I); + for (const forbidden of ['mean', 'mid', 'midpoint', 'average', 'mostLikely', 'centre', 'center']) { + expect(ops).not.toContain(forbidden); + } + }); +}); diff --git a/tests/oracle.test.ts b/tests/oracle.test.ts new file mode 100644 index 0000000..d58e513 --- /dev/null +++ b/tests/oracle.test.ts @@ -0,0 +1,122 @@ +/** + * SPEC-07 — the oracle cases (vignette §9), the independent correctness leg. + * Expected values are HAND-COMPUTED constants, typed from the vignette, and are + * NEVER regenerated from the scorer (an oracle regenerated from the code under + * test is not an oracle — vignette §9 rule). + */ +import fc from 'fast-check'; +import { describe, expect, it } from 'vitest'; +import type { Band, Comparator, VerdictBand } from '../src/generated/types.js'; +import * as I from '../src/interval.js'; +import { marginBand, verdictFor } from '../src/score.js'; + +const B = (lo: number, hi: number, unit = 'step'): Band => ({ lo, hi, unit }); + +// The shared oracle metric: strait_open_step = start + duration(A) + duration(B). +const straitOpenStep = (start: number, a: Band, b: Band): Band => + I.add(I.scalar(start, 'step'), I.add(a, b)); + +describe('Oracle O-1 — interval sum', () => { + it('start 2 · A[4,6] · B[3,5] ⇒ [9,13] exactly', () => { + expect(straitOpenStep(2, B(4, 6), B(3, 5))).toEqual(B(9, 13)); + }); +}); + +describe('Oracle O-2 — verdict at a clear margin', () => { + it('[9,13] vs at_most 28 ⇒ robust, margin [15,19]', () => { + const value = B(9, 13); + const margin = marginBand('at_most', 28, value); + expect(margin).toEqual(B(15, 19)); + expect(verdictFor(margin)).toBe('robust'); + }); +}); + +describe('Oracle O-3 — band-straddling threshold and the four-stop scale', () => { + it('[9,13] vs at_most 12 ⇒ neither robust nor violated (tight)', () => { + const v = verdictFor(marginBand('at_most', 12, B(9, 13))); + expect(v).toBe('tight'); + expect(v).not.toBe('robust'); + expect(v).not.toBe('violated'); + }); + + it('sweeping at_most T over 8→14, the verdict changes only at the band edges 9 and 13', () => { + const value = B(9, 13); // band edges are lo=9 and hi=13 + const verdict = (t: number) => verdictFor(marginBand('at_most', t, value)); + const seq = [8, 9, 10, 11, 12, 13, 14].map(verdict); + expect(seq).toEqual([ + 'violated', // T=8 — below the band: best case fails + 'tight', // T=9 — lower band edge (m_hi crosses 0) + 'tight', // T=10 ┐ + 'tight', // T=11 ├ strictly inside the band: constant + 'tight', // T=12 ┘ + 'marginal', // T=13 — upper band edge (m_lo crosses 0): the boundary verdict + 'robust', // T=14 — above the band: worst case satisfies with room + ]); + // The continuous breakpoints are the band edges 9 and 13, and NOTHING changes + // strictly inside the band — the honesty O-3 pins (marginal is the boundary + // value AT the upper edge, not an interior transition). + expect(verdict(10)).toBe(verdict(11)); + expect(verdict(11)).toBe(verdict(12)); + for (let t = 9; t <= 13; t++) { + const v = verdict(t); + expect(v).not.toBe('robust'); // the band spans the threshold: never "definitely fine" + expect(v).not.toBe('violated'); // and never "definitely failed" + } + expect(verdict(8)).toBe('violated'); // decisive only strictly below the band + expect(verdict(14)).toBe('robust'); // decisive only strictly above it + }); +}); + +describe('Oracle O-4 — containment under widening (candidate G6)', () => { + it('re-running O-1 with A widened [4,6]→[3,7] ⇒ [8,14] ⊇ [9,13]', () => { + const wide = straitOpenStep(2, B(3, 7), B(3, 5)); + expect(wide).toEqual(B(8, 14)); + expect(I.contains(wide, straitOpenStep(2, B(4, 6), B(3, 5)))).toBe(true); + }); + + const anyBand = (unit: string) => + fc + .tuple(fc.integer({ min: -50, max: 50 }), fc.integer({ min: 0, max: 40 })) + .map(([lo, span]) => B(lo, lo + span, unit)); + + it('property: widening any input never narrows the output, and every point-realisation is contained', () => { + fc.assert( + fc.property(anyBand('step'), anyBand('step'), fc.integer({ min: 0, max: 20 }), fc.integer({ min: 0, max: 20 }), (a, b, growLo, growHi) => { + const start = 2; + const base = straitOpenStep(start, a, b); + const aWide = B(a.lo - growLo, a.hi + growHi); + const wide = straitOpenStep(start, aWide, b); + // (1) inclusion monotonicity + expect(I.contains(wide, base)).toBe(true); + // (2) soundness: any point in the inputs scores inside the output band + for (const pa of [a.lo, a.hi, Math.floor((a.lo + a.hi) / 2)]) { + for (const pb of [b.lo, b.hi]) { + expect(I.member(base, start + pa + pb)).toBe(true); + } + } + }), + { numRuns: 300 }, + ); + }); + + it('property: widening moves a verdict TOWARD uncertainty (tight), never away from it', () => { + // Honest direction: more input uncertainty never buys a more DECISIVE verdict. + // `tight` is maximal uncertainty; robust and violated are equally decisive (of + // opposite outcomes), marginal is one step in. Widening moves m_lo down and m_hi + // up, so the verdict can only slide toward tight from either end — decisiveness + // (distance from tight) is non-increasing, and robust never flips to violated. + const decisiveness: Record = { tight: 0, marginal: 1, robust: 2, violated: 2 }; + const comparators: Comparator[] = ['at_most', 'at_least', 'by_step', 'never']; + fc.assert( + fc.property(anyBand('step'), fc.integer({ min: -60, max: 60 }), fc.constantFrom(...comparators), fc.integer({ min: 0, max: 15 }), fc.integer({ min: 0, max: 15 }), (v, threshold, comparator, growLo, growHi) => { + const base = verdictFor(marginBand(comparator, threshold, v)); + const wide = verdictFor(marginBand(comparator, threshold, B(v.lo - growLo, v.hi + growHi))); + expect(decisiveness[wide]).toBeLessThanOrEqual(decisiveness[base]); + // widening can never flip a satisfied verdict into a violated one, or vice versa + if (base === 'robust') expect(wide).not.toBe('violated'); + if (base === 'violated') expect(wide).not.toBe('robust'); + }), + { numRuns: 300 }, + ); + }); +}); diff --git a/tests/s2Matrix.test.ts b/tests/s2Matrix.test.ts new file mode 100644 index 0000000..9943b8a --- /dev/null +++ b/tests/s2Matrix.test.ts @@ -0,0 +1,56 @@ +import { describe, expect, it } from 'vitest'; +import type { CommitmentVerdict } from '../src/generated/types.js'; +import { s2Matrix, type S2Cell } from '../src/components/s2Matrix.js'; + +const V = (commitment: string, verdict: CommitmentVerdict['verdict'], margin?: { lo: number; hi: number; unit: string }): CommitmentVerdict => ({ + logical_id: `V-P1-${commitment}-BASE`, + version: 1, + plan: 'P1', + commitment, + scenario: 'BASE', + world_stamp: 'abc', + verdict, + engine_version: '0.1.0', + ...(margin ? { margin } : {}), +}); + +const rows: S2Cell[] = [ + { + plan: 'P1', + verdicts: [ + V('C1', 'robust', { lo: 8, hi: 8, unit: 'step' }), + V('C2', 'robust', { lo: 6, hi: 6, unit: 'step' }), + V('C3', 'marginal', { lo: 0, hi: 0, unit: 'district cells fired upon' }), + V('C4', 'tight', { lo: -24, hi: 0, unit: 'band-hours' }), + V('C5', 'marginal'), + V('C6', 'robust', { lo: 8, hi: 8, unit: 'step' }), + ], + }, +]; + +describe('s2Matrix — the honest matrix (ui-design §2)', () => { + const html = s2Matrix(['C1', 'C2', 'C3', 'C4', 'C5', 'C6'], rows); + + it('renders a four-stop chip per cell', () => { + for (const stop of ['robust', 'marginal', 'tight']) { + expect(html).toContain(`>${stop}`); + } + expect((html.match(/assay-verdict-chip/g) ?? []).length).toBe(6); + }); + + it('shows the margin band only on hover (a title), never as a decimal in the cell face', () => { + expect(html).toContain('title="margin -24–0 band-hours"'); // C4 margin on hover + // the cell FACE carries only the four-stop word — no numeric margin leaks into it + const faces = [...html.matchAll(/assay-verdict-chip[^>]*>([^<]*) m[1]); + for (const face of faces) { + expect(face).not.toMatch(/\d/); // no digit in any verdict cell face (G2, no decimals) + expect(['robust', 'marginal', 'tight', 'violated']).toContain(face); + } + }); + + it('places columns in the given commitment order', () => { + const order = ['C1', 'C2', 'C3', 'C4', 'C5', 'C6'].map((c) => html.indexOf(`>${c}`)); + expect(order).toEqual([...order].sort((a, b) => a - b)); + expect(order.every((i) => i >= 0)).toBe(true); + }); +}); diff --git a/tests/score.test.ts b/tests/score.test.ts new file mode 100644 index 0000000..0981e15 --- /dev/null +++ b/tests/score.test.ts @@ -0,0 +1,216 @@ +import { readFileSync } from 'node:fs'; +import { beforeEach, describe, expect, it } from 'vitest'; +import type { + Commitment, + CommitmentVerdict, + KnowledgeObject, + Plan, + ScenarioCOA, + VignetteConfig, +} from '../src/generated/types.js'; +import { KnowledgeService } from '../src/knowledge.js'; +import { CompileService } from '../src/compile.js'; +import { ScoreService } from '../src/score.js'; +import { isRefusal, type ScoreResult } from '../src/seam.js'; +import type { Ref } from '../src/store.js'; + +const load = (name: string): T[] => + JSON.parse(readFileSync(new URL(`../fixtures/${name}.json`, import.meta.url), 'utf8')) as T[]; + +const knowledge = load('knowledge'); +const coas = load('coas'); +const commitments = load('commitments'); +const plans = load('plans'); +const config = JSON.parse( + readFileSync(new URL('../fixtures/vignette-config.json', import.meta.url), 'utf8'), +) as VignetteConfig; + +const byId = new Map(knowledge.map((k) => [k.logical_id, k])); +const K = (id: string): KnowledgeObject => structuredClone(byId.get(id)!); +const answered = (id: string): KnowledgeObject => ({ ...K(id), status: 'answered' }); +const ref = (id: string): Ref => ({ logical_id: id, content_hash: '' }); +const BASE = ['K1', 'K2', 'K3', 'K4', 'K6', 'K7', 'K8', 'K9']; +const ENGINE = '0.1.0'; + +interface Rig { + svc: KnowledgeService; + scorer: ScoreService; + planRef: (id: string) => Ref; + worldRef: (scenario?: string) => Promise; +} + +async function setup(): Promise { + const svc = new KnowledgeService(); + for (const id of BASE) await svc.create(answered(id)); + await svc.create(K('K12a')); + await svc.create(K('K12b')); + svc.contest('K12a', 'K12b'); + svc.resolve('K12a', 'defector debrief corroborated; manifests predate the drawdown'); + for (const coa of coas) await svc.store.put(coa as unknown as Record); + for (const c of commitments) await svc.store.put(c as unknown as Record); + const planRefs = new Map(); + for (const p of plans) planRefs.set(p.logical_id, await svc.store.put(p as unknown as Record)); + + const compiler = new CompileService({ knowledge: svc }); + const worldRef = async (scenario?: string): Promise => { + const req = { knowledge: [...BASE, 'K12a'].map(ref), config, engine_version: ENGINE, ...(scenario ? { scenario } : {}) }; + const r = await compiler.compile(req); + if (isRefusal(r)) throw new Error(`compile refused: ${r.reason}`); + return r.world; + }; + + const scorer = new ScoreService({ store: svc.store, trace: svc.trace, config, commitments }); + return { svc, scorer, planRef: (id) => planRefs.get(id)!, worldRef }; +} + +const ok = (r: ScoreResult) => { + if (isRefusal(r)) throw new Error(`expected success, got refusal ${r.reason}`); + return r; +}; + +describe('US3 — the honest matrix over real Meridian output', () => { + let rig: Rig; + beforeEach(async () => { + rig = await setup(); + }); + + it('scores one four-stop verdict per commitment C1–C6, each carrying a margin band', async () => { + const world = await rig.worldRef(); + const r = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + + expect(r.verdicts.map((v) => v.commitment)).toEqual(['C1', 'C2', 'C3', 'C4', 'C5', 'C6']); + const stops = ['robust', 'marginal', 'tight', 'violated']; + for (const v of r.verdicts) { + expect(stops).toContain(v.verdict); + expect(v.plan).toBe('P1'); + expect(v.world_stamp).toBeTruthy(); + // every non-severed verdict carries a margin Band (shown on demand, G2) + if (v.margin) expect(v.margin).toMatchObject({ lo: expect.any(Number), hi: expect.any(Number), unit: expect.any(String) }); + } + }); + + it('spreads verdicts across the four-stop scale (a real matrix, not all one colour)', async () => { + const world = await rig.worldRef(); + const r = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + const distinct = new Set(r.verdicts.map((v) => v.verdict)); + expect(distinct.size).toBeGreaterThanOrEqual(3); // robust, marginal, tight all present for P1 + }); + + it('writes a scored_from edge from each verdict to the world — a complete backward chain (G3)', async () => { + const world = await rig.worldRef(); + const before = rig.svc.trace.edges.length; + const r = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + + const scoredFrom = rig.svc.trace.edges.filter((e) => e.edge_type === 'scored_from' && e.to_hash === world.content_hash); + expect(scoredFrom.length).toBe(r.verdicts.length + r.scores.length); + expect(rig.svc.trace.edges.length).toBeGreaterThan(before); + // the world itself has incoming compiled_into edges from named knowledge, so a + // backward walk verdict → world → knowledge terminates at named owners (G3). + const compiledInto = rig.svc.trace.edges.filter((e) => e.edge_type === 'compiled_into' && e.to_hash === world.content_hash); + expect(compiledInto.length).toBeGreaterThan(0); + }); + + it('thesis C: P1 (strait-early) is healthier on C1/C2 under R1 than under R2', async () => { + const worldR1 = await rig.worldRef('R1'); + const worldR2 = await rig.worldRef('R2'); + const r1 = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world: worldR1, scenario: 'R1', engine_version: ENGINE })); + const r2 = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world: worldR2, scenario: 'R2', engine_version: ENGINE })); + + const stop = (v: CommitmentVerdict[], c: string) => v.find((x) => x.commitment === c)!.verdict; + // Under R2 the mined strait collapses BROOM's sweep and PACKHORSE's transit. + expect(stop(r2.verdicts, 'C2')).toBe('violated'); + expect(stop(r2.verdicts, 'C1')).toBe('violated'); + // Under R1 (threat concentrated in the port district, strait physically clear) they hold. + expect(['robust', 'marginal', 'tight']).toContain(stop(r1.verdicts, 'C2')); + expect(['robust', 'marginal', 'tight']).toContain(stop(r1.verdicts, 'C1')); + }); +}); + +describe('US4 — determinism', () => { + it('same plan + world + overrides ⇒ byte-identical stamp and identical verdicts', async () => { + const rig = await setup(); + const world = await rig.worldRef(); + const a = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + const b = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + expect(a.stamp).toBe(b.stamp); + expect(a.verdicts).toEqual(b.verdicts); + expect(a.scores).toEqual(b.scores); + }); + + it('commitment order does not affect the stamp or the (canonically ordered) verdicts', async () => { + const rig = await setup(); + const world = await rig.worldRef(); + const a = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + + const shuffled = new ScoreService({ + store: rig.svc.store, + trace: rig.svc.trace, + config, + commitments: [...commitments].reverse(), + }); + const b = ok(await shuffled.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + expect(b.verdicts.map((v) => v.commitment)).toEqual(['C1', 'C2', 'C3', 'C4', 'C5', 'C6']); + expect(b.stamp).toBe(a.stamp); + }); +}); + +describe('US5 — perturbation hook and comparability guard', () => { + it('knowledge_overrides re-scores without persisting the override (nothing forked)', async () => { + const rig = await setup(); + const world = await rig.worldRef(); + const worldsBefore = countWorlds(rig.svc); + + const plain = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world, scenario: 'BASE', engine_version: ENGINE })); + const storeAfterPlain = rig.svc.store.size; + + const rig2 = await setup(); + const world2 = await rig2.worldRef(); + const overridden = ok( + await rig2.scorer.score({ + plan: rig2.planRef('P1'), + world: world2, + scenario: 'BASE', + engine_version: ENGINE, + knowledge_overrides: [{ ref: ref('K6'), answer: { lo: 2, hi: 10, unit: 'sorties/day' } }], + }), + ); + // No new CompiledWorld and no new knowledge object: the override is not stored. + expect(countWorlds(rig2.svc)).toBe(worldsBefore); + // The same count of artefacts is written whether or not an override is present — + // only the returned verdicts/scores join the store, never the override itself. + expect(rig2.svc.store.size).toBe(storeAfterPlain); + // The override affects the scored answer: it may not make any verdict MORE decisive. + const c4Plain = plain.verdicts.find((v) => v.commitment === 'C4')!.verdict; + const c4Over = overridden.verdicts.find((v) => v.commitment === 'C4')!.verdict; + const decisive = { tight: 0, marginal: 1, robust: 2, violated: 2 } as const; + expect(decisive[c4Over]).toBeLessThanOrEqual(decisive[c4Plain]); + }); + + it('refuses stamp_mismatch when the world was compiled for a different COA (comparability guard)', async () => { + const rig = await setup(); + const worldR2 = await rig.worldRef('R2'); + const result = await rig.scorer.score({ plan: rig.planRef('P1'), world: worldR2, scenario: 'R1', engine_version: ENGINE }); + + expect(isRefusal(result)).toBe(true); + if (isRefusal(result)) { + expect(result.reason).toBe('stamp_mismatch'); + expect(result.offending.map((o) => o.logical_id)).toContain(worldR2.logical_id); + } + }); + + it('a plan that fails a must is a SUCCESS with a violated verdict, not a refusal', async () => { + const rig = await setup(); + const worldR2 = await rig.worldRef('R2'); + const r = ok(await rig.scorer.score({ plan: rig.planRef('P1'), world: worldR2, scenario: 'R2', engine_version: ENGINE })); + expect(r.verdicts.some((v) => v.verdict === 'violated')).toBe(true); // honest output, not a decline + }); +}); + +function countWorlds(svc: KnowledgeService): number { + // count distinct CompiledWorld logical ids present (W-BASE, W-R1, ...) + let n = 0; + for (const id of ['W-BASE', 'W-R1', 'W-R2', 'W-R3', 'W-R3m']) { + if (svc.store.versions(id).length > 0) n += 1; + } + return n; +}