US5 + US4, docs PR previews & Phase 8 polish (generation stays on Python 3.9)

.github/workflows/ci.yml

@@ -25,5 +25,21 @@ jobs:
       - name: Verify (lint + tests)
         run: make verify
 
+      - name: Drift gate (generated models + schema docs match the schema)
+        # Regenerate on the same Python 3.9 target and fail if the committed artifacts differ.
+        # xsdata is pinned (==25.7, the last 3.9-supporting line) so generation is byte-stable.
+        # See docs/decisions/0008.
+        run: |
+          make generate
+          make gen-schema-docs
+          paths="src/acoustic_dataset/models docs/reference/schema"
+          if [ -n "$(git status --porcelain -- $paths)" ]; then
+            echo "::error::Generated artifacts are stale. Run 'make generate' and 'make gen-schema-docs' and commit."
+            git --no-pager diff -- $paths
+            git status --porcelain -- $paths
+            exit 1
+          fi
+          echo "Generated models and schema docs are in sync with the schema."
+
       - name: Build HTML docs (strict)
         run: make docs

.github/workflows/deploy-docs.yml

@@ -9,7 +9,7 @@ on:
   # Allow manual runs from the Actions tab.
   workflow_dispatch:
 
-# Needed so `mkdocs gh-deploy` can push the built site to the gh-pages branch.
+# Needed so the deploy action can push the built site to the gh-pages branch.
 permissions:
   contents: write
 
@@ -23,9 +23,6 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-        with:
-          # Full history so gh-deploy can update the gh-pages branch cleanly.
-          fetch-depth: 0
 
       - name: Set up Python 3.9
         # Docs build on the latest 3.9.x patch; the exact-3.9.4 pin is a target-system
@@ -37,7 +34,17 @@ jobs:
       - name: Install docs dependencies
         run: python -m pip install -e ".[docs]"
 
-      - name: Build (strict) and publish to gh-pages
-        # gh-deploy builds with --strict, force-pushes to gh-pages, and writes .nojekyll
-        # so GitHub serves the MkDocs HTML as-is (no Jekyll processing).
-        run: mkdocs gh-deploy --force --no-history --strict
+      - name: Build (strict)
+        run: |
+          mkdocs build --strict   # output -> site/
+          touch site/.nojekyll    # serve the MkDocs HTML as-is (no Jekyll processing)
+
+      - name: Publish to gh-pages (preserving PR previews)
+        # NOT `mkdocs gh-deploy` — that force-replaces the whole branch and would wipe the
+        # `pr-preview/` directory the PR-preview workflow publishes into. clean-exclude keeps it.
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          folder: site
+          branch: gh-pages
+          clean: true
+          clean-exclude: pr-preview/

.github/workflows/pr-preview.yml

@@ -0,0 +1,45 @@
+name: Docs PR preview
+
+# Build this PR's docs and publish a live preview to gh-pages under pr-preview/pr-<N>/,
+# posting (and updating) a sticky comment with the link. The preview is torn down when the
+# PR closes. The main deploy (deploy-docs.yml) excludes pr-preview/ so it never wipes these.
+
+on:
+  pull_request:
+    types: [opened, reopened, synchronize, closed]
+
+# Same-repo PRs get a write-scoped token (preview deploys + comments). Fork PRs get a
+# read-only token and will skip the deploy — that's expected and safe.
+permissions:
+  contents: write
+  pull-requests: write
+
+concurrency:
+  group: pr-preview-${{ github.event.pull_request.number }}
+  cancel-in-progress: true
+
+jobs:
+  preview:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.9
+        if: github.event.action != 'closed'
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.9"
+
+      - name: Build the docs site (strict)
+        if: github.event.action != 'closed'
+        run: |
+          python -m pip install -e ".[docs]"
+          mkdocs build --strict   # output -> site/
+          touch site/.nojekyll
+
+      - name: Deploy / update / tear down the PR preview
+        uses: rossjrw/pr-preview-action@v1
+        with:
+          source-dir: site
+          preview-branch: gh-pages
+          umbrella-dir: pr-preview

Makefile

@@ -17,8 +17,9 @@ bootstrap: ## Install deps + generate models. Run by the devcontainer.
 	$(MAKE) generate
 	@echo "Bootstrap complete. Run 'make verify' to check the environment."
 
-verify: ## Run the full test suite + lint — the single 'is the environment good?' command.
+verify: ## Run lint + type-check + tests — the single 'is the environment good?' command.
 	ruff check src tests
+	mypy
 	pytest
 
 docs: ## Build the attractive HTML docs site (strict — fails on broken links).
@@ -30,8 +31,8 @@ docs-serve: ## Live-preview the HTML docs at http://localhost:8000
 generate: ## Regenerate typed models from schema/*.xsd via xsdata.
 	python -m acoustic_dataset.cli generate
 
-gen-schema-docs: ## Generate schema reference pages + Mermaid ERD from the enriched XSD. (Phase 1 task)
-	@echo "Not yet implemented — tracked in specs/001-codespace-xml-scaffold/tasks.md"; exit 1
+gen-schema-docs: ## Generate schema reference pages + Mermaid ERD from the enriched XSD.
+	python -m acoustic_dataset.cli gen-schema-docs
 
 pipeline: ## End-to-end: map example input -> objects -> XML -> validate -> round-trip.
 	python -m acoustic_dataset.cli pipeline
@@ -42,5 +43,5 @@ COMPARE_REFERENCE ?= examples/reference/trial_known_good.xml
 compare: ## Migration-safety diff: $(COMPARE_GENERATED) vs $(COMPARE_REFERENCE) (override the vars).
 	python -m acoustic_dataset.cli compare "$(COMPARE_GENERATED)" "$(COMPARE_REFERENCE)"
 
-bundle: ## Produce distribution bundle (data + schema + generated models). (Phase 1 task)
-	@echo "Not yet implemented — tracked in specs/001-codespace-xml-scaffold/tasks.md"; exit 1
+bundle: pipeline ## Produce distribution bundle (data + schema + generated models) in build/dist.
+	python -m acoustic_dataset.cli bundle

README.md

@@ -39,7 +39,7 @@ specification, plan, and design artifacts are under
 | Path | What it is |
 |---|---|
 | `.devcontainer/` | Codespaces environment (Python 3.9.4) |
-| `schema/` | The XSD contract (placeholder until the real one lands) |
+| `schema/` | The XSD contract |
 | `src/acoustic_dataset/` | The pipeline package (`models/` is generated, never hand-edited) |
 | `examples/` | Example calculation input + `reference/` known-good files |
 | `tests/` | Unit, integration, and golden-file tests |

docs/concepts/pipeline-data-flow.md

@@ -28,10 +28,11 @@ intermediate (no CSV, no pickle) is needed to get testability; that whole chain
 
 ## The entities as an ER diagram
 
-This is the kind of **Mermaid ERD** the docs render — here drawn by hand for the *placeholder*
-domain. Once the real enriched XSD lands, `make gen-schema-docs` produces the equivalent
-diagram **automatically from the schema** (see [the example schema ERD](../reference/schema-erd.md)
-and [ADR 0009](../decisions/0009-mkdocs-material-mermaid-html-docs.md)).
+This is the kind of **Mermaid ERD** the docs render — here drawn by hand for the data-flow story
+(it deliberately includes pipeline entities like the golden and reference files). The
+**[schema reference](../reference/schema/index.md)** ERD, by contrast, is produced
+**automatically from the schema** by `make gen-schema-docs`
+([ADR 0009](../decisions/0009-mkdocs-material-mermaid-html-docs.md)).
 
 ```mermaid
 erDiagram
@@ -69,15 +70,15 @@ erDiagram
     }
     SCHEMA {
         string xsd_path
-        string status "PLACEHOLDER"
+        string version "0.2.0"
     }
 ```
 
 ## Reading the entities
 
 | Entity | What it is | Key rule |
 |---|---|---|
-| **CalculationResult** | Output of the acoustic seams; placeholder for the real recalculation | Produced by discrete, testable functions |
+| **CalculationResult** | Output of the acoustic seams (the recalculation/resampling step) | Produced by discrete, testable functions |
 | **Populated domain objects** | Generated-model instances after the single mapping | The assertion boundary; the one place logic lives |
 | **Platform XML** | The validated, round-tripped Phase 1 deliverable | Must pass both gates before it's trusted |
 | **Golden file** | Trusted expected output | Drives the semantic gate; changed deliberately |

docs/concepts/two-verification-gates.md

@@ -53,7 +53,7 @@ Distinct from both gates is the **schema-valid-but-different** check. When repla
 generator, new output can be perfectly schema-valid yet differ from files a consumer already
 depends on (perhaps relying on a quirk of the old hand-rolled output). The migration-safety
 comparison diffs new output against a known-good **reference** file to surface exactly this.
-See [Swap in the real schema](../how-to/swap-in-the-real-schema.md) and
+See [Change the schema](../how-to/change-the-schema.md) and
 [ADR 0004](../decisions/0004-two-gate-verification.md).
 
 ## Why split them at all?

docs/concepts/typed-vs-dicts.md

@@ -0,0 +1,78 @@
+# Typed objects vs. dictionaries
+
+> **Explanation** — the input is loaded as JSON (nested `dict`s), but the pipeline maps it once
+> onto typed objects generated from the schema and stores data in those. This page sets out what
+> strong typing constrains at the point data is stored. See
+> [ADR 0002](../decisions/0002-drop-csv-pickle-and-write_xml.md).
+
+## Storing data in a dictionary
+
+A `dict` places no constraints on what it holds: keys are arbitrary strings and values are `Any`.
+
+```python
+record = {}
+record["sourceLevel"] = 215.0      # any key, any value type
+record["sorceLevel"] = 9999        # a misspelled key is just another entry
+record["sourceLevel"] = "loud"     # a string replaces the number, with no objection
+```
+
+The structure exists only by convention. A misspelled key, a wrong value type, or an omitted
+field is stored as readily as correct data, so a mistake surfaces later — when something reads
+the value, when the XML fails validation, or not at all.
+
+## Storing data in a typed object
+
+The generated dataclasses declare which fields exist and the type of each, so the shape of what
+is stored is defined up front:
+
+```python
+from decimal import Decimal
+from acoustic_dataset.models.acoustic_dataset import Sector
+
+Sector(bearing=Decimal("30.000"), level=Decimal("134.000"))   # the declared fields
+Sector(bering=Decimal("30.000"), level=Decimal("134.000"))    # TypeError: unexpected 'bering'
+```
+
+- A name that is not a declared field is rejected when the object is constructed (`TypeError`),
+  where a `dict` would store it under a new key.
+- Each field has a declared type — here `Decimal`, as the schema specifies. A type checker
+  (`mypy`, run by `make verify`) reports a wrong-typed value before the code runs:
+
+  ```python
+  Sector(bearing="thirty", level=Decimal("134.000"))   # mypy: incompatible type "str"
+  ```
+
+- The fields and their documentation are generated from the schema, so the stored object follows
+  the contract rather than an ad-hoc shape.
+
+## Checking values as they are stored
+
+Field names and types define the *shape*. The single mapping (`to_model`) is where calculation
+output is stored into these objects, and it also enforces the schema's numeric **ranges** at that
+point:
+
+```python
+import dataclasses
+from acoustic_dataset import acoustics
+from acoustic_dataset.mapping import to_model, MappingError
+
+result = acoustics.calculate_from_file("examples/calculation_input.json")
+# Decibels are bounded to [-200, 300]; attempt to store an impossible source level:
+bad = dataclasses.replace(
+    result, active_sonar=dataclasses.replace(result.active_sonar, source_level_db=9999.0)
+)
+to_model(bad)        # MappingError — rejected as it is stored, not left for a later stage
+```
+
+A `dict` would hold `9999` and pass it on; the typed boundary rejects it.
+
+## Summary
+
+| At the point data is stored | `dict` | Typed object |
+|---|---|---|
+| Field names | any string accepted | declared; an unknown name is a `TypeError` |
+| Value types | `Any` | declared (e.g. `Decimal`), checked by `mypy` |
+| Out-of-range values | stored as-is | rejected by the mapping (`MappingError`) |
+| Relationship to the schema | convention only | generated from it |
+
+The behaviours above are checked in `tests/unit/test_typed_vs_dict.py`.

docs/decisions/0001-schema-driven-generation-with-xsdata.md

@@ -1,6 +1,6 @@
 # ADR 0001: Generate typed data classes from the XSD with xsdata
 
-- **Status**: Accepted (provisional — pending validation against the real XSD)
+- **Status**: Accepted
 - **Date**: 2026-06-13
 - **Deciders**: Project team (to confirm with the author/the consumer)
 

docs/decisions/0005-placeholder-schema-runnable-now.md

@@ -1,6 +1,8 @@
 # ADR 0005: Ship a labelled placeholder schema so the pipeline runs today
 
-- **Status**: Accepted (provisional — replaced by the real XSD when it lands)
+- **Status**: Superseded (2026-06-14) — the schema authored here has been adopted as the
+  authoritative contract; there is no separate "real" XSD pending. Evolve it via
+  [Change the schema](../how-to/change-the-schema.md).
 - **Date**: 2026-06-13
 - **Deciders**: Project team
 
@@ -18,7 +20,7 @@ We will author a small but representative **placeholder** `schema/acoustic_datas
 (a banded, typed numeric structure with `xs:documentation` on terms) and a matching example
 calculation, both **clearly labelled as placeholders**, so the whole pipeline — generate →
 map → serialise → validate → round-trip — is demonstrably runnable now. The schema sits at a
-single documented swap location (see [the how-to](../how-to/swap-in-the-real-schema.md)).
+single documented location (see [the how-to](../how-to/change-the-schema.md)).
 
 ## Consequences
 
@@ -49,5 +51,9 @@ single documented swap location (see [the how-to](../how-to/swap-in-the-real-sch
 
 ## Notes / revisit triggers
 
-- When the real XSD arrives, follow the swap how-to; this ADR is then **superseded** by the
-  reality of the real contract, and the placeholder is deleted.
+- **Superseded.** Rather than waiting for a separate "real" XSD, the schema authored here has
+  been adopted as the authoritative contract (it exercises the hard half — banded/typed
+  numerics, enforced cardinalities, type extension — so it earned the promotion). It is no
+  longer treated as a placeholder; evolve it like any contract via
+  [Change the schema](../how-to/change-the-schema.md). The decision to *bootstrap* with a
+  runnable schema (rather than stall on an external dependency) still stands and paid off.

docs/decisions/0008-generated-models-no-drift.md

@@ -19,11 +19,25 @@ as a **generated artifact**:
 
 - Generated files carry a "do not edit" header.
 - `ruff`/`mypy` exclude the generated package (it is not hand-maintained code).
-- CI **regenerates** from the XSD and fails if the working tree differs
-  (`git diff --exit-code`), catching any hand-edit or stale output.
+- CI **regenerates** from the XSD and fails if the working tree differs, catching any
+  hand-edit or stale output.
 - The same discipline extends to the generated HTML schema reference and ERD
   ([ADR 0009](0009-mkdocs-material-mermaid-html-docs.md)).
 
+For the drift gate to be reliable rather than flaky, **generation must be byte-reproducible on
+the Python 3.9.4 target** ([ADR 0007](0007-pin-python-3-9-4.md)):
+
+- `xsdata` is **pinned to `==25.7`** — the last release line that supports Python 3.9 (26.x
+  requires ≥3.10). The drift gate runs on Python 3.9, the same as the rest of CI; there is no
+  separate "generation toolchain" Python.
+- `make generate` passes explicit `--no-slots --no-kw-only --no-union-type
+  --no-postponed-annotations` flags, because xsdata otherwise toggles those by the *running*
+  interpreter's version (`config.py` checks `sys.version_info`), which would make the committed
+  models depend on who generated them.
+- xsdata's docstring wrapping still differs between Python **minor** versions (3.9 vs 3.11), so
+  models **must be regenerated on Python 3.9.x** (the devcontainer, or `uv python install
+  3.9.4`). The schema-reference generator is pure-Python string building and is version-independent.
+
 ## Consequences
 
 ### Positive
@@ -36,7 +50,7 @@ as a **generated artifact**:
 
 - A schema change produces a large generated diff in the PR (noise, but honest).
 - Contributors must learn "edit the schema or the mapping, never the models." Documented in
-  [Swap in the real schema](../how-to/swap-in-the-real-schema.md).
+  [Change the schema](../how-to/change-the-schema.md).
 
 ## Alternatives considered
 

docs/decisions/index.md

@@ -27,7 +27,7 @@ To add one, follow [Add a decision record](../how-to/add-a-decision-record.md) u
 | [0002](0002-drop-csv-pickle-and-write_xml.md) | Drop the CSV, the pickle hand-off, and hand-rolled `write_xml.py` | Accepted |
 | [0003](0003-xmlschema-as-validation-gate.md) | Use `xmlschema` as the structural validation gate | Accepted |
 | [0004](0004-two-gate-verification.md) | Verify with two gates: structural (mechanical) + semantic (human) | Accepted |
-| [0005](0005-placeholder-schema-runnable-now.md) | Ship a labelled placeholder schema so the pipeline runs today | Accepted |
+| [0005](0005-placeholder-schema-runnable-now.md) | Ship a labelled placeholder schema so the pipeline runs today | Superseded |
 | [0006](0006-codespaces-with-local-fallback.md) | GitHub Codespaces as primary env, with a portable local fallback | Accepted |
 | [0007](0007-pin-python-3-9-4.md) | Pin to exactly Python 3.9.4 to match the target system | Accepted |
 | [0008](0008-generated-models-no-drift.md) | Treat bindings as generated artifacts; fail CI on drift | Accepted |

docs/glossary.md

@@ -46,7 +46,7 @@ comparison, not by validation alone.
 
 **ERD (Entity-Relationship Diagram)** — a diagram of entities and how they relate. We
 render these with **Mermaid** and generate them from the enriched XSD so they cannot drift
-from the contract. See the [example ERD](reference/schema-erd.md).
+from the contract. See the [generated schema reference](reference/schema/index.md).
 
 ## Tooling
 

docs/how-to/build-the-docs-site.md

@@ -45,14 +45,14 @@ erDiagram
 ````
 
 See the [pipeline ERD](../concepts/pipeline-data-flow.md) and the
-[example schema ERD](../reference/schema-erd.md) for working examples.
+[generated schema ERD](../reference/schema/index.md) for working examples.
 
 ## Schema reference & ERD are *generated*
 
-The pages under `reference/` that describe the schema are produced from the **enriched XSD**
-by `make gen-schema-docs` (a Phase 1 task), not hand-written — so they can't drift from the
-contract. Until that step is implemented, the [example ERD](../reference/schema-erd.md) is a
-hand-drawn stand-in showing the intended output.
+The [schema reference](../reference/schema/index.md) — the entity tables and the Mermaid ERD —
+is produced from the **enriched XSD** by `make gen-schema-docs`, not hand-written, so it can't
+drift from the contract. Regenerate it after any schema change; the CI drift gate fails if the
+committed page is stale.
 
 ## Linking rules (to keep `--strict` happy)