Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,11 @@ gen-schema-docs: ## Generate schema reference pages + Mermaid ERD from the enric
pipeline: ## End-to-end: map example input -> objects -> XML -> validate -> round-trip.
python -m acoustic_dataset.cli pipeline

compare: ## Migration-safety diff of generated output vs a known-good reference. (Phase 1 task)
@echo "Not yet implemented — tracked in specs/001-codespace-xml-scaffold/tasks.md"; exit 1
COMPARE_GENERATED ?= build/acoustic_dataset.xml
COMPARE_REFERENCE ?= examples/reference/trial_known_good.xml

compare: ## Migration-safety diff: $(COMPARE_GENERATED) vs $(COMPARE_REFERENCE) (override the vars).
python -m acoustic_dataset.cli compare "$(COMPARE_GENERATED)" "$(COMPARE_REFERENCE)"

bundle: ## Produce distribution bundle (data + schema + generated models). (Phase 1 task)
@echo "Not yet implemented — tracked in specs/001-codespace-xml-scaffold/tasks.md"; exit 1
61 changes: 61 additions & 0 deletions docs/how-to/run-the-migration-safety-comparison.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# How to run the migration-safety comparison

> **How-to** — check freshly generated XML against a trusted known-good file and catch the
> dangerous *schema-valid-but-different* case.

Schema validity proves the output fits the contract; it does **not** prove it says the same
thing as a file a consumer already depends on. The `compare` command closes that gap
([ADR 0004](../decisions/0004-two-gate-verification.md), FR-015).

## Steps

1. **Get a known-good reference.** Drop a trusted prior-process file (e.g. one of the
consumer's trial outputs) into `examples/reference/`. A shipped placeholder,
`examples/reference/trial_known_good.xml`, is there as a template.

2. **Produce fresh output.**
```bash
make pipeline # writes build/acoustic_dataset.xml
```

3. **Compare.**
```bash
make compare # build/acoustic_dataset.xml vs examples/reference/trial_known_good.xml
```
Point it at other files by overriding the variables:
```bash
make compare COMPARE_GENERATED=build/acoustic_dataset.xml \
COMPARE_REFERENCE=examples/reference/your-trial-file.xml
```
Or call the CLI directly:
```bash
python -m acoustic_dataset.cli compare <generated.xml> <reference.xml>
```

## How to read the result

- **Clean match** → prints `match: …` and exits `0`. The documents are identical once
cosmetic noise is removed.
- **Meaningful difference** → prints a unified diff (reference → generated) and exits non-zero,
so it fails CI and scripts rather than passing silently.

## What counts as cosmetic (ignored) vs meaningful (caught)

The comparison canonicalises both sides first, so these **never** cause a failure:

| Ignored (cosmetic) | Example |
|---|---|
| Attribute order | `a="1" b="2"` vs `b="2" a="1"` |
| Whitespace / indentation | pretty-printed vs single-line |
| Namespace **prefix** | default `xmlns=…` vs `ds:` prefix (same URI) |
| Comments | a `<!-- trial file -->` banner |

Anything that changes the **data** — an element value, a missing or extra element, a different
structure — is reported as a difference. For instance, a radiated-noise `Level` of `134.000`
that becomes `144.000` is still inside the schema's decibel band (so it stays schema-valid) but
is surfaced as a one-line diff.

!!! note "Why prefixes and comments are ignored"
A reference file from another tool may be serialised in a different style. The equality
decision is fully namespace-aware; only the *prefix label* and documentation comments are
treated as presentation, so style differences don't drown out the differences that matter.
47 changes: 47 additions & 0 deletions docs/onboarding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Onboarding

> **Start point for a new contributor.** This single page gets you to a green build and
> shows you where everything lives. For the guided, explained walkthrough, follow the
> [Start here tutorial](tutorials/01-start-here.md); for a command-by-command validation
> run, see the [quickstart](https://github.com/deepbluecltd/python-party/blob/main/specs/001-codespace-xml-scaffold/quickstart.md).

## Reach green

**GitHub Codespace (primary):** on the repository page, **Code → Codespaces → Create
codespace**. Provisioning runs `make bootstrap` for you (installs the toolchain on Python
3.9.4 and generates the models). When the terminal is ready:

```bash
make verify # lint + tests — confirm the environment is green
make pipeline # produce build/acoustic_dataset.xml (schema-valid, round-trip-equal)
```

**Local fallback:** you need Python 3.9.x and `make`, then run `make bootstrap` yourself
before the same `make verify` / `make pipeline`. Both paths reach the *same* green state
([ADR 0006](decisions/0006-codespaces-with-local-fallback.md)). Run `make help` to see every target.

## Where things live

| You're looking for… | It's here |
|---|---|
| The contract (the XSD) — **swap location** for the real schema | `schema/acoustic_dataset.xsd` ([how-to](how-to/swap-in-the-real-schema.md)) |
| The scientific seams (named, testable calc functions) | `src/acoustic_dataset/acoustics/` |
| The **one** place calc output becomes typed objects | `src/acoustic_dataset/mapping.py` |
| Generated models (never hand-edited — regenerate) | `src/acoustic_dataset/models/` |
| Example calculation input (placeholder) | `examples/calculation_input.json` |
| Tests (unit / integration / golden) | `tests/` |
| The plan & design artifacts | `specs/001-codespace-xml-scaffold/` (`spec.md`, `plan.md`, `tasks.md`) |
| Why each choice was made | [Decision records](decisions/index.md) |

## Build the mental model

Read these, in order — they're short:

1. [Schema as the contract](concepts/schema-as-contract.md) — the idea everything follows from.
2. [The two verification gates](concepts/two-verification-gates.md) — why schema-valid isn't the same as correct.
3. [Pipeline data flow](concepts/pipeline-data-flow.md) — how input becomes validated XML.

!!! tip "What's done, what's next"
The environment, the end-to-end pipeline, and the migration-safety `compare` are in
place. Remaining pipeline work (generated schema docs/ERD and the distribution bundle)
is tracked in `specs/001-codespace-xml-scaffold/tasks.md`.
6 changes: 3 additions & 3 deletions docs/reference/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@
| `make verify` | Lint (`ruff`) + tests (`pytest`) | ✅ |
| `make docs` | Build static HTML site (`mkdocs build --strict`) → `site/` | ✅ |
| `make docs-serve` | Live-preview docs at <http://localhost:8000> | ✅ |
| `make generate` | Regenerate models from `schema/*.xsd` via `xsdata` | ⏳ Phase 1 task |
| `make generate` | Regenerate models from `schema/*.xsd` via `xsdata` | |
| `make gen-schema-docs` | Generate schema reference + Mermaid ERD from the enriched XSD | ⏳ Phase 1 task |
| `make pipeline` | End-to-end: map → serialise → validate → round-trip | ⏳ Phase 1 task |
| `make compare` | Migration-safety diff vs a known-good reference | ⏳ Phase 1 task |
| `make pipeline` | End-to-end: map → serialise → validate → round-trip | |
| `make compare` | Migration-safety diff vs a known-good reference | |
| `make bundle` | Distribution bundle (data + schema + models) | ⏳ Phase 1 task |

⏳ targets currently print a pointer to `specs/001-codespace-xml-scaffold/tasks.md` and exit
Expand Down
Loading
Loading