diff --git a/docs/concepts/pipeline-data-flow.md b/docs/concepts/pipeline-data-flow.md index b876ba2..59e4ac3 100644 --- a/docs/concepts/pipeline-data-flow.md +++ b/docs/concepts/pipeline-data-flow.md @@ -6,25 +6,24 @@ ```mermaid flowchart TD - Input["Example calculation input
(examples/calculation_input.json)"] - Acoustics["Acoustic seams
(named, testable functions)"] - Result["CalculationResult"] - Mapping["mapping.py
(the ONE mapping)"] - Objects["Populated domain objects
(generated from XSD)"] + Input["Calculation input
(examples/calculation_input.json)"] + Build["build.py
(acoustic seams compute &
populate the schema object)"] + Objects["Schema data object
(Platform, generated from XSD)"] XML["Emitted Platform XML"] Golden["Golden file"] Reference["Known-good reference"] - Input --> Acoustics --> Result --> Mapping --> Objects --> XML + Input --> Build --> Objects --> XML Objects -. "tests assert here" .-> Golden XML -. "structural gate" .-> XML XML -. "migration safety" .-> Reference ``` -The **populated domain objects, before serialisation, are the typed testable boundary** — -tests assert on them directly, or diff the serialised XML against a golden file. No separate -intermediate (no CSV, no pickle) is needed to get testability; that whole chain was removed -(ADR 0002). +The acoustic seams compute the values and the builder populates **one schema data object** +directly — there is no intermediate domain hierarchy built only to be converted (ADR 0010). +That object, before serialisation, is the **typed testable boundary**: tests assert on it +directly, or diff the serialised XML against a golden file. No separate intermediate (no CSV, +no pickle) is needed to get testability; that whole chain was removed (ADR 0002). ## The entities as an ER diagram @@ -36,8 +35,7 @@ This is the kind of **Mermaid ERD** the docs render — here drawn by hand for t ```mermaid erDiagram - CALCULATION_INPUT ||--|| CALCULATION_RESULT : "produces (via acoustic seams)" - CALCULATION_RESULT ||--|| PLATFORM : "mapped once onto" + CALCULATION_INPUT ||--|| PLATFORM : "built into (via acoustic seams)" PLATFORM ||--|| CHARACTERISTICS : describes PLATFORM ||--o{ RADIATED_BAND : "radiates across (10 bands)" RADIATED_BAND ||--o{ SECTOR : "all round (12 sectors)" @@ -78,8 +76,8 @@ erDiagram | Entity | What it is | Key rule | |---|---|---| -| **CalculationResult** | Output of the acoustic seams (the recalculation/resampling step) | Produced by discrete, testable functions | -| **Populated domain objects** | Generated-model instances after the single mapping | The assertion boundary; the one place logic lives | +| **Acoustic seams** | The named, testable calculation functions (`band_centre_hz`, …) | Pure `float` arithmetic; feed the builder | +| **Schema data object** | The generated `Platform` the builder populates directly | The assertion boundary; built to meet the schema | | **Platform XML** | The validated, round-tripped Phase 1 deliverable | Must pass both gates before it's trusted | | **Golden file** | Trusted expected output | Drives the semantic gate; changed deliberately | | **Reference file** | Prior-process output | Drives migration-safety comparison | diff --git a/docs/concepts/typed-vs-dicts.md b/docs/concepts/typed-vs-dicts.md index 0a80ff9..1cff96f 100644 --- a/docs/concepts/typed-vs-dicts.md +++ b/docs/concepts/typed-vs-dicts.md @@ -12,13 +12,12 @@ The pipeline turns the calculation into **one typed data object that meets the s typed, and documented by the contract, so values never have to live in a loosely-typed `dict`. ```python -from acoustic_dataset import acoustics -from acoustic_dataset.mapping import to_model +from acoustic_dataset import build input_path = "examples/calculation_input.json" -# Produce the schema's data object from the calculation input: -platform = to_model(acoustics.calculate_from_file(input_path)) +# Build the schema's data object directly from the input: +platform = build.build_platform_from_file(input_path) # It is generated from the XSD; explore it by attribute. # The IDE autocompletes each step and the values carry @@ -87,32 +86,25 @@ Sector(bering=Decimal("30.000"), level=Decimal("134.000")) ## Checking values as they are stored -Field names and types define the *shape*. The single mapping (`to_model`) is where calculation -output is stored into these objects, and it also enforces the schema's numeric **ranges** at that -point: +Field names and types define the *shape*. The builder is where each value is stored into the +schema object, and it also enforces the schema's numeric **ranges** at that point — a value that +does not meet the schema is rejected before serialisation: ```python -import dataclasses -from acoustic_dataset import acoustics -from acoustic_dataset.mapping import to_model, MappingError +from acoustic_dataset import acoustics, build -input_path = "examples/calculation_input.json" -result = acoustics.calculate_from_file(input_path) - -# Decibels are bounded to [-200, 300]. -# Force an impossible source level, then map it: -bad = dataclasses.replace( - result, - active_sonar=dataclasses.replace( - result.active_sonar, source_level_db=9999.0 - ), -) -to_model(bad) -# -> MappingError: rejected as it is stored, +data = acoustics.load_input("examples/calculation_input.json") + +# Decibels are bounded to [-200, 300]. Force an impossible +# source level into the input, then build the schema object: +data["sensors"]["active"]["sourceLevelDb"] = 9999.0 +build.build_platform(data) +# -> MappingError: rejected as it is built, # not left for a later stage ``` -A `dict` would hold `9999` and pass it on; the typed boundary rejects it. +A `dict` would hold `9999` and pass it on; the builder rejects it because it does not meet the +schema. ## Summary @@ -120,7 +112,7 @@ A `dict` would hold `9999` and pass it on; the typed boundary rejects it. |---|---|---| | Field names | any string accepted | declared; an unknown name is a `TypeError` | | Value types | `Any` | declared (e.g. `Decimal`), checked by `mypy` | -| Out-of-range values | stored as-is | rejected by the mapping (`MappingError`) | +| Out-of-range values | stored as-is | rejected by the builder (`MappingError`) | | Relationship to the schema | convention only | generated from it | | What downstream stages receive | a shape to trust on faith | a declared structure, all the way to XML | diff --git a/docs/decisions/0002-drop-csv-pickle-and-write_xml.md b/docs/decisions/0002-drop-csv-pickle-and-write_xml.md index a988f17..ba42aa3 100644 --- a/docs/decisions/0002-drop-csv-pickle-and-write_xml.md +++ b/docs/decisions/0002-drop-csv-pickle-and-write_xml.md @@ -48,5 +48,9 @@ serialised once to XML. ## Notes / revisit triggers +- **Refined by [ADR 0010](0010-build-schema-object-directly.md)**: the "single mapping from + calculation output onto schema-generated objects" is replaced by building the schema object + *directly* (no intermediate domain hierarchy). The removal of CSV/pickle/`write_xml` and the + serialise-once decision here still stand. - If profiling later shows the single in-process flow can't hold the full corpus in memory, revisit with a streaming approach — but not by resurrecting CSV/pickle. diff --git a/docs/decisions/0008-generated-models-no-drift.md b/docs/decisions/0008-generated-models-no-drift.md index 01bb9cb..a7aa5c4 100644 --- a/docs/decisions/0008-generated-models-no-drift.md +++ b/docs/decisions/0008-generated-models-no-drift.md @@ -49,7 +49,7 @@ the Python 3.9.4 target** ([ADR 0007](0007-pin-python-3-9-4.md)): ### Negative / trade-offs - A schema change produces a large generated diff in the PR (noise, but honest). -- Contributors must learn "edit the schema or the mapping, never the models." Documented in +- Contributors must learn "edit the schema or the builder, never the models." Documented in [Change the schema](../how-to/change-the-schema.md). ## Alternatives considered diff --git a/docs/decisions/0010-build-schema-object-directly.md b/docs/decisions/0010-build-schema-object-directly.md index 8fb8526..3635664 100644 --- a/docs/decisions/0010-build-schema-object-directly.md +++ b/docs/decisions/0010-build-schema-object-directly.md @@ -1,8 +1,8 @@ # ADR 0010: Build the schema's data object directly (drop the intermediate domain hierarchy) -- **Status**: Proposed +- **Status**: Accepted - **Date**: 2026-06-16 -- **Deciders**: Project team (pending stakeholder confirmation of the defensible-process framing) +- **Deciders**: Project team (stakeholder-defensibility framing confirmed) ## Context diff --git a/docs/how-to/change-the-schema.md b/docs/how-to/change-the-schema.md index 8d9eec8..7c0cb5b 100644 --- a/docs/how-to/change-the-schema.md +++ b/docs/how-to/change-the-schema.md @@ -28,8 +28,8 @@ redesign*: models, validation, bindings and the schema docs all regenerate from Produces the reference pages and the Mermaid ERD from the schema (ADR 0009). -4. **Update the mapping.** - `src/acoustic_dataset/mapping.py` is the **one place** that knows element names — update it +4. **Update the builder.** + `src/acoustic_dataset/build.py` is the **one place** that knows element names — update it for any added/renamed/retyped fields. Generation code does *not* change. 5. **Update the example and golden file.** diff --git a/docs/onboarding.md b/docs/onboarding.md index 1c2acf9..db19b27 100644 --- a/docs/onboarding.md +++ b/docs/onboarding.md @@ -26,7 +26,7 @@ before the same `make verify` / `make pipeline`. Both paths reach the *same* gre |---|---| | The contract (the XSD) | `schema/acoustic_dataset.xsd` ([how to change it](how-to/change-the-schema.md)) | | The scientific seams (named, testable calc functions) | `src/acoustic_dataset/acoustics/` | -| The **one** place calc output becomes typed objects | `src/acoustic_dataset/mapping.py` | +| The **one** place the schema object is built | `src/acoustic_dataset/build.py` | | Generated models (never hand-edited — regenerate) | `src/acoustic_dataset/models/` | | Example calculation input | `examples/calculation_input.json` | | Tests (unit / integration / golden) | `tests/` | diff --git a/docs/reference/schema/index.md b/docs/reference/schema/index.md index b1e9eef..450487e 100644 --- a/docs/reference/schema/index.md +++ b/docs/reference/schema/index.md @@ -248,14 +248,14 @@ A validated document produced by `make pipeline` from `examples/calculation_inpu ## Working with the typed objects -The pipeline maps calculation output **once** onto the generated dataclasses; tests assert on those typed objects (the testable boundary) and they serialise straight to XML: +The pipeline builds the calculation **directly** into the generated dataclasses; tests assert on those typed objects (the testable boundary) and they serialise straight to XML: ```python -from acoustic_dataset import acoustics, serialize -from acoustic_dataset.mapping import to_model +from acoustic_dataset import build, serialize -result = acoustics.calculate_from_file("examples/calculation_input.json") -platform = to_model(result) # a generated Platform object +platform = build.build_platform_from_file( # a generated Platform object + "examples/calculation_input.json" +) platform.name # 'Reference Platform A' platform.characteristics.draft # Decimal('7.500') diff --git a/src/acoustic_dataset/acoustics/__init__.py b/src/acoustic_dataset/acoustics/__init__.py index 06f75c2..38c3667 100644 --- a/src/acoustic_dataset/acoustics/__init__.py +++ b/src/acoustic_dataset/acoustics/__init__.py @@ -1,4 +1,4 @@ -"""Acoustic calculation seams: ``calculation_input.json`` -> ``CalculationResult``. +"""Acoustic calculation seams: the pure functions that compute the dataset's values. The recalculation/resampling-onto-bands work (FR-012): each physical step is a **discrete, named, testable function** with the engineering "how it's computed" documented here in code @@ -12,86 +12,20 @@ directivity lobe around the platform), sampled every 30 degrees, and * one derived sensor figure (an active sonar's maximum echo range from its source level). -It produces typed, banded numbers for the pipeline to carry. The dataclasses below are -deliberately named with a ``Result`` suffix so they never collide with the *generated* model -classes of the same domain. +These functions return **plain ``float``s** — the arithmetic, not a data structure. The builder +in :mod:`acoustic_dataset.build` calls them to populate the schema-generated data object +directly (see ADR 0010); there is no intermediate domain hierarchy. """ from __future__ import annotations import json import math -from dataclasses import dataclass from pathlib import Path - -@dataclass(frozen=True) -class SectorResult: - """Radiated noise in one 30-degree bearing sector of a band (plain Python types).""" - - bearing_deg: float - level_db: float - - -@dataclass(frozen=True) -class BandResult: - """Directional radiated noise for one frequency band.""" - - index: int - centre_frequency_hz: float - sectors: list[SectorResult] - - -@dataclass(frozen=True) -class CharacteristicsResult: - """The platform's physical characteristics.""" - - draft_m: float - length_m: float - weight_t: float - year_introduced: int - - -@dataclass(frozen=True) -class ActiveSonarResult: - """An active sonar: identity, transmit figures, and a derived maximum echo range.""" - - name: str - manufacturer: str - operating_frequency_hz: float - source_level_db: float - beamwidth_deg: float - pulse_length_s: float - max_range_m: float - - -@dataclass(frozen=True) -class PassiveSonarResult: - """A passive (listen-only) sonar: identity plus reception figures.""" - - name: str - manufacturer: str - operating_frequency_hz: float - array_gain_db: float - detection_threshold_db: float - bearing_accuracy_deg: float - - -@dataclass(frozen=True) -class CalculationResult: - """The output of the acoustic seams: one platform's reference data.""" - - schema_version: str - name: str - generated_utc: str - characteristics: CharacteristicsResult - bands: list[BandResult] - active_sonar: ActiveSonarResult - passive_sonars: list[PassiveSonarResult] - - # ----- individual seam functions (each independently unit-tested) ----- + def band_centre_hz(base_hz: float, ratio: float, index: int) -> float: """Centre frequency of the ``index``-th band (1-based) of a geometric ladder. @@ -143,86 +77,9 @@ def bearings(step_deg: float) -> list[float]: return [round(step_deg * k, 6) for k in range(count)] -# ----- orchestration over the seams ----- - -def _calculate_bands(spec: dict) -> list[BandResult]: - """Synthesise the directional radiated-noise bands from the high-level spec.""" - base_hz = float(spec["baseFrequencyHz"]) - ratio = float(spec["bandRatio"]) - band_count = int(spec["bandCount"]) - base_level = float(spec["baseLevelDb"]) - rolloff = float(spec["rolloffDbPerOctave"]) - peak_bearing = float(spec["directivity"]["peakBearingDeg"]) - amplitude = float(spec["directivity"]["amplitudeDb"]) - sampled_bearings = bearings(float(spec["bearingStepDeg"])) - - bands: list[BandResult] = [] - for index in range(1, band_count + 1): - centre = band_centre_hz(base_hz, ratio, index) - rolloff_db = spectral_rolloff_db(centre, base_hz, rolloff) - sectors = [ - SectorResult( - bearing_deg=bearing, - level_db=radiated_level_db( - base_level, rolloff_db, directivity_db(bearing, peak_bearing, amplitude) - ), - ) - for bearing in sampled_bearings - ] - bands.append(BandResult(index=index, centre_frequency_hz=centre, sectors=sectors)) - return bands - - -def _active_sonar(spec: dict) -> ActiveSonarResult: - return ActiveSonarResult( - name=str(spec["name"]), - manufacturer=str(spec["manufacturer"]), - operating_frequency_hz=float(spec["operatingFrequencyHz"]), - source_level_db=float(spec["sourceLevelDb"]), - beamwidth_deg=float(spec["beamwidthDeg"]), - pulse_length_s=float(spec["pulseLengthSeconds"]), - max_range_m=active_max_range_m( - float(spec["sourceLevelDb"]), float(spec["detectionThresholdDb"]) - ), - ) - - -def _passive_sonar(spec: dict) -> PassiveSonarResult: - return PassiveSonarResult( - name=str(spec["name"]), - manufacturer=str(spec["manufacturer"]), - operating_frequency_hz=float(spec["operatingFrequencyHz"]), - array_gain_db=float(spec["arrayGainDb"]), - detection_threshold_db=float(spec["detectionThresholdDb"]), - bearing_accuracy_deg=float(spec["bearingAccuracyDeg"]), - ) - - -def calculate(data: dict) -> CalculationResult: - """Run the seams over a parsed input document, producing a ``CalculationResult``.""" - characteristics = data["characteristics"] - sensors = data["sensors"] - return CalculationResult( - schema_version=str(data.get("schemaVersion", "0.2.0")), - name=str(data["name"]), - generated_utc=str(data["generatedUtc"]), - characteristics=CharacteristicsResult( - draft_m=float(characteristics["draftMetres"]), - length_m=float(characteristics["lengthMetres"]), - weight_t=float(characteristics["weightTonnes"]), - year_introduced=int(characteristics["yearIntroduced"]), - ), - bands=_calculate_bands(data["radiatedNoise"]), - active_sonar=_active_sonar(sensors["active"]), - passive_sonars=[_passive_sonar(p) for p in sensors["passive"]], - ) +# ----- input loading ----- def load_input(path: Path) -> dict: """Read and parse a calculation-input JSON file.""" return json.loads(Path(path).read_text(encoding="utf-8")) - - -def calculate_from_file(path: Path) -> CalculationResult: - """Convenience: load an input file and run the seams over it.""" - return calculate(load_input(path)) diff --git a/src/acoustic_dataset/build.py b/src/acoustic_dataset/build.py new file mode 100644 index 0000000..3719aa0 --- /dev/null +++ b/src/acoustic_dataset/build.py @@ -0,0 +1,237 @@ +"""Build the schema's data object directly from calculation input (ADR 0010). + +This is the **one builder** that produces schema-typed objects. It computes the dataset's +values with the pure seam functions in :mod:`acoustic_dataset.acoustics` and populates the +generated ``Platform`` / ``RadiatedBand`` / ``Sector`` / ... classes directly — there is no +intermediate domain hierarchy that is built only to be converted. + +It enforces, at the point of construction, that every value **meets the schema**: + +1. Numbers are converted to the schema's ``Decimal`` type and quantised to a fixed scale, so + the serialised XML is deterministic (stable golden file). +2. Any value that does not fit a declared schema band/type raises a :class:`MappingError` + *here*, before serialisation — so the pipeline never emits schema-valid-but-wrong XML, and + the failure points at the offending location. The XSD validation gate then re-proves + conformance on the serialised document. +""" + +from __future__ import annotations + +from decimal import ROUND_HALF_EVEN, Decimal +from pathlib import Path + +from xsdata.models.datatype import XmlDateTime + +from acoustic_dataset import acoustics +from acoustic_dataset.models.acoustic_dataset import ( + ActiveSonar, + Directional, + PassiveSonar, + Platform, + PlatformCharacteristics, + RadiatedBand, + RadiatedNoise, + Sector, + SensorSuite, +) + +# Declared schema bands (kept in step with schema/acoustic_dataset.xsd). These mirror the +# XSD facets so a violation is caught at build time with a clear, location-aware message. +_DECIBELS_RANGE: tuple[Decimal, Decimal] = (Decimal("-200"), Decimal("300")) +_DEGREES_RANGE: tuple[Decimal, Decimal] = (Decimal("0"), Decimal("360")) # inclusive both ends +_BEARING_RANGE: tuple[Decimal, Decimal] = (Decimal("0"), Decimal("360")) # high end EXCLUSIVE +_YEAR_RANGE: tuple[int, int] = (1900, 2100) +_NON_NEGATIVE = Decimal("0") + +# Fixed quantisation keeps serialised output deterministic across platforms (stable golden). +_Q_GENERAL = Decimal("0.001") + + +class MappingError(ValueError): + """Raised when a calculated value does not fit a declared schema band/type.""" + + +def _dec(value: float) -> Decimal: + return Decimal(str(value)).quantize(_Q_GENERAL, rounding=ROUND_HALF_EVEN) + + +def _require_in_range( + value: Decimal, low: Decimal, high: Decimal, *, where: str, field: str +) -> Decimal: + if value < low or value > high: + raise MappingError(f"{where}: {field}={value} is outside the schema band [{low}, {high}]") + return value + + +def _require_below( + value: Decimal, low: Decimal, high: Decimal, *, where: str, field: str +) -> Decimal: + """Half-open band ``[low, high)`` — used for ``Bearing`` (0 <= x < 360).""" + if value < low or value >= high: + raise MappingError(f"{where}: {field}={value} is outside the schema band [{low}, {high})") + return value + + +def _require_min(value: Decimal, low: Decimal, *, where: str, field: str) -> Decimal: + if value < low: + raise MappingError(f"{where}: {field}={value} is below the schema minimum {low}") + return value + + +def _require_int_in_range(value: int, low: int, high: int, *, where: str, field: str) -> int: + if value < low or value > high: + raise MappingError(f"{where}: {field}={value} is outside the schema band [{low}, {high}]") + return value + + +def _build_characteristics(spec: dict) -> PlatformCharacteristics: + where = "characteristics" + return PlatformCharacteristics( + draft=_require_min( + _dec(float(spec["draftMetres"])), _NON_NEGATIVE, where=where, field="Draft" + ), + length=_require_min( + _dec(float(spec["lengthMetres"])), _NON_NEGATIVE, where=where, field="Length" + ), + weight=_require_min( + _dec(float(spec["weightTonnes"])), _NON_NEGATIVE, where=where, field="Weight" + ), + year_introduced=_require_int_in_range( + int(spec["yearIntroduced"]), *_YEAR_RANGE, where=where, field="YearIntroduced" + ), + ) + + +def _build_sector(band_index: int, bearing_deg: float, level_db: float) -> Sector: + where = f"band {band_index} bearing {bearing_deg:g}" + return Sector( + bearing=_require_below(_dec(bearing_deg), *_BEARING_RANGE, where=where, field="Bearing"), + level=_require_in_range(_dec(level_db), *_DECIBELS_RANGE, where=where, field="Level"), + ) + + +def _build_bands(spec: dict) -> list[RadiatedBand]: + """Synthesise the directional radiated-noise bands and build them into schema objects.""" + base_hz = float(spec["baseFrequencyHz"]) + ratio = float(spec["bandRatio"]) + band_count = int(spec["bandCount"]) + base_level = float(spec["baseLevelDb"]) + rolloff = float(spec["rolloffDbPerOctave"]) + peak_bearing = float(spec["directivity"]["peakBearingDeg"]) + amplitude = float(spec["directivity"]["amplitudeDb"]) + sampled_bearings = acoustics.bearings(float(spec["bearingStepDeg"])) + + bands: list[RadiatedBand] = [] + for index in range(1, band_count + 1): + centre = acoustics.band_centre_hz(base_hz, ratio, index) + rolloff_db = acoustics.spectral_rolloff_db(centre, base_hz, rolloff) + sectors = [ + _build_sector( + index, + bearing, + acoustics.radiated_level_db( + base_level, + rolloff_db, + acoustics.directivity_db(bearing, peak_bearing, amplitude), + ), + ) + for bearing in sampled_bearings + ] + bands.append( + RadiatedBand( + centre_frequency=_require_min( + _dec(centre), _NON_NEGATIVE, where=f"band {index}", field="CentreFrequency" + ), + directional=Directional(sector=sectors), + index=index, + ) + ) + return bands + + +def _build_active(spec: dict) -> ActiveSonar: + where = "active sonar" + source_level = float(spec["sourceLevelDb"]) + max_range = acoustics.active_max_range_m(source_level, float(spec["detectionThresholdDb"])) + return ActiveSonar( + name=str(spec["name"]), + manufacturer=str(spec["manufacturer"]), + operating_frequency=_require_min( + _dec(float(spec["operatingFrequencyHz"])), + _NON_NEGATIVE, + where=where, + field="OperatingFrequency", + ), + source_level=_require_in_range( + _dec(source_level), *_DECIBELS_RANGE, where=where, field="SourceLevel" + ), + beamwidth=_require_in_range( + _dec(float(spec["beamwidthDeg"])), *_DEGREES_RANGE, where=where, field="Beamwidth" + ), + pulse_length=_require_min( + _dec(float(spec["pulseLengthSeconds"])), + _NON_NEGATIVE, + where=where, + field="PulseLength", + ), + max_range=_require_min(_dec(max_range), _NON_NEGATIVE, where=where, field="MaxRange"), + ) + + +def _build_passive(ordinal: int, spec: dict) -> PassiveSonar: + where = f"passive sonar {ordinal}" + return PassiveSonar( + name=str(spec["name"]), + manufacturer=str(spec["manufacturer"]), + operating_frequency=_require_min( + _dec(float(spec["operatingFrequencyHz"])), + _NON_NEGATIVE, + where=where, + field="OperatingFrequency", + ), + array_gain=_require_in_range( + _dec(float(spec["arrayGainDb"])), *_DECIBELS_RANGE, where=where, field="ArrayGain" + ), + detection_threshold=_require_in_range( + _dec(float(spec["detectionThresholdDb"])), + *_DECIBELS_RANGE, + where=where, + field="DetectionThreshold", + ), + bearing_accuracy=_require_in_range( + _dec(float(spec["bearingAccuracyDeg"])), + *_DEGREES_RANGE, + where=where, + field="BearingAccuracy", + ), + ) + + +def build_platform(data: dict) -> Platform: + """Build a populated, schema-conformant ``Platform`` from a parsed input document. + + Computes the dataset's values and constructs the generated model objects directly, + converting to the schema's types, quantising, and range-checking on the way (raising + :class:`MappingError` on any value that does not meet the schema). + """ + bands = _build_bands(data["radiatedNoise"]) + if not bands: + raise MappingError("calculation produced no bands; nothing to build") + return Platform( + schema_version=str(data.get("schemaVersion", "0.2.0")), + name=str(data["name"]), + generated_utc=XmlDateTime.from_string(str(data["generatedUtc"])), + characteristics=_build_characteristics(data["characteristics"]), + radiated_noise=RadiatedNoise(band=bands), + sensors=SensorSuite( + active=_build_active(data["sensors"]["active"]), + passive=[ + _build_passive(i, p) for i, p in enumerate(data["sensors"]["passive"], start=1) + ], + ), + ) + + +def build_platform_from_file(path: Path) -> Platform: + """Convenience: load an input file and build the schema's ``Platform`` from it.""" + return build_platform(acoustics.load_input(path)) diff --git a/src/acoustic_dataset/cli.py b/src/acoustic_dataset/cli.py index 0635c0e..703c432 100644 --- a/src/acoustic_dataset/cli.py +++ b/src/acoustic_dataset/cli.py @@ -33,15 +33,13 @@ def cmd_generate(args: argparse.Namespace) -> int: def cmd_pipeline(args: argparse.Namespace) -> int: - """Map input -> objects -> XML -> validate -> round-trip; write only if both gates pass.""" - from acoustic_dataset import acoustics, serialize, validate - from acoustic_dataset.mapping import MappingError, to_model + """Build the schema object -> XML -> validate -> round-trip; write only if both gates pass.""" + from acoustic_dataset import build, serialize, validate - result = acoustics.calculate_from_file(args.input) try: - model = to_model(result) - except MappingError as exc: - print(f"error: mapping rejected a value before serialisation: {exc}", file=sys.stderr) + model = build.build_platform_from_file(args.input) + except build.MappingError as exc: + print(f"error: build rejected a value before serialisation: {exc}", file=sys.stderr) return 1 xml = serialize.to_xml(model) diff --git a/src/acoustic_dataset/mapping.py b/src/acoustic_dataset/mapping.py deleted file mode 100644 index 1cd6358..0000000 --- a/src/acoustic_dataset/mapping.py +++ /dev/null @@ -1,194 +0,0 @@ -"""The single mapping: ``CalculationResult`` -> populated generated models. - -This is the **one place schema-aware logic lives** (contracts/pipeline-contract.md). It is -the only place to touch when the schema changes; nothing else references schema element names. - -Two rules it enforces: - -1. The mapping happens **exactly once** — no reshape-twice, no intermediate CSV/pickle/string. -2. Any value that does not fit a declared schema band/type is raised as a - :class:`MappingError` *here*, before serialisation — so the pipeline never emits - schema-valid-but-wrong XML, and the failure points at the offending location. -""" - -from __future__ import annotations - -from decimal import ROUND_HALF_EVEN, Decimal - -from xsdata.models.datatype import XmlDateTime - -from acoustic_dataset.acoustics import ( - ActiveSonarResult, - BandResult, - CalculationResult, - CharacteristicsResult, - PassiveSonarResult, - SectorResult, -) -from acoustic_dataset.models.acoustic_dataset import ( - ActiveSonar, - Directional, - PassiveSonar, - Platform, - PlatformCharacteristics, - RadiatedBand, - RadiatedNoise, - Sector, - SensorSuite, -) - -# Declared schema bands (kept in step with schema/acoustic_dataset.xsd). These mirror the -# XSD facets so a violation is caught at mapping time with a clear, location-aware message. -_DECIBELS_RANGE: tuple[Decimal, Decimal] = (Decimal("-200"), Decimal("300")) -_DEGREES_RANGE: tuple[Decimal, Decimal] = (Decimal("0"), Decimal("360")) # inclusive both ends -_BEARING_RANGE: tuple[Decimal, Decimal] = (Decimal("0"), Decimal("360")) # high end EXCLUSIVE -_YEAR_RANGE: tuple[int, int] = (1900, 2100) -_NON_NEGATIVE = Decimal("0") - -# Fixed quantisation keeps serialised output deterministic across platforms (stable golden). -_Q_GENERAL = Decimal("0.001") - - -class MappingError(ValueError): - """Raised when a calculated value does not fit a declared schema band/type.""" - - -def _dec(value: float) -> Decimal: - return Decimal(str(value)).quantize(_Q_GENERAL, rounding=ROUND_HALF_EVEN) - - -def _require_in_range( - value: Decimal, low: Decimal, high: Decimal, *, where: str, field: str -) -> Decimal: - if value < low or value > high: - raise MappingError( - f"{where}: {field}={value} is outside the schema band [{low}, {high}]" - ) - return value - - -def _require_below( - value: Decimal, low: Decimal, high: Decimal, *, where: str, field: str -) -> Decimal: - """Half-open band ``[low, high)`` — used for ``Bearing`` (0 <= x < 360).""" - if value < low or value >= high: - raise MappingError( - f"{where}: {field}={value} is outside the schema band [{low}, {high})" - ) - return value - - -def _require_min(value: Decimal, low: Decimal, *, where: str, field: str) -> Decimal: - if value < low: - raise MappingError(f"{where}: {field}={value} is below the schema minimum {low}") - return value - - -def _require_int_in_range(value: int, low: int, high: int, *, where: str, field: str) -> int: - if value < low or value > high: - raise MappingError( - f"{where}: {field}={value} is outside the schema band [{low}, {high}]" - ) - return value - - -def _map_characteristics(result: CharacteristicsResult) -> PlatformCharacteristics: - where = "characteristics" - return PlatformCharacteristics( - draft=_require_min(_dec(result.draft_m), _NON_NEGATIVE, where=where, field="Draft"), - length=_require_min(_dec(result.length_m), _NON_NEGATIVE, where=where, field="Length"), - weight=_require_min(_dec(result.weight_t), _NON_NEGATIVE, where=where, field="Weight"), - year_introduced=_require_int_in_range( - result.year_introduced, *_YEAR_RANGE, where=where, field="YearIntroduced" - ), - ) - - -def _map_sector(band_index: int, sector: SectorResult) -> Sector: - where = f"band {band_index} bearing {sector.bearing_deg:g}" - return Sector( - bearing=_require_below( - _dec(sector.bearing_deg), *_BEARING_RANGE, where=where, field="Bearing" - ), - level=_require_in_range( - _dec(sector.level_db), *_DECIBELS_RANGE, where=where, field="Level" - ), - ) - - -def _map_band(band: BandResult) -> RadiatedBand: - centre = _require_min( - _dec(band.centre_frequency_hz), _NON_NEGATIVE, - where=f"band {band.index}", field="CentreFrequency", - ) - return RadiatedBand( - centre_frequency=centre, - directional=Directional(sector=[_map_sector(band.index, s) for s in band.sectors]), - index=band.index, - ) - - -def _map_active(result: ActiveSonarResult) -> ActiveSonar: - where = "active sonar" - return ActiveSonar( - name=result.name, - manufacturer=result.manufacturer, - operating_frequency=_require_min( - _dec(result.operating_frequency_hz), _NON_NEGATIVE, - where=where, field="OperatingFrequency", - ), - source_level=_require_in_range( - _dec(result.source_level_db), *_DECIBELS_RANGE, where=where, field="SourceLevel" - ), - beamwidth=_require_in_range( - _dec(result.beamwidth_deg), *_DEGREES_RANGE, where=where, field="Beamwidth" - ), - pulse_length=_require_min( - _dec(result.pulse_length_s), _NON_NEGATIVE, where=where, field="PulseLength" - ), - max_range=_require_min( - _dec(result.max_range_m), _NON_NEGATIVE, where=where, field="MaxRange" - ), - ) - - -def _map_passive(ordinal: int, result: PassiveSonarResult) -> PassiveSonar: - where = f"passive sonar {ordinal}" - return PassiveSonar( - name=result.name, - manufacturer=result.manufacturer, - operating_frequency=_require_min( - _dec(result.operating_frequency_hz), _NON_NEGATIVE, - where=where, field="OperatingFrequency", - ), - array_gain=_require_in_range( - _dec(result.array_gain_db), *_DECIBELS_RANGE, where=where, field="ArrayGain" - ), - detection_threshold=_require_in_range( - _dec(result.detection_threshold_db), *_DECIBELS_RANGE, - where=where, field="DetectionThreshold", - ), - bearing_accuracy=_require_in_range( - _dec(result.bearing_accuracy_deg), *_DEGREES_RANGE, - where=where, field="BearingAccuracy", - ), - ) - - -def to_model(result: CalculationResult) -> Platform: - """Map a ``CalculationResult`` onto a populated ``Platform`` (raises on bad values).""" - if not result.bands: - raise MappingError("calculation produced no bands; nothing to map") - return Platform( - schema_version=result.schema_version, - name=result.name, - generated_utc=XmlDateTime.from_string(result.generated_utc), - characteristics=_map_characteristics(result.characteristics), - radiated_noise=RadiatedNoise(band=[_map_band(b) for b in result.bands]), - sensors=SensorSuite( - active=_map_active(result.active_sonar), - passive=[ - _map_passive(i, p) for i, p in enumerate(result.passive_sonars, start=1) - ], - ), - ) diff --git a/src/acoustic_dataset/schema_docs.py b/src/acoustic_dataset/schema_docs.py index d25d3b7..e1a2ebd 100644 --- a/src/acoustic_dataset/schema_docs.py +++ b/src/acoustic_dataset/schema_docs.py @@ -324,11 +324,10 @@ def _render_examples(example_input: Path) -> list[str]: a trimmed sample document, typed-object usage, and a field *derived* from elementary physics. """ - from acoustic_dataset import acoustics, serialize - from acoustic_dataset.mapping import to_model + from acoustic_dataset import acoustics, build, serialize raw = acoustics.load_input(example_input) - platform = to_model(acoustics.calculate(raw)) + platform = build.build_platform(raw) xml = serialize.to_xml(platform) characteristics = platform.characteristics @@ -370,15 +369,15 @@ def _render_examples(example_input: Path) -> list[str]: "", "## Working with the typed objects", "", - "The pipeline maps calculation output **once** onto the generated dataclasses; tests " + "The pipeline builds the calculation **directly** into the generated dataclasses; tests " "assert on those typed objects (the testable boundary) and they serialise straight to XML:", "", "```python", - "from acoustic_dataset import acoustics, serialize", - "from acoustic_dataset.mapping import to_model", + "from acoustic_dataset import build, serialize", "", - 'result = acoustics.calculate_from_file("examples/calculation_input.json")', - "platform = to_model(result) # a generated Platform object", + 'platform = build.build_platform_from_file( # a generated Platform object', + ' "examples/calculation_input.json"', + ")", "", f"platform.name # {platform.name!r}", f"platform.characteristics.draft # Decimal('{characteristics.draft}')", diff --git a/tests/integration/test_gates.py b/tests/integration/test_gates.py index b6359e0..0472079 100644 --- a/tests/integration/test_gates.py +++ b/tests/integration/test_gates.py @@ -1,17 +1,14 @@ -"""Structural-gate tests (T020): schema validity + round-trip, and the mapping guard. +"""Structural-gate tests (T020): schema validity + round-trip, and the builder guard. These encode the contract that schema-validity alone is not sufficient, and that a value -outside a declared band is rejected at mapping time — never emitted as schema-valid-but-wrong. +outside a declared band is rejected at build time — never emitted as schema-valid-but-wrong. """ from __future__ import annotations -import dataclasses - import pytest -from acoustic_dataset import acoustics, serialize, validate -from acoustic_dataset.mapping import MappingError, to_model +from acoustic_dataset import acoustics, build, serialize, validate def test_golden_passes_both_structural_gates(golden_path, schema_path): @@ -23,14 +20,14 @@ def test_golden_passes_both_structural_gates(golden_path, schema_path): def test_pipeline_output_passes_both_gates(input_path, schema_path): - xml = serialize.to_xml(to_model(acoustics.calculate_from_file(input_path))) + xml = serialize.to_xml(build.build_platform_from_file(input_path)) report = validate.validate(xml, schema_path) assert report.ok def test_schema_invalid_document_is_caught(input_path, schema_path): # Push a bearing past the schema's [0, 360) band: must fail the structural gate. - xml = serialize.to_xml(to_model(acoustics.calculate_from_file(input_path))) + xml = serialize.to_xml(build.build_platform_from_file(input_path)) bad = xml.replace("0.000", "400.000", 1) assert bad != xml, "expected to find a bearing to tamper with" report = validate.validate(bad, schema_path) @@ -39,10 +36,9 @@ def test_schema_invalid_document_is_caught(input_path, schema_path): assert any("schema" in e for e in report.errors) -def test_mapping_rejects_out_of_band_value_before_serialisation(input_path): - result = acoustics.calculate_from_file(input_path) +def test_build_rejects_out_of_band_value_before_serialisation(input_path): + data = acoustics.load_input(input_path) # Force an impossible source level past the schema's decibel band: the guard must fire. - tampered_active = dataclasses.replace(result.active_sonar, source_level_db=9999.0) - tampered = dataclasses.replace(result, active_sonar=tampered_active) - with pytest.raises(MappingError, match="SourceLevel"): - to_model(tampered) + data["sensors"]["active"]["sourceLevelDb"] = 9999.0 + with pytest.raises(build.MappingError, match="SourceLevel"): + build.build_platform(data) diff --git a/tests/integration/test_pipeline.py b/tests/integration/test_pipeline.py index b781376..aa31e0a 100644 --- a/tests/integration/test_pipeline.py +++ b/tests/integration/test_pipeline.py @@ -1,7 +1,7 @@ """Integration test for the pipeline (T019). Asserts on BOTH boundaries the spec cares about (FR-010): -1. the populated domain objects produced by the single mapping, and +1. the schema data object the builder populates directly, and 2. the serialised XML, diffed against the trusted golden file (the semantic gate). """ @@ -11,17 +11,16 @@ from lxml import etree -from acoustic_dataset import acoustics, serialize -from acoustic_dataset.mapping import to_model +from acoustic_dataset import build, serialize def _build_xml(input_path): - result = acoustics.calculate_from_file(input_path) - return result, serialize.to_xml(to_model(result)) + model = build.build_platform_from_file(input_path) + return model, serialize.to_xml(model) def test_populated_objects_carry_the_expected_typed_values(input_path): - model = to_model(acoustics.calculate_from_file(input_path)) + model = build.build_platform_from_file(input_path) assert model.schema_version == "0.2.0" diff --git a/tests/integration/test_schema_docs.py b/tests/integration/test_schema_docs.py index 32ae1c8..f81d56a 100644 --- a/tests/integration/test_schema_docs.py +++ b/tests/integration/test_schema_docs.py @@ -72,7 +72,7 @@ def test_example_document_section_has_a_sample_xml(tmp_path, schema_path, input_ def test_typed_objects_section_shows_real_values(tmp_path, schema_path, input_path): md = _generate(tmp_path, schema_path, input_path) assert "## Working with the typed objects" in md - assert "to_model(result)" in md + assert "build.build_platform_from_file(" in md assert "'Reference Platform A'" in md # the real platform name assert "Decimal('118850.223')" in md diff --git a/tests/unit/test_acoustics.py b/tests/unit/test_acoustics.py index 8f2ba1c..0e913f9 100644 --- a/tests/unit/test_acoustics.py +++ b/tests/unit/test_acoustics.py @@ -49,20 +49,3 @@ def test_bearings_sample_all_round_in_30_degree_sectors(): 0.0, 30.0, 60.0, 90.0, 120.0, 150.0, 180.0, 210.0, 240.0, 270.0, 300.0, 330.0 ] assert acoustics.bearings(90.0) == [0.0, 90.0, 180.0, 270.0] - - -def test_calculate_produces_ten_bands_of_twelve_sectors(input_path): - result = acoustics.calculate_from_file(input_path) - assert [b.index for b in result.bands] == list(range(1, 11)) - assert all(len(b.sectors) == 12 for b in result.bands) - assert [s.bearing_deg for s in result.bands[0].sectors][:3] == [0.0, 30.0, 60.0] - - -def test_calculate_synthesises_characteristics_and_sensors(input_path): - result = acoustics.calculate_from_file(input_path) - assert result.schema_version == "0.2.0" - assert result.characteristics.year_introduced == 1998 - assert result.active_sonar.name == "AS-900 Echo" - assert result.active_sonar.max_range_m == pytest.approx(10.0 ** (203.0 / 40.0)) - assert len(result.passive_sonars) == 2 - assert result.passive_sonars[1].manufacturer == "Marine Acoustics Ltd" diff --git a/tests/unit/test_build.py b/tests/unit/test_build.py new file mode 100644 index 0000000..db9fcc1 --- /dev/null +++ b/tests/unit/test_build.py @@ -0,0 +1,32 @@ +"""Unit tests for the builder: calculation input -> a schema-conformant ``Platform`` (ADR 0010). + +The acoustic seams (tested in ``test_acoustics.py``) are exercised here through the builder, +which constructs the generated model objects directly — there is no intermediate hierarchy. +""" + +from __future__ import annotations + +from decimal import Decimal + +from acoustic_dataset import build + + +def test_build_produces_ten_bands_of_twelve_sectors(input_path): + platform = build.build_platform_from_file(input_path) + bands = platform.radiated_noise.band + assert [b.index for b in bands] == list(range(1, 11)) + assert all(len(b.directional.sector) == 12 for b in bands) + assert [s.bearing for s in bands[0].directional.sector][:3] == [ + Decimal("0.000"), Decimal("30.000"), Decimal("60.000") + ] + + +def test_build_synthesises_characteristics_and_sensors(input_path): + platform = build.build_platform_from_file(input_path) + assert platform.schema_version == "0.2.0" + assert platform.characteristics.year_introduced == 1998 + assert platform.sensors.active.name == "AS-900 Echo" + # The derived max range falls straight out of the sonar equation, quantised to mm. + assert platform.sensors.active.max_range == Decimal("118850.223") + assert len(platform.sensors.passive) == 2 + assert platform.sensors.passive[1].manufacturer == "Marine Acoustics Ltd" diff --git a/tests/unit/test_typed_vs_dict.py b/tests/unit/test_typed_vs_dict.py index 6ce726c..2bf89cf 100644 --- a/tests/unit/test_typed_vs_dict.py +++ b/tests/unit/test_typed_vs_dict.py @@ -6,13 +6,11 @@ from __future__ import annotations -import dataclasses from decimal import Decimal import pytest -from acoustic_dataset import acoustics -from acoustic_dataset.mapping import MappingError, to_model +from acoustic_dataset import acoustics, build from acoustic_dataset.models.acoustic_dataset import Sector @@ -33,16 +31,14 @@ def test_an_unknown_field_is_rejected_when_constructing(): def test_a_stored_field_has_the_schema_declared_type(input_path): - platform = to_model(acoustics.calculate_from_file(input_path)) + platform = build.build_platform_from_file(input_path) level = platform.radiated_noise.band[0].directional.sector[0].level assert isinstance(level, Decimal) # stored as the schema's Decimal, not a bare float def test_out_of_range_value_is_rejected_as_it_is_stored(input_path): - result = acoustics.calculate_from_file(input_path) + data = acoustics.load_input(input_path) # The schema bounds Decibels to [-200, 300]; a dict would carry 9999 straight through. - bad = dataclasses.replace( - result, active_sonar=dataclasses.replace(result.active_sonar, source_level_db=9999.0) - ) - with pytest.raises(MappingError, match="SourceLevel"): - to_model(bad) + data["sensors"]["active"]["sourceLevelDb"] = 9999.0 + with pytest.raises(build.MappingError, match="SourceLevel"): + build.build_platform(data)