Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 12 additions & 14 deletions docs/concepts/pipeline-data-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,24 @@

```mermaid
flowchart TD
Input["Example calculation input<br/>(examples/calculation_input.json)"]
Acoustics["Acoustic seams<br/>(named, testable functions)"]
Result["CalculationResult"]
Mapping["mapping.py<br/>(the ONE mapping)"]
Objects["Populated domain objects<br/>(generated from XSD)"]
Input["Calculation input<br/>(examples/calculation_input.json)"]
Build["build.py<br/>(acoustic seams compute &<br/>populate the schema object)"]
Objects["Schema data object<br/>(Platform, generated from XSD)"]
XML["Emitted Platform XML"]
Golden["Golden file"]
Reference["Known-good reference"]

Input --> Acoustics --> Result --> Mapping --> Objects --> XML
Input --> Build --> Objects --> XML
Objects -. "tests assert here" .-> Golden
XML -. "structural gate" .-> XML
XML -. "migration safety" .-> Reference
```

The **populated domain objects, before serialisation, are the typed testable boundary** —
tests assert on them directly, or diff the serialised XML against a golden file. No separate
intermediate (no CSV, no pickle) is needed to get testability; that whole chain was removed
(ADR 0002).
The acoustic seams compute the values and the builder populates **one schema data object**
directly — there is no intermediate domain hierarchy built only to be converted (ADR 0010).
That object, before serialisation, is the **typed testable boundary**: tests assert on it
directly, or diff the serialised XML against a golden file. No separate intermediate (no CSV,
no pickle) is needed to get testability; that whole chain was removed (ADR 0002).

## The entities as an ER diagram

Expand All @@ -36,8 +35,7 @@ This is the kind of **Mermaid ERD** the docs render — here drawn by hand for t

```mermaid
erDiagram
CALCULATION_INPUT ||--|| CALCULATION_RESULT : "produces (via acoustic seams)"
CALCULATION_RESULT ||--|| PLATFORM : "mapped once onto"
CALCULATION_INPUT ||--|| PLATFORM : "built into (via acoustic seams)"
PLATFORM ||--|| CHARACTERISTICS : describes
PLATFORM ||--o{ RADIATED_BAND : "radiates across (10 bands)"
RADIATED_BAND ||--o{ SECTOR : "all round (12 sectors)"
Expand Down Expand Up @@ -78,8 +76,8 @@ erDiagram

| Entity | What it is | Key rule |
|---|---|---|
| **CalculationResult** | Output of the acoustic seams (the recalculation/resampling step) | Produced by discrete, testable functions |
| **Populated domain objects** | Generated-model instances after the single mapping | The assertion boundary; the one place logic lives |
| **Acoustic seams** | The named, testable calculation functions (`band_centre_hz`, …) | Pure `float` arithmetic; feed the builder |
| **Schema data object** | The generated `Platform` the builder populates directly | The assertion boundary; built to meet the schema |
| **Platform XML** | The validated, round-tripped Phase 1 deliverable | Must pass both gates before it's trusted |
| **Golden file** | Trusted expected output | Drives the semantic gate; changed deliberately |
| **Reference file** | Prior-process output | Drives migration-safety comparison |
Expand Down
42 changes: 17 additions & 25 deletions docs/concepts/typed-vs-dicts.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,12 @@ The pipeline turns the calculation into **one typed data object that meets the s
typed, and documented by the contract, so values never have to live in a loosely-typed `dict`.

```python
from acoustic_dataset import acoustics
from acoustic_dataset.mapping import to_model
from acoustic_dataset import build

input_path = "examples/calculation_input.json"

# Produce the schema's data object from the calculation input:
platform = to_model(acoustics.calculate_from_file(input_path))
# Build the schema's data object directly from the input:
platform = build.build_platform_from_file(input_path)

# It is generated from the XSD; explore it by attribute.
# The IDE autocompletes each step and the values carry
Expand Down Expand Up @@ -87,40 +86,33 @@ Sector(bering=Decimal("30.000"), level=Decimal("134.000"))

## Checking values as they are stored

Field names and types define the *shape*. The single mapping (`to_model`) is where calculation
output is stored into these objects, and it also enforces the schema's numeric **ranges** at that
point:
Field names and types define the *shape*. The builder is where each value is stored into the
schema object, and it also enforces the schema's numeric **ranges** at that point — a value that
does not meet the schema is rejected before serialisation:

```python
import dataclasses
from acoustic_dataset import acoustics
from acoustic_dataset.mapping import to_model, MappingError
from acoustic_dataset import acoustics, build

input_path = "examples/calculation_input.json"
result = acoustics.calculate_from_file(input_path)

# Decibels are bounded to [-200, 300].
# Force an impossible source level, then map it:
bad = dataclasses.replace(
result,
active_sonar=dataclasses.replace(
result.active_sonar, source_level_db=9999.0
),
)
to_model(bad)
# -> MappingError: rejected as it is stored,
data = acoustics.load_input("examples/calculation_input.json")

# Decibels are bounded to [-200, 300]. Force an impossible
# source level into the input, then build the schema object:
data["sensors"]["active"]["sourceLevelDb"] = 9999.0
build.build_platform(data)
# -> MappingError: rejected as it is built,
# not left for a later stage
```

A `dict` would hold `9999` and pass it on; the typed boundary rejects it.
A `dict` would hold `9999` and pass it on; the builder rejects it because it does not meet the
schema.

## Summary

| At the point data is stored | `dict` | Typed object |
|---|---|---|
| Field names | any string accepted | declared; an unknown name is a `TypeError` |
| Value types | `Any` | declared (e.g. `Decimal`), checked by `mypy` |
| Out-of-range values | stored as-is | rejected by the mapping (`MappingError`) |
| Out-of-range values | stored as-is | rejected by the builder (`MappingError`) |
| Relationship to the schema | convention only | generated from it |
| What downstream stages receive | a shape to trust on faith | a declared structure, all the way to XML |

Expand Down
4 changes: 4 additions & 0 deletions docs/decisions/0002-drop-csv-pickle-and-write_xml.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,9 @@ serialised once to XML.

## Notes / revisit triggers

- **Refined by [ADR 0010](0010-build-schema-object-directly.md)**: the "single mapping from
calculation output onto schema-generated objects" is replaced by building the schema object
*directly* (no intermediate domain hierarchy). The removal of CSV/pickle/`write_xml` and the
serialise-once decision here still stand.
- If profiling later shows the single in-process flow can't hold the full corpus in memory,
revisit with a streaming approach — but not by resurrecting CSV/pickle.
2 changes: 1 addition & 1 deletion docs/decisions/0008-generated-models-no-drift.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ the Python 3.9.4 target** ([ADR 0007](0007-pin-python-3-9-4.md)):
### Negative / trade-offs

- A schema change produces a large generated diff in the PR (noise, but honest).
- Contributors must learn "edit the schema or the mapping, never the models." Documented in
- Contributors must learn "edit the schema or the builder, never the models." Documented in
[Change the schema](../how-to/change-the-schema.md).

## Alternatives considered
Expand Down
4 changes: 2 additions & 2 deletions docs/decisions/0010-build-schema-object-directly.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# ADR 0010: Build the schema's data object directly (drop the intermediate domain hierarchy)

- **Status**: Proposed
- **Status**: Accepted
- **Date**: 2026-06-16
- **Deciders**: Project team (pending stakeholder confirmation of the defensible-process framing)
- **Deciders**: Project team (stakeholder-defensibility framing confirmed)

## Context

Expand Down
4 changes: 2 additions & 2 deletions docs/how-to/change-the-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ redesign*: models, validation, bindings and the schema docs all regenerate from
Produces the reference pages and the Mermaid ERD from the schema
(ADR 0009).

4. **Update the mapping.**
`src/acoustic_dataset/mapping.py` is the **one place** that knows element names — update it
4. **Update the builder.**
`src/acoustic_dataset/build.py` is the **one place** that knows element names — update it
for any added/renamed/retyped fields. Generation code does *not* change.

5. **Update the example and golden file.**
Expand Down
2 changes: 1 addition & 1 deletion docs/onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ before the same `make verify` / `make pipeline`. Both paths reach the *same* gre
|---|---|
| The contract (the XSD) | `schema/acoustic_dataset.xsd` ([how to change it](how-to/change-the-schema.md)) |
| The scientific seams (named, testable calc functions) | `src/acoustic_dataset/acoustics/` |
| The **one** place calc output becomes typed objects | `src/acoustic_dataset/mapping.py` |
| The **one** place the schema object is built | `src/acoustic_dataset/build.py` |
| Generated models (never hand-edited — regenerate) | `src/acoustic_dataset/models/` |
| Example calculation input | `examples/calculation_input.json` |
| Tests (unit / integration / golden) | `tests/` |
Expand Down
10 changes: 5 additions & 5 deletions docs/reference/schema/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,14 +248,14 @@ A validated document produced by `make pipeline` from `examples/calculation_inpu

## Working with the typed objects

The pipeline maps calculation output **once** onto the generated dataclasses; tests assert on those typed objects (the testable boundary) and they serialise straight to XML:
The pipeline builds the calculation **directly** into the generated dataclasses; tests assert on those typed objects (the testable boundary) and they serialise straight to XML:

```python
from acoustic_dataset import acoustics, serialize
from acoustic_dataset.mapping import to_model
from acoustic_dataset import build, serialize

result = acoustics.calculate_from_file("examples/calculation_input.json")
platform = to_model(result) # a generated Platform object
platform = build.build_platform_from_file( # a generated Platform object
"examples/calculation_input.json"
)

platform.name # 'Reference Platform A'
platform.characteristics.draft # Decimal('7.500')
Expand Down
155 changes: 6 additions & 149 deletions src/acoustic_dataset/acoustics/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Acoustic calculation seams: ``calculation_input.json`` -> ``CalculationResult``.
"""Acoustic calculation seams: the pure functions that compute the dataset's values.

The recalculation/resampling-onto-bands work (FR-012): each physical step is a **discrete,
named, testable function** with the engineering "how it's computed" documented here in code
Expand All @@ -12,86 +12,20 @@
directivity lobe around the platform), sampled every 30 degrees, and
* one derived sensor figure (an active sonar's maximum echo range from its source level).

It produces typed, banded numbers for the pipeline to carry. The dataclasses below are
deliberately named with a ``Result`` suffix so they never collide with the *generated* model
classes of the same domain.
These functions return **plain ``float``s** — the arithmetic, not a data structure. The builder
in :mod:`acoustic_dataset.build` calls them to populate the schema-generated data object
directly (see ADR 0010); there is no intermediate domain hierarchy.
"""

from __future__ import annotations

import json
import math
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class SectorResult:
"""Radiated noise in one 30-degree bearing sector of a band (plain Python types)."""

bearing_deg: float
level_db: float


@dataclass(frozen=True)
class BandResult:
"""Directional radiated noise for one frequency band."""

index: int
centre_frequency_hz: float
sectors: list[SectorResult]


@dataclass(frozen=True)
class CharacteristicsResult:
"""The platform's physical characteristics."""

draft_m: float
length_m: float
weight_t: float
year_introduced: int


@dataclass(frozen=True)
class ActiveSonarResult:
"""An active sonar: identity, transmit figures, and a derived maximum echo range."""

name: str
manufacturer: str
operating_frequency_hz: float
source_level_db: float
beamwidth_deg: float
pulse_length_s: float
max_range_m: float


@dataclass(frozen=True)
class PassiveSonarResult:
"""A passive (listen-only) sonar: identity plus reception figures."""

name: str
manufacturer: str
operating_frequency_hz: float
array_gain_db: float
detection_threshold_db: float
bearing_accuracy_deg: float


@dataclass(frozen=True)
class CalculationResult:
"""The output of the acoustic seams: one platform's reference data."""

schema_version: str
name: str
generated_utc: str
characteristics: CharacteristicsResult
bands: list[BandResult]
active_sonar: ActiveSonarResult
passive_sonars: list[PassiveSonarResult]


# ----- individual seam functions (each independently unit-tested) -----


def band_centre_hz(base_hz: float, ratio: float, index: int) -> float:
"""Centre frequency of the ``index``-th band (1-based) of a geometric ladder.

Expand Down Expand Up @@ -143,86 +77,9 @@ def bearings(step_deg: float) -> list[float]:
return [round(step_deg * k, 6) for k in range(count)]


# ----- orchestration over the seams -----

def _calculate_bands(spec: dict) -> list[BandResult]:
"""Synthesise the directional radiated-noise bands from the high-level spec."""
base_hz = float(spec["baseFrequencyHz"])
ratio = float(spec["bandRatio"])
band_count = int(spec["bandCount"])
base_level = float(spec["baseLevelDb"])
rolloff = float(spec["rolloffDbPerOctave"])
peak_bearing = float(spec["directivity"]["peakBearingDeg"])
amplitude = float(spec["directivity"]["amplitudeDb"])
sampled_bearings = bearings(float(spec["bearingStepDeg"]))

bands: list[BandResult] = []
for index in range(1, band_count + 1):
centre = band_centre_hz(base_hz, ratio, index)
rolloff_db = spectral_rolloff_db(centre, base_hz, rolloff)
sectors = [
SectorResult(
bearing_deg=bearing,
level_db=radiated_level_db(
base_level, rolloff_db, directivity_db(bearing, peak_bearing, amplitude)
),
)
for bearing in sampled_bearings
]
bands.append(BandResult(index=index, centre_frequency_hz=centre, sectors=sectors))
return bands


def _active_sonar(spec: dict) -> ActiveSonarResult:
return ActiveSonarResult(
name=str(spec["name"]),
manufacturer=str(spec["manufacturer"]),
operating_frequency_hz=float(spec["operatingFrequencyHz"]),
source_level_db=float(spec["sourceLevelDb"]),
beamwidth_deg=float(spec["beamwidthDeg"]),
pulse_length_s=float(spec["pulseLengthSeconds"]),
max_range_m=active_max_range_m(
float(spec["sourceLevelDb"]), float(spec["detectionThresholdDb"])
),
)


def _passive_sonar(spec: dict) -> PassiveSonarResult:
return PassiveSonarResult(
name=str(spec["name"]),
manufacturer=str(spec["manufacturer"]),
operating_frequency_hz=float(spec["operatingFrequencyHz"]),
array_gain_db=float(spec["arrayGainDb"]),
detection_threshold_db=float(spec["detectionThresholdDb"]),
bearing_accuracy_deg=float(spec["bearingAccuracyDeg"]),
)


def calculate(data: dict) -> CalculationResult:
"""Run the seams over a parsed input document, producing a ``CalculationResult``."""
characteristics = data["characteristics"]
sensors = data["sensors"]
return CalculationResult(
schema_version=str(data.get("schemaVersion", "0.2.0")),
name=str(data["name"]),
generated_utc=str(data["generatedUtc"]),
characteristics=CharacteristicsResult(
draft_m=float(characteristics["draftMetres"]),
length_m=float(characteristics["lengthMetres"]),
weight_t=float(characteristics["weightTonnes"]),
year_introduced=int(characteristics["yearIntroduced"]),
),
bands=_calculate_bands(data["radiatedNoise"]),
active_sonar=_active_sonar(sensors["active"]),
passive_sonars=[_passive_sonar(p) for p in sensors["passive"]],
)
# ----- input loading -----


def load_input(path: Path) -> dict:
"""Read and parse a calculation-input JSON file."""
return json.loads(Path(path).read_text(encoding="utf-8"))


def calculate_from_file(path: Path) -> CalculationResult:
"""Convenience: load an input file and run the seams over it."""
return calculate(load_input(path))
Loading
Loading