Physics driver l2 by yiluchen1066 · Pull Request #1301 · C2SM/icon4py

yiluchen1066 · 2026-06-04T15:18:56Z

No description provided.

msimberg

This is not really a review, but more questions and comments about minor and major things. Mostly for my own understanding and reminder to revisit as the PR evolves, but adding the comments in case someone else has thought about them already and has answers.

msimberg · 2026-06-11T10:24:30Z

+    def __call__(
+        self, state: dict[str, model.DataField], time_step: datetime.datetime
+    ) -> dict[str, model.DataField]:


Question for myself:I suppose this is one of the critical interfaces, though the signature could likely be tightened to something resembling state: (Something?)State, time_step: datetime.datetime? Or the whole class should inherit from a protocol.

Minor note as well that may need broader discussion: If this is indeed the protocol, my preference is to give this a name rather than using the generic __call__ operator, but I also know we already use __call__ extensively, so may be me still not being used to __call__ being used everywhere.

And further thinking out loud: Could this be based on a more generic TendencyAdaptor? The muphys-specific parts seem quite minimal and the interesting logic seems to be the _to_tendency parts.

and time_step should be datetime.timedelta if we decide to use timedeltas for parametrizing (as opposed to the integer time steps)

msimberg · 2026-06-11T13:55:00Z

+        self,
+        prognostic: prognostics.PrognosticState,
+        outputs: dict[str, fa.CellKField[ta.wpfloat]],
+        dt: float,


I guess this is in seconds? I wonder if we can add a bit more type information to this (something from datetime?) and just convert to the correct units as late as possible (i.e. when passing into gt4py programs).

after a separate discussion with @msimberg we came up with this for the standalone driver:
https://github.com/C2SM/icon4py/pull/1304/changes#diff-1374141d85d171d3fa2c5711b656072708d65ff1080beaf0cdeedcc64f105bd7R37

should we use it here too?

jcanton · 2026-06-15T13:15:26Z

@@ -0,0 +1,3 @@
+# icon4py-atmosphere-physics-interface
+
+The L2 physics orchestrator (`PhysicsDriver`) and its process registry (`PhysicsProcess`, `ProcessTimeControl`, `ForcingMode`) for icon4py, modelled on ICON-MPIM's AES physics interface (`aes_phy_main`).


let's not use two or more names for the same thing ;-)
see discussion in #icon4py-dev

Also, this is still referencing the L2 naming. Please check if there are other places that need to have the LN naming changed.

jcanton · 2026-06-15T13:18:57Z

+    return dict(
+        standard_name=f"tendency_of_{base['standard_name']}",
+        units=f"{base['units']} s-1",
+        kind="tendency",
+    )


also from a cheap-friend:

🟡 risk: _tendency_of returns dict(...) not model.FieldMetaData — TypedDict construction bypasses key validation. Use explicit model.FieldMetaData(standard_name=..., units=..., kind="tendency")

jcanton

some comments here and there

Copilot

Pull request overview

This PR introduces an L2 physics orchestration layer (PhysicsDriver) as a new workspace package and wires it into the standalone driver to enable running fast-physics (currently: muphys) after dynamics/diffusion/transport, along with supporting metadata and small math utilities needed by the new adapters.

Changes:

Added new icon4py-atmosphere-physics-interface package containing PhysicsDriver, PhysicsProcess, ProcessTimeControl, and ForcingMode.
Integrated a muphys physics process into model/standalone_driver (granule initialization + per-timestep call site) and added muphys state/component/config/data adapters.
Extended common field metadata with a kind tag and added shared CF metadata registries for microphysics precipitation diagnostics and generic tendencies.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
uv.lock	Adds new workspace member + dependency entries for `icon4py-atmosphere-physics-interface` and updates standalone_driver deps.
tach.toml	Registers physics_interface in the dependency graph; standalone_driver now depends on muphys + physics_interface.
pyproject.toml	Adds physics_interface to workspace dependencies, mypy paths, and pythonpath.
model/standalone_driver/src/icon4py/model/standalone_driver/standalone_driver.py	Injects `ModelTimeVariables` and invokes physics granule each timestep.
model/standalone_driver/src/icon4py/model/standalone_driver/driver_utils.py	Initializes `PhysicsDriver` granule and constructs a muphys `PhysicsProcess`.
model/standalone_driver/src/icon4py/model/standalone_driver/driver_states.py	Refactors `ModelTimeVariables` initialization and time-step count derivation.
model/standalone_driver/src/icon4py/model/standalone_driver/config.py	Adds `muphys` config field (placeholder; parsing TODOs).
model/standalone_driver/pyproject.toml	Adds dependencies on muphys and the new physics-interface package.
model/common/tests/common/components/unit_tests/init.py	Adds package marker/license header for new tests package path.
model/common/tests/common/components/init.py	Adds package marker/license header for new tests package path.
model/common/src/icon4py/model/common/states/model.py	Adds optional `kind` metadata field for physics outputs (tendency vs diagnostic).
model/common/src/icon4py/model/common/states/data.py	Adds shared CF metadata for microphysics precipitation diagnostics and generic tendencies.
model/common/src/icon4py/model/common/math/stencils/generic_math_operations.py	Adds GT4Py programs for `a + coeff*b` and copy operations used by physics adapters.
model/common/src/icon4py/model/common/math/operators.py	Adds underlying field_operators for `a + coeff*b` and copy.
model/common/src/icon4py/model/common/components/physics_state.py	Introduces `PhysicsState` protocol (driver-facing state adapter interface).
model/atmosphere/subgrid_scale_physics/physics_interface/tests/physics_interface/unit_tests/init.py	Initializes physics_interface unit test package (no tests yet).
model/atmosphere/subgrid_scale_physics/physics_interface/tests/physics_interface/init.py	Initializes physics_interface tests package.
model/atmosphere/subgrid_scale_physics/physics_interface/src/icon4py/model/atmosphere/subgrid_scale_physics/physics_interface/physics_driver.py	Adds `PhysicsDriver` orchestrator and time-control primitives.
model/atmosphere/subgrid_scale_physics/physics_interface/src/icon4py/model/atmosphere/subgrid_scale_physics/physics_interface/init.py	Package metadata/version for physics_interface.
model/atmosphere/subgrid_scale_physics/physics_interface/README.md	Brief package-level description of the new orchestrator.
model/atmosphere/subgrid_scale_physics/physics_interface/pyproject.toml	New package definition for `icon4py-atmosphere-physics-interface`.
model/atmosphere/subgrid_scale_physics/muphys/tests/muphys/unit_tests/init.py	Initializes muphys unit test package path (no unit tests added here).
model/atmosphere/subgrid_scale_physics/muphys/src/icon4py/model/atmosphere/subgrid_scale_physics/muphys/state.py	Adds muphys `PhysicsState` adapter to gather/scatter prognostic state and apply tendencies.
model/atmosphere/subgrid_scale_physics/muphys/src/icon4py/model/atmosphere/subgrid_scale_physics/muphys/data.py	Defines muphys component input/output field metadata mappings.
model/atmosphere/subgrid_scale_physics/muphys/src/icon4py/model/atmosphere/subgrid_scale_physics/muphys/config.py	Adds `MuphysConfig` (currently only `qnc`).
model/atmosphere/subgrid_scale_physics/muphys/src/icon4py/model/atmosphere/subgrid_scale_physics/muphys/component.py	Adds `MuphysComponent` wrapper that runs muphys and converts state updates to tendencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        for proc in self._processes:
+            state = proc.state
+            state.gather_from_prognostic(prognostic, tracers)
+            tc = proc.time_control
+            if not tc.enable_process:
+                continue
+            if not tc.is_in_window(simulation_current_datetime):
+                # outside the process window: no forcing
+                continue
+            # Compute on a firing (active) step, and also on the first in-window step -- when
+            # there is nothing cached to recycle yet. Otherwise reuse the last computed forcing.
+            if tc.is_active(simulation_current_datetime) or proc.name not in self._recycle_cache:
+                # compute
+                outputs = proc.component(state.as_component_input(), simulation_current_datetime)
+                self._recycle_cache[proc.name] = outputs
+            else:
+                # recycle
+                outputs = self._recycle_cache[proc.name]
+            state.scatter_to_prognostic(prognostic, outputs, dt)


jcanton · 2026-06-25T15:05:20Z

+        elapsed = (simulation_current_datetime - self.start_date).total_seconds()
+        interval_s = self.interval.total_seconds()
+        quotient = elapsed / interval_s
+        return abs(quotient - round(quotient)) == 0.0


no, do my version ;-)

+    from icon4py.model.common.states import model
+
+_SPECIES = ("v", "c", "r", "s", "i", "g")
+# TODO (Yilu): avoid duplicating specieis


+# TODO (Yilu): this can belong to a standalone file (with the exception of ForcingMode.APPLY)?
+@dataclasses.dataclass(frozen=True)
+class ProcessTimeControl:
+    """icon4py analogue of the per-process fields in AES `aes_phy_tc`.
+


msimberg

Just a few minor additoinal comments and todos.

msimberg · 2026-06-25T14:01:53Z

+
+from icon4py.model.atmosphere.subgrid_scale_physics.muphys import data as muphys_data
+from icon4py.model.atmosphere.subgrid_scale_physics.muphys.core.definitions import Q
+from icon4py.model.atmosphere.subgrid_scale_physics.muphys.driver import run_full_muphys


Nit, maybe:

Suggested change

from icon4py.model.atmosphere.subgrid_scale_physics.muphys.driver import run_full_muphys

from icon4py.model.atmosphere.subgrid_scale_physics.muphys.driver import muphys_driver

?

I see run_full_muphys was used elsewhere already, but I don't think it hurts to clean up this name.

msimberg · 2026-06-25T14:03:57Z

+            new_field=new,
+            tendency=out,
+            horizontal_start=0,
+            horizontal_end=self._ncells,


Have you checked if this needs to be computed all the way to _ncells or would the local patch be sufficient?

msimberg · 2026-06-25T14:05:12Z

+        return cast(
+            "dict[str, model.DataField]",


Why is the cast needed?

msimberg · 2026-06-25T14:05:42Z

+    """Configuration for the muphys microphysics component.
+    """
+
+    qnc: float


Just noting so it's not forgotten: was this a namelist option or a field in the end?

it's a hardcoded float in fortran :-)
so we make it a configurable float, but not read from namelists

msimberg · 2026-06-25T14:19:09Z

+_ACTIVE_TOLERANCE = 1e-6  # relative tolerance on the modulo, in fractions of `interval`
+
+
+@dataclasses.dataclass(frozen=True)
+class ProcessTimeControl:
+    """icon4py analogue of the per-process fields in AES `aes_phy_tc`.
+
+    Mirrors `mo_aes_phy_main.f90` semantics:
+      - `interval`   (`dt_xxx`): zero means the process is disabled.
+      - `start_date` (`sd_xxx`), `end_date` (`ed_xxx`): half-open
+        `[start, end)` window during which the process exists at all.
+      - `forcing_mode` (`fc_xxx`): DIAGNOSTIC (compute only) or APPLY.
+    """
+
+    interval: datetime.timedelta
+    start_date: datetime.datetime
+    end_date: datetime.datetime
+    forcing_mode: ForcingMode = ForcingMode.APPLY
+
+    def is_enabled(self) -> bool:
+        return self.interval > datetime.timedelta(0)
+
+    def is_in_window(self, now: datetime.datetime) -> bool:
+        return self.start_date <= now < self.end_date
+
+    def is_active(self, now: datetime.datetime) -> bool:
+        """True if the process's mtime-event fires on the given step.
+
+        Equivalent to AES `isCurrentEventActive(ev_xxx, datetime)`. Uses a
+        small relative tolerance to absorb datetime/timedelta floating-point
+        jitter accumulated over many steps.
+
+        """
+        if now < self.start_date or not self.is_enabled():
+            return False
+        elapsed = (now - self.start_date).total_seconds()
+        interval_s = self.interval.total_seconds()
+        quotient = elapsed / interval_s
+        return abs(quotient - round(quotient)) < _ACTIVE_TOLERANCE


Just noting that this I would like to see moved to a separate file and split from forcing_mode in this PR.

msimberg · 2026-06-25T14:20:31Z

+        elapsed = (simulation_current_datetime - self.start_date).total_seconds()
+        interval_s = self.interval.total_seconds()
+        quotient = elapsed / interval_s
+        return abs(quotient - round(quotient)) == 0.0


I forget if we discussed whether or not this is allowed to run on non-integer-multiples of timesteps? If it should only be integer multiples, this needs at the very least a TODO/issue.

no, only integer multiples, but with datetime comparisons, not converting to floats

msimberg · 2026-06-25T14:21:35Z

@@ -0,0 +1,3 @@
+# icon4py-atmosphere-physics-interface
+
+The L2 physics orchestrator (`PhysicsDriver`) and its process registry (`PhysicsProcess`, `ProcessTimeControl`, `ForcingMode`) for icon4py, modelled on ICON-MPIM's AES physics interface (`aes_phy_main`).


Also, this is still referencing the L2 naming. Please check if there are other places that need to have the LN naming changed.

msimberg · 2026-06-25T14:23:04Z

+    # TODO (Yilu): we can add a post_int()
+    # TODO (Yilu): use the name from ModelTimeVariables
+


jcanton

looking good

jcanton · 2026-06-25T13:49:47Z

-class MuphysGranule:
-    """L4 per-process adapter wrapping the muphys microphysics program."""
+class MuphysComponent:
+    # TODO (Yilu): later on this should be cleared that it inherit from the component protocol


why not now already?

The Component protocol seems to need changes (it's got some generic type parameters where it's unclear if it makes sense to keep or not; in principle more structured checking is good, but I'm not about what requirements it puts on those who inherit from it). I've started investigations to make it usable (with a friend), but I think it's best not to mix it into this PR, so far it looks like a not entirely trivial change.

jcanton · 2026-06-25T13:57:19Z

        backend: gtx_typing.Backend | None = None,
        *,
-        muphys_step: Callable[..., Any] | None = None,
+        step: Callable[..., Any] | None = None,


ignorance of mine and not finding this:
is this ever passed in as argument? or should we rather remove it from the args and just set self._step = run_full_muphys.setup_muphys below?

jcanton · 2026-06-25T14:06:37Z

+    """Configuration for the muphys microphysics component.
+    """
+
+    qnc: float


add default value

I think this could already use the facilities that @DropD has set up.

jcanton · 2026-06-25T14:07:21Z

+    "dz": model.FieldMetaData(
+        standard_name="layer_thickness", units="m"
+    ),


get this guy from metrics attributes ddqdz as discussed

but also check if fortran for some reason re-computes it in a different way, just in case

jcanton · 2026-06-25T14:11:54Z

+def _require(field: fa.CellKField[ta.wpfloat] | None, name: str) -> fa.CellKField[ta.wpfloat]:
+    """Return ``field``, or raise if it is inactive (``None``).

-@gtx.field_operator
-def _add_scaled_tendency(
-    field: fa.CellKField[ta.wpfloat],
-    tendency: fa.CellKField[ta.wpfloat],
-    dt: ta.wpfloat,
-) -> fa.CellKField[ta.wpfloat]:
-    return field + tendency * dt
-
-
-@gtx.program(grid_type=gtx.GridType.UNSTRUCTURED)
-def apply_tendency(  # noqa: PLR0917  # stencil params referenced in domain specs must stay positional
-    field: fa.CellKField[ta.wpfloat],
-    tendency: fa.CellKField[ta.wpfloat],
-    dt: ta.wpfloat,
-    result: fa.CellKField[ta.wpfloat],
-    horizontal_start: gtx.int32,
-    horizontal_end: gtx.int32,
-    vertical_start: gtx.int32,
-    vertical_end: gtx.int32,
-) -> None:
-    """``result = field + tendency * dt`` (Lie-Trotter split-update).
-
-    Pass ``result=field`` for an in-place update, or a distinct buffer to keep
-    the original (e.g. computing the new temperature without clobbering ``te``).
+    muphys needs all six moisture species; ``TracerState`` fields are optional
+    (a tracer may be inactive per ``TracerConfig``), so we fail loudly here rather
+    than feed ``None`` into the microphysics.
    """
-    _add_scaled_tendency(
-        field,
-        tendency,
-        dt,
-        out=result,
-        domain={
-            dims.CellDim: (horizontal_start, horizontal_end),
-            dims.KDim: (vertical_start, vertical_end),
-        },
-    )
+    if field is None:
+        raise ValueError(f"muphys requires tracer '{name}' to be active in the TracerState")
+    return field


you can use TracerState.active_names() instead of a specific helper here

do you mean TracerState.active_fields() right?

jcanton · 2026-06-25T14:50:54Z

        self.rho = prognostic.rho
-        self.q = {s: prognostic.tracer[_STUB_TRACER_INDEX[s]] for s in _SPECIES}
-
+        self.q = {


more generic: do you need this to be a dictionary?
or can this be a TracerState instead so you avoid passing around 6 separate variables with string identifiers. then you unpack it to the individual tracers only when calling the actual muphys.run()

Related to the above and #1301 (comment), but orthogonal to your actual comment about TracerState, I want to note for context: together with formalizing the Component protocol I'd like to formalize the State concept/protocol whatever, so that we don't pass around plain dicts (the State might just be dict-like, but we should give it a name and a shape). My comment needs no further action in this PR, @jcanton's probably does.

msimberg · 2026-06-26T09:41:00Z

Mandatory Tests

Before merging, run the merge pipeline with cscs-ci run merge. Merging is blocked unless this pipeline passes.

When developing, you can test your changes on CSCS CI before merge with the default pipeline: cscs-ci run default. This will run a default subset of tests.

You can pass options to override pipeline variables, for example:
...

Now that the above reminder is updated I'd like to encourage you to try out the new default pipeline to run only tests that you've added in this PR or that you're affecting.

github-actions · 2026-06-26T15:03:05Z

Mandatory Tests

Before merging, run the merge pipeline with cscs-ci run merge. Merging is blocked unless this pipeline passes.

When developing, you can test your changes on CSCS CI before merge with the default pipeline: cscs-ci run default. This will run a default subset of tests.

You can pass options to override pipeline variables, for example:

cscs-ci run default;BACKENDS=gtfn_cpu;LEVELS=unit
cscs-ci run default;MODEL_SUBPACKAGES=common:driver;SESSIONS=model
Avoid running the pipeline for all tests when you are developing.

Available options are:

SESSIONS: model, model_mpi, or tools (correspond to nox sessions)
MODEL_SUBSETS: datatest, basic, or stencils (correspond to nox session selections)
MODEL_SUBPACKAGES: subpackages for non-MPI tests (last component, e.g. diffusion, standalone_driver)
MODEL_MPI_SUBPACKAGES: subpackages for MPI tests (as above)
BACKENDS: backends
GRIDS: grids for stencil tests (simple, icon_regional, or icon_global)
LEVELS: testing level for non-stencil tests (any, unit, or integration)

See scripts/python/generate_ci_pipeline.py and noxfile.py for available values for each option.

The all pipeline can be run with cscs-ci run all. This will run all icon4py tests in CSCS CI which can be expensive. This pipeline runs on a schedule on main, and can be run when extensive validation is needed (e.g. before releases).

Optional Tests

To run benchmarks you can use:

cscs-ci run benchmark-bencher

For more detailed information please look at CI in the EXCLAIM universe.

yiluchen1066 and others added 15 commits June 4, 2026 17:13

first commit: implement physics interface (WIP)

717b3e2

pre commit: adding licence

e0bfe9e

Merge branch 'main' into physics_driver_l2

c38a902

add physics state (MuphysState)

1466c21

add the wrapper for muphys

a3bb29a

rename the muphys granule

66617a4

up + add the datatest for muphys granule wrapper

60c9c6a

fix the test

9f193b9

some small fixes and renaming

47f0aa7

move the muphysState to muphys

b6d166a

move the physics_driver to a physics_interface package

c325c5f

move the physics_driver to a physics_interface package (more)

b035382

small fix

730893f

some small clean ups

f74672b

Merge branch 'main' into physics_driver_l2

2c61eae

yiluchen1066 marked this pull request as ready for review June 11, 2026 10:01

yiluchen1066 requested review from egparedes, havogt, jcanton and msimberg June 11, 2026 10:01

msimberg reviewed Jun 11, 2026

View reviewed changes

msimberg mentioned this pull request Jun 12, 2026

feat(standalone_driver): single-node NetCDF/UGRID output via the common io module #1307

Merged

yiluchen1066 and others added 2 commits June 12, 2026 15:38

solve the inplace update in muphys, input_properties and some mypy stuff

e720c40

Merge branch 'main' into physics_driver_l2

4722175