Skip to content

Deduplicate ExperimentConfig/read_config; move initial_condition to common#1353

Open
jcanton wants to merge 2 commits into
mainfrom
deduplicate_read_config
Open

Deduplicate ExperimentConfig/read_config; move initial_condition to common#1353
jcanton wants to merge 2 commits into
mainfrom
deduplicate_read_config

Conversation

@jcanton

@jcanton jcanton commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

from a friend:

Closes #1320

What

ExperimentConfig and the Fortran-namelist config reader were duplicated between standalone_driver and testing, each carrying a # NOTE: duplicate ... to avoid circular imports marker. The duplication existed because testing depended on standalone_driver (for DriverConfig/InitialConditionConfig) — the wrong direction, since the standalone driver should sit at the top of the dependency graph with nothing depending on it.

Changes

  • New icon4py.model.experiment_config package holding the single ExperimentConfig, DriverConfig, and the shared read_experiment_config reader. It sits above the atmosphere packages but below both testing and standalone_driver, so neither depends on the other — the testing -> standalone_driver edge is gone.
  • standalone_driver.config.read_config and testing.datatest_utils.create_experiment_configuration are now thin wrappers over the shared reader.
  • Moved the initial_condition subpackage to common/initial_condition/, mirroring common/topography/, so the IC config dataclasses stay next to their compute functions. This required relocating StaticFieldFactories (a NamedTuple of common factories) to common/states/static_fields.py.
  • Dropped the now-unused serialbox4py dependency from standalone_driver (its only user moved to common, which already depends on it).
  • Removed the now-resolved # type: ignore[arg-type] / TODO(1320) workarounds in the standalone-driver initial-condition tests, since both sides now share a single ExperimentConfig type.

…ommon

`ExperimentConfig` and the Fortran-namelist reader were duplicated between
`standalone_driver` and `testing`, each marked "duplicate ... to avoid circular
imports". The duplication existed because `testing` depended on
`standalone_driver` (for `DriverConfig`/`InitialConditionConfig`), which is the
wrong direction: the standalone driver should sit at the top of the dependency
graph, with nothing depending on it.

Changes:
- New `icon4py.model.experiment_config` package holds the single `ExperimentConfig`,
  `DriverConfig`, and the shared `read_experiment_config` reader. It sits above the
  atmosphere packages but below both `testing` and `standalone_driver`, so neither
  depends on the other. The `testing -> standalone_driver` edge is removed.
- `standalone_driver.config.read_config` and
  `testing.datatest_utils.create_experiment_configuration` become thin wrappers
  over the shared reader.
- Move the `initial_condition` subpackage to `common/initial_condition/`, mirroring
  `common/topography/`, so the IC configs stay next to their compute functions.
  This required moving `StaticFieldFactories` (a NamedTuple of common factories)
  to `common/states/static_fields.py`.
- Drop the now-unused `serialbox4py` dependency from `standalone_driver` (its only
  user moved to `common`, which already depends on it).
- Remove the now-resolved `# type: ignore[arg-type]` / `TODO(1320)` workarounds in
  the standalone driver initial-condition tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jcanton jcanton requested review from DropD and msimberg June 26, 2026 19:24
@jcanton

jcanton commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

@DropD I thought of getting this one in before it risks becoming annoying for your config PRs, hopefully it doesn't annoy you now, but if so we can postpone it to a later time

@github-actions

Copy link
Copy Markdown

Mandatory Tests

Before merging, run the merge pipeline with cscs-ci run merge. Merging is blocked unless this pipeline passes.

When developing, you can test your changes on CSCS CI before merge with the default pipeline: cscs-ci run default. This will run a default subset of tests.

You can pass options to override pipeline variables, for example:

  • cscs-ci run default;BACKENDS=gtfn_cpu;LEVELS=unit
  • cscs-ci run default;MODEL_SUBPACKAGES=common:driver;SESSIONS=model
    Avoid running the pipeline for all tests when you are developing.

Available options are:

  • SESSIONS: model, model_mpi, or tools (correspond to nox sessions)
  • MODEL_SUBSETS: datatest, basic, or stencils (correspond to nox session selections)
  • MODEL_SUBPACKAGES: subpackages for non-MPI tests (last component, e.g. diffusion, standalone_driver)
  • MODEL_MPI_SUBPACKAGES: subpackages for MPI tests (as above)
  • BACKENDS: backends
  • GRIDS: grids for stencil tests (simple, icon_regional, or icon_global)
  • LEVELS: testing level for non-stencil tests (any, unit, or integration)

See scripts/python/generate_ci_pipeline.py and noxfile.py for available values for each option.

The all pipeline can be run with cscs-ci run all. This will run all icon4py tests in CSCS CI which can be expensive. This pipeline runs on a schedule on main, and can be run when extensive validation is needed (e.g. before releases).

Optional Tests

To run benchmarks you can use:

  • cscs-ci run benchmark-bencher

For more detailed information please look at CI in the EXCLAIM universe.

@jcanton

jcanton commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

cscs-ci run default;BACKENDS=gtfn_cpu;LEVELS=unit:integration

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes duplicated experiment-configuration code by introducing a shared icon4py.model.experiment_config package, and relocates initial-condition functionality into icon4py.model.common to fix an inverted dependency (testing -> standalone_driver) in the monorepo.

Changes:

  • Add model/experiment_config with the canonical ExperimentConfig/DriverConfig and a shared read_experiment_config(...) reader; update tach.toml, workspace deps, and mypy paths accordingly.
  • Replace duplicated readers in standalone_driver.config.read_config and testing.datatest_utils.create_experiment_configuration with thin wrappers over the shared reader.
  • Move initial-condition code (and StaticFieldFactories) into icon4py.model.common, and update standalone-driver code/tests to import from the new locations.

Reviewed changes

Copilot reviewed 21 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
uv.lock Adds workspace member + dependency for icon4py-experiment-config; removes standalone-driver’s direct serialbox4py dep.
tach.toml Adds icon4py.model.experiment_config module + adjusts dependency edges for testing and standalone_driver.
pyproject.toml Registers new workspace member, dependency, and mypy/pytest paths; updates mypy override for moved IC package.
model/testing/src/icon4py/model/testing/definitions.py Removes duplicated ExperimentConfig and imports the shared one.
model/testing/src/icon4py/model/testing/datatest_utils.py Replaces local config assembly with read_experiment_config(...).
model/standalone_driver/tests/standalone_driver/mpi_tests/test_parallel_initial_conditions.py Updates IC imports and removes now-unneeded type-ignore/TODO.
model/standalone_driver/tests/standalone_driver/integration_tests/test_initial_conditions.py Updates IC imports and removes now-unneeded type-ignore/TODO.
model/standalone_driver/src/icon4py/model/standalone_driver/standalone_driver.py Switches IC imports to icon4py.model.common.initial_condition.
model/standalone_driver/src/icon4py/model/standalone_driver/driver_states.py Moves StaticFieldFactories to common and imports it from there.
model/standalone_driver/src/icon4py/model/standalone_driver/config.py Re-exports shared config types and delegates read_config(...) to shared reader.
model/standalone_driver/pyproject.toml Drops direct serialbox4py dependency.
model/experiment_config/src/icon4py/model/experiment_config/reader.py New shared Fortran-namelist config reader assembling ExperimentConfig.
model/experiment_config/src/icon4py/model/experiment_config/config.py New shared config dataclasses and type aliases for experiments/driver.
model/experiment_config/src/icon4py/model/experiment_config/init.py New package metadata/version module.
model/experiment_config/src/icon4py/model/experiment_config/py.typed Marks package as typed.
model/experiment_config/README.md New package documentation.
model/experiment_config/pyproject.toml New package build metadata and dependencies.
model/common/src/icon4py/model/common/topography/from_file.py Updates documentation reference due to IC move.
model/common/src/icon4py/model/common/states/static_fields.py New home for StaticFieldFactories.
model/common/src/icon4py/model/common/initial_condition/from_file.py New file-based initial-condition implementation under common.
model/common/src/icon4py/model/common/initial_condition/config.py Updates IC config + dispatcher to use new common IC subpackage layout.
model/common/src/icon4py/model/common/initial_condition/analytical/utils.py Adds shared analytical-IC helper utilities under common.
model/common/src/icon4py/model/common/initial_condition/analytical/jablonowski_williamson.py Updates imports/types to use common IC utilities + StaticFieldFactories.
model/common/src/icon4py/model/common/initial_condition/analytical/gauss3d.py Updates imports/types to use common IC utilities + StaticFieldFactories.
model/common/src/icon4py/model/common/initial_condition/analytical/init.py Adds package init for analytical IC submodule.
model/common/src/icon4py/model/common/initial_condition/init.py Re-exports InitialConditionConfig and create from the new common location.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 51 to 55
ignored so that this function shares the same call signature as the
analytical initial-condition functions (``jablonowski_williamson``,
``gauss3d``, …) and can be stored transparently in
:attr:`~icon4py.model.standalone_driver.initial_condition.InitialConditionConfig.create`.
:attr:`~icon4py.model.common.initial_condition.create`.
"""
Comment on lines +8 to +11
- `ExperimentConfig` / `DriverConfig` / `ProfilingStats` (`config.py`)
- the initial-condition configuration dataclasses (`initial_condition_config.py`)
- `read_experiment_config(...)`, which assembles an `ExperimentConfig` from a directory
of serialized Fortran namelists (`reader.py`)
@jcanton jcanton requested a review from havogt June 26, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

remove the duplication of ExperimentConfig and the readers

2 participants