refactor: change asset topology setup#518
Conversation
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
There was a problem hiding this comment.
Pull request overview
This PR refactors the asset-topology data model usage across the repo to align with the newer “lean topology” representation: Topology now stores station records in raw_stations plus canonical assets/asset_bays, and reconstructs MaterializedStation views via Topology.materialize_stations().
Changes:
- Updated action-set compression/expansion and many solver/importer call sites to operate on
raw_stations+materialize_stations()rather than embedded station assets. - Updated import/export pipelines (pandapower + powsybl/network-graph) to build topologies using
RawStationand canonicalized assets/asset bays. - Refreshed a large portion of tests and fixtures/data files to match the new topology structure.
Reviewed changes
Copilot reviewed 56 out of 56 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/interfaces_pkg/tests/test_stored_action_set.py | Updates stored-action-set tests for RawStation/MaterializedStation usage. |
| packages/interfaces_pkg/tests/test_stored_action_set_performance.py | Updates performance test generator to build topologies from raw_stations and canonical assets. |
| packages/interfaces_pkg/tests/test_asset_topology.py | Adjusts asset-topology tests to new model + adds coverage for station split detection and materialization behavior. |
| packages/interfaces_pkg/tests/test_asset_topology_helpers.py | Updates helper tests to use MaterializedStation and RawStation and adds topology build helpers. |
| packages/interfaces_pkg/src/toop_engine_interfaces/stored_action_set.py | Switches action-set APIs to MaterializedStation and materializes from Topology when needed. |
| packages/interfaces_pkg/src/toop_engine_interfaces/asset_topology_helpers.py | Refactors helper functions to operate on MaterializedStation and adds RawStation support for connected-asset resolution. |
| packages/interfaces_pkg/README.md | Documents new raw_stations + materialize_stations() usage (naming needs adjustment). |
| packages/importer_pkg/tests/pypowsybl_import/test_preprocessing.py | Adds regression test for filtering split stations from relevant_subs. |
| packages/importer_pkg/tests/pandapower/test_preprocessing_pandapower.py | Updates pandapower preprocessing test to use a real Topology instance. |
| packages/importer_pkg/tests/pandapower/test_asset_topology_pandapower.py | Updates pandapower asset-topology tests for MaterializedStation/raw_stations. |
| packages/importer_pkg/tests/network_graph/test_powsybl_station_to_graph.py | Updates network-graph tests for MaterializedStation and materialized station counts. |
| packages/importer_pkg/tests/exporter/test_asset_topology_to_ucte.py | Updates exporter tests to use materialized stations (one test needs topology rebuild after mutation). |
| packages/importer_pkg/src/toop_engine_importer/pypowsybl_import/preprocessing.py | Filters split stations out of relevant_subs and reorders preprocessing steps to apply the filter before saving masks/N-1 definition. |
| packages/importer_pkg/src/toop_engine_importer/pypowsybl_import/init.py | Renames exported API symbol to get_raw_stations_and_assets. |
| packages/importer_pkg/src/toop_engine_importer/pandapower_import/preprocessing.py | Updates preprocess_net_step2 to rewrite raw_stations immutably via copy_topology_with_updates. |
| packages/importer_pkg/src/toop_engine_importer/pandapower_import/asset_topology.py | Builds Topology(raw_stations, assets, asset_bays) by canonicalizing station-derived payloads. |
| packages/importer_pkg/src/toop_engine_importer/network_graph/powsybl_station_to_graph.py | Produces Topology(raw_stations, assets, asset_bays) from materialized station extraction; adds busbar expansion logic. |
| packages/importer_pkg/src/toop_engine_importer/exporter/asset_topology_to_ucte.py | Exports using asset_topology.materialize_stations() and updates type hints to MaterializedStation. |
| packages/grid_helpers_pkg/tests/powsybl/test_powsybl_asset_topo.py | Updates powsybl helper tests for RawStation + canonical assets extraction. |
| packages/grid_helpers_pkg/src/toop_engine_grid_helpers/powsybl/powsybl_asset_topo.py | Refactors station extraction to return (raw_stations, topology_assets) and updates topology construction accordingly. |
| packages/grid_helpers_pkg/src/toop_engine_grid_helpers/powsybl/example_grids.py | Updates extracted topology generation to populate raw_stations, assets, and asset_bays. |
| packages/dc_solver_pkg/tests/test_example_grids.py | Updates assertions to use materialize_stations() counts. |
| packages/dc_solver_pkg/tests/preprocessing/test_write_aux_data.py | Switches monitored element counting to raw_stations. |
| packages/dc_solver_pkg/tests/preprocessing/test_realize_assignment.py | Updates station model references to MaterializedStation. |
| packages/dc_solver_pkg/tests/preprocessing/test_preprocess.py | Updates preprocessing tests to use raw_stations and topology copy helpers. |
| packages/dc_solver_pkg/tests/preprocessing/test_preprocess_switching.py | Updates switching preprocessing tests to use MaterializedStation and materialization calls. |
| packages/dc_solver_pkg/tests/preprocessing/test_preprocess_bb_outage.py | Updates outage preprocessing tests for MaterializedStation and realized-station usage. |
| packages/dc_solver_pkg/tests/postprocessing/test_apply_asset_topo_powsybl.py | Updates postprocess tests to use materialize_stations() and raw-station based topology copies. |
| packages/dc_solver_pkg/tests/postprocessing/test_apply_asset_topo_pandapower.py | Updates pandapower apply tests to use materialize_stations(). |
| packages/dc_solver_pkg/tests/jax/test_busbar_outage.py | Refactors busbar-outage tests to use realized stations instead of deprecated assignment helpers. |
| packages/dc_solver_pkg/tests/export/test_station_switch_updates.py | Updates export tests to use materialized stations and raw-station topology copies. |
| packages/dc_solver_pkg/tests/export/test_export.py | Updates export tests to use materialized stations and raw-station topology copies. |
| packages/dc_solver_pkg/tests/export/test_disconnection_switch_updates.py | Updates disconnection export tests to use materialize_stations() and raw-station topology copies. |
| packages/dc_solver_pkg/tests/export/test_asset_topology_to_dgs.py | Updates DGS export tests to operate on materialized stations. |
| packages/dc_solver_pkg/tests/deprecated/assignment.py | Renames station type usage to MaterializedStation. |
| packages/dc_solver_pkg/tests/conftest.py | Updates fixtures to build MaterializedStation objects. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/preprocess.py | Builds simplified topology via topology_from_materialized_stations and switches loops to materialize_stations(). |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/preprocess_switching.py | Canonicalizes asset metadata updates via topology-level updates (copy_topology_with_updates). |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/preprocess_station_realisations.py | Updates injection aggregation to resolve assets from RawStation via topology-owned assets. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/preprocess_bb_outage.py | Updates outage preprocessing to use materialized stations from topology. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/network_data.py | Updates network-data validation + relevant-station selection to operate on raw_stations and MaterializedStation realizations. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/action_set.py | Updates injection-topology computation to resolve assets from RawStation with topology assets. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/postprocess/realize_assignment.py | Updates postprocessing realization typing and validation to MaterializedStation. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/postprocess/postprocess_powsybl.py | Builds changed-station topology via topology_from_materialized_stations and fixes station-id lookup via raw_stations. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/postprocess/postprocess_pandapower.py | Switches realized-station type to AppliedStation and constructs realized topology via topology_from_materialized_stations. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/postprocess/apply_asset_topo_powsybl.py | Refactors powsybl application to accept (Topology, RawStation) and materialize internally when needed. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/postprocess/apply_asset_topo_pandapower.py | Updates pandapower apply path to use MaterializedStation + AppliedStation. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/export/station_switch_updates.py | Updates switching-diff export to use MaterializedStation and materialize_stations() for baselines. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/export/export.py | Updates export entry point typing and changed-station resolution for MaterializedStation. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/export/disconnection_switch_updates.py | Resolves disconnections against materialize_stations() output rather than topology.stations. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/example_grids.py | Builds random example topologies via topology_from_materialized_stations. |
| packages/dc_solver_pkg/src/toop_engine_dc_solver/example_classes.py | Updates example node-breaker topology JSON construction to new schema (raw_stations, canonical assets/asset_bays). |
| docs/interfaces/asset_topology.md | Updates documentation for the lean-topology model (class names still need alignment). |
| data/test_grid_node_breaker/initial_topology/asset_topology.json | Updates on-disk topology fixture to raw_stations + canonical assets/asset_bays schema. |
Comments suppressed due to low confidence (1)
packages/importer_pkg/tests/exporter/test_asset_topology_to_ucte.py:579
- In this test,
topology_stations = topology_model.materialize_stations()returns stations whoseassetsare deep-copied fromtopology_model.assets. Mutatingtopology_stations[0].assets[1].grid_model_idtherefore does not updatetopology_model, soasset_topo_to_uct(asset_topology=topology_model, ...)won’t see the intended asset-id change. Rebuildtopology_modelfrom the mutated materialized stations before exporting.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: LeonHilf <149764770+LeonHilf@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: LeonHilf <149764770+LeonHilf@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: LeonHilf <149764770+LeonHilf@users.noreply.github.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: LeonHilf <149764770+LeonHilf@users.noreply.github.com>
bpetrickNezk
left a comment
There was a problem hiding this comment.
it is not clear to me how I get the Asset bay for:
- we go to list[StationAssetConnection] e.g. in a station and get the switches
does work, but looks also it can lead to some bugs if the order is wrong? - how do I get the two endpoints of a branch? I did not find this directly
this is also in the StationAssetConnection? So one needs to loop over all connections to get the two that are relevant?
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
|
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
Signed-off-by: Leonard Hilfrich <Leonard.Hilfrich@50Hertz.com>
nicow-elia
left a comment
There was a problem hiding this comment.
Some comments, I will look more into it tomorrow.
There was a problem hiding this comment.
Do we need anything in this file still?
| """Injection disconnections as ``(station_id, injection_index)`` tuples.""" | ||
|
|
||
|
|
||
| class AppliedStation(BaseModel): |
There was a problem hiding this comment.
Maybe call it RealizedStation in line with RealizedTopology?
|
|
||
| return self | ||
|
|
||
| def materialize_stations(self) -> list[MaterializedStation]: |
There was a problem hiding this comment.
I'd prefer it to have in the interface file only the model definition and validators and a heavy translation logic such as this one to live in another file. I would expect this logic to be bug-heavy and often changing, while hopefully the interface definition isn't
| return None | ||
|
|
||
|
|
||
| class MaterializedStation(_StationStructure): |
There was a problem hiding this comment.
So as I see it - the only difference between a raw station and a materialized station is that the materialized station has the AssetBay populated in the AssetConnections. I think it is a bit verbose to introduce a whole new station class for this, I'd rather say that we have an AssetConnection base model and two deriving classes MaterializedAssetConnection and Raw(currently Station)AssetConnection. The Station just has a list of AssetConnections and a property that does an isinstance check to find out if the asset bays have been populated?
This way I was just trying to get through Applied, Materialized and Raw Stations that all describe the same thing and I got lost a bit.




Checklist
Please check if the PR fulfills these requirements:
Does this PR already have an issue describing the problem?
Fixes #
What is the new behavior (if this is a feature change)?
Does this PR introduce a breaking change?