Skip to content

Add system test for KPI integration in simulator workflow#175

Merged
jarsarasty merged 2 commits into
mainfrom
system-test-for-KPI-simulator-integration
Jun 17, 2026
Merged

Add system test for KPI integration in simulator workflow#175
jarsarasty merged 2 commits into
mainfrom
system-test-for-KPI-simulator-integration

Conversation

@jarsarasty

@jarsarasty jarsarasty commented Mar 19, 2026

Copy link
Copy Markdown
Contributor
  • Update docker-compose.yml to use simulator-worker 0.0.30beta2 (kpi-calculator 0.4.2 integration; 0.0.29 was never released as stable)
  • Add test__simulator__kpis_present_in_output asserting KPIs are present in instance[0].area after a full pipeline run, with CAPEX verified against the ESDL costInformation and system_lifetime exercised with a non-default value (25 years)
  • Add EXCLUDE_KPIS path set and pass it to compare_esdl in existing simulator tests so KPI churn does not break snapshot comparisons
  • Increase simulator test timeouts from 60s to 120s (100s to 160s for the delete test): KPI calculation runs after each simulation, adding ~30-60s per job

@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch 2 times, most recently from 107ce00 to 9eccd6f Compare March 19, 2026 09:06
@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch 2 times, most recently from c0e145e to 4a46805 Compare March 20, 2026 08:01
@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch 2 times, most recently from a7d52f7 to 487ce99 Compare April 13, 2026 09:30
@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch from 487ce99 to 9093540 Compare May 12, 2026 12:38
- Add test__simulator__kpis_present_in_output asserting KPIs are present
  in instance[0].area after a full pipeline run, with CAPEX verified
  against the ESDL costInformation and system_lifetime exercised with a
  non-default value (25 years)
- Update docker-compose.yml to use simulator-worker 0.0.30beta2
  (kpi-calculator 0.4.2 integration; 0.0.29 was never released as stable)
- Add EXCLUDE_KPIS path set and pass it to compare_esdl in existing
  simulator tests so KPI churn does not break snapshot comparisons
- Increase simulator test timeouts to 120s to account for KPI calculation
  time added after simulation
@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch 4 times, most recently from b8f1bf2 to 9c3b6d3 Compare May 12, 2026 14:47
@jarsarasty jarsarasty requested a review from MarkTNO May 21, 2026 12:10

@MarkTNO MarkTNO left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tnx for adding the test! I added a couple of comments.
I am surprised that the timeout needs to be increased by that much. KPI's calculations should be very quick?

Comment thread system_tests/src/test_workflows_steps.py Outdated
Comment thread docker-compose.yml Outdated
@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch from 9c3b6d3 to a8b4d45 Compare May 28, 2026 07:07
@jarsarasty

Copy link
Copy Markdown
Contributor Author

Tnx for adding the test! I added a couple of comments. I am surprised that the timeout needs to be increased by that much. KPI's calculations should be very quick?

Thanks for your review, @MarkTNO!

You're right that KPI calculation itself is fast. After investigating the logs, the extra time is mainly due to overhead from verbose logging in both the simulator core and the kpi-calculator, as well as multiple database queries per job in the kpi-calculator. We will address this in future releases. For now the increased timeout gives us enough headroom to run the tests.

- Replace simulator_ates_short_run.esdl with test_ates.esdl from simulator-worker
  to fix negative mass flow caused by reversed ATES pipe directions
- Update snapshot test__simulator__multiple_ates_run.esdl to match new ESDL output
- Improve KPI assertions in test__simulator__kpis_present_in_output: check exact
  KPI name set, assert CAPEX/OPEX values against ESDL cost data, and verify
  Production/Demand energy entries are positive
- Update docker-compose.yml to use simulator-worker 0.0.30
@jarsarasty jarsarasty force-pushed the system-test-for-KPI-simulator-integration branch from a8b4d45 to 19d0df1 Compare June 17, 2026 12:28
@jarsarasty jarsarasty merged commit c94bfc3 into main Jun 17, 2026
1 check passed
@jarsarasty jarsarasty deleted the system-test-for-KPI-simulator-integration branch June 17, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants