Skip to content

Add multi‑commodity support to device scheduler#1946

Merged
Flix6x merged 129 commits into
mainfrom
feat/multi-commodity
Jul 8, 2026
Merged

Add multi‑commodity support to device scheduler#1946
Flix6x merged 129 commits into
mainfrom
feat/multi-commodity

Conversation

@Ahmad-Wahid

@Ahmad-Wahid Ahmad-Wahid commented Feb 3, 2026

Copy link
Copy Markdown
Contributor

This PR combines all the work done so far on supporting:

The PR description could use a rewrite, but for lack of time I point you to the descriptions of the above PRs.
The original description for this PR follows below, but I want to point out that this chain of PRs did not end up introducing a new gas-price field. Instead, gas prices can be modelled by wrapping whatever existing dict flex-context you have into a list, and extending that list with a second entry as follows:

{
  "flex-context":  [
    <your old dict flex-context>,
    {
      "commodity": "gas",
      "consumption-price": "<your gas price>"
    }
  ]
}

Original description

This PR adds initial support for multi‑commodity commitments (e.g. gas and electricity) in the device_scheduler, building on the grouped‑devices functionality. Flow commitments can now be tagged with a commodity, and EMS‑level flow constraints are applied per commodity by aggregating the relevant devices.

TODO

  • Specify how commodity is encoded per device in the flex‑model
  • Exclude non‑matching commodities (e.g. gas devices) from EMS electricity commitments
  • Extend flex‑model schema with commodity metadata
  • add gas-price field to flex-context
  • set up gas commitment using gas-powered devices and gas-price
  • Update scheduling.rst
  • Add tutorial examples for multi‑commodity scheduling

closes https://github.com/SeitaBV/simulations/issues/987

Flix6x and others added 26 commits December 5, 2025 16:50
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…ontext

Signed-off-by: F.N. Claessen <felix@seita.nl>
…formulation

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…and map them to the respective group id

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…chemas

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Flix6x and others added 4 commits April 26, 2026 15:46
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ce series with scheduler index

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
@nhoening nhoening modified the milestones: 0.33.0, 0.34.0 May 23, 2026
Flix6x added 2 commits June 3, 2026 10:10
…n-gas-and-electricity

# Conflicts:
#	documentation/changelog.rst
…lectricity' into feat/multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

# Conflicts:
#	flexmeasures/data/schemas/scheduling/metadata.py
#	flexmeasures/ui/static/openapi-specs.json
Base automatically changed from feat/switching-between-gas-and-electricity to main June 3, 2026 08:49
Flix6x and others added 3 commits June 3, 2026 10:52
# Conflicts:
#	flexmeasures/data/models/planning/__init__.py
#	flexmeasures/data/models/planning/tests/test_commitments.py
#	flexmeasures/data/tests/test_scheduling_simultaneous.py
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…tion

Context:
- Test test_all_metadata_fields_are_documented was failing because these fields were not documented
- Part of multi-commodity feature development
Change:
- Added ``commodity`` field to the storage flex-model table
- Added ``gas-price`` field to the flex-context table
@Ahmad-Wahid Ahmad-Wahid marked this pull request as ready for review June 25, 2026 09:04
@Ahmad-Wahid Ahmad-Wahid requested a review from Flix6x June 25, 2026 09:04
Ahmad-Wahid and others added 5 commits July 7, 2026 14:37
* feat: add stock-id field in Storage and DB flex model schemas

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: build stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: get stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add a test case for multi feed stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add support for shared storage

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove the breakpoint

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: update the test case for two devices with shared stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add assertions with clear reasons

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Add support for multi-device charging of shared storage

Introduce stock_groups mapping to link multiple devices to a shared SOC.

Aggregate stock delta across devices sharing the same battery.

Update stock change calculation to use combined device flows.

Add device-to-group and group-to-devices lookup for efficient shared stock computation.

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: sum all devices soc contribution, and use individual device efficiencies

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update test case for multi feed stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* expect to charge the battery early to see the effect of fully discharge

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update the assert statements according to the scheduler results

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: first step in resolving merge conflicts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: code annotation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: not all flex-models have sensors

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: static method has no self

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove inapplicable fields for stock model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: fix interpretation of test results

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: move initialization of ems_constraints

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: resolve merge conflicts on _build_soc_schedule, copied from Ahmad

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: remove redundant code block

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: use "state-of-charge" key instead of "sensor" key for stock models

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip StockCommitment for device models that outsource their stock model to a separately modeled device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: old flex models that describe a device that serves both as a feeder and stock are both categorized as device models and stock models

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: model stock devices using the state-of-charge field instead of the sensor field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: identify asset to merge with db flex-model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: validation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: flex-model setup in test

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: create stock group

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* use soc-sensor in case of missing power sensor and also correct stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: create stock model for a model which has itself stock

* update the assert statements

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove stock-id field

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: correct the stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: remove unneccessary test function

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: shared soc-gain, soc-usage, soc-minima and soc-maxima

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: shared StockCommitment for preferring a full SoC

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add "test" test case

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix commodity-level commitments by grouping devices and aligning device series with scheduler index

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: use net energy costs instead of individual device costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: comment out the buggy lines

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: merge conflicts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: add device-model in groups if it's missing

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: restore SOC constraints and state-of-charge handling broken by multi-feed-stock refactor

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: Split flex-context settings by commodity for dynamic capacity scheduling (#2172)

* dev: Support commodity-specific prices and site capacities in storage scheduler

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Add commodity-specific flex-context schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Add dynamic commodity prices and split flex-context settings to capacity scheduling test

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: create a shared schema for flex-context and commodity-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the test case to have inflexible-devices-sensors for each commodity-flex-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: loop over flex-context fields and choose all fields except 'gas-price' for electricity as commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add inflexible-device-sensors to the gas commodity model

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: remove self

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: fall back to deprecated price fields

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: typo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: store commitment costs on job meta

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: clarify which job is which

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test expectation: the battery could save more?

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: price window should match scheduling window

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: comment out unreasoned check

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test expectation; apparently the battery could save more?

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: flake8

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: exclude commodities field from flex-context schema referencing a dedicated issue

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: only save commitment costs on job if we have a job to save it on

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: inflexible devices are electricity devices by default

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: no more need for backwards-compatibility of the temporary gas-price field (only used during development)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: optional dict key

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: keep ems-constraints and fix the test cases (#2233)

* fix: keep ems-constraints and fix the test cases

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Update flexmeasures/data/models/planning/storage.py

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* fix: update the comment and raise value error if ems_constraints_group is not passed

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: only raise in case of multiple EMS constraint DataFrames

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: the wait for marshmallow-code/apispec#999 is over

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: allow any commodity, with electricity and gas serving as examples

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove unreleased flex-context field for gas price

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: add all relaxation fields to the list of fields to ignore when moving old flex-context fields into the electricity commodity context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: gas_price is no longer a field (remove reference to unreleased field)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: just treat the whole old flex-context as the electricity flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: update openapi-specs.json

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: list the commodity field first rather than last

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: commodity is a field in both flex-model and flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: flex-model commodity can also be more than just electricity and gas

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: remove mention of gas-price field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: adjust scheduling section for multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: adjust field descriptions for multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: list the commodity field first rather than last, also in flex-model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: wrong data-key due to wrong indentation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: explain the "commodities" field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: no need for ill-formatted in-line code formatting

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* remove: devices do not have to be storages anymore

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set default flex-context commodity to electricity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: preserve field order in case schema is made OpenAPI compatible

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: default flex-model and flex-context to empty list and empty dict, respectively, because it is possible that the entire flex-config is described in the db instead of the trigger message

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Feature: flex-context per commodity as list (#2235)

* feat: support passing flex-context as a list (one flex-context per commodity)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set default flex-context commodity to electricity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: preserve field order in case schema is made OpenAPI compatible

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: reduce documented nesting when defining a flex-context per commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todos

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: flake8

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* scheduling: complete schema refactoring per PR #2235

- Add SensorReferenceSchema import
- Override relax_constraints default to True in CommodityFlexContextSchema
- Remove duplicate breach price and relax fields from FlexContextSchema
- Remove duplicate set_default_breach_prices method from FlexContextSchema

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* tests: add comprehensive tests for schema refactoring

- Test aggregate-consumption and aggregate-production fields
- Test SharedSchema fields accessible in FlexContextSchema
- Test CommodityFlexContextSchema relax_constraints defaults to True
- Test shared currency logic for flex-context listings
- Test breach prices in both schemas
- Add noqa comment for existing unused variable

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* scheduling: add shared currency validation for commodity contexts

- Add validator to check prices share same currency across all commodity contexts
- Fix test data to use actual fixture sensor names
- All new tests now passing

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* docs: add aggregate fields to documentation and update tests

- Add aggregate-consumption and aggregate-production to scheduling.rst
- Exclude COMMODITY_FLEX_CONTEXT and COMMODITY_FLEX_MODEL from doc test (already documented as "commodity")
- Exclude aggregate fields from UI test (not yet supported in UI)

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* chore: upgrade openapi-specs.json

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: move _try_to_convert_price_units to SharedSchema

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: some fields cannot use source filters; now instead of just having validators in place to forbid that, we actually stop having those fields in the schemas of those fields

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: mypy

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove redundant tests

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: minimize diff

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: default flex-model and flex-context to empty lists, because it is possible that the entire flex-config is described in the db instead of the trigger message (this cherry-pick was adapted, because the flex-context moved from being documented as a dict to a list)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: prefer unique sensor IDs in examples

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: clarify cross-reference

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: UI support for picking a
sensor for aggregate-consumption sensor and/or aggregate-production

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: for some reason, for flex-context fields, the sensor-only property is defined in the html in 3 places

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test coverage for UI support of aggregate-consumption and aggregate-production

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add deprecation instructions for aggregate-power

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: move down the aggregate-power field documentation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* use different compatible units.

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update the comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* chore: update openapi-specs.json again

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add multi feed stock tutorial

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove extra chart explanation

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update flexmeasures version in openapi-specs

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* feat: add multi commodity tutorial

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Apply my own suggestions from code review

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: add comprehensive coverage for aggregate-consumption/production flex-context fields
- Add test_asset_trigger_and_get_aggregate_schedule to verify aggregate sensors get
  populated (currently xfail as StorageScheduler doesn't yet populate them)
- Add test_asset_trigger_with_multi_commodity_flex_context to verify aggregate sensors
  work with multi-commodity flex-context format (list of commodity dicts)
- Document that aggregate sensor population is missing logic in StorageScheduler
- Both tests verify the infrastructure and schema support works correctly

* fix: compute per-commodity aggregate power flows for aggregate-consumption and aggregate-production sensors

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Include per-commodity inflexible devices in aggregate schedules

_compute_commodity_aggregate_schedules only added the top-level inflexible
devices to the electricity commodity, so a commodity's own inflexible demand
(e.g. a heat load) was excluded from its aggregate-consumption schedule. The
aggregate then reflected only that commodity's flexible devices. Mirror the
device enumeration used in _prepare so each commodity context's
inflexible-device-sensors are appended to that commodity at the matching
ems_schedule indices.

* test: update aggregate sensor tests to verify implementation
- Remove xfail markers from both tests now that StorageScheduler populates aggregate sensors
- Update test docstrings to reflect actual functionality
- Tests now verify that aggregate-consumption and aggregate-production sensors receive data
- First test covers single aggregate pair with dual devices
- Second test covers aggregates with multiple devices scheduling together

* feat: test commodities referenced in only the flex-model or only the flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test logs should explain why response was unexpected

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: AssetTriggerSchema accepts list-like flex-contexts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip aggregate computation for unused commodities
When a multi-commodity flex_context includes a commodity (e.g., heat)
but no devices exist for that commodity, the sum() of an empty list
returns integer 0 instead of a pandas Series. This caused AttributeError
when calling .round() on the result later in compute().
Fixed by checking if any devices contribute to the commodity before
attempting to aggregate. If no devices exist for a commodity, skip
creating an aggregate schedule for it.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>
Co-authored-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* docs: prefer storage when talking about the device, and stock when talking about the device state

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: discuss a more realistic case for the multi-feed-storage example; add cross-references between tutorials

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: F.N. Claessen <claessen@seita.nl>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: F.N. Claessen <claessen@seita.nl>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>
# Conflicts:
#	documentation/changelog.rst
#	flexmeasures/api/v3_0/assets.py
#	flexmeasures/data/models/planning/storage.py
#	flexmeasures/ui/static/openapi-specs.json
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…generic_asset_public_root_name_key

Signed-off-by: F.N. Claessen <claessen@seita.nl>
…vices in _prepare

The commodity_to_devices mapping and the num_flexible_devices count
were built from the re-copied flex_model, which includes stock-model
entries. This overran the sensors list (IndexError at sensor_d =
sensors[d]) whenever the flex-model contained a stock entry, and gave
stock entries bogus device indices in commodity groups. Enumerate
device_models instead, keeping the earlier num_flexible_devices =
len(device_models) and matching _compute_commodity_aggregate_schedules,
which already uses self._device_models.

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

# Conflicts:
#	flexmeasures/data/models/planning/storage.py
#	flexmeasures/data/services/scheduling.py
Copilot AI added a commit that referenced this pull request Jul 8, 2026
Context:
- PR #2072 added scheduling_result output to StorageScheduler.compute()
  when return_multiple=True, and added _compute_unresolved_targets.
- In single-sensor mode (self.sensor is set), flex_model entries lack a
  "sensor" key, so _compute_unresolved_targets was skipping every device
  and returning empty unresolved/resolved lists.

Changes:
- storage.py: fall back to self.sensor in _compute_unresolved_targets
  when the flex_model entry has no "sensor" key (single-sensor mode)
- test_commitments.py: update schedule-count assertions (+1 for the new
  scheduling_result entry added by PR #2072)

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
…2274)

Context:
- PR #2072 added scheduling_result output to StorageScheduler.compute()
  when return_multiple=True, and added _compute_unresolved_targets.
- In single-sensor mode (self.sensor is set), flex_model entries lack a
  "sensor" key, so _compute_unresolved_targets was skipping every device
  and returning empty unresolved/resolved lists.

Changes:
- storage.py: fall back to self.sensor in _compute_unresolved_targets
  when the flex_model entry has no "sensor" key (single-sensor mode)
- test_commitments.py: update schedule-count assertions (+1 for the new
  scheduling_result entry added by PR #2072)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

@Flix6x Flix6x left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@Flix6x Flix6x merged commit 003c145 into main Jul 8, 2026
13 checks passed
@Flix6x Flix6x deleted the feat/multi-commodity branch July 8, 2026 15:58
Flix6x added a commit that referenced this pull request Jul 8, 2026
…) ahead of v1 (#2271)

* docs: new section describing commitments

Signed-off-by: F.N. Claessen <felix@seita.nl>

* docs: rewrite the overview in commitments section

Signed-off-by: F.N. Claessen <felix@seita.nl>

* docs: keep ems variables, but explain them as representing the site context

Signed-off-by: F.N. Claessen <felix@seita.nl>

* docs: cross-reference the commitments section and the linear problem formulation

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: cross-references

Signed-off-by: F.N. Claessen <felix@seita.nl>

* docs: add commitments section to index

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feat: add a util function for printing out commitments in a tabulated format

* refactor: move pretty printing method to class

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feat: Commitment supports device groups

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feat: start testing device grouping

Signed-off-by: F.N. Claessen <felix@seita.nl>

* dev: test multi-feed

Signed-off-by: F.N. Claessen <felix@seita.nl>

* update the ids of devices to be integers

* feat: function that group commitment quantities

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add commitments for multi group

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: get unique list of devices for a frame column

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: create util functions that extract devices for a list of values and map them to the respective group id

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: create a series for a list of grouped devices

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* drop outdated comments

* use commitment costs and add asserts for electricity and gas

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* and an assert for commodity costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add an extra assert on costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* support commodity based EMS flow commitments and grouped devices

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Add commodity field and support multi-device commitments

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* test shared buffer and multi-comodity commitments

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove hard check for commodity to make backward compatible

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the function to support backward compatibility

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add commodity field to the flexmodel and DBstorage-flex-model schemas

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: use devices as index rather than time series

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: exclude gas-power devices from electricity commitments

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add gas-price field to the Flex-context schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* apply black

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add a test case for two flexible devices with commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* use expected datatypes

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: split commitments per commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: split commitments per commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Revert "use expected datatypes"

This reverts commit b22c6d7.

* feat: add a test case for different commodities

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: do not produce gas

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: create a flow commitment for prefering to charge sooner devices

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add soc constraints for boiler

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add some assert statments

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update and add new assertions with clear explanation

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the docstring

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: move tiny-price-slope decleration out of the for loop

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Revert "refactor: move tiny-price-slope decleration out of the for loop"

This reverts commit 2becd02.

* refactor: move tiny-price-slope decleration out of the for loop

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add data_key attr

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add missing commodity description and it's field in ui flexmodel schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add missing gas-price field in UI Flexcontext schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: wrong timezone; the test relied on the preference to charge sooner and discharge later, rather than on the EPEX price transition, as the inline test documentation advertised

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: move preference to charge sooner and discharge later into a StockCommitment to prefer being full

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: test case no longer relies on arbitrage opportunity coming from artificial price slope

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: check for optimal schedule

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: prefer a full storage earlier over later

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: update commitment name and inline comments

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: touch up test explanation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test case given preference for a full battery

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: clean up comment

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: model the preference to curtail later within the same StockCommitment, using a tiny price slope to prefer a fuller SoC sooner rather than later, by lowering penalties later

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: reduce tiny price slope

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: delete duplicate changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: fix broken link

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Revert "fix: reduce tiny price slope"

This reverts commit bf16e63.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: soc unit conversion

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: adapt test to check for 1 hour of free energy at 15-min scheduling resolution

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: check curtailment preference per distinct device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set tight tolerance for HiGHS solver

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: merge if-blocks

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test_two_flexible_assets_with_commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: check curtailment preference per distinct device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set tight tolerance for HiGHS solver

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: merge if-blocks

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: use iloc

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: diminish tiny price slope by number of planning steps

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: always diminish tiny price slope by number of planning steps, such that its relative weight does not grow with the number of steps

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: increment StorageScheduler version

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: use approximation to compare battery and heat pump costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the assert statements with prefered to charge battery sooner than later

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update the expected ev and battery costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add device id to get costs for the given device

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: rewrite test to not rely on multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: rewrite test to prove that only when both HPs share a single commitment does the optimizer treat their stocks as a combined resource

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: do not coerce device_group into a time series

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: pretty_print is not specifically for FlowCommitments

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add (back) inline dev notes

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: strengthen asserts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add check for exact electricity costs expected

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: sum over electricity costs; and move to StockCommitment to model preference for full soc sooner

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: replace vague assert with explicit asserts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: lose confusing comment

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: backwards compatibility in case no device is specified;

The two changes together are the backwards-compatible default that was missing: when no device is specified (device=None), to_frame() must emit NaN in the device_group column (not crash), so the optimizer routes the commitment through ems_flow_commitment_equalities exactly as it did before device_group was introduced.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: when device is present but device_group is absent, fall back to the pre-existing behaviour — each device is its own group

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: update expectations of how costs are shared between EV and battery

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix commodity-level commitments by grouping devices and aligning device series with scheduler index

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the test cases for net commodity consumption and production

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update the test cases according device level costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* docs/scheduling: add COMMODITY and GAS_PRICE metadata field documentation
Context:
- Test test_all_metadata_fields_are_documented was failing because these fields were not documented
- Part of multi-commodity feature development
Change:
- Added ``commodity`` field to the storage flex-model table
- Added ``gas-price`` field to the flex-context table

* Feat: Support shared storage (multi‑feed stock) (#2001)

* feat: add stock-id field in Storage and DB flex model schemas

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: build stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: get stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add a test case for multi feed stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add support for shared storage

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove the breakpoint

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: update the test case for two devices with shared stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add assertions with clear reasons

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Add support for multi-device charging of shared storage

Introduce stock_groups mapping to link multiple devices to a shared SOC.

Aggregate stock delta across devices sharing the same battery.

Update stock change calculation to use combined device flows.

Add device-to-group and group-to-devices lookup for efficient shared stock computation.

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: sum all devices soc contribution, and use individual device efficiencies

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update test case for multi feed stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* expect to charge the battery early to see the effect of fully discharge

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update the assert statements according to the scheduler results

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: first step in resolving merge conflicts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: code annotation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: not all flex-models have sensors

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: static method has no self

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove inapplicable fields for stock model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: fix interpretation of test results

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: move initialization of ems_constraints

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: resolve merge conflicts on _build_soc_schedule, copied from Ahmad

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: remove redundant code block

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: use "state-of-charge" key instead of "sensor" key for stock models

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip StockCommitment for device models that outsource their stock model to a separately modeled device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: old flex models that describe a device that serves both as a feeder and stock are both categorized as device models and stock models

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: model stock devices using the state-of-charge field instead of the sensor field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: identify asset to merge with db flex-model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: validation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: flex-model setup in test

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: create stock group

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* use soc-sensor in case of missing power sensor and also correct stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: create stock model for a model which has itself stock

* update the assert statements

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove stock-id field

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: correct the stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: remove unneccessary test function

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: shared soc-gain, soc-usage, soc-minima and soc-maxima

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: shared StockCommitment for preferring a full SoC

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add "test" test case

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix commodity-level commitments by grouping devices and aligning device series with scheduler index

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: use net energy costs instead of individual device costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: comment out the buggy lines

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: merge conflicts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: add device-model in groups if it's missing

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: restore SOC constraints and state-of-charge handling broken by multi-feed-stock refactor

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: Split flex-context settings by commodity for dynamic capacity scheduling (#2172)

* dev: Support commodity-specific prices and site capacities in storage scheduler

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Add commodity-specific flex-context schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Add dynamic commodity prices and split flex-context settings to capacity scheduling test

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: create a shared schema for flex-context and commodity-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the test case to have inflexible-devices-sensors for each commodity-flex-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: loop over flex-context fields and choose all fields except 'gas-price' for electricity as commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add inflexible-device-sensors to the gas commodity model

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: remove self

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: fall back to deprecated price fields

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: typo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: store commitment costs on job meta

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: clarify which job is which

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test expectation: the battery could save more?

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: price window should match scheduling window

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: comment out unreasoned check

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test expectation; apparently the battery could save more?

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: flake8

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: exclude commodities field from flex-context schema referencing a dedicated issue

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: only save commitment costs on job if we have a job to save it on

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: inflexible devices are electricity devices by default

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: no more need for backwards-compatibility of the temporary gas-price field (only used during development)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: optional dict key

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: keep ems-constraints and fix the test cases (#2233)

* fix: keep ems-constraints and fix the test cases

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Update flexmeasures/data/models/planning/storage.py

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* fix: update the comment and raise value error if ems_constraints_group is not passed

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: only raise in case of multiple EMS constraint DataFrames

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: the wait for marshmallow-code/apispec#999 is over

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: allow any commodity, with electricity and gas serving as examples

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove unreleased flex-context field for gas price

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: add all relaxation fields to the list of fields to ignore when moving old flex-context fields into the electricity commodity context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: gas_price is no longer a field (remove reference to unreleased field)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: just treat the whole old flex-context as the electricity flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: update openapi-specs.json

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: list the commodity field first rather than last

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: commodity is a field in both flex-model and flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: flex-model commodity can also be more than just electricity and gas

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: remove mention of gas-price field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: adjust scheduling section for multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: adjust field descriptions for multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: list the commodity field first rather than last, also in flex-model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: wrong data-key due to wrong indentation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: explain the "commodities" field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: no need for ill-formatted in-line code formatting

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* remove: devices do not have to be storages anymore

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set default flex-context commodity to electricity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: preserve field order in case schema is made OpenAPI compatible

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: default flex-model and flex-context to empty list and empty dict, respectively, because it is possible that the entire flex-config is described in the db instead of the trigger message

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Feature: flex-context per commodity as list (#2235)

* feat: support passing flex-context as a list (one flex-context per commodity)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set default flex-context commodity to electricity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: preserve field order in case schema is made OpenAPI compatible

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: reduce documented nesting when defining a flex-context per commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todos

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: flake8

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* scheduling: complete schema refactoring per PR #2235

- Add SensorReferenceSchema import
- Override relax_constraints default to True in CommodityFlexContextSchema
- Remove duplicate breach price and relax fields from FlexContextSchema
- Remove duplicate set_default_breach_prices method from FlexContextSchema

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* tests: add comprehensive tests for schema refactoring

- Test aggregate-consumption and aggregate-production fields
- Test SharedSchema fields accessible in FlexContextSchema
- Test CommodityFlexContextSchema relax_constraints defaults to True
- Test shared currency logic for flex-context listings
- Test breach prices in both schemas
- Add noqa comment for existing unused variable

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* scheduling: add shared currency validation for commodity contexts

- Add validator to check prices share same currency across all commodity contexts
- Fix test data to use actual fixture sensor names
- All new tests now passing

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* docs: add aggregate fields to documentation and update tests

- Add aggregate-consumption and aggregate-production to scheduling.rst
- Exclude COMMODITY_FLEX_CONTEXT and COMMODITY_FLEX_MODEL from doc test (already documented as "commodity")
- Exclude aggregate fields from UI test (not yet supported in UI)

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* chore: upgrade openapi-specs.json

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: move _try_to_convert_price_units to SharedSchema

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: some fields cannot use source filters; now instead of just having validators in place to forbid that, we actually stop having those fields in the schemas of those fields

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: mypy

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove redundant tests

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: minimize diff

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: default flex-model and flex-context to empty lists, because it is possible that the entire flex-config is described in the db instead of the trigger message (this cherry-pick was adapted, because the flex-context moved from being documented as a dict to a list)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: prefer unique sensor IDs in examples

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: clarify cross-reference

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: UI support for picking a
sensor for aggregate-consumption sensor and/or aggregate-production

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: for some reason, for flex-context fields, the sensor-only property is defined in the html in 3 places

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test coverage for UI support of aggregate-consumption and aggregate-production

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add deprecation instructions for aggregate-power

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: move down the aggregate-power field documentation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* use different compatible units.

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update the comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* chore: update openapi-specs.json again

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add multi feed stock tutorial

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove extra chart explanation

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update flexmeasures version in openapi-specs

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* feat: add multi commodity tutorial

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Apply my own suggestions from code review

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: add comprehensive coverage for aggregate-consumption/production flex-context fields
- Add test_asset_trigger_and_get_aggregate_schedule to verify aggregate sensors get
  populated (currently xfail as StorageScheduler doesn't yet populate them)
- Add test_asset_trigger_with_multi_commodity_flex_context to verify aggregate sensors
  work with multi-commodity flex-context format (list of commodity dicts)
- Document that aggregate sensor population is missing logic in StorageScheduler
- Both tests verify the infrastructure and schema support works correctly

* fix: compute per-commodity aggregate power flows for aggregate-consumption and aggregate-production sensors

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Include per-commodity inflexible devices in aggregate schedules

_compute_commodity_aggregate_schedules only added the top-level inflexible
devices to the electricity commodity, so a commodity's own inflexible demand
(e.g. a heat load) was excluded from its aggregate-consumption schedule. The
aggregate then reflected only that commodity's flexible devices. Mirror the
device enumeration used in _prepare so each commodity context's
inflexible-device-sensors are appended to that commodity at the matching
ems_schedule indices.

* test: update aggregate sensor tests to verify implementation
- Remove xfail markers from both tests now that StorageScheduler populates aggregate sensors
- Update test docstrings to reflect actual functionality
- Tests now verify that aggregate-consumption and aggregate-production sensors receive data
- First test covers single aggregate pair with dual devices
- Second test covers aggregates with multiple devices scheduling together

* feat: test commodities referenced in only the flex-model or only the flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test logs should explain why response was unexpected

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: AssetTriggerSchema accepts list-like flex-contexts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip aggregate computation for unused commodities
When a multi-commodity flex_context includes a commodity (e.g., heat)
but no devices exist for that commodity, the sum() of an empty list
returns integer 0 instead of a pandas Series. This caused AttributeError
when calling .round() on the result later in compute().
Fixed by checking if any devices contribute to the commodity before
attempting to aggregate. If no devices exist for a commodity, skip
creating an aggregate schedule for it.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>
Co-authored-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* docs: prefer storage when talking about the device, and stock when talking about the device state

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: discuss a more realistic case for the multi-feed-storage example; add cross-references between tutorials

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: F.N. Claessen <claessen@seita.nl>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: F.N. Claessen <claessen@seita.nl>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix(tests): unique public asset names in commitment tests to satisfy generic_asset_public_root_name_key

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: consolidate currency validation across commodity flex-contexts

Replace the broken split("/")-based cross-commodity currency validator
(which skipped Sensor-valued prices and never actually normalized units
correctly) with a robust comparison of each commodity context's already
normalized shared_currency_unit, computed by the inherited
_try_to_convert_price_units validator. Also cross-check the top-level
flex-context's currency against per-commodity contexts, and remove the
now-redundant relax-constraints override on CommodityFlexContextSchema.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat!: default top-level relax-constraints to True

Flip the default of SharedSchema.relax_constraints (used by the
top-level FlexContextSchema) to True, matching the default already used
for per-commodity CommodityFlexContextSchema entries. This changes
behaviour for existing single-commodity users who did not explicitly
set relax-constraints: constraints are now relaxed (with default breach
prices) by default. See changelog for details.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: support per-commodity inflexible-device-sensors and all-gas portfolios

_prepare() only read the top-level inflexible-device-sensors key, so
sensors declared inside a commodity context got no device constraints
and did not count against that commodity's site capacity, even though
_compute_commodity_aggregate_schedules already assumed they were
enumerated. Collect and constrain each commodity context's inflexible
sensors too, appended after the top-level (electricity) ones, in
listed order, matching that enumeration. Extract the shared
commodity-to-device-index reconstruction into
_reconstruct_commodity_to_devices()/_electricity_device_indices() to
avoid duplicating this logic.

Also switch commodity_to_devices["electricity"] += ... to setdefault(),
so a flex-model without any electricity device (e.g. all-gas) no longer
raises a KeyError.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: default device commodity to electricity and match commitment commodity

flex_model_d["commodity"] raised a KeyError for devices without an
explicit commodity; default to "electricity" instead. Also only apply a
commitment spec to devices of a matching commodity (defaulting the
commitment's commodity to "electricity" too), so a future
CommitmentSchema commodity field can be wired through without further
changes here.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: fix dead cross-commodity currency check in list-form flex-context

shared_currency_unit was assigned before the is-None check on every
iteration, so the mismatch branch could never fire when loading the
list-of-dicts flex-context form. Compare against the previously seen
currency instead.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: restrict aggregate-power to electricity devices

aggregate-power previously summed every device's schedule regardless of
commodity, mixing e.g. gas devices into an electricity aggregate. Sum
only the flexible and inflexible devices belonging to the electricity
commodity, per decision. Also drop the resolved todo comment and the
unused commented-out sensors parameter of
_compute_commodity_aggregate_schedules and its call site.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: reject duplicate commodities in flex-context commodities list

_get_commodity_contexts (storage.py) keys commodity contexts by
commodity, so a duplicate entry in the `commodities` list silently
overwrote an earlier one. Reject duplicates explicitly.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: reject non-electricity semantics in the single-dict flex-context

A single flex-context dict only supports the electricity commodity.
Reject a top-level `commodity` key with any value other than
"electricity", and reject the case where the `commodities` list is
combined with commodity-specific top-level SharedSchema fields, since
it is then ambiguous which commodity those fields belong to.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: raise ValidationError for malformed flex-context input

Extend the existing normalize_flex_context_format pre_load hook to
raise a proper 422 ValidationError when the normalized flex-context is
neither a dict nor a list of dicts, instead of letting downstream code
fail with a TypeError.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: drop dead 'types' comment blocks in UI_FLEX_CONTEXT_SCHEMA

Per-commodity UI editing of aggregate-consumption/aggregate-production
field types is a follow-up; remove the commented-out placeholders.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: add schema-level regression tests for hardening fixes

Cover duplicate commodities in the commodities list, non-electricity
semantics in the single-dict flex-context form, mixing commodities with
top-level shared fields, and malformed flex-context input rejected with
a ValidationError instead of a TypeError.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: add scheduler-level regression tests for gas-only and per-commodity inflexible sensors

Add test_all_gas_flex_model_without_electricity_device, covering a
flex-model with no electricity device (guarding the setdefault fix),
and test_per_commodity_inflexible_device_sensors, covering an
inflexible-device-sensor declared inside a (non-electricity) commodity
context.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: changelog entry for multi-commodity hardening fixes

Document the correctness fixes and the relax-constraints default
change under the v1.0.0 section.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: count only device models, not stock entries, when enumerating devices in _prepare

The commodity_to_devices mapping and the num_flexible_devices count
were built from the re-copied flex_model, which includes stock-model
entries. This overran the sensors list (IndexError at sensor_d =
sensors[d]) whenever the flex-model contained a stock entry, and gave
stock entries bogus device indices in commodity groups. Enumerate
device_models instead, keeping the earlier num_flexible_devices =
len(device_models) and matching _compute_commodity_aggregate_schedules,
which already uses self._device_models.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: declare commodity field on FlexContextSchema

The single-dict electricity-only check never ran: FlexContextSchema
declares no commodity field and marshmallow raises 'Unknown field.'
for it before schema-level validators fire. Declare the field
(load_default 'electricity') with a OneOf validator carrying a
helpful message pointing to the commodities list. Not part of the
documented UI/OpenAPI fields.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: tolerate commodities list combined with top-level flex-context fields

The mixed-fields rejection caused false positives: in the API path, a
multi-commodity list is normalized to {"commodities": [...]} and
collect_flex_config then dict-merges the asset's db-stored flex-context
fields (e.g. site-power-capacity) at the top level, so any asset with
stored electricity flex-context fields would get a 422. Remove the
rejection; semantics stay as designed: top-level fields serve as the
electricity context only when the commodities list has no electricity
entry (see _get_commodity_contexts). Documented in the changelog and
as a code comment.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: use units_are_convertible for scheduler-side currency check; fix stale comment

Match the schema-side currency comparison in _deserialize_flex_context,
and update a test comment still claiming relax_constraints defaults to
False.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: strengthen per-commodity inflexible sensor test; cover electricity-only aggregate indices

Assert via an aggregate-consumption sensor that the gas commodity's
inflexible load (8 kW) is included in the aggregate schedule on top of
the flexible boiler's schedule; before the fix, the per-commodity
inflexible sensor's device index was silently dropped, so this
assertion would have failed. Also add a DB-free unit test that
_electricity_device_indices (used by the aggregate-power sum) covers
only electricity devices, flexible and inflexible.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: keep deprecated price sensors working and fix breach-price error attribution

Two regressions from defaulting relax-constraints to True:

1. The 'please switch to consumption-price' guard counted the
   load_default-filled relax_constraints as an explicitly used new-style
   field, breaking the deprecated consumption/production-price-sensor
   path with a hard 422. Count only fields explicitly present in the
   original input.

2. Default breach prices were filled even when the shared currency unit
   was derived from a mis-united price field (e.g. '6 kWh' passed as a
   breach price), causing DBFlexContextSchema unit validation to flag
   the filled soc-*-breach-price fields instead of the actual offending
   field. Skip the fills when the deprecated price sensor fields are
   used (legacy behaviour unchanged) or when the derived shared currency
   is not an actual currency.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip commodity device groups without devices in _prepare

An empty group could be created for a commodity that no device refers
to (e.g. the always-created electricity entry in an all-gas flex-model,
or a commodity context listed in the flex-context but unused by the
flex-model). Such groups demanded a consumption price and produced
empty-device commitments and EMS constraint groups, making the problem
unbounded. Skip them: they need no prices, commitments or constraints.

Also normalize inflexible_device_sensors to a list (tests bypassing the
schema may pass dict_values, which broke concatenation with the
per-commodity inflexible sensors), and fix the DataSource keyword
(type, not source_type) in the new per-commodity inflexible sensor
test.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: pin relax-constraints False in tests asserting hard-capacity semantics

test_capacity, test_battery_power_capacity_as_sensor and
test_device_power_capacity_uses_directional_capacity_before_site_fallback
assert the hard constraint bounds (derivative min/max DataFrames) equal
the contracted capacities. With relax-constraints defaulting to True,
default breach prices are filled and the hard bound becomes the physical
capacity (contracted capacities are then only softly penalized), so
these tests must opt out explicitly.

Note the default breach prices (10000/kW site, 100/kW device, plus soc
prices) dwarf typical energy prices, so optimal schedules still respect
contracted capacities in ordinary price-arbitrage cases; the relaxation
only kicks in when the problem would otherwise be infeasible.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: exclude the commodity field from the UI flex-context schema guard

The new FlexContextSchema.commodity field (single-dict form is
electricity-only) is deliberately not exposed in the UI, like the
relax-* developer fields.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: fix sign convention of inflexible gas load beliefs

Power sensors store consumption as negative values by default;
get_power_values flips the sign to the scheduler's consumption-positive
convention. The test wrote +8 kW beliefs, which entered the problem as
8 kW production, making the aggregate 1 - 8 = -7 kW instead of the
expected 1 + 8 = 9 kW. Record the load as -8 kW. The per-commodity
inflexible path in _prepare shares the exact same get_power_values loop
as the top-level path, so no production-code change is needed.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: reorder and consolidate changelog entries per self-review

Move the relax-constraints breaking-change entry to the top of its
section, and fold the PR #2271 hardening-fix summary into the existing
multi-commodity feature entry (appending the PR reference) instead of
listing it as a separate bugfix bullet.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: restore intentional 'types' notes on UI_FLEX_CONTEXT_SCHEMA aggregate fields

Per self-review: these commented-out notes on aggregate-consumption
and aggregate-production are intentional placeholders for planned
future UI support, not dead code to remove.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: resolve CI failures from merge conflict between #1946 and #2072 (#2274)

Context:
- PR #2072 added scheduling_result output to StorageScheduler.compute()
  when return_multiple=True, and added _compute_unresolved_targets.
- In single-sensor mode (self.sensor is set), flex_model entries lack a
  "sensor" key, so _compute_unresolved_targets was skipping every device
  and returning empty unresolved/resolved lists.

Changes:
- storage.py: fall back to self.sensor in _compute_unresolved_targets
  when the flex_model entry has no "sensor" key (single-sensor mode)
- test_commitments.py: update schedule-count assertions (+1 for the new
  scheduling_result entry added by PR #2072)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Co-authored-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>
@Flix6x Flix6x added the AI Infra label Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants