fix: harden multi-commodity flex-context (correctness + API decisions) ahead of v1#2271
Merged
Conversation
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…ontext Signed-off-by: F.N. Claessen <felix@seita.nl>
…formulation Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…and map them to the respective group id Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…chemas Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Extend the existing normalize_flex_context_format pre_load hook to raise a proper 422 ValidationError when the normalized flex-context is neither a dict nor a list of dicts, instead of letting downstream code fail with a TypeError. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Per-commodity UI editing of aggregate-consumption/aggregate-production field types is a follow-up; remove the commented-out placeholders. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Cover duplicate commodities in the commodities list, non-electricity semantics in the single-dict flex-context form, mixing commodities with top-level shared fields, and malformed flex-context input rejected with a ValidationError instead of a TypeError. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…dity inflexible sensors Add test_all_gas_flex_model_without_electricity_device, covering a flex-model with no electricity device (guarding the setdefault fix), and test_per_commodity_inflexible_device_sensors, covering an inflexible-device-sensor declared inside a (non-electricity) commodity context. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Document the correctness fixes and the relax-constraints default change under the v1.0.0 section. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…vices in _prepare The commodity_to_devices mapping and the num_flexible_devices count were built from the re-copied flex_model, which includes stock-model entries. This overran the sensors list (IndexError at sensor_d = sensors[d]) whenever the flex-model contained a stock entry, and gave stock entries bogus device indices in commodity groups. Enumerate device_models instead, keeping the earlier num_flexible_devices = len(device_models) and matching _compute_commodity_aggregate_schedules, which already uses self._device_models. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Resolve conflict in _prepare: keep the base branch's device_models-based enumeration (excluding stock entries) combined with this branch's setdefault-based electricity grouping and per-commodity inflexible device handling. Signed-off-by: F.N. Claessen <claessen@seita.nl>
The single-dict electricity-only check never ran: FlexContextSchema declares no commodity field and marshmallow raises 'Unknown field.' for it before schema-level validators fire. Declare the field (load_default 'electricity') with a OneOf validator carrying a helpful message pointing to the commodities list. Not part of the documented UI/OpenAPI fields. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ields
The mixed-fields rejection caused false positives: in the API path, a
multi-commodity list is normalized to {"commodities": [...]} and
collect_flex_config then dict-merges the asset's db-stored flex-context
fields (e.g. site-power-capacity) at the top level, so any asset with
stored electricity flex-context fields would get a 422. Remove the
rejection; semantics stay as designed: top-level fields serve as the
electricity context only when the commodities list has no electricity
entry (see _get_commodity_contexts). Documented in the changelog and
as a code comment.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…; fix stale comment Match the schema-side currency comparison in _deserialize_flex_context, and update a test comment still claiming relax_constraints defaults to False. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ity-only aggregate indices Assert via an aggregate-consumption sensor that the gas commodity's inflexible load (8 kW) is included in the aggregate schedule on top of the flexible boiler's schedule; before the fix, the per-commodity inflexible sensor's device index was silently dropped, so this assertion would have failed. Also add a DB-free unit test that _electricity_device_indices (used by the aggregate-power sum) covers only electricity devices, flexible and inflexible. Signed-off-by: F.N. Claessen <claessen@seita.nl>
… attribution Two regressions from defaulting relax-constraints to True: 1. The 'please switch to consumption-price' guard counted the load_default-filled relax_constraints as an explicitly used new-style field, breaking the deprecated consumption/production-price-sensor path with a hard 422. Count only fields explicitly present in the original input. 2. Default breach prices were filled even when the shared currency unit was derived from a mis-united price field (e.g. '6 kWh' passed as a breach price), causing DBFlexContextSchema unit validation to flag the filled soc-*-breach-price fields instead of the actual offending field. Skip the fills when the deprecated price sensor fields are used (legacy behaviour unchanged) or when the derived shared currency is not an actual currency. Signed-off-by: F.N. Claessen <claessen@seita.nl>
An empty group could be created for a commodity that no device refers to (e.g. the always-created electricity entry in an all-gas flex-model, or a commodity context listed in the flex-context but unused by the flex-model). Such groups demanded a consumption price and produced empty-device commitments and EMS constraint groups, making the problem unbounded. Skip them: they need no prices, commitments or constraints. Also normalize inflexible_device_sensors to a list (tests bypassing the schema may pass dict_values, which broke concatenation with the per-commodity inflexible sensors), and fix the DataSource keyword (type, not source_type) in the new per-commodity inflexible sensor test. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…mantics test_capacity, test_battery_power_capacity_as_sensor and test_device_power_capacity_uses_directional_capacity_before_site_fallback assert the hard constraint bounds (derivative min/max DataFrames) equal the contracted capacities. With relax-constraints defaulting to True, default breach prices are filled and the hard bound becomes the physical capacity (contracted capacities are then only softly penalized), so these tests must opt out explicitly. Note the default breach prices (10000/kW site, 100/kW device, plus soc prices) dwarf typical energy prices, so optimal schedules still respect contracted capacities in ordinary price-arbitrage cases; the relaxation only kicks in when the problem would otherwise be infeasible. Signed-off-by: F.N. Claessen <claessen@seita.nl>
The new FlexContextSchema.commodity field (single-dict form is electricity-only) is deliberately not exposed in the UI, like the relax-* developer fields. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Power sensors store consumption as negative values by default; get_power_values flips the sign to the scheduler's consumption-positive convention. The test wrote +8 kW beliefs, which entered the problem as 8 kW production, making the aggregate 1 - 8 = -7 kW instead of the expected 1 + 8 = 9 kW. Record the load as -8 kW. The per-commodity inflexible path in _prepare shares the exact same get_power_values loop as the top-level path, so no production-code change is needed. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Documentation build overview
31 files changed ·
|
Flix6x
commented
Jul 8, 2026
Move the relax-constraints breaking-change entry to the top of its section, and fold the PR #2271 hardening-fix summary into the existing multi-commodity feature entry (appending the PR reference) instead of listing it as a separate bugfix bullet. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…regate fields Per self-review: these commented-out notes on aggregate-consumption and aggregate-production are intentional placeholders for planned future UI support, not dead code to remove. Signed-off-by: F.N. Claessen <claessen@seita.nl>
1 task
Signed-off-by: F.N. Claessen <claessen@seita.nl> # Conflicts: # flexmeasures/data/models/planning/storage.py # flexmeasures/data/services/scheduling.py
…2274) Context: - PR #2072 added scheduling_result output to StorageScheduler.compute() when return_multiple=True, and added _compute_unresolved_targets. - In single-sensor mode (self.sensor is set), flex_model entries lack a "sensor" key, so _compute_unresolved_targets was skipping every device and returning empty unresolved/resolved lists. Changes: - storage.py: fall back to self.sensor in _compute_unresolved_targets when the flex_model entry has no "sensor" key (single-sensor mode) - test_commitments.py: update schedule-count assertions (+1 for the new scheduling_result entry added by PR #2072) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
…ulti-commodity-hardening
…ardening # Conflicts: # documentation/changelog.rst # flexmeasures/data/models/planning/storage.py # flexmeasures/data/models/planning/tests/test_commitments.py # flexmeasures/data/schemas/scheduling/__init__.py # flexmeasures/data/schemas/tests/test_scheduling.py # flexmeasures/ui/tests/test_utils.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Hardening of the multi-commodity flex-context feature (follow-up to #2172) ahead of the v1 release: correctness fixes plus API/schema decisions.
Correctness fixes
StorageScheduler._deserialize_flex_context(the mismatch branch could never fire), and consolidate currency validation: per-contextshared_currency_units are now compared directly (also covers sensor-valued prices, which the previoussplit("/")-based check skipped).KeyErrorfor portfolios without any electricity device (all-gas/heat sites); empty commodity device groups are now skipped instead of demanding prices.inflexible-device-sensorsdeclared inside a commodity context: they now get device constraints and count against that commodity's site capacity, matching the enumeration order assumed by the aggregate schedules (previously they were silently dropped).commoditytoelectricityinconvert_to_commitmentsinstead of raisingKeyError.aggregate-powernow sums only electricity devices (flexible + inflexible), no longer mixing in gas flows.API/schema decisions
relax-constraintsnow defaults toTrueat the top level too (breaking behaviour change; see changelog). Tests asserting hard-capacity semantics now passrelax-constraints: Falseexplicitly. Default breach-price fills are skipped for non-currency units and when only deprecated price-sensor fields are used, keeping legacy behaviour unchanged.commoditieslist are rejected.commoditykey but only with valueelectricity(helpful error points to thecommoditieslist form).commoditieslist with top-level flex-context fields is tolerated: the db-stored flex-context is dict-merged on top level in the API path, and top-level fields serve as the electricity fallback when the list has no electricity entry.flex-contextpayloads of the wrong type now fail with a 422 validation error instead of a downstreamTypeError.Regression tests
All-gas flex-model; per-commodity inflexible sensors included in constraints and aggregates; currency mismatch across commodity contexts; device without
commoditykey; single-dict flex-context commodity validation; electricity-only device index selection (DB-free unit test).documentation/changelog.rstHow to test
Related Items
Follow-up to #2172; prepares #1946 for release.