v0.33.1

FlexMeasures 0.33.1

This is a patch release in the 0.33.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.33.0...v0.33.1

Copy assets, account sources

Summary

Version v0.33 of FlexMeasures adds the ability to copy assets and supports filtering beliefs data by source much better.

Also, we improved many small things as usual. One interesting change is that pushing data into FlexMeasures now is not blocking API handling instances from accepting other requests. Another is more fine-grained monitoring of failed tasks and absent users.

Read more on these features on the FlexMeasures blog.

This release consists of more than 30 PRs:

What's Changed

• Return 422 ValidationError for zero-duration single-value posts on non-instantaneous sensors by @Copilot in #2116
• Fix/asset context page bugs by @Flix6x in #2117
• upgrade dependencies after release of v0.32 by @nhoening in #2114
• Navigate to parent asset after deleting a child asset; show deletion toast by @Copilot in #2119
• Feat/sensor data source account filtering by @BelhsanHmida in #2083
• Always enable RQ embedded scheduler in flexmeasures jobs run-worker by @Ragnar-the-mighty in #2112
• feat: Cleanup of stale sensor references by @joshuaunity in #2106
• Feat/copy assets by @joshuaunity in #2017
• Add "Copy this asset" button to the asset properties page by @Copilot in #2121
• fix: default stats window is for the selected period by @Flix6x in #2129
• Check read permissions for regressor sensors when triggering forecasts by @Copilot in #2096
• Fix/ajax dialogue regression by @joshuaunity in #2124
• Fix account PATCH endpoint rejecting valid requests on accounts with a consultancy set, and show attributes on the account page by @Copilot in #2140
• fix attributes being overwritten in asset form by @nhoening in #2138
• Feat/clean up old rolling forecasting code by @BelhsanHmida in #2082
• Feat: check flex config permissions schedule by @BelhsanHmida in #2125
• Add GET /api/v3_0/sources endpoint for accessible data sources and types by @Copilot in #2126
• Chore/update to timely beliefs v3.5.5 by @Flix6x in #2148
• Add unified job status endpoint GET /api/v3_0/jobs/ by @Copilot in #2141
• Feat: sensor data source type filter by @BelhsanHmida in #2127
• Fix/drop unchanged beliefs per event by @Flix6x in #2150
• Fix forecast regressor filtering by belief time by @Ragnar-the-mighty in #2134
• Feat/monitoring more fine grained by @nhoening in #2158
• Chore(deps): Bump urllib3 from 2.6.3 to 2.7.0 by @dependabot[bot] in #2161
• Fix: include belief time in fixed-value records for chart data by @joshuaunity in #2147
• Standardize resolution formatting across API endpoints for consistent response payloads by @joshuaunity in #2152
• fix: reporter crashing on KeyError: 'Level source not found' by @Flix6x in #2159
• Fix/build device soc values by @Flix6x in #2160
• Fix auth check when running CLI command with flask by @Flix6x in #2169
• Fix auth issues: child asset creation, account page access by @Copilot in #2163
• feat: add account_id filter for sources in search_beliefs by @Copilot in #2065
• Fix: All use of raw json when updating flex_configs by @joshuaunity in #2162
• Feat/validate source type filter by @Flix6x in #2151
• Feat/speed up stats panel on sensor page by @Flix6x in #2173
• Fix/sensor data deletion window by @Flix6x in #2175
• Fix/migrate darts package by @BelhsanHmida in #2180
• Feat/efficiencyfield support sensor by @joshuaunity in #2142
• Fix: visually distinguish data sources beyond source name by @BelhsanHmida in #2185
• Move sensor data ingestion to job queue by @Copilot in #2101
• fix the links with sed between endpoint listing and endpoint details by @nhoening in #1753
• Fix/force new job when scheduling an asset (tree) by @Flix6x in #2182
• Fix sensor forecast button to send train-start from available history by @Copilot in #2187
• Docs/fix various api doc issues by @nhoening in #2122
• Fix: DST transition handling by supporting both datetime and pandas Timestamp objects by @Ahmad-Wahid in #2197
• Create .github/instructions folder with cross-cutting topic instruction files by @Copilot in #2198
• Harden RQ Dashboard job views against non-JSON-serializable job data by @Copilot in #2200
• Fix StorageScheduler crash on devices without sensor in flex-model by @saerts-gp in #2085
• Derive seasonal lags from sensor resolution by @BelhsanHmida in #2157
• docs: improve docstring for asset copy endpoint by @nhoening in #2204
• CLI command to inspect a job by @nhoening in #2202
• Chore/avoid autouse fixtures by @Flix6x in #2206
• uploading/posting beliefs works without Redis (again) by @nhoening in #2207
• Fail clearly on insufficient forecasting history by translating Darts fit errors by @Copilot in #2192
• Select regressor beliefs per (event_start, regressor) in forecasting covariate assembly by @Copilot in #2155
• Handle standardized sensors_to_show plots in flexmeasures show asset by @Copilot in #2189
• feat: allow setting a consumption/production sensor in the flex-model of an asset by @Flix6x in #2190
• docs: improve API units section with conversion examples and Pint reference by @Copilot in #2211
• Catch 404 errors in Sentry interation without exc_info as well by @nhoening in #2212
• Fix/storage efficiency validation using consumption/production field by @Flix6x in #2213

New Contributors

• @saerts-gp made their first contribution in #2085

Full Changelog: v0.32.0...v0.33.0

v0.32.3

FlexMeasures 0.32.3

This is a patch release in the 0.32.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.32.2...v0.32.3

v0.32.2

FlexMeasures 0.32.2

This is a patch release in the 0.32.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.32.1...v0.32.2

v0.32.1

FlexMeasures 0.32.1

This is a patch release in the 0.32.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.32.0...v0.32.1

v0.32: Flex-config on graphs

Summary

Version v0.32 of FlexMeasures improves the UX of building dashboards with information in the flex-context and flex-model.

Also, we added functionality on the sensor page: a button to create forecasts and a button to delete data. Plus, the layout of actions on the sensor page is now nicer & more usable.

Here are some other notable features:

improving hovering the data points on graphs
switch dev environment to uv (faster installations)
Annotations (see v0.31 how they look like in the UI) can be sent via the API

Read more on https://flexmeasures.io/v032-flex-config-on-graphs/

What's Changed

• Fix/copilot setup steps shares steps with lint and test by @Flix6x in #1998
• Docs/missing flex config fields by @Flix6x in #2003
• Feat: redirect to RTD if local docs missing by @Flix6x in #2006
• Fix: forecasting CLI for v0.31 by @Flix6x in #2007
• Add if_exists=True to index drops in migration b8f3cda5e023 by @Ahmad-Wahid in #2009
• Fix/resample to instantaneous for scheduling by @Flix6x in #2012
• Support saving state-of-charge schedules to sensors with % unit by @Copilot in #1996
• Add unit query parameter to GET schedule endpoint for unit conversion by @Copilot in #1993
• fix: ensure context attribute exists before accessing it by @BelhsanHmida in #2014
• refactor: migrate from pip-tools to uv for dependency management by @TeaDrinkingProgrammer in #1973
• fix: do not fail API endpoint listing if an endpoint has no docstring by @nhoening in #2032
• feat: show sensor attributes on sensor page by @Flix6x in #2015
• Fix/small forecasting pipeline fixes by @BelhsanHmida in #2011
• chore: re-add the bcrypt dependency by @nhoening in #2029
• feat: Adapting flex-config to SensorsToShowSchema by @joshuaunity in #1904
• update contact details on get-in-touch page by @nhoening in #2022
• Upgrade some dependencies for security reasons by @nhoening in #2037
• Expand audit logging for password life-cycle by @nhoening in #2036
• Stop creating new toy assets when restarting the docker-compose stack by @Copilot in #2018
• Add FlexMeasures-Version and API-Version response headers by @Copilot in #2021
• Feat: full SoC preference by @Flix6x in #2023
• Make review lead main agent contact point by @Flix6x in #2066
• Fail fast in forecasting when omitted --start resolves to a predict_start after --end by @Copilot in #2064
• Fix lead agent initialization by @Copilot in #2068
• Feat/handle security error by @Flix6x in #2054
• Filter 404 errors from Sentry reports by default by @Copilot in #2071
• fix: regression in upsampling speed by updating timely-beliefs to v3.5.4 by @Flix6x in #2060
• Feat: save_to_db no longer attempts to save duplicate beliefs by @Flix6x in #2059
• Add POST endpoints for creating annotations on accounts, assets, and sensors by @Copilot in #1968
• Enhance JSON field change formatting for audit logs by @joshuaunity in #2055
• Feat/forecasting belief time by @Flix6x in #1978
• Sensor page: Stop side-panels using vertical space when closed by @nhoening in #2033
• docs: add Available Agents section to AGENTS.md so specialist subagents are discoverable by @Copilot in #2090
• Add "Create forecast" button to sensor page by @Copilot in #1985
• Feat/dynamic soc at start by @BelhsanHmida in #2026
• Fix drop_unchanged prior-belief selection regression by @Ragnar-the-mighty in #2086
• Fix: multiple regressors bug by @BelhsanHmida in #2088
• fix: DST bug in processing time series segments by @Flix6x in #2069
• Feat: JSON editor for sensor/asset/account attributes by @Flix6x in #2093
• Feat/show vertical graph lines by @joshuaunity in #2079
• Refactor/remove existing beliefs by @Flix6x in #2098
• Support coupling data sources to accounts, and preserve user ID and account ID references in audit logs and data sources by @Copilot in #2058
• Docs/new page about scripting and hems example by @nhoening in #2097
• Fix asset status page crash on old sensors_to_show format by @Copilot in #2103
• Add "Delete data" action to sensor page with filtered DELETE endpoint by @Copilot in #2095
• docs: document fallback schedule mechanism and 303 See Other redirects by @Copilot in #2107
• Fix: scheduling none soc mixed assets by @BelhsanHmida in #2108
• Fix: wrong ID type for fetching a schedule via OpenAPI by @Flix6x in #2109

New Contributors

• @TeaDrinkingProgrammer made their first contribution in #1973

Full Changelog: v0.31.0...v0.32.0

v0.31.3

FlexMeasures 0.31.3

This is a patch release in the 0.31.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.31.2...v0.31.3

v0.31.2

FlexMeasures 0.31.2

This is a patch release in the 0.31.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.31.1...v0.31.2

v0.31.1

FlexMeasures 0.31.1

This is a patch release in the 0.31.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our changelog for a list of all the changes.

Full Changelog: v0.31.0...v0.31.1

v0.31: Forecasting API & data annotations

Summary

Version v0.31 of FlexMeasures adds API endpoints to control forecasting and (brings back) visual annotations on time series data.

Here are other notable features:

• Schedules can now be returned in a custom resolution 
• The upload form (on the sensor page is improved): allow uploading data with different units (different resolution already supported), data can have gaps and better error handling
• Step-by-step replay in graphs
• The SwaggerUI API page can now be customized (incl. link to signup page)
• The 2FA code form (step of login) has improved UX
• JSON attributes (on assets, sensors) are now searchable (by code)
• Python3.9 has been deprecated
• We started using Copilot agents in PRs (for development)

What's Changed

• chore: upgrade dependencies for v0.31 by @nhoening in #1847
• Return meaningful results from the Train-Predict pipeline compute method by @BelhsanHmida in #1822
• Chore(deps): Bump urllib3 from 2.5.0 to 2.6.0 in requirements by @dependabot[bot] in #1850
• add OpenAPI general info & description, add two config setting for hosts by @nhoening in #1851
• Chore/no more SingleBlockManager warnings by @Flix6x in #1858
• fix: Fixed broken UI flex model fields by @joshuaunity in #1860
• Fix: duplicate beliefs by storing non-floored belief_time in forecasting pipeline by @BelhsanHmida in #1855
• feat: make listing public assets on account page optional by @nhoening in #1872
• Clean Base.html by @joshuaunity in #1868
• chore(db): Migrate JSON fields to JSONB, so they can be used to look up sensors, attributes, sources by @nhoening in #1867
• Feat/add trigger forecasting api endpoint by @BelhsanHmida in #1813
• Chore: more resilient docker compose setup by @nhoening in #1790
• Feat/less verbose error logs by @Flix6x in #1883
• add FLEXMEASURES_SIGNUP_PAGE option and improve OpenAPI intro by @nhoening in #1889
• Feat/index assets given a root node and/or depth by @Flix6x in #1874
• Fix sensor creation form by @nhoening in #1876
• Feat/add fetch forecasts endpoint by @BelhsanHmida in #1823
• Fix/multiple temperature sensors in chart by @Flix6x in #1890
• Fix/keep around finished forecasting jobs longer by @Flix6x in #1870
• add fields param to the asset-listing endpoints, controlling fields in response by @nhoening in #1884
• feat (CLI): schedule at a certain belief time by @Flix6x in #1871
• Feat/CLI scheduling dry-run by @Flix6x in #1892
• Fix/new sensors in server timezone by @Flix6x in #1894
• Two UX improvements on the 2FA form by @nhoening in #1875
• Step by Step replay graph by @joshuaunity in #1898
• Fix/delete beliefs source filter by @BelhsanHmida in #1903
• feat: allow plugins to register forecasters by @Flix6x in #1863
• Fix/include public assets on account page by @Flix6x in #1899
• Editing sensor timezone from UI by @joshuaunity in #1900
• fix: enforce canonical order before concatenation by @Flix6x in #1905
• fix: pass start and end dates to getAssetKPIs() on graphs page by @nhoening in #1909
• Add unit field for sensor data upload form by @joshuaunity in #1836
• Fix: flex-context dialogue is empty when flex-context has two booleans with the same value by @nhoening in #1907
• Bring back CLI tests to GitHub Actions by @nhoening in #778
• Docs/fix workflow and coverage badges by @Flix6x in #1913
• CI/doctests included in coverage by @Flix6x in #1914
• Include supported but un-editable flex-config fields in modal dialogues (UI editors) by @joshuaunity in #1915
• Add save buttons to flex-config modal dialogues by @joshuaunity in #1916
• Extra security layer for checking breached passwords by @joshuaunity in #1919
• Fixed broken toast message with sensor data upload form by @joshuaunity in #1921
• Fix/return validation errors instead of db errors by @Flix6x in #1923
• feat: upload allows gaps in data by @BelhsanHmida in #1918
• Fix/mutate job state when using flexmeasures jobs run-job by @Flix6x in #1929
• Fix/single job exception handler by @Flix6x in #1930
• feat: let custom schedulers return custom results by @Flix6x in #1933
• fix: simplify_index should not change the input frame inplace by @Flix6x in #1931
• Feat/show zero values in bar charts by @Flix6x in #1932
• Feat/variable peak pricing by @Flix6x in #1835
• Feat/register custom scheduler by name by @Flix6x in #1924
• Add test-specialist agent configuration for FlexMeasures testing patterns by @Copilot in #1940
• Create agent coordinator and initial specialist roster with codebase-specific domain knowledge by @Copilot in #1942
• Add YAML frontmatter to agent files for GitHub recognition by @Copilot in #1944
• fix: stroke encoding showed undefined color field by @Flix6x in #1936
• Fix: StorageScheduler to correctly combine multiple sessions targeting the same sensor by @Ahmad-Wahid in #1948
• feat: allow seeing the complete datetime in audit log by @joshuaunity in #1949
• feat: show statistics of the job queueing system by @Flix6x in #1838
• Feat/schedule resolution by @Flix6x in #1857
• feat: support storing the aggregate power schedule by @Flix6x in #1736
• Fix/sensor only fields in swagger by @Flix6x in #1951
• Agent instructions: Enforce dev environment usage and commit discipline by @Copilot in #1945
• Fix API test isolation: set fs_authn_via for Flask-Security session auth by @Copilot in #1960
• CI/setup agent test environment by @Flix6x in #1962
• Fix/syntax for copilot-setup-steps.yml by @Flix6x in #1963
• CI/agent lessons from PR 1961 by @Flix6x in #1964
• Do not support Py3.9 or mention it as the minimum requirement by @nhoening in #1965
• Add latest dependabot alert suggestions by @nhoening in #1959
• Fix/annotations on sensor page by @Flix6x in #1967
• Docs: fix the ReadTheDocs pipeline by @nhoening in #1972
• feat/default datasource types by @nhoening in #1938
• Docs/commitments section by @Flix6x in #1849
• Feat/optional training start by @Flix6x in #1917
• Feat/trigger field order by @Flix6x in #1982
• feat: allow update of parent asset on UI by @joshuaunity in #1957
• Feat: Add resolution to sensors list on asset context page by @nhoening in #1986
• Test: Add forecasting config tests by @BelhsanHmida in #1987
• Fix JSON serialization error on forecasting job pages by @Copilot in https://gith...