diff --git a/documentation/api/change_log.rst b/documentation/api/change_log.rst index 19803bca98..99cb239e06 100644 --- a/documentation/api/change_log.rst +++ b/documentation/api/change_log.rst @@ -5,10 +5,16 @@ API change log .. note:: The FlexMeasures API follows its own versioning scheme. This is also reflected in the URL (e.g. `/api/v3_0`), allowing developers to upgrade at their own pace. -v3.0-31 | 2026-06-01 +v3.0-32 | 2026-06-18 """""""""""""""""""" + - Support filtering assets and sensors by ID prefix in the ``filter`` query parameter of ``GET /assets``, ``GET /assets//sensors`` and ``GET /sensors``. +- **Field canonicalization** for background job tracking: + * The ``job_id`` field is now the canonical way to identify background jobs returned by `/sensors//schedules/trigger`, `/assets//schedules/trigger`, and `/sensors//forecasts/trigger` endpoints. If applicable, the triggered response now also returns a ``job_results_url`` pointing to the sensor-specific results endpoint, alongside the generic ``job_monitor_url``. + * Legacy `schedule` field (in scheduling endpoints) and `forecast` field (in forecasting endpoints) are deprecated but remain in responses for backward compatibility. All responses include a ``_deprecated_fields`` object with machine-readable migration guidance for each deprecated field, including the recommended replacement and removal date. + -- Support filtering assets and sensors by ID prefix in the ``filter`` query parameter of ``GET /assets``, ``GET /assets//sensors`` and ``GET /sensors``. +v3.0-31 | 2026-06-01 +"""""""""""""""""""" - Added a unified job status endpoint ``GET /api/v3_0/jobs/`` that returns the current execution status and a human-readable result message for any background job (scheduling, forecasting, etc.) identified by its UUID. - Switched from ``force_new_job_creation`` to ``force-new-job-creation`` (maintaining backwards compatibility) and added the field to `/assets/(id)/schedules/trigger` (POST) endpoint, too. - Support both snake_case and kebab-case fields in `/sensors//data` (GET), while only documenting the kebab-case ones. diff --git a/documentation/api/introduction.rst b/documentation/api/introduction.rst index caa047d716..c59fe65055 100644 --- a/documentation/api/introduction.rst +++ b/documentation/api/introduction.rst @@ -155,6 +155,91 @@ Here is a client-side code example in Python for handling 303 redirects (this me print(f"Failed to fetch fallback schedule: {response.status_code} {response.text}") return response +.. _api_background_jobs: + +Background job monitoring +-------------------------- + +Several API endpoints queue background jobs for asynchronous processing (scheduling, forecasting, data ingestion) and return a ``202 Accepted`` response. +These responses include a ``job_id`` field (the canonical identifier) that clients can use to monitor job progress and retrieve results. +They also include both ``job_monitor_url`` for generic status monitoring and (if applicable) ``job_results_url`` for the sensor-specific results endpoint. + +**Example 202 Accepted response from a scheduling endpoint:** + +.. code-block:: json + + { + "status": "ACCEPTED", + "job_id": "364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_monitor_url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_results_url": "/api/v3_0/sensors/3/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "message": "Request has been accepted for processing." + } + +**Monitoring job status:** + +Clients should use the ``job_id`` to query the unified job status endpoint: + +.. code-block:: bash + + GET /api/v3_0/jobs/ + +This returns the current execution status and a human-readable result message. For example: + +.. code-block:: python + + import requests + import time + + def wait_for_job(job_id, job_monitor_url, timeout=300, poll_interval=5): + """Wait for a background job to complete and return the result. + + Parameters + ---------- + job_id : str + The UUID of the background job. + job_monitor_url : str + The URL to query for job status (e.g., "/api/v3_0/jobs/"). + timeout : int + Maximum seconds to wait for job completion. + poll_interval : int + Seconds between status checks. + """ + start_time = time.time() + while time.time() - start_time < timeout: + response = requests.get(job_monitor_url) + if not response.ok: + raise RuntimeError(f"Failed to query job status: {response.status_code} {response.text}") + + job_data = response.json() + status = job_data.get("status") + + if status in ("PENDING", "RUNNING"): + print(f"Job {job_id} is still {status.lower()}...") + time.sleep(poll_interval) + elif status == "SUCCEEDED": + return job_data.get("result") + else: # Failed, error, etc. + raise RuntimeError(f"Job failed with status {status}: {job_data.get('message')}") + + raise TimeoutError(f"Job {job_id} did not complete within {timeout} seconds") + +.. note:: + + For **schedules**, after the job completes successfully, use the job ID (same value as the legacy ``schedule`` field) to retrieve the actual schedule or follow the returned ``job_results_url``: + + .. code-block:: bash + + GET /api/v3_0/sensors//schedules/ + + For **forecasts**, after the job completes successfully, use the job ID to retrieve the forecast or follow the returned ``job_results_url``: + + .. code-block:: bash + + GET /api/v3_0/sensors//forecasts/ + + Both of these endpoints will also return `202 Accepted` if the job is still being computed, so clients can continue to poll them directly if they prefer. + .. _api_deprecation: Deprecation and sunset @@ -166,6 +251,38 @@ For more information on our multi-stage deprecation approach and available optio .. _api_deprecation_clients: +Deprecated response fields +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In addition to deprecating entire endpoints, we sometimes deprecate individual fields in API responses while maintaining backward compatibility by including both the legacy and canonical fields. +When this happens, responses include a ``_deprecated_fields`` object containing machine-readable information about deprecated fields. + +For example, when scheduling endpoints switched from ``schedule`` to ``job_id`` as the canonical field identifier for background jobs: + +.. code-block:: json + + { + "status": "ACCEPTED", + "job_id": "364bfd06-c1fa-430b-8d25-8f5a547651fb", + "schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_monitor_url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_results_url": "/api/v3_0/sensors/3/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "_deprecated_fields": { + "schedule": { + "use": "job_id", + "deprecated_since": "1.0.0", + "removal_date": "2.0.0", + "note": "The 'schedule' response field is deprecated; use 'job_id' instead" + } + } + } + +Clients should: + +- Use the ``_deprecated_fields`` object to identify which fields are deprecated in their version. +- Migrate to use the canonical field names (indicated by the ``"use"`` field). +- Plan upgrades before the ``"removal_date"`` to avoid breakage when the deprecated fields are removed in a future API version. + Clients ^^^^^^^ diff --git a/documentation/changelog.rst b/documentation/changelog.rst index 6e8d585933..e85111420c 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -7,6 +7,9 @@ FlexMeasures Changelog v1.0.0 | July XX, 2026 ============================ +.. warning:: As of this release we standardize asynchronous job responses to use the `job_id` field and return HTTP `202 Accepted` when a background job is queued. Legacy response fields such as ``schedule`` and ``forecast`` will be deprecated; clients should migrate to `job_id` (see the Infrastructure / Support section below for migration details). + + New features ------------- * In the UI, asset and sensor lists can be filtered by ID prefix through API-backed search fields [see `PR #2231 `_] @@ -18,6 +21,8 @@ New features Infrastructure / Support ---------------------- +* Standardize job-trigger API responses to return `202 Accepted` and a canonical `job_id` field; legacy response fields such as ``schedule`` and ``forecast`` are preserved for backward-compatibility but marked deprecated with a migration guide in the API docs (see issue #2171). + * Upgraded dependencies [see `PR #1485 `_ and `PR #2215 `_] * Prepare the ``device_scheduler`` to deal with commitments per device group [see `PR #1934 `_] * Documentation section on the modelling choice for recording measurements, forecasts and schedules under one or multiple sensors [see `PR #2217 `_] diff --git a/documentation/tut/toy-example-expanded.rst b/documentation/tut/toy-example-expanded.rst index 3922ad9a3e..9ccd602ae5 100644 --- a/documentation/tut/toy-example-expanded.rst +++ b/documentation/tut/toy-example-expanded.rst @@ -119,6 +119,10 @@ This will have an effect on the available headroom for the battery, given the `` } } + .. note:: + The API returns a **202 Accepted** response containing a ``job_id`` field. + Use this UUID to monitor the job status via :ref:`api_background_jobs`. + .. tab:: FlexMeasures Client Using the `FlexMeasures Client `_: diff --git a/documentation/tut/toy-example-from-scratch.rst b/documentation/tut/toy-example-from-scratch.rst index e4a9d40d04..94f1a9b0fd 100644 --- a/documentation/tut/toy-example-from-scratch.rst +++ b/documentation/tut/toy-example-from-scratch.rst @@ -90,6 +90,20 @@ There is more information being used by the scheduler, such as the battery's cap } .. note:: You can try this right in Swagger UI, too! You should find it at `http://localhost:5000/api/v3_0/docs `_ after starting FlexMeasures locally. + + The API returns a **202 Accepted** response with a ``job_id`` field to track the scheduling job: + + .. code-block:: json + + { + "status": "ACCEPTED", + "job_id": "364bfd06-c1fa-430b-8d25-8f5a547651fb", + "schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_monitor_url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_results_url": "/api/v3_0/sensors/2/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb" + } + + **Important:** Use the ``job_id`` field to reference the scheduling job. You can also follow the returned ``job_results_url`` to fetch the schedule results once they are ready. For more details, see :ref:`api_background_jobs`. .. tab:: FlexMeasures Client diff --git a/flexmeasures/api/common/responses.py b/flexmeasures/api/common/responses.py index 5b161ffa27..3a9294a61a 100644 --- a/flexmeasures/api/common/responses.py +++ b/flexmeasures/api/common/responses.py @@ -6,6 +6,7 @@ from flask import url_for from flexmeasures.auth.error_handling import FORBIDDEN_MSG, FORBIDDEN_STATUS_CODE +from flexmeasures.utils.time_utils import server_now p = inflect.engine() @@ -383,16 +384,45 @@ def request_processed(message: str) -> ResponseTuple: def request_accepted_for_processing( job_id: str, message: str = "Request has been accepted for processing.", + legacy_key: str | None = None, + deprecated_since: str | None = None, + removal_date: str | None = None, + job_results_url: str | None = None, ) -> ResponseTuple: - return ( - dict( - status="ACCEPTED", - message=message, - job_monitor_url=url_for("JobAPI:get_job_status", uuid=job_id), - job_id=job_id, - ), - 202, + """ + Standard 202 response when a background job is accepted. + + Optional backwards-compatibility: if `legacy_key` is provided the response + will include the job id under that legacy key (e.g. `schedule` or + `forecast`). The response will also include a `_deprecated_fields` object + with guidance for migrating to `job_id`. + + Optional `job_results_url` may be supplied to provide a direct link to the + sensor-specific job results endpoint, e.g. `/api/v3_0/sensors//schedules/`. + """ + resp: dict[str, object] = dict( + status="ACCEPTED", + message=message, + job_monitor_url=url_for("JobAPI:get_job_status", uuid=job_id), + job_id=job_id, ) + if job_results_url: + resp["job_results_url"] = job_results_url + + if legacy_key: + # keep legacy key for backwards compatibility + resp[legacy_key] = job_id + # include machine-readable deprecation info so clients can detect it + resp["_deprecated_fields"] = { + legacy_key: dict( + use="job_id", + deprecated_since=deprecated_since or server_now().date().isoformat(), + removal_date=removal_date, + note=(f"The '{legacy_key}' response field is deprecated; use 'job_id'"), + ) + } + + return resp, 202 def request_too_large(message: str) -> ResponseTuple: diff --git a/flexmeasures/api/v3_0/assets.py b/flexmeasures/api/v3_0/assets.py index 60a7c29bda..6dcd1e9aaf 100644 --- a/flexmeasures/api/v3_0/assets.py +++ b/flexmeasures/api/v3_0/assets.py @@ -67,7 +67,7 @@ ) from flexmeasures.api.common.responses import ( unprocessable_entity, - request_processed, + request_accepted_for_processing, ) from flexmeasures.api.common.schemas.users import AccountIdField from flexmeasures.api.common.schemas.assets import default_response_fields @@ -1479,8 +1479,8 @@ def trigger_schedule( site-consumption-capacity: {sensor: 32} responses: - 200: - description: PROCESSED + 202: + description: ACCEPTED (Scheduling job queued for processing) content: application/json: schema: @@ -1488,14 +1488,25 @@ def trigger_schedule( examples: successful_response: description: | - This message indicates that the scheduling request has been processed without any error. + This message indicates that the scheduling request has been accepted for processing (202 Accepted). A scheduling job has been created with some Universally Unique Identifier (UUID), which will be picked up by a worker. - The given UUID may be used to obtain the resulting schedule for each flexible device: see [/sensors/schedules/](#/Sensors/get_api_v3_0_sensors__id__schedules__uuid_). + The given UUID is returned in the canonical `job_id` field. + For backward-compatibility, the legacy `schedule` field is also included but is deprecated; + use `job_id` instead. The `_deprecated_fields` object provides migration guidance. + The given UUID may be used to obtain the resulting schedule for each flexible device via [/sensors/schedules/](#/Sensors/get_api_v3_0_sensors__id__schedules__uuid_). value: - status: PROCESSED + status: ACCEPTED + job_id: "364bfd06-c1fa-430b-8d25-8f5a547651fb" schedule: "364bfd06-c1fa-430b-8d25-8f5a547651fb" - message: "Request has been processed." + job_monitor_url: "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb" + message: "Request has been accepted for processing." + _deprecated_fields: + schedule: + use: "job_id" + deprecated_since: "1.0.0" + removal_date: "2.0.0" + note: "The 'schedule' response field is deprecated; use 'job_id'" 400: description: INVALID_DATA 401: @@ -1536,9 +1547,9 @@ def trigger_schedule( except ValueError as err: return unprocessable_entity(str(err)) - response = dict(schedule=job.id) - d, s = request_processed() - return dict(**response, **d), s + # Keep legacy `schedule` key for backward compatibility; prefer `job_id`. + d, s = request_accepted_for_processing(job.id, legacy_key="schedule") + return d, s @route("//kpis", methods=["GET"]) @use_kwargs( diff --git a/flexmeasures/api/v3_0/sensors.py b/flexmeasures/api/v3_0/sensors.py index 1c5e53d653..6d5ab559bd 100644 --- a/flexmeasures/api/v3_0/sensors.py +++ b/flexmeasures/api/v3_0/sensors.py @@ -19,6 +19,7 @@ from sqlalchemy import delete, select, or_ from flexmeasures.api.common.responses import ( + request_accepted_for_processing, request_processed, unrecognized_event, unknown_forecast, @@ -969,24 +970,59 @@ def trigger_schedule( site-peak-consumption-price: "260 EUR/MW" site-peak-production-price: "260 EUR/MW" responses: - 200: - description: PROCESSED + 202: + description: ACCEPTED (Scheduling job queued for processing) content: application/json: schema: type: object + properties: + job_id: + type: string + description: UUID of the queued scheduling job (canonical field; use this). + status: + type: string + enum: ["ACCEPTED"] + message: + type: string + job_monitor_url: + type: string + format: uri + description: URL to query the generic job status API. + job_results_url: + type: string + format: uri + description: URL to fetch the schedule results for this job once it is complete. + schedule: + type: string + description: (DEPRECATED) UUID of the queued scheduling job. Use `job_id` instead. + _deprecated_fields: + type: object + description: Machine-readable mapping of deprecated fields with migration guidance. examples: schedule_created: summary: Schedule response description: | - This message indicates that the scheduling request has been processed without any error. + This message indicates that the scheduling request has been accepted for processing (202 Accepted). A scheduling job has been created with some Universally Unique Identifier (UUID), which will be picked up by a worker. + The given UUID is returned in the canonical `job_id` field. + For backward-compatibility, the legacy `schedule` field is also included but is deprecated; + use `job_id` instead. The `_deprecated_fields` object provides migration guidance. The given UUID may be used to obtain the resulting schedule: see /sensors//schedules/. value: - status: "PROCESSED" + status: "ACCEPTED" + job_id: "364bfd06-c1fa-430b-8d25-8f5a547651fb" schedule: "364bfd06-c1fa-430b-8d25-8f5a547651fb" - message: "Request has been processed." + job_monitor_url: "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb" + job_results_url: "/api/v3_0/sensors/3/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb" + message: "Request has been accepted for processing." + _deprecated_fields: + schedule: + use: "job_id" + deprecated_since: "1.0.0" + removal_date: "2.0.0" + note: "The 'schedule' response field is deprecated; use 'job_id'" 400: description: INVALID_DATA 401: @@ -1034,9 +1070,15 @@ def trigger_schedule( db.session.commit() - response = dict(schedule=job.id) - d, s = request_processed() - return dict(**response, **d), s + # Keep legacy `schedule` key for backward compatibility; prefer `job_id`. + d, s = request_accepted_for_processing( + job.id, + legacy_key="schedule", + job_results_url=url_for( + "SensorAPI:get_schedule", id=sensor.id, uuid=job.id + ), + ) + return dict(**d), s # mark endpoint as deprecated trigger_schedule.after_request = lambda response: add_deprecation_header(response) @@ -1187,6 +1229,27 @@ def get_schedule( # noqa: C901 start: "2015-06-02T10:00:00+00:00" duration: "PT45M" unit: "MW" + 202: + description: Scheduling job status while the job is still running + content: + application/json: + schema: + type: object + properties: + status: + type: string + enum: ["QUEUED", "STARTED", "DEFERRED"] + description: Processing status of the scheduling job. + + message: + type: string + description: Human-readable message about current job processing. + examples: + started: + summary: Scheduling job is still running + value: + status: "STARTED" + message: "The scheduling job is currently running." 400: description: INVALID_TIMEZONE, INVALID_DOMAIN, UNKNOWN_SCHEDULE, UNRECOGNIZED_CONNECTION_GROUP 401: @@ -1243,7 +1306,23 @@ def get_schedule( # noqa: C901 _external=True, ), ) + elif job.is_failed: + return unknown_schedule(job_status_description(job, scheduler_info_msg)) else: + if current_app.config.get("FLEXMEASURES_API_SUNSET_ACTIVE"): + job_status = job.get_status() + job_status_name = ( + job_status.upper() + if isinstance(job_status, str) + else job_status.name + ) + return ( + dict( + status=job_status_name, + message=job_status_description(job, scheduler_info_msg), + ), + 202, + ) return unknown_schedule(job_status_description(job, scheduler_info_msg)) schedule_start = job.kwargs["start"] @@ -1878,25 +1957,48 @@ def trigger_forecast(self, id: int, **params): start: "2026-01-15T00:00:00+01:00" duration: "P2D" responses: - 200: - description: PROCESSED + 202: + description: ACCEPTED (Forecasting job queued for processing) content: application/json: schema: type: object properties: - forecast: + job_id: type: string - description: UUID of the queued forecasting job. + description: UUID of the queued forecasting job (canonical field; use this). status: type: string - enum: ["PROCESSED"] + enum: ["ACCEPTED"] message: type: string + job_monitor_url: + type: string + format: uri + description: URL to query the generic job status API. + job_results_url: + type: string + format: uri + description: URL to fetch the forecast results for this job once it is complete. + forecast: + type: string + description: (DEPRECATED) UUID of the queued forecasting job. Use `job_id` instead. + _deprecated_fields: + type: object + description: Machine-readable mapping of deprecated fields with migration guidance. example: - forecast: "b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b" - status: "PROCESSED" + status: "ACCEPTED" + job_id: "b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b" message: "Forecasting job has been queued." + job_monitor_url: "/api/v3_0/jobs/b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b" + job_results_url: "/api/v3_0/sensors/3/forecasts/b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b" + forecast: "b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b" + _deprecated_fields: + forecast: + use: "job_id" + deprecated_since: "1.0.0" + removal_date: "2.0.0" + note: "The 'forecast' response field is deprecated; use 'job_id' instead" 401: description: UNAUTHORIZED 403: @@ -1937,8 +2039,15 @@ def trigger_forecast(self, id: int, **params): current_app.logger.exception("Forecast job failed to enqueue.") return unprocessable_entity(str(e)) - d, s = request_processed() - return dict(forecast=pipeline_returns["job_id"], **d), s + # Keep legacy `forecast` key for backward compatibility; prefer `job_id`. + d, s = request_accepted_for_processing( + pipeline_returns["job_id"], + legacy_key="forecast", + job_results_url=url_for( + "SensorAPI:get_forecast", id=id, uuid=pipeline_returns["job_id"] + ), + ) + return dict(**d), s @route("//forecasts/", methods=["GET"]) @use_kwargs( diff --git a/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py b/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py index aac1b80ffa..b864b97759 100644 --- a/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py +++ b/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py @@ -95,7 +95,7 @@ def test_asset_trigger_and_get_schedule( json=message, ) print("Server responded with:\n%s" % trigger_schedule_response.json) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 job_id = trigger_schedule_response.json["schedule"] # look for scheduling jobs in queue diff --git a/flexmeasures/api/v3_0/tests/test_forecasting_api_fresh_db.py b/flexmeasures/api/v3_0/tests/test_forecasting_api_fresh_db.py index 6b5a643e15..86175c8d2f 100644 --- a/flexmeasures/api/v3_0/tests/test_forecasting_api_fresh_db.py +++ b/flexmeasures/api/v3_0/tests/test_forecasting_api_fresh_db.py @@ -48,7 +48,10 @@ def test_trigger_and_fetch_forecasts( trigger_res = client.post( trigger_url, json=payload, headers={"Authorization": token} ) - assert trigger_res.status_code == 200, trigger_res.json + assert trigger_res.status_code == 202, trigger_res.json + assert trigger_res.json["job_results_url"] == url_for( + "SensorAPI:get_forecast", id=sensor_0.id, uuid=trigger_res.json["forecast"] + ) trigger_json = trigger_res.get_json() wrap_up_job = app.queues["forecasting"].fetch_job(trigger_json["forecast"]) diff --git a/flexmeasures/api/v3_0/tests/test_jobs_api.py b/flexmeasures/api/v3_0/tests/test_jobs_api.py index 28454db63f..335160177f 100644 --- a/flexmeasures/api/v3_0/tests/test_jobs_api.py +++ b/flexmeasures/api/v3_0/tests/test_jobs_api.py @@ -110,7 +110,7 @@ def test_get_job_status_queued( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] # immediately query the generic job endpoint – job is still queued @@ -156,7 +156,7 @@ def test_get_job_status_started( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] # simulate the job being picked up by a worker @@ -205,7 +205,7 @@ def test_get_job_status_finished( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] # run the scheduling job @@ -266,7 +266,7 @@ def fail_with_assertion(self): url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message_for_trigger_schedule(), ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] work_on_rq( @@ -302,7 +302,7 @@ def test_get_job_status_failed_infeasible_schedule_includes_exc_info( url_for("SensorAPI:trigger_schedule", id=charging_station.id), json=message, ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] work_on_rq( diff --git a/flexmeasures/api/v3_0/tests/test_sensor_schedules.py b/flexmeasures/api/v3_0/tests/test_sensor_schedules.py index 6928c452ab..d4c28e4cd1 100644 --- a/flexmeasures/api/v3_0/tests/test_sensor_schedules.py +++ b/flexmeasures/api/v3_0/tests/test_sensor_schedules.py @@ -143,7 +143,7 @@ def test_trigger_schedule_preserves_flex_model_datetimes_in_job_kwargs( json=message, ) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 assert len(app.queues["scheduling"]) == 1 job = app.queues["scheduling"].jobs[0] @@ -184,7 +184,7 @@ def test_trigger_schedule_preserves_flex_model_datetimes_when_flooring_disabled( ) sensor.attributes = original_attributes - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 assert len(app.queues["scheduling"]) == 1 job = app.queues["scheduling"].jobs[0] @@ -298,7 +298,7 @@ def test_trigger_and_get_schedule_with_unknown_prices( ) print("Server responded with:\n%s" % trigger_schedule_response.json) check_deprecation(trigger_schedule_response, deprecation=None, sunset=None) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 job_id = trigger_schedule_response.json["schedule"] # look for scheduling jobs in queue @@ -337,6 +337,47 @@ def test_trigger_and_get_schedule_with_unknown_prices( assert "prices unknown" in get_schedule_response.json["message"].lower() +@pytest.mark.parametrize( + "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True +) +def test_get_schedule_unfinished_job_returns_202_when_sunset_active( + app, + add_battery_assets, + keep_scheduling_queue_empty, + requesting_user, +): + sensor = add_battery_assets["Test battery"].sensors[0] + original_sunset_active = app.config.get("FLEXMEASURES_API_SUNSET_ACTIVE") + app.config["FLEXMEASURES_API_SUNSET_ACTIVE"] = True + + with app.test_client() as client: + trigger_schedule_response = client.post( + url_for("SensorAPI:trigger_schedule", id=sensor.id), + json=message_for_trigger_schedule(), + ) + assert trigger_schedule_response.status_code == 202 + job_id = trigger_schedule_response.json["schedule"] + + get_schedule_response = client.get( + url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id), + ) + + assert get_schedule_response.status_code == 202 + assert get_schedule_response.json["status"] in {"QUEUED", "STARTED", "DEFERRED"} + assert "message" in get_schedule_response.json + + app.config["FLEXMEASURES_API_SUNSET_ACTIVE"] = False + with app.test_client() as client: + get_schedule_response_old = client.get( + url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id), + ) + + app.config["FLEXMEASURES_API_SUNSET_ACTIVE"] = original_sunset_active + + assert get_schedule_response_old.status_code == 400 + assert get_schedule_response_old.json["status"] == unknown_schedule()[0]["status"] + + @pytest.mark.parametrize( "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True ) @@ -410,7 +451,7 @@ def test_get_schedule_fallback( ) # check that the call is successful - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 job_id = trigger_schedule_response.json["schedule"] # look for scheduling jobs in queue @@ -567,7 +608,7 @@ def test_get_schedule_fallback_not_redirect( ) # check that the call is successful - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 job_id = trigger_schedule_response.json["schedule"] # look for scheduling jobs in queue @@ -655,7 +696,7 @@ def test_get_schedule_with_unit( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_schedule_response.status_code == 200, trigger_schedule_response.json + assert trigger_schedule_response.status_code == 202, trigger_schedule_response.json job_id = trigger_schedule_response.json["schedule"] # process the scheduling queue diff --git a/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py b/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py index 4ee841a502..d8eefa689e 100644 --- a/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py +++ b/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py @@ -73,7 +73,12 @@ def test_trigger_and_get_schedule( json=message, ) print("Server responded with:\n%s" % trigger_schedule_response.json) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 + assert trigger_schedule_response.json["job_results_url"] == url_for( + "SensorAPI:get_schedule", + id=sensor.id, + uuid=trigger_schedule_response.json["schedule"], + ) job_id = trigger_schedule_response.json["schedule"] # look for scheduling jobs in queue @@ -263,7 +268,7 @@ def test_trigger_schedule_uses_state_of_charge_sensor_for_soc_at_start( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 job_id = trigger_schedule_response.json["schedule"] work_on_rq(app.queues["scheduling"], exc_handler=handle_scheduling_exception) @@ -481,7 +486,7 @@ def test_price_sensor_priority( json=message, ) print("Server responded with:\n%s" % trigger_schedule_response.json) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 # Patch TimedBelief.search method with patch.object( @@ -572,7 +577,7 @@ def test_inflexible_device_sensors_priority( json=message, ) print("Server responded with:\n%s" % trigger_schedule_response.json) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 with patch( "flexmeasures.data.models.planning.storage.get_power_values", @@ -653,7 +658,7 @@ def test_multiple_contracts( json=message, ) print("Server responded with:\n%s" % trigger_schedule_response.json) - assert trigger_schedule_response.status_code == 200 + assert trigger_schedule_response.status_code == 202 job_id = trigger_schedule_response.json["schedule"] # process the scheduling queue @@ -860,7 +865,7 @@ def test_get_schedule_sign_convention_json_flex_model( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] work_on_rq(app.queues["scheduling"], exc_handler=handle_scheduling_exception) @@ -967,7 +972,7 @@ def test_get_schedule_sign_convention_db_flex_model( url_for("SensorAPI:trigger_schedule", id=sensor.id), json=message, ) - assert trigger_response.status_code == 200 + assert trigger_response.status_code == 202 job_id = trigger_response.json["schedule"] work_on_rq(app.queues["scheduling"], exc_handler=handle_scheduling_exception) diff --git a/flexmeasures/ui/static/openapi-specs.json b/flexmeasures/ui/static/openapi-specs.json index fd62b861ac..f58f90f0b9 100644 --- a/flexmeasures/ui/static/openapi-specs.json +++ b/flexmeasures/ui/static/openapi-specs.json @@ -821,6 +821,40 @@ } } }, + "202": { + "description": "Scheduling job status while the job is still running", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "string", + "enum": [ + "QUEUED", + "STARTED", + "DEFERRED" + ], + "description": "Processing status of the scheduling job." + }, + "message": { + "type": "string", + "description": "Human-readable message about current job processing." + } + } + }, + "examples": { + "started": { + "summary": "Scheduling job is still running", + "value": { + "status": "STARTED", + "message": "The scheduling job is currently running." + } + } + } + } + } + }, "400": { "description": "INVALID_TIMEZONE, INVALID_DOMAIN, UNKNOWN_SCHEDULE, UNRECOGNIZED_CONNECTION_GROUP" }, @@ -1353,32 +1387,61 @@ } }, "responses": { - "200": { - "description": "PROCESSED", + "202": { + "description": "ACCEPTED (Forecasting job queued for processing)", "content": { "application/json": { "schema": { "type": "object", "properties": { - "forecast": { + "job_id": { "type": "string", - "description": "UUID of the queued forecasting job." + "description": "UUID of the queued forecasting job (canonical field; use this)." }, "status": { "type": "string", "enum": [ - "PROCESSED" + "ACCEPTED" ] }, "message": { "type": "string" + }, + "job_monitor_url": { + "type": "string", + "format": "uri", + "description": "URL to query the generic job status API." + }, + "job_results_url": { + "type": "string", + "format": "uri", + "description": "URL to fetch the forecast results for this job once it is complete." + }, + "forecast": { + "type": "string", + "description": "(DEPRECATED) UUID of the queued forecasting job. Use `job_id` instead." + }, + "_deprecated_fields": { + "type": "object", + "description": "Machine-readable mapping of deprecated fields with migration guidance." } } }, "example": { + "status": "ACCEPTED", + "job_id": "b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b", + "message": "Forecasting job has been queued.", + "job_monitor_url": "/api/v3_0/jobs/b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b", + "job_results_url": "/api/v3_0/sensors/3/forecasts/b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b", "forecast": "b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b", - "status": "PROCESSED", - "message": "Forecasting job has been queued." + "_deprecated_fields": { + "forecast": { + "use": "job_id", + "deprecated_since": "1.0.0", + "removal_date": "2.0.0", + "note": "The 'forecast' response field is deprecated; use 'job_id' instead" + } + } } } } @@ -1503,21 +1566,65 @@ } }, "responses": { - "200": { - "description": "PROCESSED", + "202": { + "description": "ACCEPTED (Scheduling job queued for processing)", "content": { "application/json": { "schema": { - "type": "object" + "type": "object", + "properties": { + "job_id": { + "type": "string", + "description": "UUID of the queued scheduling job (canonical field; use this)." + }, + "status": { + "type": "string", + "enum": [ + "ACCEPTED" + ] + }, + "message": { + "type": "string" + }, + "job_monitor_url": { + "type": "string", + "format": "uri", + "description": "URL to query the generic job status API." + }, + "job_results_url": { + "type": "string", + "format": "uri", + "description": "URL to fetch the schedule results for this job once it is complete." + }, + "schedule": { + "type": "string", + "description": "(DEPRECATED) UUID of the queued scheduling job. Use `job_id` instead." + }, + "_deprecated_fields": { + "type": "object", + "description": "Machine-readable mapping of deprecated fields with migration guidance." + } + } }, "examples": { "schedule_created": { "summary": "Schedule response", - "description": "This message indicates that the scheduling request has been processed without any error.\nA scheduling job has been created with some Universally Unique Identifier (UUID),\nwhich will be picked up by a worker.\nThe given UUID may be used to obtain the resulting schedule: see /sensors//schedules/.\n", + "description": "This message indicates that the scheduling request has been accepted for processing (202 Accepted).\nA scheduling job has been created with some Universally Unique Identifier (UUID),\nwhich will be picked up by a worker.\nThe given UUID is returned in the canonical `job_id` field.\nFor backward-compatibility, the legacy `schedule` field is also included but is deprecated;\nuse `job_id` instead. The `_deprecated_fields` object provides migration guidance.\nThe given UUID may be used to obtain the resulting schedule: see /sensors//schedules/.\n", "value": { - "status": "PROCESSED", + "status": "ACCEPTED", + "job_id": "364bfd06-c1fa-430b-8d25-8f5a547651fb", "schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb", - "message": "Request has been processed." + "job_monitor_url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "job_results_url": "/api/v3_0/sensors/3/schedules/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "message": "Request has been accepted for processing.", + "_deprecated_fields": { + "schedule": { + "use": "job_id", + "deprecated_since": "1.0.0", + "removal_date": "2.0.0", + "note": "The 'schedule' response field is deprecated; use 'job_id'" + } + } } } } @@ -3986,8 +4093,8 @@ } }, "responses": { - "200": { - "description": "PROCESSED", + "202": { + "description": "ACCEPTED (Scheduling job queued for processing)", "content": { "application/json": { "schema": { @@ -3995,11 +4102,21 @@ }, "examples": { "successful_response": { - "description": "This message indicates that the scheduling request has been processed without any error.\nA scheduling job has been created with some Universally Unique Identifier (UUID),\nwhich will be picked up by a worker.\nThe given UUID may be used to obtain the resulting schedule for each flexible device: see [/sensors/schedules/](#/Sensors/get_api_v3_0_sensors__id__schedules__uuid_).\n", + "description": "This message indicates that the scheduling request has been accepted for processing (202 Accepted).\nA scheduling job has been created with some Universally Unique Identifier (UUID),\nwhich will be picked up by a worker.\nThe given UUID is returned in the canonical `job_id` field.\nFor backward-compatibility, the legacy `schedule` field is also included but is deprecated;\nuse `job_id` instead. The `_deprecated_fields` object provides migration guidance.\nThe given UUID may be used to obtain the resulting schedule for each flexible device via [/sensors/schedules/](#/Sensors/get_api_v3_0_sensors__id__schedules__uuid_).\n", "value": { - "status": "PROCESSED", + "status": "ACCEPTED", + "job_id": "364bfd06-c1fa-430b-8d25-8f5a547651fb", "schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb", - "message": "Request has been processed." + "job_monitor_url": "/api/v3_0/jobs/364bfd06-c1fa-430b-8d25-8f5a547651fb", + "message": "Request has been accepted for processing.", + "_deprecated_fields": { + "schedule": { + "use": "job_id", + "deprecated_since": "1.0.0", + "removal_date": "2.0.0", + "note": "The 'schedule' response field is deprecated; use 'job_id'" + } + } } } }