feat(gateway): typed DTO contract for HTTP responses and OpenAPI schema

README.md

@@ -98,7 +98,7 @@ Beyond faults, medkit exposes the full ROS 2 graph through REST:
 | **Software Updates** | Async prepare/execute lifecycle with pluggable backends |
 | **Authentication** | JWT-based RBAC (viewer, operator, configurator, admin) |
 | **Logs** | Log entries and configuration |
-| **Docs** | OpenAPI 3.1.0 spec and Swagger UI at `/api/v1/docs` |
+| **Docs** | OpenAPI 3.1.0 spec and Swagger UI at `/api/v1/docs` - schemas are generated from typed C++ structs so the spec always matches the wire format |
 
 On the [roadmap](https://selfpatch.github.io/ros2_medkit/roadmap.html): entity lifecycle control, mode management, communication logs.
 

docs/tutorials/openapi.rst

@@ -77,6 +77,28 @@ When disabled, all ``/docs`` endpoints return HTTP 501.
 
 See :doc:`/config/server` for the full parameter reference.
 
+How Schemas Are Generated
+--------------------------
+
+The ``components/schemas`` object in every ``/docs`` response is generated
+automatically from the DTO registry. Each response and request type in the
+gateway is declared as a plain C++ struct with a ``constexpr dto_fields<T>``
+descriptor tuple. The ``SchemaWriter<T>`` visitor folds over this tuple at
+compile time to produce the OpenAPI JSON Schema entry, and the
+``AllDtos`` registry in ``dto/registry.hpp`` lists every named type so that
+``collect_component_schemas()`` can populate the full schema map without
+any hand-written schema factories.
+
+The same descriptor is used for serialization (``JsonWriter<T>``) and
+request-body validation (``JsonReader<T>``), so the wire shape and the
+published schema are always derived from the same source. Genuinely dynamic
+payloads - such as live ROS 2 message data and free-form fault environment
+records - are typed as ``nlohmann::json`` members and appear in the schema
+as unconstrained objects (``{}``).
+
+For the full design of the DTO contract layer, see
+:doc:`/design/ros2_medkit_gateway/dto_contract`.
+
 See Also
 --------
 

docs/tutorials/plugin-system.rst

@@ -98,7 +98,7 @@ Writing a Plugin
      tl::expected<std::vector<std::string>, UpdateBackendErrorInfo>
        list_updates(const UpdateFilter& filter) override { /* ... */ }
 
-     tl::expected<nlohmann::json, UpdateBackendErrorInfo>
+     tl::expected<dto::UpdateDetail, UpdateBackendErrorInfo>
        get_update(const std::string& id) override { /* ... */ }
 
      tl::expected<void, UpdateBackendErrorInfo>
@@ -201,7 +201,7 @@ A self-contained plugin implementing UpdateProvider (copy-paste starting point):
        return std::vector<std::string>{};
      }
 
-     tl::expected<nlohmann::json, UpdateBackendErrorInfo>
+     tl::expected<dto::UpdateDetail, UpdateBackendErrorInfo>
      get_update(const std::string& id) override {
        return tl::make_unexpected(
          UpdateBackendErrorInfo{UpdateBackendError::NotFound, "not found: " + id});

src/ros2_medkit_gateway/CHANGELOG.rst

@@ -7,12 +7,67 @@ Unreleased
 
 **Breaking Changes:**
 
+* Typed router refactor. ``HandlerContext`` no longer carries
+  ``send_json`` / ``send_error`` / ``send_plugin_error`` / ``send_dto`` /
+  ``parse_body``: handlers return ``http::Result<TResponse>`` and the
+  framework owns response writing through ``RouteRegistry``. The raw
+  ``void(httplib::Request, httplib::Response)`` ``RouteRegistry`` lambda
+  overloads are removed - call sites must use the typed
+  ``reg.get<T>`` / ``reg.post<TBody, T>`` / ``reg.del<T>`` overloads, the
+  multi-shape ``reg.post_alternates<TBody, TAlt...>`` /
+  ``reg.del_alternates<TAlt...>``, or one of the named escape hatches
+  (``reg.sse`` / ``reg.binary_download`` / ``reg.multipart_upload<T>`` /
+  ``reg.static_asset`` / ``reg.docs_endpoint`` / ``reg.docs_subtree``).
+  ``static_assert(dto::has_dto_shape_v<T>)`` gates every typed overload, so
+  non-DTO return types fail at compile time. The plugin ABI is unaffected:
+  ``PluginResponse`` keeps its ``send_json`` / ``send_error`` surface and
+  now routes through the same internal ``http::detail::write_json_body``
+  primitive as the framework, so plugin wire format is unchanged
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* Provider ABI typed. ``FaultProvider``, ``DataProvider``,
+  ``OperationProvider``, and ``UpdateProvider::get_update`` return typed
+  DTO envelopes (``FaultListResult`` / ``FaultDetailResult`` /
+  ``FaultClearResult`` / the matching ``Data*Result`` and
+  ``Operation*Result`` shapes / ``UpdateStatusResult``) instead of raw
+  ``tl::expected<nlohmann::json, ErrorInfo>``. The wire bytes are
+  byte-identical because each envelope wraps an opaque ``content`` object
+  emitted verbatim by ``JsonWriter``; commercial and out-of-tree plugins
+  must wrap their existing JSON in the matching envelope type
+  (mechanical: ``Result.content = std::move(json_payload)``). The plugin
+  ABI itself (``PluginRoute`` shape, ``PluginResponse`` ctor, plugin api
+  version) is locked by ``test_plugin_abi_conformance`` and is unchanged
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* ``SchemaWriter`` emits optional DTO fields as
+  ``anyOf: [<inner>, {type: "null"}]`` (the OpenAPI 3.1 idiom) instead of
+  ``nullable: true``. Generated clients see ``T | null`` for every optional
+  field rather than ``T | undefined``. Wire format is unchanged - the
+  gateway still omits absent optional fields, and ``JsonReader`` continues
+  to accept absent fields; the schema change only opts the published spec
+  into round-tripping a literal ``null`` value cleanly for clients that
+  prefer to send one. The ``rtmaps_medkit`` variant is explicitly NOT
+  covered by this PR - its handlers continue to run on the pre-typed
+  HandlerContext surface and will be migrated separately
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* Synchronous operation-execution service-call failures
+  (``POST /api/v1/{entity-path}/operations/{id}/executions`` when the underlying
+  ROS 2 service call fails) now return the standard SOVD ``GenericError`` envelope
+  (``{"error_code": "vendor-error", "vendor_code":
+  "x-medkit-ros2-service-unavailable", "message": "Service call failed", ...}``,
+  HTTP status 500 unchanged) instead of the previous bespoke nested
+  ``{"error": {"code", "message", "details"}}`` object. This aligns the one
+  remaining non-standard error path with every other gateway error; clients that
+  parsed ``error.code`` / ``error.details`` for this specific failure must read
+  ``vendor_code`` / ``parameters`` instead
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
 * ``ros2_medkit_msgs/srv/ClearFault`` request gains a ``bool skip_correlation_auto_clear`` field (see the per-entity fault scope entry below for the in-tree motivation). Adding a request field changes the service type hash, so out-of-tree callers that invoke the service directly (for example ``ros2 service call /fault_manager/clear_fault ros2_medkit_msgs/srv/ClearFault ...`` as documented in the ``ros2_medkit_fault_manager`` README) must rebuild against the new ``ros2_medkit_msgs`` to keep talking to ``fault_manager``. The in-tree gateway client and server are updated together (`#395 <https://github.com/selfpatch/ros2_medkit/issues/395>`_)
 * Per-entity fault routes are now correctly scoped to the entity's hosted apps. ``GET /api/v1/{entity-path}/faults/{fault_code}``, ``DELETE /api/v1/{entity-path}/faults/{fault_code}``, ``GET /api/v1/{entity-path}/faults``, and ``DELETE /api/v1/{entity-path}/faults`` previously fell back to a prefix match against the entity's ``namespace_path``; when that was empty (host-derived / synthetic components, manifest components without a ``namespace`` field, Areas, Functions, and Apps with a wildcard ``ros_binding.namespace_pattern``) the scope filter was silently disabled and the routes exposed - and on ``DELETE``, cleared - faults reported by apps that belonged to entirely different entities. All four handlers now resolve the addressed entity to its hosted-app FQN set (via the new ``HandlerContext::resolve_entity_source_fqns`` helper) and apply a strict all-sources scope check: a fault counts as in scope only when **every** entry in its ``reporting_sources`` is owned by the entity (exact FQN match, or strict path-child via ``<fqn>/...``). Per-fault routes return ``404 Resource Not Found`` for any fault that fails the check; collection routes return an empty ``items`` array. The underlying ``GetFault.srv`` contract is unchanged; ``ClearFault.srv`` gains a new ``skip_correlation_auto_clear`` request flag so per-entity DELETE can opt out of cascade-clearing correlated symptom fault codes that may live in other entities. Per-entity collection responses no longer include the global ``muted_count`` / ``cluster_count`` / ``muted_faults`` / ``clusters`` correlation metadata; those remain on the global ``GET /api/v1/faults`` route. Behavior changes visible to clients: (a) faults reported by apps outside the addressed entity are no longer returned or cleared via that entity's route, (b) **mixed-source** faults that include at least one out-of-entity reporter are likewise rejected with ``404`` on per-fault routes and excluded from per-entity collection responses (use the global ``GET /api/v1/faults`` to see them), (c) per-entity DELETE no longer cascade-clears correlated symptoms outside the entity (`#395 <https://github.com/selfpatch/ros2_medkit/issues/395>`_)
 * ``GET /api/v1/updates/{id}/status`` no longer returns ``404`` for a registered-but-idle package; ``POST /api/v1/updates`` now seeds a ``pending`` status, so the endpoint returns ``200 {"status": "pending"}`` immediately after registration. ``404`` is reserved for packages that are not registered. Clients that used ``404`` as a signal for "registered but nothing started yet" must adapt (`#378 <https://github.com/selfpatch/ros2_medkit/issues/378>`_)
 
 **Features:**
 
+* Typed ``fan_out_collection<T>`` aggregating helper replaces raw-JSON ``merge_peer_items`` on the typed collection routes (data, operations, config, logs). Peer items are decoded via ``dto::JsonReader<T>``; items that fail validation are removed from the merged ``items`` array, recorded in ``x-medkit.peer_dropped_items`` with the JsonReader error plus a best-effort ``source_id``, and logged at ``WARN``. Items that parse successfully are re-serialized through the local ``dto::JsonWriter<T>``, so any peer-supplied fields outside the local DTO schema are dropped from the merged response (the previous raw passthrough preserved unknown peer fields verbatim). Previously, malformed peer items silently disappeared into the merged response; fleet operators can now detect inter-gateway schema drift directly on the wire (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* ``Collection<T, XMedkitT>`` is now a 2-parameter template. Domain list endpoints (faults, config, data, logs) reference their richer per-domain collection x-medkit struct (``FaultListXMedkit``, ``ConfigListXMedkit``, ``DataListXMedkit``, ``LogListXMedkit``) directly in the published schema instead of the generic ``XMedkitCollection``, so generated clients see aggregation counts, peer provenance, and ``peer_dropped_items`` from the schema (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* New ``opaque_object("key", &T::field)`` DTO field descriptor in ``dto/contract.hpp``. Binds a ``nlohmann::json`` member as a typed "any JSON object" field: ``JsonWriter`` emits it verbatim, ``JsonReader`` rejects scalars / arrays / null, ``SchemaWriter`` emits ``{type: object, additionalProperties: true, x-medkit-opaque: true}``. Used for fields whose runtime shape is decided by an upstream component the gateway cannot introspect (live ROS message payloads, plugin-defined fault envelopes, action results) (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
 * ``GET /api/v1/faults/stream`` event payloads now carry an optional ``x-medkit`` SOVD payload-extension object with ``entity_type`` and ``entity_id`` fields. When the gateway can resolve the fault's first reporting source back to a SOVD entity (via the manifest-mode linking index, or a runtime-mode last-segment match against an existing App), consumers can hit ``/{entity_type}/{entity_id}/bulk-data/rosbags/{fault_code}`` directly instead of HEAD-probing every entity. Resolution is snapshotted at event arrival, so a discovery refresh between enqueue and stream-out cannot retroactively change the entity reported to consumers. The ``x-medkit`` object is omitted entirely when no entity can be resolved, so existing SSE consumers ignore the addition (`#380 <https://github.com/selfpatch/ros2_medkit/issues/380>`_)
 * Plugin API version bumped to v7. Adds ``PluginContext::notify_entities_changed(EntityChangeScope)`` lifecycle hook for plugins that mutate the entity surface at runtime; default no-op keeps v6 source code compiling unchanged against v7 headers. Binary compatibility is not provided: the plugin loader uses a strict equality check on ``plugin_api_version()``, so out-of-tree plugins must be recompiled (`#376 <https://github.com/selfpatch/ros2_medkit/issues/376>`_)
 * New ``discovery.manifest.fragments_dir`` parameter: gateway scans the directory for ``*.yaml`` / ``*.yml`` fragment files on every manifest load / reload and merges apps, components, and functions on top of the base manifest. Fragments are forbidden from declaring top-level ``areas``, ``metadata``, ``discovery``, ``scripts``, ``capabilities``, or ``lock_overrides`` - those stay in the base manifest. Presence of any forbidden key (including empty-valued ones like ``areas: []``) is reported as a ``FRAGMENT_FORBIDDEN_FIELD`` validation error that fails the load / reload. Unknown top-level keys (typos such as ``app:`` vs ``apps:``) are ignored with a warning log. Files merged in alphabetical order for deterministic duplicate-id errors (`#376 <https://github.com/selfpatch/ros2_medkit/issues/376>`_)

src/ros2_medkit_gateway/CMakeLists.txt

@@ -171,6 +171,7 @@ add_library(gateway_ros2 STATIC
   src/discovery/merge_pipeline.cpp
   src/fault_manager_paths.cpp
   src/gateway_node.cpp
+  src/http/fan_out_helpers.cpp
   src/http/handlers/auth_handlers.cpp
   src/http/handlers/bulkdata_handlers.cpp
   src/http/handlers/config_handlers.cpp
@@ -631,9 +632,25 @@ if(BUILD_TESTING)
   target_link_libraries(test_handler_context gateway_ros2)
   medkit_set_test_domain(test_handler_context)
 
-  # Add x-medkit extension tests
-  ament_add_gtest(test_x_medkit test/test_x_medkit.cpp)
-  target_link_libraries(test_x_medkit gateway_ros2)
+  # DTO contract core (pure C++17, no ROS node)
+  ament_add_gtest(test_dto_contract test/test_dto_contract.cpp)
+  target_link_libraries(test_dto_contract gateway_core)
+
+  # DTO Collection<T, XMedkit> template + fan-out observability fields
+  ament_add_gtest(test_collection test/test_collection.cpp)
+  target_link_libraries(test_collection gateway_core)
+
+  # DTO opaque_object machinery (pure C++17, no ROS node)
+  ament_add_gtest(test_opaque_object test/test_opaque_object.cpp)
+  target_link_libraries(test_opaque_object gateway_core)
+
+  # Typed router foundational types (pure C++17, no ROS node)
+  ament_add_gtest(test_typed_router test/test_typed_router.cpp)
+  target_link_libraries(test_typed_router gateway_core)
+
+  # Framework response-writing primitives (pure C++17, no ROS node)
+  ament_add_gtest(test_primitives test/test_primitives.cpp)
+  target_link_libraries(test_primitives gateway_core)
 
   # Add rate limiter tests
   ament_add_gtest(test_rate_limiter test/test_rate_limiter.cpp)
@@ -798,6 +815,16 @@ if(BUILD_TESTING)
   ament_add_gtest(test_plugin_http_types test/test_plugin_http_types.cpp)
   target_link_libraries(test_plugin_http_types gateway_ros2)
 
+  # Plugin ABI conformance tests
+  # Pins the public plugin ABI (PluginRoute, PluginRequest, PluginResponse,
+  # PLUGIN_API_VERSION) via static_assert + loads test_gateway_plugin.so via
+  # the production PluginLoader entry point. Locks in the wire-format contract
+  # so refactors that would break commercial out-of-tree plugins (UDS, OPC-UA,
+  # Uptane OTA, Mender OTA) fail loudly here.
+  ament_add_gtest(test_plugin_abi_conformance test/test_plugin_abi_conformance.cpp)
+  target_link_libraries(test_plugin_abi_conformance gateway_ros2)
+  medkit_target_dependencies(test_plugin_abi_conformance ament_index_cpp)
+
   # Log manager tests
   # Dedicated ROS_DOMAIN_ID to prevent cross-talk with concurrent integration tests
   ament_add_gtest(test_log_manager test/test_log_manager.cpp)
@@ -837,6 +864,10 @@ if(BUILD_TESTING)
   ament_add_gtest(test_route_registry test/test_route_registry.cpp)
   target_link_libraries(test_route_registry gateway_ros2)
 
+  # Typed RouteRegistry overload tests (PR-403 commit 4 typed router wrappers)
+  ament_add_gtest(test_typed_route_registry test/test_typed_route_registry.cpp)
+  target_link_libraries(test_typed_route_registry gateway_core)
+
   # Docs handlers tests (OpenAPI /docs endpoint handlers)
   ament_add_gtest(test_docs_handlers test/test_docs_handlers.cpp)
   target_link_libraries(test_docs_handlers gateway_ros2)
@@ -929,7 +960,6 @@ if(BUILD_TESTING)
       test_manifest_manager
       test_capability_builder
       test_handler_context
-      test_x_medkit
       test_rate_limiter
       test_auth_config
       test_data_access_manager
@@ -956,6 +986,7 @@ if(BUILD_TESTING)
       test_plugin_loader
       test_plugin_manager
       test_plugin_http_types
+      test_plugin_abi_conformance
       test_log_manager
       test_log_handlers
       test_merge_pipeline
@@ -967,6 +998,7 @@ if(BUILD_TESTING)
       test_path_resolver
       test_capability_generator
       test_route_registry
+      test_typed_route_registry
       test_docs_handlers
       test_lock_manager
       test_lock_handlers