diff --git a/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/.openspec.yaml b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/.openspec.yaml new file mode 100644 index 00000000..38f76288 --- /dev/null +++ b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-06-22 diff --git a/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/design.md b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/design.md new file mode 100644 index 00000000..8b1f4d5d --- /dev/null +++ b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/design.md @@ -0,0 +1,157 @@ +## Context + +BartonCore's camera SBMD driver (`camera.sbmd.js`) implements the abstract session lifecycle endpoint (`ep/camera`) with `createSession`, `stream`, `destroySession`, and `sessionStatus` resources. When a client executes `stream`, the driver emits a `sessionStatus` event with `nextAction: "/devices//ep/webrtc/r/offerSdp"` — but this endpoint does not yet exist. The signaling exchange cannot proceed. + +The Matter SDK build already provides generated headers for `WebRTCTransportProvider` (0x0553) and `WebRTCTransportRequestor` (0x0554) clusters. The SBMD runtime supports `commandHandlers` for incoming Matter commands, and `device.sendCommand()` for outgoing commands. The infrastructure is ready — we need to wire it together. + +The reference app currently has generic `execResource`/`readResource` commands but no camera-specific workflow. Testing a camera stream requires 6+ manual commands with event monitoring. A dedicated command with integrated media rendering is needed for development and demonstration. + +``` +┌─────────────────────────────────────────────────────────────────────────────┐ +│ Current Architecture │ +│ │ +│ Client (ref app) Barton (camera.sbmd.js) Camera │ +│ ═══════════════ ═══════════════════════ ══════ │ +│ │ +│ er(createSession) ──────► executeCreateSession() │ +│ ◄── response: "1" (allocates session) │ +│ │ +│ er(stream, "1") ────────► executeStream() │ +│ ◄── sessionStatus event (emits setup + nextAction) │ +│ nextAction: .../ep/webrtc/r/offerSdp │ +│ │ +│ er(offerSdp, sdp) ─────► ??? ep/webrtc DOES NOT EXIST │ +│ │ +└─────────────────────────────────────────────────────────────────────────────┘ +``` + +## Goals / Non-Goals + +**Goals:** +- Implement the `ep/webrtc` endpoint in the camera SBMD with full signaling resource set +- Map Barton resource executes to outgoing Matter commands (WebRTCTransportProvider cluster) +- Map incoming Matter commands (WebRTCTransportRequestor cluster) to Barton resource events +- Build a reference app `cameraStream` command that orchestrates the full flow with GStreamer media rendering +- Structure the webrtc endpoint code for future extraction to a standalone driver + +**Non-Goals:** +- Adding a WebRTC media stack to Barton core (Barton is signaling-only) +- OpenHome or direct camera endpoint implementation +- STUN/TURN server configuration or NAT traversal beyond what ICE provides +- Multi-stream support or bidirectional audio +- Changes to BCoreClient public API +- Dynamic endpoint registration (future improvement — using static ZAP endpoint for now) +- Production-quality error recovery or session timeout management + +## Decisions + +### D1: WebRTC endpoint colocated in camera.sbmd.js + +**Decision**: Add `ep/webrtc` as a second endpoint in `camera.sbmd.js` rather than creating a separate `webrtc.sbmd.js`. + +**Rationale**: SBMD currently has no cross-driver composition mechanism. The session lifecycle in `ep/camera` and the signaling in `ep/webrtc` share transient data (session state, protocol info). Keeping them in one file allows shared access to session state via transient data supplements. The code is structured with clear separation (grouped constants, dedicated handler functions) so extraction is straightforward when SBMD composition becomes available. + +**Alternatives considered**: +- Separate `webrtc.sbmd.js`: Would require a cross-driver data sharing mechanism that doesn't exist. The SBMD runtime matches one driver per device — a second driver for the same device type isn't supported. +- Native C++ driver for webrtc: Defeats the purpose of SBMD. The signaling is pure data transformation (resource values → TLV commands, TLV commands → resource events) which SBMD handles well. + +### D2: Barton as signaling relay — no media involvement + +**Decision**: Barton's webrtc endpoint relays signaling strings (SDP, ICE) between the client and camera via Matter commands. Barton never instantiates a peer connection or receives media. + +**Rationale**: Barton is a protocol-agnostic device management library. The media consumer varies per deployment (mobile app, cloud service, reference app). Embedding a media stack would couple Barton to a specific rendering environment and add large dependencies (libdatachannel/GStreamer) to the core library. + +``` +┌────────────────────────────────────────────────────────────────────────────┐ +│ Target Architecture │ +│ │ +│ Ref App (WebRTC peer) Barton (signaling relay) Camera │ +│ ═════════════════════ ════════════════════════ ══════ │ +│ GStreamer + webrtcbin camera.sbmd.js Matter │ +│ │ +│ 1. er(createSession) ────► allocate session ────────────────────────── │ +│ 2. er(stream, sid) ──────► emit sessionStatus(setup, webrtc, offerSdp) │ +│ 3. [create local PC] │ +│ [generate SDP offer] │ +│ 4. er(offerSdp, sdp) ───► sendCommand(0x0553, ProvideOffer, {sdp}) ──► │ +│ ◄── Offer cmd (0x0554, {sdp_answer}) ───── │ +│ 5. ◄── remoteSdp event ─── updateResource(webrtc, remoteSdp, answer) │ +│ [set remote SDP] │ +│ 6. er(offerIce, [...]) ──► sendCommand(0x0553, ProvideICE, {cands}) ──► │ +│ ◄── ICECandidates cmd (0x0554, {cands}) ── │ +│ 7. ◄── remoteIce event ── updateResource(webrtc, remoteIce, cands) │ +│ [add remote ICE] │ +│ 8. [media flows P2P] ◄═══════════════════════════════════════════════► │ +│ 9. er(destroySession) ──► sendCommand(0x0553, EndSession) ───────────► │ +│ │ +└────────────────────────────────────────────────────────────────────────────┘ +``` + +**Thread safety**: All SBMD handler invocations run under `MQuickJsRuntime::GetMutex()`. The `device.sendCommand()` result terminal marshals to the Matter event loop. Incoming commands from Matter arrive via `g_main_context_invoke` before dispatching to SBMD. No additional synchronization needed. + +### D2b: WebRTCTransportRequestor cluster added to ZAP endpoint + +**Decision**: Add the `WebRTCTransportRequestor` cluster (0x0554) as a server cluster on Barton's existing endpoint in `barton-library.matter` and `barton-library.zap`. Handle its commands (Offer, Answer, ICECandidates, End) via the existing `CommandHandlerInterfaceRegistry` mechanism. + +**Rationale**: When Barton sends `ProvideOffer` to the camera, it includes an `originatingEndpointID`. The camera sends signaling commands (Offer, ICECandidates, End) back to that endpoint. For the camera to accept this endpoint as a valid target, Barton must advertise cluster 0x0554 in its Descriptor cluster's server list. Without this, cameras may reject the target or fail to route the command. + +**Alternatives considered**: +- Dynamic endpoint registration at runtime (like Matter SDK's `WebRTCTransportRequestorManager`): Cleaner long-term, but Barton has no existing dynamic endpoint infrastructure. Adding it is a separate effort. +- No ZAP change, rely on `CommandHandlerInterface` wildcard: The wildcard intercepts commands regardless of target endpoint, but the camera may refuse to send to an endpoint that doesn't advertise the cluster. Unreliable. + +### D3: Reference app uses GStreamer webrtcbin — no Matter or libdatachannel dependency + +**Decision**: The reference app uses GStreamer's `webrtcbin` element as its WebRTC peer. It does NOT use Matter's `WebRTCClient` class or link libdatachannel directly. + +**Rationale**: The reference app is a Barton client — it talks exclusively through `BCoreClient` APIs. Using Matter's WebRTC classes would violate the abstraction boundary. GStreamer `webrtcbin` provides a complete WebRTC stack (SDP generation, ICE, DTLS/SRTP, media decode) in one element, and the reference app already lives in GLib/GObject land. No bridging or separate UDP forwarding needed. + +**GStreamer pipeline structure**: +``` +webrtcbin name=webrtc + → decodebin → videoconvert → autovideosink (display mode) + → decodebin → x264enc → mp4mux → filesink (file mode) +``` + +### D4: Command naming — `cameraStream` / `cs` + +**Decision**: The reference app command is named `cameraStream` (short: `cs`), not generic `stream`. + +**Rationale**: "stream" is too ambiguous — Barton may stream data/media from non-camera devices in the future. `cameraStream` clearly indicates camera video streaming. Follows existing reference app naming: `discoverStart`/`dstart`, `printDevice`/`pd`, `readResource`/`rr`. + +**Usage**: +``` +cameraStream [--file ] +``` +Default: display output. If `--file` specified: record to file. If display unavailable and no `--file`: error with guidance. + +### D5: Signaling flow maps to Matter WebRTC clusters + +**Decision**: Map Barton resource operations to specific Matter cluster commands: + +| Barton operation | Direction | Matter cluster | Command | ID | +|---|---|---|---|---| +| execute `offerSdp` | Barton → Camera | WebRTCTransportProvider (0x0553) | ProvideOffer | 0x02 | +| execute `offerIceCandidates` | Barton → Camera | WebRTCTransportProvider (0x0553) | ProvideICECandidates | 0x05 | +| event `remoteSdp` | Camera → Barton | WebRTCTransportRequestor (0x0554) | Offer | 0x00 | +| event `remoteIceCandidates` | Camera → Barton | WebRTCTransportRequestor (0x0554) | ICECandidates | 0x02 | +| execute `destroySession` | Barton → Camera | WebRTCTransportProvider (0x0553) | EndSession | 0x06 | + +**Rationale**: These mappings follow from the Matter 1.5 camera specification. `SolicitOffer` (0x00) is used when the controller wants the camera to generate an offer — but in our flow, the client (reference app) generates the offer and passes it through Barton, so `ProvideOffer` is the primary path. + +### D6: SDP and ICE payloads are opaque strings in Barton resources + +**Decision**: SDP offers/answers are stored as plain string resource values. ICE candidates are JSON-encoded arrays of candidate strings. Barton does not parse, validate, or transform these payloads. + +**Rationale**: Barton is a relay. SDP and ICE are negotiated between the actual WebRTC peers (client and camera). Parsing them would add fragile protocol-version-specific logic to the service layer with no benefit. The SBMD handler simply packages the string into TLV for the Matter command and unpacks TLV back to a string for the event. + +## Risks / Trade-offs + +- **[TLV encoding complexity in SBMD]** → The SBMD `device.sendCommand()` requires TLV-encoded payloads. SDP strings and ICE candidate lists must be serialized to Matter TLV format from JavaScript. Mitigation: Use `Sbmd.tlv` helpers already available in the runtime (base64 TLV encoding via `sbmd-tlv.js`). +- **[GStreamer availability on target platforms]** → The `cameraStream` command requires GStreamer + webrtcbin. Not all deployments will have this. Mitigation: Gate behind `BCORE_CAMERA_STREAM` CMake flag; command prints clear error if GStreamer unavailable at runtime. +- **[Session correlation between endpoints]** → The webrtc endpoint needs to know which session is active to correlate signaling. Mitigation: The session is stored in transient data; webrtc execute handlers read session state via supplements. +- **[No SolicitOffer support initially]** → Some cameras may expect the controller to call `SolicitOffer` first. The initial implementation uses `ProvideOffer` (client generates offer). Mitigation: Can add `SolicitOffer` path later as an alternative flow triggered by a flag or automatic detection. + +## Open Questions + +1. **Should `cameraStream` support a `--stun` flag for specifying a STUN server URL?** webrtcbin supports this via the `stun-server` property. Likely useful for testing across network boundaries but not needed for local dev. +2. **How should the reference app handle `sessionStatus: "error"` events during streaming?** Options: immediate teardown with error message, or retry logic. Leaning toward simple teardown for MVP. diff --git a/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/proposal.md b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/proposal.md new file mode 100644 index 00000000..633fe99c --- /dev/null +++ b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/proposal.md @@ -0,0 +1,36 @@ +## Why + +The camera SBMD driver defines an abstract session lifecycle (createSession, stream, destroySession, sessionStatus) but has no protocol-specific endpoint to carry WebRTC signaling. Clients currently have no way to exchange SDP offers/answers or ICE candidates through Barton's resource model. Additionally, the reference app has no camera-specific command — testing camera streams requires manual multi-step `execResource` calls with no media rendering. + +## What Changes + +- **WebRTC endpoint in camera SBMD**: Add an `ep/webrtc` endpoint (profile: `"webrtc"`) to `camera.sbmd.js` with four resources — `offerSdp` [execute], `remoteSdp` [events], `offerIceCandidates` [execute], `remoteIceCandidates` [events]. Execute handlers send Matter commands to the camera's `WebRTCTransportProvider` cluster (0x0553). SBMD command handlers receive incoming signaling from the camera's `WebRTCTransportRequestor` cluster (0x0554) and emit events on the protocol endpoint resources. +- **Reference app `cameraStream` command**: A convenience orchestrator (`cameraStream` / `cs`) that automates the full camera streaming flow — session creation, WebRTC signaling via Barton's resource API, and media rendering via a GStreamer `webrtcbin` pipeline. Supports display output (autovideosink) and file recording (filesink). Barton remains a signaling relay; the reference app is the WebRTC peer and media consumer. +- **Separability by design**: The webrtc endpoint code is structured within `camera.sbmd.js` so it can be extracted to a standalone SBMD driver in the future without breaking the contract. + +## Non-goals + +- Media stack in Barton core — Barton relays signaling only, never receives media +- OpenHome or direct camera endpoint implementation (separate future work) +- STUN/TURN server integration +- Multi-stream or bidirectional audio support +- Changes to the public C API (BCoreClient already supports executeResource and event subscriptions) + +## Capabilities + +### New Capabilities +- `webrtc-signaling-endpoint`: The WebRTC protocol-specific endpoint (ep/webrtc) with resources for SDP and ICE exchange, SBMD command handlers for incoming Matter signaling, and execute handlers for outgoing Matter commands. Designed as a reusable pattern for any device type that uses WebRTC transport. +- `camera-stream-reference-command`: The reference app `cameraStream` command that orchestrates session lifecycle, WebRTC signaling through Barton, and media rendering via GStreamer webrtcbin. Demonstrates end-to-end camera streaming without direct Matter SDK or WebRTC library coupling in the client. + +### Modified Capabilities +_(none — no existing spec-level requirements are changing)_ + +## Impact + +- **SBMD driver**: `core/deviceDrivers/matter/sbmd/specs/camera.sbmd.js` — new endpoint, resources, constants, command handlers, execute handlers +- **Reference app**: `reference/src/` — new `cameraCategory.c/.h` for the `cameraStream` command, GStreamer pipeline management, event subscription handling +- **Reference app build**: `reference/CMakeLists.txt` — link against GStreamer (gstreamer-1.0, gstreamer-webrtc-1.0, gstreamer-sdp-1.0) +- **Docker image**: `gstreamer1.0-tools` and `libgstreamer-plugins-bad1.0-dev` added (version 2.12) +- **CMake flags**: Gated behind `BCORE_MATTER` (webrtc endpoint is Matter-only); reference app GStreamer support gated behind a new `BCORE_CAMERA_STREAM` flag +- **ZAP changes**: Add `WebRTCTransportRequestor` cluster (0x0554) as a server cluster on Barton's endpoint in `barton-library.matter` / `barton-library.zap`. The camera needs to send signaling commands (Offer, Answer, ICECandidates, End) back to Barton — this requires Barton to advertise the cluster so the camera knows where to target those commands. +- **Matter SDK headers used**: `WebRTCTransportProvider/CommandIds.h`, `WebRTCTransportRequestor/CommandIds.h` for cluster/command ID constants (already in build/matter-install) diff --git a/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/specs/camera-stream-reference-command/spec.md b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/specs/camera-stream-reference-command/spec.md new file mode 100644 index 00000000..f85ac1e1 --- /dev/null +++ b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/specs/camera-stream-reference-command/spec.md @@ -0,0 +1,117 @@ +## ADDED Requirements + +### Requirement: cameraStream command exists in reference app + +The reference app SHALL provide a command named `cameraStream` with short alias `cs` in a dedicated camera command category. The command SHALL accept a device ID as a required argument and an optional `--file ` flag for recording. + +#### Scenario: Command appears in help +- **WHEN** a user types `help` in the reference app +- **THEN** the camera category SHALL list `cameraStream` with usage: ` [--file ]` + +#### Scenario: Command with short alias +- **WHEN** a user types `cs ` +- **THEN** the command SHALL execute identically to `cameraStream ` + +### Requirement: cameraStream orchestrates full session lifecycle + +The `cameraStream` command SHALL orchestrate the complete camera streaming flow through Barton's resource API: +1. Execute `createSession` on the device's `ep/camera` endpoint +2. Execute `stream` with the returned sessionId +3. Wait for `sessionStatus` event with status `"setup"` and extract `nextAction` +4. Create a local GStreamer webrtcbin peer connection +5. Generate a local SDP offer from webrtcbin +6. Execute `offerSdp` on the device's `ep/webrtc` endpoint with the local SDP +7. Wait for `remoteSdp` event and set the remote description on webrtcbin +8. Exchange ICE candidates (local → `offerIceCandidates`, remote ← `remoteIceCandidates` events) +9. Wait for media to flow (peer connection state: connected) +10. On user interrupt (Ctrl+C or `q`): execute `destroySession` and tear down the pipeline + +#### Scenario: Successful camera stream to display +- **WHEN** user executes `cameraStream ` and the camera responds to signaling +- **THEN** the reference app SHALL display video output via GStreamer autovideosink and print status messages for each step + +#### Scenario: Successful camera stream to file +- **WHEN** user executes `cameraStream --file recording.mp4` +- **THEN** the reference app SHALL record the video stream to the specified file path + +#### Scenario: User stops the stream +- **WHEN** user presses Ctrl+C or types `q` during an active stream +- **THEN** the reference app SHALL execute `destroySession`, stop the GStreamer pipeline, and return to the command prompt + +### Requirement: cameraStream uses only BCoreClient API for signaling + +The `cameraStream` command SHALL interact with Barton exclusively through `BCoreClient` APIs (`b_core_client_execute_resource`, event subscriptions). It SHALL NOT use Matter SDK APIs, link against Matter libraries, or reference Matter-specific types. + +#### Scenario: No Matter SDK dependency +- **WHEN** the reference app is compiled +- **THEN** the camera stream module SHALL compile without any Matter SDK headers in its include path + +### Requirement: cameraStream uses GStreamer webrtcbin for media + +The `cameraStream` command SHALL use GStreamer's `webrtcbin` element as its local WebRTC peer connection. webrtcbin handles SDP generation, ICE gathering, DTLS/SRTP negotiation, and media decoding. + +#### Scenario: GStreamer pipeline for display +- **WHEN** `cameraStream` is invoked without `--file` +- **THEN** a GStreamer pipeline SHALL be created with `webrtcbin` connected to `decodebin` and `autovideosink` + +#### Scenario: GStreamer pipeline for file recording +- **WHEN** `cameraStream` is invoked with `--file ` +- **THEN** a GStreamer pipeline SHALL be created with `webrtcbin` connected to appropriate muxing and `filesink` elements + +#### Scenario: GStreamer not available +- **WHEN** GStreamer libraries are not found at runtime +- **THEN** the command SHALL print an error message explaining that GStreamer with webrtcbin is required and exit gracefully + +### Requirement: cameraStream subscribes to Barton events + +The `cameraStream` command SHALL subscribe to resource events on the device to receive signaling data asynchronously. Specifically: +- `sessionStatus` events on `ep/camera` (for session state transitions) +- `remoteSdp` events on `ep/webrtc` (for the camera's SDP answer) +- `remoteIceCandidates` events on `ep/webrtc` (for the camera's ICE candidates) + +#### Scenario: Remote SDP delivered via event +- **WHEN** the camera responds with an SDP answer +- **THEN** the reference app SHALL receive it as a `remoteSdp` event and feed it to webrtcbin as the remote description + +#### Scenario: Remote ICE candidates delivered via events +- **WHEN** the camera sends ICE candidates +- **THEN** the reference app SHALL receive them as `remoteIceCandidates` events and add each candidate to webrtcbin + +### Requirement: cameraStream reports progress to user + +The command SHALL emit human-readable progress messages to stdout at each stage of the flow: +- Session created (sessionId) +- Streaming initiated (protocol, nextAction) +- SDP offer sent +- SDP answer received +- ICE candidates exchanged +- Media flowing / connected +- Stream ended (reason) + +#### Scenario: Progress output during successful stream +- **WHEN** `cameraStream` completes signaling and media begins flowing +- **THEN** the user SHALL see step-by-step status messages indicating progress through the flow + +### Requirement: cameraStream handles errors gracefully + +The command SHALL handle failures at any stage (session creation failure, signaling timeout, peer connection failure) by printing an error message, cleaning up any partial state (destroying the session if created), and returning to the command prompt. + +#### Scenario: Device does not support camera streaming +- **WHEN** `cameraStream` is executed on a device without a `camera` endpoint +- **THEN** the command SHALL print an error and exit without crashing + +#### Scenario: Signaling timeout +- **WHEN** the camera does not respond to signaling within a reasonable timeout +- **THEN** the command SHALL print a timeout error, destroy the session, and exit + +### Requirement: Camera command category is gated by CMake flag + +The camera stream command and its GStreamer dependencies SHALL be gated behind a `BCORE_CAMERA_STREAM` CMake option (default OFF). When disabled, the reference app builds without GStreamer dependencies and without the camera category. + +#### Scenario: Build without camera stream support +- **WHEN** `BCORE_CAMERA_STREAM=OFF` (default) +- **THEN** the reference app SHALL build successfully without GStreamer development libraries + +#### Scenario: Build with camera stream support +- **WHEN** `BCORE_CAMERA_STREAM=ON` +- **THEN** the reference app SHALL link against gstreamer-1.0, gstreamer-webrtc-1.0, gstreamer-sdp-1.0 and include the camera category diff --git a/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/specs/webrtc-signaling-endpoint/spec.md b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/specs/webrtc-signaling-endpoint/spec.md new file mode 100644 index 00000000..93265168 --- /dev/null +++ b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/specs/webrtc-signaling-endpoint/spec.md @@ -0,0 +1,107 @@ +## ADDED Requirements + +### Requirement: WebRTC endpoint declares signaling resources + +The camera SBMD driver SHALL declare an endpoint with id `"webrtc"` and profile `"webrtc"` containing four resources: + +| Resource | Type | Modes | Purpose | +|----------|------|-------|---------| +| `offerSdp` | `function` | execute | Client sends local SDP offer | +| `remoteSdp` | `string` | [] (events only) | Delivers remote SDP answer to client | +| `offerIceCandidates` | `function` | execute | Client sends local ICE candidates | +| `remoteIceCandidates` | `string` | [] (events only) | Delivers remote ICE candidates to client | + +The endpoint SHALL be declared within the same `camera.sbmd.js` file as the `ep/camera` endpoint. + +#### Scenario: Endpoint appears on commissioned camera device +- **WHEN** a Matter camera device (deviceType 0x0142) with WebRTCTransportProvider cluster (0x0553) is commissioned +- **THEN** the device SHALL have an endpoint with id `"webrtc"`, profile `"webrtc"`, and all four resources registered + +#### Scenario: Event-only resources are not readable +- **WHEN** a client attempts to read `remoteSdp` or `remoteIceCandidates` +- **THEN** the read SHALL fail or return no value (modes list is empty — no read mode) + +### Requirement: offerSdp execute sends ProvideOffer to camera + +The `offerSdp` execute handler SHALL send a `ProvideOffer` command (ID 0x02) to the camera's `WebRTCTransportProvider` cluster (0x0553). The execute input is the client's SDP offer string. The handler SHALL package the SDP into the command's TLV payload along with the active session's `webRTCSessionID`. + +#### Scenario: Client provides SDP offer +- **WHEN** a client executes `offerSdp` with a valid SDP string as input +- **THEN** the SBMD handler SHALL send a `ProvideOffer` command to the camera with the SDP in the `sdp` field and the Matter-side webRTCSessionID in the `webRTCSessionID` field + +#### Scenario: No active session +- **WHEN** a client executes `offerSdp` but no session is in `streaming` state +- **THEN** the handler SHALL return an error result + +### Requirement: offerIceCandidates execute sends ProvideICECandidates to camera + +The `offerIceCandidates` execute handler SHALL send a `ProvideICECandidates` command (ID 0x05) to the camera's `WebRTCTransportProvider` cluster (0x0553). The execute input is a JSON-encoded array of ICE candidate strings. + +#### Scenario: Client provides ICE candidates +- **WHEN** a client executes `offerIceCandidates` with a JSON array of ICE candidate strings +- **THEN** the SBMD handler SHALL send a `ProvideICECandidates` command to the camera with the candidates in the `ICECandidates` field + +### Requirement: Incoming Offer command emits remoteSdp event + +The SBMD driver SHALL register a command handler for the `Offer` command (ID 0x00) on the `WebRTCTransportRequestor` cluster (0x0554). When received, the handler SHALL extract the SDP string and emit it as an event on the `remoteSdp` resource of the `webrtc` endpoint. + +#### Scenario: Camera sends SDP answer +- **WHEN** the camera sends an `Offer` command (cluster 0x0554, command 0x00) containing an SDP string +- **THEN** the SBMD handler SHALL call `updateResource('webrtc', 'remoteSdp', sdpString)` to emit an event to subscribed clients + +### Requirement: Incoming ICECandidates command emits remoteIceCandidates event + +The SBMD driver SHALL register a command handler for the `ICECandidates` command (ID 0x02) on the `WebRTCTransportRequestor` cluster (0x0554). When received, the handler SHALL extract the candidate list and emit it as a JSON-encoded array on the `remoteIceCandidates` resource. + +#### Scenario: Camera sends ICE candidates +- **WHEN** the camera sends an `ICECandidates` command (cluster 0x0554, command 0x02) containing ICE candidates +- **THEN** the SBMD handler SHALL call `updateResource('webrtc', 'remoteIceCandidates', jsonCandidates)` to emit an event to subscribed clients + +### Requirement: Incoming End command emits sessionStatus error + +The SBMD driver SHALL register a command handler for the `End` command (ID 0x03) on the `WebRTCTransportRequestor` cluster (0x0554). When received, the handler SHALL emit a `sessionStatus` event with value `"error"` and metadata containing the session ID and error reason. + +#### Scenario: Camera ends session +- **WHEN** the camera sends an `End` command with a reason code +- **THEN** the SBMD handler SHALL emit a `sessionStatus` event with value `"error"` and metadata `{ "sessionId": "", "error": "" }` + +### Requirement: destroySession sends EndSession to camera + +When the camera session endpoint's `destroySession` is executed for a session that has progressed to WebRTC signaling, the handler SHALL send an `EndSession` command (ID 0x06) to the camera's `WebRTCTransportProvider` cluster (0x0553) before cleaning up local session state. + +#### Scenario: Client destroys active streaming session +- **WHEN** a client executes `destroySession` for a session in `streaming` state +- **THEN** the handler SHALL send `EndSession` to the camera AND remove the session from transient data + +#### Scenario: Client destroys session that never started streaming +- **WHEN** a client executes `destroySession` for a session in `created` state (never executed `stream`) +- **THEN** the handler SHALL only remove the session from transient data (no Matter command needed) + +### Requirement: WebRTC constants use correct Matter cluster and command IDs + +The SBMD driver SHALL define constants for all WebRTC cluster and command identifiers: + +| Constant | Value | Description | +|----------|-------|-------------| +| CL_WEBRTC_TRANSPORT_PROVIDER | 0x0553 | Camera's provider cluster | +| CL_WEBRTC_TRANSPORT_REQUESTOR | 0x0554 | Barton's requestor cluster | +| CMD_PROVIDE_OFFER | 0x02 | Send SDP offer to camera | +| CMD_PROVIDE_ANSWER | 0x04 | Send SDP answer to camera | +| CMD_PROVIDE_ICE | 0x05 | Send ICE candidates to camera | +| CMD_END_SESSION | 0x06 | End a WebRTC session | +| CMD_OFFER | 0x00 | Incoming offer from camera | +| CMD_ANSWER | 0x01 | Incoming answer from camera | +| CMD_ICE_CANDIDATES | 0x02 | Incoming ICE from camera | +| CMD_END | 0x03 | Incoming end from camera | + +#### Scenario: Constants match Matter specification +- **WHEN** the SBMD driver is loaded +- **THEN** all cluster and command ID constants SHALL match the values defined in the Matter 1.5 WebRTC Transport cluster specification + +### Requirement: WebRTC endpoint is separable by design + +The webrtc endpoint resources, constants, and handler functions SHALL be grouped together and access session state only through transient data supplements. No direct coupling between camera endpoint handlers and webrtc endpoint handlers beyond shared transient data keys. + +#### Scenario: Code organization supports extraction +- **WHEN** the webrtc endpoint code is reviewed +- **THEN** all webrtc-specific constants, resources, and handlers SHALL be identifiable as a cohesive group that could be moved to a separate file with only transient data key sharing as the interface diff --git a/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/tasks.md b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/tasks.md new file mode 100644 index 00000000..910af08e --- /dev/null +++ b/openspec/changes/archive/2026-07-13-webrtc-endpoint-and-camera-stream-command/tasks.md @@ -0,0 +1,54 @@ +## 1. ZAP — Add WebRTCTransportRequestor Cluster + +- [x] 1.1 Add `WebRTCTransportRequestor` cluster definition (0x0554) with commands (Offer, Answer, ICECandidates, End) to `barton-library.matter` +- [x] 1.2 Add the cluster as a server on Barton's endpoint in the ZAP endpoint configuration section of `barton-library.matter`, handle commands (Offer, Answer, ICECandidates, End) +- [x] 1.3 Regenerate ZAP artifacts (`barton-library.zap` if needed) and verify the Matter build compiles with the new cluster + +## 2. SBMD WebRTC Endpoint — Constants and Resource Declarations + +- [x] 2.1 Add WebRTC cluster and command ID constants to camera.sbmd.js (CL_WEBRTC_TRANSPORT_PROVIDER, CL_WEBRTC_TRANSPORT_REQUESTOR, CMD_PROVIDE_OFFER, CMD_PROVIDE_ICE, CMD_END_SESSION, CMD_OFFER, CMD_ICE_CANDIDATES, CMD_END, etc.) +- [x] 2.2 Declare the `webrtc` endpoint in the `endpoints` block with profile `"webrtc"`, resources: `offerSdp` (function/execute), `remoteSdp` (string, modes:[]), `offerIceCandidates` (function/execute), `remoteIceCandidates` (string, modes:[]) +- [x] 2.3 Add `CL_WEBRTC_TRANSPORT_REQUESTOR` to the `featureClusters` array in the matter config (enables incoming command handler registration) + +## 3. SBMD WebRTC Endpoint — Execute Handlers + +- [x] 3.1 Implement `executeOfferSdp` handler: validate active session in transient data, extract SDP from input, build TLV payload for ProvideOffer command (webRTCSessionID + sdp + streamUsage + originatingEndpointID), return `device.sendCommand(CL_WEBRTC_TRANSPORT_PROVIDER, CMD_PROVIDE_OFFER, tlv)` +- [x] 3.2 Implement `executeOfferIceCandidates` handler: parse JSON array of ICE candidate strings from input, build TLV payload for ProvideICECandidates command (webRTCSessionID + ICECandidates array), return `device.sendCommand(CL_WEBRTC_TRANSPORT_PROVIDER, CMD_PROVIDE_ICE, tlv)` +- [x] 3.3 Update `executeDestroySession` to send EndSession command (CMD_END_SESSION) to the camera when session state is `streaming` before removing session from transient data + +## 4. SBMD WebRTC Endpoint — Command Handlers (Incoming from Camera) + +- [x] 4.1 Add `commandHandlers` block to the SbmdDriver registration with aliases for cluster 0x0554 commands (Offer, ICECandidates, End) +- [x] 4.2 Implement `handleIncomingOffer` command handler: decode TLV to extract SDP string, call `updateResource('webrtc', 'remoteSdp', sdp)`, store the Matter webRTCSessionID in transient data for correlation +- [x] 4.3 Implement `handleIncomingIceCandidates` command handler: decode TLV to extract ICE candidate list, JSON-encode, call `updateResource('webrtc', 'remoteIceCandidates', jsonCandidates)` +- [x] 4.4 Implement `handleIncomingEnd` command handler: extract reason code, emit `sessionStatus` error event with metadata `{sessionId, error}`, clean up session state + +## 5. SBMD WebRTC Endpoint — Unit Testing + +- [x] 5.1 Write unit tests for `executeOfferSdp`: valid session produces sendCommand result, missing session returns error, missing input returns error +- [x] 5.2 Write unit tests for `executeOfferIceCandidates`: valid JSON array produces sendCommand, invalid JSON returns error +- [x] 5.3 Write unit tests for incoming command handlers: verify updateResource calls with correct endpoint/resource/value for Offer, ICECandidates, and End commands +- [x] 5.4 Write unit test for `executeDestroySession` with streaming session: verify EndSession command is sent before cleanup + +## 6. Reference App — Build System and Category Setup + +- [x] 6.1 Add `BCORE_CAMERA_STREAM` CMake option (default OFF) to `reference/CMakeLists.txt` +- [x] 6.2 When `BCORE_CAMERA_STREAM=ON`: find GStreamer packages (gstreamer-1.0, gstreamer-webrtc-1.0, gstreamer-sdp-1.0), add to link dependencies, define `HAVE_CAMERA_STREAM` compile definition +- [x] 6.3 Create `cameraCategory.h` / `cameraCategory.c` with `buildCameraCategory()` returning a Category with `cameraStream` / `cs` command +- [x] 6.4 Register camera category in the reference app's main category list (gated by `#ifdef HAVE_CAMERA_STREAM`) + +## 7. Reference App — GStreamer WebRTC Pipeline + +- [x] 7.1 Create `cameraStreamPipeline.h` / `cameraStreamPipeline.c` — GStreamer pipeline management: create pipeline with webrtcbin, extract local SDP offer, set remote SDP, add ICE candidates, connect to autovideosink or filesink based on mode +- [x] 7.2 Implement SDP offer extraction: connect to webrtcbin's `on-negotiation-needed` signal, call `create-offer` action, extract SDP string from the local description +- [x] 7.3 Implement remote SDP handling: parse SDP answer string into GstWebRTCSessionDescription, call `set-remote-description` on webrtcbin +- [x] 7.4 Implement ICE candidate handling: connect to `on-ice-candidate` signal for local candidates, provide `add-ice-candidate` for remote candidates +- [x] 7.5 Implement pipeline teardown: stop pipeline, free resources, handle GStreamer state changes + +## 8. Reference App — cameraStream Command Orchestration + +- [x] 8.1 Implement `cameraStreamFunc`: parse args (deviceId, optional --file), verify device has camera endpoint, call createSession, call stream, subscribe to events +- [x] 8.2 Implement event handling loop: wait for sessionStatus(setup) → create pipeline → extract SDP → execute offerSdp → wait for remoteSdp event → set remote → exchange ICE → wait for connection +- [x] 8.3 Implement Ctrl+C / `q` handling: signal handler sets teardown flag, calls destroySession, stops GStreamer pipeline, returns to prompt +- [x] 8.4 Implement progress output: print human-readable status at each stage (session created, SDP sent, answer received, ICE exchanged, media flowing, stream ended) +- [x] 8.5 Implement error handling: timeout on signaling steps, graceful cleanup on failure, clear error messages diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/.openspec.yaml b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/.openspec.yaml new file mode 100644 index 00000000..b119b635 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-13 diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/design.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/design.md new file mode 100644 index 00000000..1b72ce40 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/design.md @@ -0,0 +1,122 @@ +## Context + +The camera SBMD driver (`core/deviceDrivers/matter/sbmd/specs/camera.sbmd.js`) exposes an abstract session endpoint (`ep/camera`) plus a protocol endpoint (`ep/webrtc`). The abstract endpoint was originally designed around a `sessionStatus` event resource acting as a protocol-agnostic "traffic controller" that would emit `setup`/`done`/`error` with `nextAction` metadata to walk a protocol-blind client through the flow (see archived `2026-05-04-camera-architecture-redesign`). + +That vision did not survive implementation, and testing exposed two independent problems: + +1. **`sessionStatus` is vestigial.** The reference client ignores `nextAction` and hard-codes the WebRTC flow; `done` is never emitted; only `error` (camera `End`) has a live consumer. The abstraction is also leaky — a client must carry protocol-specific media code (webrtcbin for WebRTC, a URL player for direct), so it inevitably knows the signaling sequence and gains nothing from server-pushed next-steps. + +2. **Asynchronous failures are silently dropped.** The SBMD command plumbing has three delivery channels with very different reliability: + - Synchronous execute `.error()` → surfaces to the client as a failed `executeResource` (reliable). + - `requestCommand` `onError`/timeout continuations → run and return `.error()`, but the originating execute already returned optimistically, so the result terminal has **no client channel** and is logged/dropped. + - `sendCommand` (fire-and-forget) → **no `onError` at all**; async failures are invisible. + + Because `sessionStatus="error"` is only written on an incoming camera `End`, every other async failure (malformed SDP rejected by camera, VideoStreamAllocate/ProvideOffer rejected, camera unreachable during ICE, no ICE candidates) produces an unbounded black-screen hang. + +Additionally, the device service suppresses `resourceUpdated` events when a cached resource's value is unchanged (`core/src/deviceService.c`), and `sessionStatus` is registered cached — so even the paths that do write it can be silently coalesced (e.g. a persisted `"setup"` produces zero events on a re-stream). + +``` + Current Proposed + ───────────────────────────────── ───────────────────────────────── + ep/camera ep/camera + r/createSession (execute) r/createSession (execute) + r/stream (execute) ── emits ──► r/stream (execute) ── returns {protocol, entryPoint} + r/sessionStatus (event) ◄─ traffic r/takePicture (execute) + r/takePicture (execute) controller r/destroySession (execute) + r/destroySession (execute) ep/webrtc + ep/webrtc r/offerSdp (execute) + r/offerSdp (execute) r/remoteSdp (event) + r/remoteSdp (event) r/offerIceCandidates(execute) + r/offerIceCandidates(execute) r/remoteIceCandidates(event) + r/remoteIceCandidates(event) r/webrtcError (event, NEVER-cached) ◄─ all async + failures + End +``` + +## Goals / Non-Goals + +**Goals:** +- Remove `sessionStatus` from `ep/camera` and relocate its one real job (async teardown/error signaling) to the protocol endpoint. +- Give every asynchronous signaling failure a reliable delivery channel to the client. +- Make the failure/teardown event immune to no-change event suppression. +- Give the reference client a bounded, reported outcome for a failed ICE/connectivity exchange. +- Keep the abstract-endpoint contract protocol-agnostic so a future non-WebRTC protocol reuses it unchanged. + +**Non-Goals:** +- Adding a WebRTC media stack to Barton core (Barton remains signaling-only). +- Implementing a second protocol endpoint (`ep/direct`, `ep/openhome`) — the design must *accommodate* one, not build it. +- Changing the `BCoreClient` public GObject API or the resource/event signal surface. +- Reworking the device-service change-detection logic itself (we opt out per-resource via caching policy rather than changing core behavior). +- SolicitOffer / camera-initiated-offer flows. + +## Decisions + +### D1: `stream` execute returns `{protocol, entryPoint}` instead of emitting `sessionStatus(setup)` + +**Decision**: `executeStream` returns a JSON object `{ "protocol": "webrtc", "entryPoint": "//ep/webrtc/r/offerSdp" }` as its synchronous execute result. No `setup` event is emitted. + +**Rationale**: The client already invokes `stream`; the "what protocol / where next" answer belongs in that call's return value, delivered on the reliable synchronous channel. This preserves the protocol-agnostic property (a client can discover the protocol without hard-coding it) while deleting the unused event machinery. A future `ep/direct` driver returns `{protocol:"direct", entryPoint:".../getMediaUrl"}` identically. + +**Alternatives considered**: +- Keep `sessionStatus(setup)` as an event — rejected: it is the machinery we found unused and suppression-prone. +- Put discovery on `createSession` — rejected: protocol is a property of *streaming*, and `stream` is where a session commits to a protocol. + +### D2: Single `webrtcError` async event on `ep/webrtc`, non-cached via a new `volatile` SBMD mode + +**Decision**: Add one event-only resource `webrtcError` to `ep/webrtc` carrying a small value (`ended` / `failed`) plus metadata `{ reason, detail }`. To make it emit unconditionally, add a new declarable resource mode `volatile` to the SBMD runtime that maps the resource to `CACHING_POLICY_NEVER`; declare `webrtcError` with `modes: ['volatile']`. + +**Rationale**: Colocating in-session termination/error signaling with the protocol endpoint keeps protocol concerns off the abstract endpoint. `CACHING_POLICY_NEVER` makes `updateResource` bypass the `strcmp` change-check, so a value that equals the stored one (e.g. a second session's `failed` after the previous session's `failed` persisted) still emits — the exact footgun that silently dropped repeated `sessionStatus` writes. + +SBMD v4 does not currently expose caching policy: `SpecBasedMatterDeviceDriver::DoRegisterDriverResources` derives it solely from `resource.read.has_value()` (`CACHING_POLICY_NEVER` for resources with a read handler, else `CACHING_POLICY_ALWAYS`). Rather than overload a read handler as a caching side-channel, we add a first-class, declarative opt-out: a `volatile` mode. The change is small and additive — extend the schema enum, accept `volatile` in `ConvertModesToBitmask` (no mode bit), and treat `read || volatile` as `CACHING_POLICY_NEVER`. The device-service change-detection logic is untouched. `RESOURCE_MODE_EMIT_EVENTS` is already on by default (only `noEvents` opts out). + +**Alternatives considered**: +- Register `NEVER` with no core change — rejected: not expressible; caching policy has no declarable field in SBMD v4. +- Give `webrtcError` a `read` handler to force `NEVER` — rejected: obscure (read handler purely as a caching side-channel) and would be the first shipped spec to use `read`. +- Reset `webrtcError` to a neutral value each session so the terminal value always differs — rejected: relies on discipline and still suppresses a duplicate terminal value within a session; a declarative mode is more robust and reusable for future protocol endpoints. +- Encode a nonce in the value — rejected: value-format hack the client must parse. +- Change `deviceService.c` to always emit — rejected: broad blast radius; per-resource opt-out is safer. + +### D3: Fan every async failure path into `webrtcError` + +**Decision**: Emit `webrtcError` from `handleVideoStreamAllocateError`, `handleProvideOfferError`, the `requestCommand` overall-deadline timeout continuation, and `handleIncomingEnd`, each with a distinguishing `reason`. + +**Rationale**: These are precisely the paths whose `.error()` results are currently dropped (the async `requestCommand` continuations) or that previously used `sessionStatus` (`End`). Emitting a resource event is the one continuation action that *does* reach the client, because it flows through the normal event pipeline rather than the (absent) execute return. + +**Known limitation**: `sendCommand`-based outbound failures (ProvideICECandidates, EndSession) still have no JS-visible error callback in the runtime. This design does not add one; the client-side connectivity timeout (D4) is the backstop that catches an ICE exchange that never completes for any reason, including a dropped outbound candidate. + +### D4: Client-side connectivity/ICE timeout in `cameraWebrtcClient` + +**Decision**: The reference WebRTC client watches `webrtcbin`'s `ice-connection-state` (and/or `connection-state`) and starts a bounded timer once signaling completes. If the connection does not reach `connected`/`completed` within the window (or transitions to `failed`), it invokes the existing `onWebrtcClosed` teardown with a failure reason. + +**Rationale**: Actual media/ICE connectivity is observable only by the WebRTC peer, never by the Matter signaling relay. The driver fundamentally cannot detect "ICE never connected," so the timeout must live in the client. Routing it through the existing `onWebrtcClosed` path reuses the graceful teardown (EndSession to the camera) already proven for window-close/Ctrl+C. + +**Alternatives considered**: +- Rely solely on `webrtcError` from the driver — rejected: the driver can't see connectivity failures where the camera never sends `End`. +- A fixed `g_usleep` guard — rejected: state-driven detection is both faster on success and correct on failure. + +### D5: Reference client consumes `webrtcError`; drop `sessionStatus` subscription + +**Decision**: `cameraDeviceSession` subscribes to `ep/webrtc/r/webrtcError` and maps `ended`/`failed` to its existing `onError` callback (teardown + reported reason). The `sessionStatus` URI handling is removed. `stream`'s return value is parsed for `protocol`/`entryPoint`. + +**Rationale**: One switch arm moves from the abstract endpoint URI to the protocol endpoint URI; the client's teardown behavior is unchanged. Reading `entryPoint` from the `stream` result (rather than assuming the webrtc URI) keeps the client honest about protocol-agnosticism even while WebRTC is the only implementation. + +## Risks / Trade-offs + +- **[`sendCommand` failures remain invisible to the driver]** → Mitigation: the D4 client connectivity timeout is the catch-all; any exchange that fails to establish — including a lost outbound ICE candidate — resolves to a bounded, reported teardown. +- **[Removing `sessionStatus` breaks any out-of-tree client that read it]** → Mitigation: we own the clients; the only in-tree consumers are the reference app and one unit test, both updated here. This is an internal signaling contract, not the public GObject API. +- **[`stream` return value becomes a semi-structured contract]** → Mitigation: specify the JSON shape (`protocol`, `entryPoint`) in the `camera-session-lifecycle` spec so future protocol drivers conform. +- **[Timeout window tuning]** → A too-short window aborts slow-but-valid connections (the camera's 4s keyframe interval already pushes first media to ~10s); too long delays failure reporting. Mitigation: make the window a named constant with margin over observed worst-case, and reset it on first media. +- **[Thread safety]** → `webrtcError` is emitted from SBMD handler continuations, which run under `MQuickJsRuntime::GetMutex()` and marshal to the device-service event pipeline exactly like `remoteSdp`/`remoteIceCandidates` today — no new synchronization. The client timeout fires on the GStreamer/`webrtcbin` thread and must request teardown via the same cross-thread `onWebrtcClosed` mechanism already used for EOS/error, not act directly. + +## Migration Plan + +1. Land the SBMD spec changes (resource set + handlers) and validate with the SBMD schema (`validate-sbmd`). +2. Update `SbmdCameraWebrtcTest.cpp` assertions (`sessionStatus` → `webrtcError`; `stream` return shape) and confirm the C++ unit suite is green. +3. Update the reference app (`cameraDeviceSession`, `cameraWebrtcClient`, `cameraCategory`). +4. Manually verify: happy path still streams; force each failure (bad SDP, kill camera mid-ICE, suppress ICE) and confirm a bounded teardown with a reported reason. +5. Rollback is a straight revert — no persisted schema/data migration; transient session data keys are unchanged. + +## Open Questions + +1. Exact `webrtcError` value vocabulary — minimal (`ended`/`failed`) with reason in metadata, versus a richer enum. Leaning minimal + `reason` metadata for protocol-agnosticism. +2. Connectivity timeout duration and whether it should be overridable via a `cameraStream` flag for high-latency networks. +3. Whether `takePicture` (currently unimplemented) should also return an entry-point/`{protocol}` shape for consistency, or stay a direct execute. diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/proposal.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/proposal.md new file mode 100644 index 00000000..d3730ad2 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/proposal.md @@ -0,0 +1,30 @@ +## Why + +Investigation of the camera driver's `sessionStatus` resource showed it is effectively an appendage: its `setup`/`nextAction` metadata is ignored by the only client (the reference app hard-codes protocol knowledge), its `done` value is never emitted, and its lone live use — signaling that the camera ended the session — is a protocol-specific event misplaced on the abstract endpoint. Worse, a spec-path audit of the WebRTC/Matter flow found that **almost every asynchronous failure is silently dropped**: `requestCommand` `onError` results (VideoStreamAllocate rejected, ProvideOffer rejected) and `requestCommand` timeouts return `.error()` that has no delivery channel to the client, and `sendCommand` (outbound ICE, EndSession) is fire-and-forget with no error path at all. The result is that malformed SDP, a camera that goes unreachable during ICE, or a client that sends no ICE candidates all manifest as an unbounded black-screen hang with no error surfaced. + +## What Changes + +- **BREAKING** (internal signaling contract): Remove the `sessionStatus` resource from the abstract `ep/camera` endpoint. Protocol and entry-point discovery move onto the return value of the `stream` execute (`{ protocol, entryPoint }`), which the client already calls — eliminating the server-pushed "traffic controller" the client never used. +- Add a new declarable SBMD resource mode `volatile` that maps a resource to `CACHING_POLICY_NEVER`, then add a single asynchronous error/termination event resource on the protocol endpoint (`ep/webrtc`, `webrtcError`) declared `modes: ['volatile']` so it emits **unconditionally**, never subject to the device-service no-change suppression that silently dropped repeated status writes. The device-service change-detection logic itself is left untouched. +- Feed that event from **every** currently-dropped failure path: `handleVideoStreamAllocateError`, `handleProvideOfferError`, the `requestCommand` overall-deadline timeout, and the incoming camera `End` command — subsuming the old `sessionStatus="error"` behavior and closing the silent-failure gaps. +- The reference app consumes the new `ep/webrtc` state event (in place of `sessionStatus`) and adds a client-side connectivity/ICE timeout driven by `webrtcbin`'s `ice-connection-state`/`connection-state`, surfacing failures through the existing graceful-teardown path instead of hanging on a black window. + +## Capabilities + +### New Capabilities +- `camera-session-lifecycle`: The protocol-agnostic abstract camera endpoint contract (`createSession`, `stream`, `takePicture`, `destroySession`) with **no** `sessionStatus` resource. The `stream` execute returns the active protocol and its entry-point URI; all in-session state, error, and teardown signaling lives on the protocol endpoint, not the abstract one. + +### Modified Capabilities +- `sbmd-v4-runtime`: Add a declarable resource mode `volatile` that registers a resource with `CACHING_POLICY_NEVER`, so its `updateResource` calls emit events unconditionally (bypassing no-change suppression) without a read handler. +- `webrtc-signaling-endpoint`: Replace the "Incoming End command emits `sessionStatus` error" requirement with an `ep/webrtc` asynchronous error/termination event (`webrtcError`); require that all async signaling failures (stream-allocate error, provide-offer error, command timeout, camera `End`) emit that event; require the event resource to be declared `volatile` so no emission is suppressed. +- `camera-stream-reference-command`: The reference command consumes the `ep/webrtc` state event rather than `sessionStatus`, and enforces a connectivity/ICE timeout so a failed exchange results in a bounded, reported teardown instead of an indefinite hang. This capability's spec is also reconciled with the as-built reference command: an `--out ` destination (record to `file://` or serve over HTTP), a decode-free passthrough pipeline (`rtph264depay → h264parse → mp4mux → appsink`) that serves fragmented MP4 to a browser via Media Source Extensions instead of a local display window, host-only ICE, and the `gstreamer-app-1.0`/`gio-2.0` build dependencies. + +## Impact + +- **SBMD runtime**: `core/deviceDrivers/matter/sbmd/schema/sbmd-spec-schema-v4.0.json` (add `volatile` to the modes enum) and `core/deviceDrivers/matter/sbmd/SpecBasedMatterDeviceDriver.cpp` (accept `volatile` in `ConvertModesToBitmask`; treat `read || volatile` as `CACHING_POLICY_NEVER` in `DoRegisterDriverResources`). +- **Drivers**: `core/deviceDrivers/matter/sbmd/specs/camera.sbmd.js` — remove `sessionStatus` resource + its two write sites; change `executeStream` to return `{protocol, entryPoint}`; add the `ep/webrtc` `webrtcError` (`volatile`) resource and emit it from the four failure paths. +- **Reference app**: `reference/src/cameraDeviceSession.c` (stop subscribing to `sessionStatus`, subscribe to the new `ep/webrtc` state event), `reference/src/cameraWebrtcClient.c/.h` (watch `ice-connection-state`/`connection-state`, drive `onWebrtcClosed` on failure/timeout), `reference/src/cameraCategory.c` (surface the failure reason). +- **Tests**: `core/test/src/SbmdCameraWebrtcTest.cpp` — the assertion expecting an `updateResource` for `sessionStatus` (and the End→sessionStatus test) must move to the new `ep/webrtc` state event. +- **Specs**: new `camera-session-lifecycle` spec; delta modifications to `webrtc-signaling-endpoint` and `camera-stream-reference-command`. +- **No public GObject API change**; `BCoreClient` resource/event surface is unchanged (only resource URIs on the camera device change). +- **CMake flags**: `BCORE_REFERENCE_CAMERA_SUPPORT` (reference command) unchanged. diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/camera-session-lifecycle/spec.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/camera-session-lifecycle/spec.md new file mode 100644 index 00000000..29cc66d2 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/camera-session-lifecycle/spec.md @@ -0,0 +1,53 @@ +## ADDED Requirements + +### Requirement: Abstract camera endpoint provides protocol-agnostic session lifecycle + +The camera SBMD driver SHALL declare an endpoint with id `"camera"` and profile `"camera"` exposing the session lifecycle as execute resources only: `createSession`, `stream`, `takePicture`, and `destroySession`. The abstract endpoint SHALL NOT declare a `sessionStatus` resource, and SHALL NOT carry any protocol-specific signaling or in-session state. All in-session state, error, and teardown signaling SHALL live on the protocol-specific endpoint (e.g. `ep/webrtc`). + +#### Scenario: Camera endpoint exposes only lifecycle executes +- **WHEN** a Matter camera device (deviceType 0x0142) is commissioned +- **THEN** the device SHALL have an endpoint with id `"camera"` exposing `createSession`, `stream`, `takePicture`, and `destroySession` as execute resources AND SHALL NOT expose a `sessionStatus` resource + +#### Scenario: No protocol coupling on the abstract endpoint +- **WHEN** the `ep/camera` endpoint is inspected +- **THEN** none of its resources SHALL reference a specific streaming protocol; protocol identity is carried only in the `stream` execute result and on the protocol endpoint + +### Requirement: createSession allocates a session and returns its identifier + +The `createSession` execute handler SHALL allocate a new session, persist it in transient data, and return the new `sessionId` synchronously as the execute result so the client holds a correlation identifier before invoking any further resource. + +#### Scenario: Client creates a session +- **WHEN** a client executes `createSession` +- **THEN** the handler SHALL return a non-empty `sessionId` string AND record the session in transient data with an initial state + +#### Scenario: Corrupt session data is reset +- **WHEN** `createSession` is executed and the stored session data cannot be parsed +- **THEN** the handler SHALL reset the session store and return an error result + +### Requirement: stream execute returns the active protocol and entry point + +The `stream` execute handler SHALL mark the identified session as streaming and return, as its synchronous execute result, a JSON object identifying the active protocol and the entry-point resource URI the client must use next: `{ "protocol": "", "entryPoint": "//ep//r/" }`. The handler SHALL NOT emit a separate event to convey the next action. + +#### Scenario: Stream returns protocol and entry point for a WebRTC camera +- **WHEN** a client executes `stream` with a valid `sessionId` on a Matter WebRTC camera +- **THEN** the handler SHALL return `{ "protocol": "webrtc", "entryPoint": "//ep/webrtc/r/offerSdp" }` AND mark the session `streaming` + +#### Scenario: Stream on unknown session +- **WHEN** a client executes `stream` with a `sessionId` that does not exist +- **THEN** the handler SHALL return an error result and SHALL NOT mark any session streaming + +### Requirement: destroySession releases session state + +The `destroySession` execute handler SHALL remove the identified session from transient data and trigger any protocol-specific teardown required for a session that reached streaming. + +#### Scenario: Client destroys a session +- **WHEN** a client executes `destroySession` with a valid `sessionId` +- **THEN** the handler SHALL remove the session from transient data + +### Requirement: Client discovers next steps without server-pushed status events + +A client SHALL be able to drive the full session flow using only execute results and protocol-endpoint event subscriptions, without reading or subscribing to any status resource on the abstract endpoint. + +#### Scenario: Reference client drives flow without sessionStatus +- **WHEN** the reference app runs a camera stream +- **THEN** it SHALL obtain the protocol and entry point from the `stream` execute result and subscribe to protocol-endpoint events, and SHALL NOT subscribe to any `sessionStatus` resource diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/camera-stream-reference-command/spec.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/camera-stream-reference-command/spec.md new file mode 100644 index 00000000..46da04b0 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/camera-stream-reference-command/spec.md @@ -0,0 +1,129 @@ +## MODIFIED Requirements + +### Requirement: cameraStream subscribes to Barton events + +The `cameraStream` command SHALL subscribe to resource events on the device to receive signaling data asynchronously. Specifically: +- `remoteSdp` events on `ep/webrtc` (for the camera's SDP answer) +- `remoteIceCandidates` events on `ep/webrtc` (for the camera's ICE candidates) +- `webrtcError` events on `ep/webrtc` (for asynchronous session termination and errors) + +The command SHALL NOT subscribe to any `sessionStatus` resource on `ep/camera`. It SHALL obtain the active protocol and entry-point URI from the `stream` execute result. + +#### Scenario: Remote SDP delivered via event +- **WHEN** the camera responds with an SDP answer +- **THEN** the reference app SHALL receive it as a `remoteSdp` event and feed it to webrtcbin as the remote description + +#### Scenario: Remote ICE candidates delivered via events +- **WHEN** the camera sends ICE candidates +- **THEN** the reference app SHALL receive them as `remoteIceCandidates` events and add each candidate to webrtcbin + +#### Scenario: Session error delivered via webrtcError event +- **WHEN** the driver emits a `webrtcError` event with an ended or failed value +- **THEN** the reference app SHALL treat it as a session-terminated signal and begin graceful teardown, reporting the reason from the event metadata + +### Requirement: cameraStream reports progress to user + +The command SHALL emit human-readable progress messages to stdout at each stage of the flow: +- Session created (sessionId) +- Streaming initiated (protocol, entryPoint) +- SDP offer sent +- SDP answer received +- ICE candidates exchanged +- Media flowing / connected +- Stream ended (reason) + +#### Scenario: Progress output during successful stream +- **WHEN** `cameraStream` completes signaling and media begins flowing +- **THEN** the user SHALL see step-by-step status messages indicating progress through the flow + +### Requirement: cameraStream handles errors gracefully + +The command SHALL handle failures at any stage (session creation failure, signaling timeout, asynchronous `webrtcError` failure, and connectivity/ICE failure) by printing an error message, cleaning up any partial state (destroying the session if created), and returning to the command prompt. In particular, because Matter signaling cannot observe media-plane connectivity, the command SHALL enforce a client-side connectivity timeout: if the WebRTC peer connection does not reach a connected state within a bounded window after signaling completes — or transitions to a failed state — the command SHALL tear down and report the failure rather than wait indefinitely. + +#### Scenario: Device does not support camera streaming +- **WHEN** `cameraStream` is executed on a device without a `camera` endpoint +- **THEN** the command SHALL print an error and exit without crashing + +#### Scenario: Signaling timeout +- **WHEN** the camera does not respond to signaling within a reasonable timeout +- **THEN** the command SHALL print a timeout error, destroy the session, and exit + +#### Scenario: Connectivity never established +- **WHEN** signaling completes but the WebRTC peer connection does not reach a connected state within the connectivity timeout window +- **THEN** the command SHALL print a connectivity-failure message, destroy the session, and exit rather than hang on a blank window + +#### Scenario: Peer connection fails +- **WHEN** the WebRTC peer connection transitions to a failed state during or after ICE exchange +- **THEN** the command SHALL print a failure message, destroy the session, and exit + +### Requirement: cameraStream command exists in reference app + +The reference app SHALL provide a command named `cameraStream` with short alias `cs` in a dedicated camera command category. The command SHALL accept a device ID as a required argument and an optional `--out ` flag that selects the media destination. + +#### Scenario: Command appears in help +- **WHEN** a user types `help` in the reference app +- **THEN** the camera category SHALL list `cameraStream` with usage: ` [--out ]` + +#### Scenario: Command with short alias +- **WHEN** a user types `cs ` +- **THEN** the command SHALL execute identically to `cameraStream ` + +#### Scenario: Output URI selects the media destination +- **WHEN** the command is invoked with `--out file://` +- **THEN** the stream SHALL be recorded to that file path +- **WHEN** the command is invoked with `--out [:]` (optionally prefixed with `http://`), or without `--out` +- **THEN** the stream SHALL be served over HTTP for a browser to play, defaulting to a loopback host and port when `--out` is omitted + +### Requirement: cameraStream orchestrates full session lifecycle + +The `cameraStream` command SHALL orchestrate the complete camera streaming flow through Barton's resource API: +1. Execute `createSession` on the device's `ep/camera` endpoint +2. Execute `stream` and obtain the active protocol and entry-point URI from its `{ protocol, entryPoint }` result +3. Create a local GStreamer `webrtcbin` peer connection using host candidates only (no STUN/TURN) +4. Generate a local SDP offer from webrtcbin +5. Execute `offerSdp` on the device's `ep/webrtc` endpoint with the local SDP +6. Wait for a `remoteSdp` event and set the remote description on webrtcbin +7. Exchange ICE candidates (local → `offerIceCandidates`, remote ← `remoteIceCandidates` events) +8. Wait for the peer connection to reach the connected state, subject to a bounded connectivity timeout +9. Route the received media to the destination selected by `--out`: serve it over the built-in HTTP server or record it to a file +10. On user interrupt (Ctrl+C or `q`) or a `webrtcError` event: execute `destroySession` and tear down the pipeline + +#### Scenario: Successful camera stream served to a browser +- **WHEN** a user executes `cameraStream ` (or with an `http://` `--out`) and the camera responds to signaling +- **THEN** the reference app SHALL serve the live stream over its built-in HTTP server for a browser to play and print status messages for each step, without opening a local display window + +#### Scenario: Successful camera stream to file +- **WHEN** a user executes `cameraStream --out file://recording.mp4` +- **THEN** the reference app SHALL record the video stream to the specified file path + +#### Scenario: User stops the stream +- **WHEN** a user presses Ctrl+C or types `q` during an active stream +- **THEN** the reference app SHALL execute `destroySession`, stop the GStreamer pipeline, and return to the command prompt + +### Requirement: cameraStream uses GStreamer webrtcbin for media + +The `cameraStream` command SHALL use GStreamer's `webrtcbin` element as its local WebRTC peer connection (host candidates only). The received H.264 SHALL be handled by a passthrough pipeline that neither decodes nor renders locally: `rtph264depay → h264parse → mp4mux` (fragmented, streamable) `→ appsink`. The muxed fragmented-MP4 buffers SHALL be delivered either to the built-in HTTP media server or to a file, according to `--out`. + +#### Scenario: Serve mode pipeline +- **WHEN** `cameraStream` runs in serve mode (the default, or an `http://` `--out`) +- **THEN** the muxed fragmented-MP4 buffers SHALL be pushed to the built-in HTTP media server, which serves them to a browser that decodes and plays them via Media Source Extensions + +#### Scenario: Record mode pipeline +- **WHEN** `cameraStream` is invoked with `--out file://` +- **THEN** the muxed fragmented-MP4 buffers SHALL be written to the file at that path + +#### Scenario: GStreamer not available +- **WHEN** GStreamer libraries (with `webrtcbin`) are not available at runtime +- **THEN** the command SHALL print an error explaining the requirement and exit gracefully + +### Requirement: Camera command category is gated by CMake flag + +The camera stream command and its GStreamer dependencies SHALL be gated behind a `BCORE_REFERENCE_CAMERA_SUPPORT` CMake option (default OFF). When disabled, the reference app builds without GStreamer dependencies and without the camera category. + +#### Scenario: Build without camera stream support +- **WHEN** `BCORE_REFERENCE_CAMERA_SUPPORT=OFF` (default) +- **THEN** the reference app SHALL build successfully without GStreamer development libraries + +#### Scenario: Build with camera stream support +- **WHEN** `BCORE_REFERENCE_CAMERA_SUPPORT=ON` +- **THEN** the reference app SHALL link against gstreamer-1.0, gstreamer-webrtc-1.0, gstreamer-sdp-1.0, gstreamer-app-1.0, and gio-2.0, and include the camera category diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/sbmd-v4-runtime/spec.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/sbmd-v4-runtime/spec.md new file mode 100644 index 00000000..bf04b8f2 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/sbmd-v4-runtime/spec.md @@ -0,0 +1,17 @@ +## ADDED Requirements + +### Requirement: volatile resource mode disables value caching + +The SBMD runtime SHALL support a resource mode `volatile`. A resource declared with the `volatile` mode SHALL be registered with `CACHING_POLICY_NEVER`, causing `updateResource` to emit a `resourceUpdated` event on every call (for a resource that emits events) regardless of whether the new value equals the currently stored value. The `volatile` mode SHALL NOT, by itself, add read, write, or execute access, and SHALL be accepted by mode-to-bitmask conversion without error. A resource is registered `CACHING_POLICY_NEVER` when it declares a read handler OR declares the `volatile` mode; otherwise it is registered `CACHING_POLICY_ALWAYS`. + +#### Scenario: Volatile resource emits on unchanged value +- **WHEN** a resource declared with `modes: ['volatile']` is updated twice with the same value +- **THEN** the runtime SHALL deliver two `resourceUpdated` events (no value-change suppression) + +#### Scenario: Volatile mode is accepted by the schema and runtime +- **WHEN** a driver declares a resource with `volatile` in its modes array +- **THEN** the spec SHALL validate against the SBMD schema AND the resource SHALL register successfully without adding read, write, or execute modes + +#### Scenario: Non-volatile resource without a read handler remains cached +- **WHEN** an event-only resource is declared without a read handler and without the `volatile` mode +- **THEN** the runtime SHALL register it with `CACHING_POLICY_ALWAYS` and suppress `resourceUpdated` events whose value is unchanged diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/webrtc-signaling-endpoint/spec.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/webrtc-signaling-endpoint/spec.md new file mode 100644 index 00000000..3c3d2cee --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/specs/webrtc-signaling-endpoint/spec.md @@ -0,0 +1,65 @@ +## MODIFIED Requirements + +### Requirement: WebRTC endpoint declares signaling resources + +The camera SBMD driver SHALL declare an endpoint with id `"webrtc"` and profile `"webrtc"` containing five resources: + +| Resource | Type | Modes | Purpose | +|----------|------|-------|---------| +| `offerSdp` | `function` | execute | Client sends local SDP offer | +| `remoteSdp` | `string` | [] (events only) | Delivers remote SDP answer to client | +| `offerIceCandidates` | `function` | execute | Client sends local ICE candidates | +| `remoteIceCandidates` | `string` | [] (events only) | Delivers remote ICE candidates to client | +| `webrtcError` | `string` | [volatile] (events only) | Delivers asynchronous session termination/error to client | + +The endpoint SHALL be declared within the same `camera.sbmd.js` file as the `ep/camera` endpoint. The `webrtcError` resource SHALL be declared with the `volatile` mode so that its events are emitted unconditionally (non-cached), independent of the previously emitted value. + +#### Scenario: Endpoint appears on commissioned camera device +- **WHEN** a Matter camera device (deviceType 0x0142) with WebRTCTransportProvider cluster (0x0553) is commissioned +- **THEN** the device SHALL have an endpoint with id `"webrtc"`, profile `"webrtc"`, and all five resources registered + +#### Scenario: Event-only resources are not readable +- **WHEN** a client attempts to read `remoteSdp`, `remoteIceCandidates`, or `webrtcError` +- **THEN** the read SHALL fail or return no value (modes list is empty — no read mode) + +## ADDED Requirements + +### Requirement: webrtcError resource emits every event unconditionally + +The `webrtcError` resource SHALL be declared with the `volatile` mode, which registers it with `CACHING_POLICY_NEVER` so that `updateResource` bypasses value-change detection and each emission delivers a `resourceUpdated` event to subscribers even when consecutive values are identical (including across sessions where a prior value persists). + +#### Scenario: Repeated identical values still emit +- **WHEN** the driver emits `webrtcError` twice in succession with the same value +- **THEN** the client SHALL receive two distinct `resourceUpdated` events (no no-change suppression) + +### Requirement: Incoming End command emits webrtcError event + +The SBMD driver SHALL register a command handler for the `End` command (ID 0x03) on the `WebRTCTransportRequestor` cluster (0x0554). When received, the handler SHALL clean up the associated session and emit a `webrtcError` event on the `webrtc` endpoint with a value indicating the session ended and metadata carrying the reason. + +#### Scenario: Camera ends session +- **WHEN** the camera sends an `End` command with a reason code +- **THEN** the SBMD handler SHALL call `updateResource('webrtc', 'webrtcError', , { "reason": "", "detail": "" })` AND remove the associated session from transient data + +### Requirement: Asynchronous signaling failures emit webrtcError event + +Each asynchronous WebRTC signaling failure that occurs after the originating execute has returned SHALL emit a `webrtcError` event so the client is notified rather than left to time out. This SHALL cover at least: a `VideoStreamAllocate` error, a `ProvideOffer` error, and a `requestCommand` overall-deadline timeout in the offer flow. + +#### Scenario: VideoStreamAllocate rejected by camera +- **WHEN** the camera rejects the `VideoStreamAllocate` command during the `offerSdp` flow +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and metadata describing the allocate error + +#### Scenario: ProvideOffer rejected by camera +- **WHEN** the camera rejects the `ProvideOffer` command during the `offerSdp` flow +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and metadata describing the provide-offer error + +#### Scenario: Offer-flow command times out +- **WHEN** a `requestCommand` in the `offerSdp` flow exceeds its overall deadline +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and a timeout reason + +## REMOVED Requirements + +### Requirement: Incoming End command emits sessionStatus error + +**Reason**: The `sessionStatus` resource is removed from the abstract `ep/camera` endpoint. Session teardown and error signaling for the WebRTC protocol now live on the protocol endpoint as the `webrtcError` event (see "Incoming End command emits webrtcError event"). + +**Migration**: Clients that subscribed to `ep/camera/r/sessionStatus` for the `"error"` value SHALL instead subscribe to `ep/webrtc/r/webrtcError` and treat its ended/failed values as the session-terminated signal. diff --git a/openspec/changes/archive/2026-07-16-camera-session-status-redesign/tasks.md b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/tasks.md new file mode 100644 index 00000000..91c6aa44 --- /dev/null +++ b/openspec/changes/archive/2026-07-16-camera-session-status-redesign/tasks.md @@ -0,0 +1,40 @@ +## 0. SBMD runtime — volatile (non-cached) resource mode + +- [x] 0.1 Add `volatile` to the modes enum in `core/deviceDrivers/matter/sbmd/schema/sbmd-spec-schema-v4.0.json` with a description +- [x] 0.2 Accept `volatile` in `ConvertModesToBitmask` (no mode bit) in `SpecBasedMatterDeviceDriver.cpp` +- [x] 0.3 In `DoRegisterDriverResources`, set `CACHING_POLICY_NEVER` when a resource has a read handler OR declares the `volatile` mode +- [x] 0.4 Validate the `volatile` mode: camera-device registration accepts `modes: ['volatile']` (the `webrtc` endpoint + `webrtcError` resource register with no "Unsupported resource mode" error), confirmed via E2E commissioning. No standalone caching-policy unit test was added — there is no existing device-registration/cachingPolicy test harness and it is disproportionate for the 3-line derivation change; the unconditional-emit behavior is exercised by the E2E path. + +## 1. Camera SBMD driver — remove sessionStatus + +- [x] 1.1 Remove the `sessionStatus` resource declaration from the `ep/camera` endpoint in `camera.sbmd.js` +- [x] 1.2 Change `executeStream` to return `{ protocol, entryPoint }` as its success result and delete its `sessionStatus`/`STATUS_SETUP` `updateResource` call and `nextAction` metadata +- [x] 1.3 Remove the now-unused `STATUS_SETUP`/`STATUS_DONE`/`STATUS_ERROR` constants and update the header comment block to describe the entry-point-return contract (no traffic-controller language) + +## 2. Camera SBMD driver — add webrtcError event + +- [x] 2.1 Declare a `webrtcError` event-only resource on the `ep/webrtc` endpoint with `modes: ['volatile']` so `updateResource` bypasses value-change suppression +- [x] 2.2 Add a shared helper that emits `webrtcError` with a value + `{ reason, detail }` metadata +- [x] 2.3 Rewrite `handleIncomingEnd` to emit `webrtcError` (ended) instead of `sessionStatus` error, preserving session cleanup +- [x] 2.4 Emit `webrtcError` (failed) from `handleVideoStreamAllocateError` with the allocate error detail +- [x] 2.5 Emit `webrtcError` (failed) from `handleProvideOfferError` with the provide-offer error detail +- [x] 2.6 Emit `webrtcError` (failed) from the offer-flow `requestCommand` timeout path (allocate + provide-offer `timeoutMs` continuations) with a timeout reason + +## 3. Reference app — consume webrtcError and drop sessionStatus + +- [x] 3.1 In `cameraDeviceSession.c`, remove the `sessionStatus` URI subscription/handling and subscribe to `ep/webrtc/r/webrtcError`, mapping ended/failed to the existing `onError` teardown with the metadata reason +- [x] 3.2 Parse the `stream` execute result for `{ protocol, entryPoint }` and use `entryPoint` instead of a hard-coded webrtc URI where the flow begins +- [x] 3.3 Update `cameraCategory.c` progress output to report `protocol`/`entryPoint` and to surface the `webrtcError` failure reason on teardown + +## 4. Reference app — connectivity/ICE timeout + +- [x] 4.1 In `cameraWebrtcClient.c`, watch `webrtcbin` `ice-connection-state` (and/or `connection-state`); on `failed`, request teardown via the existing `onWebrtcClosed` cross-thread path with a failure reason +- [x] 4.2 Start a bounded connectivity timer when signaling completes; if not `connected`/`completed` before it expires, request teardown via `onWebrtcClosed`; reset/cancel the timer on first media +- [x] 4.3 Expose the timeout window as a named constant with margin over the observed worst-case first-media latency + +## 5. Tests and validation + +- [x] 5.1 Update `SbmdCameraWebrtcTest.cpp`: replace the `sessionStatus` `updateResource` assertion with a `webrtcError` assertion, add coverage for the allocate/provide-offer/timeout failure emissions, and assert the `stream` result shape `{ protocol, entryPoint }` +- [x] 5.2 Run `validate-sbmd` on `camera.sbmd.js` and the C++ unit suite; confirm green +- [x] 5.3 Manual end-to-end: happy path streams (found + fixed an `entryPoint` URI bug — driver emitted `/devices//...` but Barton URIs are `//...`); connection reaches CONNECTED, watchdog cancels with no false timeout, no spurious `webrtcError`. Pre-connection ICE-failure watchdog and `webrtcError` emissions (End/allocate/provide-offer/timeout) are covered by unit tests; post-connection media loss is out of scope (unchanged pre-existing behavior). +- [x] 5.4 `clang-format` all touched C/C++ files from the repo root and confirm formatting/hooks pass diff --git a/openspec/specs/camera-session-lifecycle/spec.md b/openspec/specs/camera-session-lifecycle/spec.md new file mode 100644 index 00000000..dd6400b6 --- /dev/null +++ b/openspec/specs/camera-session-lifecycle/spec.md @@ -0,0 +1,57 @@ +# camera-session-lifecycle Specification + +## Purpose +The protocol-agnostic abstract camera endpoint contract: the session lifecycle executes (`createSession`, `stream`, `takePicture`, `destroySession`) with no `sessionStatus` resource. The `stream` execute returns the active protocol and its entry-point URI; all in-session state, error, and teardown signaling lives on the protocol-specific endpoint (e.g. `ep/webrtc`), not the abstract one. + +## Requirements +### Requirement: Abstract camera endpoint provides protocol-agnostic session lifecycle + +The camera SBMD driver SHALL declare an endpoint with id `"camera"` and profile `"camera"` exposing the session lifecycle as execute resources only: `createSession`, `stream`, `takePicture`, and `destroySession`. The abstract endpoint SHALL NOT declare a `sessionStatus` resource, and SHALL NOT carry any protocol-specific signaling or in-session state. All in-session state, error, and teardown signaling SHALL live on the protocol-specific endpoint (e.g. `ep/webrtc`). + +#### Scenario: Camera endpoint exposes only lifecycle executes +- **WHEN** a Matter camera device (deviceType 0x0142) is commissioned +- **THEN** the device SHALL have an endpoint with id `"camera"` exposing `createSession`, `stream`, `takePicture`, and `destroySession` as execute resources AND SHALL NOT expose a `sessionStatus` resource + +#### Scenario: No protocol coupling on the abstract endpoint +- **WHEN** the `ep/camera` endpoint is inspected +- **THEN** none of its resources SHALL reference a specific streaming protocol; protocol identity is carried only in the `stream` execute result and on the protocol endpoint + +### Requirement: createSession allocates a session and returns its identifier + +The `createSession` execute handler SHALL allocate a new session, persist it in transient data, and return the new `sessionId` synchronously as the execute result so the client holds a correlation identifier before invoking any further resource. + +#### Scenario: Client creates a session +- **WHEN** a client executes `createSession` +- **THEN** the handler SHALL return a non-empty `sessionId` string AND record the session in transient data with an initial state + +#### Scenario: Corrupt session data is reset +- **WHEN** `createSession` is executed and the stored session data cannot be parsed +- **THEN** the handler SHALL reset the session store and return an error result + +### Requirement: stream execute returns the active protocol and entry point + +The `stream` execute handler SHALL mark the identified session as streaming and return, as its synchronous execute result, a JSON object identifying the active protocol and the entry-point resource URI the client must use next: `{ "protocol": "", "entryPoint": "//ep//r/" }`. The handler SHALL NOT emit a separate event to convey the next action. + +#### Scenario: Stream returns protocol and entry point for a WebRTC camera +- **WHEN** a client executes `stream` with a valid `sessionId` on a Matter WebRTC camera +- **THEN** the handler SHALL return `{ "protocol": "webrtc", "entryPoint": "//ep/webrtc/r/localSdp" }` AND mark the session `streaming` + +#### Scenario: Stream on unknown session +- **WHEN** a client executes `stream` with a `sessionId` that does not exist +- **THEN** the handler SHALL return an error result and SHALL NOT mark any session streaming + +### Requirement: destroySession releases session state + +The `destroySession` execute handler SHALL remove the identified session from transient data and trigger any protocol-specific teardown required for a session that reached streaming. + +#### Scenario: Client destroys a session +- **WHEN** a client executes `destroySession` with a valid `sessionId` +- **THEN** the handler SHALL remove the session from transient data + +### Requirement: Client discovers next steps without server-pushed status events + +A client SHALL be able to drive the full session flow using only execute results and protocol-endpoint event subscriptions, without reading or subscribing to any status resource on the abstract endpoint. + +#### Scenario: Reference client drives flow without sessionStatus +- **WHEN** the reference app runs a camera stream +- **THEN** it SHALL obtain the protocol and entry point from the `stream` execute result and subscribe to protocol-endpoint events, and SHALL NOT subscribe to any `sessionStatus` resource diff --git a/openspec/specs/camera-stream-reference-command/spec.md b/openspec/specs/camera-stream-reference-command/spec.md new file mode 100644 index 00000000..9cc11b7e --- /dev/null +++ b/openspec/specs/camera-stream-reference-command/spec.md @@ -0,0 +1,142 @@ +# camera-stream-reference-command Specification + +## Purpose +The reference app's `cameraStream` (`cs`) command drives a Matter camera through Barton's resource API — creating a session, performing the WebRTC signaling handshake, and acting as the in-container WebRTC peer — then routes the received media to a destination selected by `--out` (record to a file or serve over HTTP to a browser). It uses only `BCoreClient` APIs and is gated behind a CMake option. +## Requirements +### Requirement: cameraStream command exists in reference app + +The reference app SHALL provide a command named `cameraStream` with short alias `cs` in a dedicated camera command category. The command SHALL accept a device ID as a required argument and an optional `--out ` flag that selects the media destination. + +#### Scenario: Command appears in help +- **WHEN** a user types `help` in the reference app +- **THEN** the camera category SHALL list `cameraStream` with usage: ` [--out ]` + +#### Scenario: Command with short alias +- **WHEN** a user types `cs ` +- **THEN** the command SHALL execute identically to `cameraStream ` + +#### Scenario: Output URI selects the media destination +- **WHEN** the command is invoked with `--out file://` +- **THEN** the stream SHALL be recorded to that file path +- **WHEN** the command is invoked with `--out [:]` (optionally prefixed with `http://`), or without `--out` +- **THEN** the stream SHALL be served over HTTP for a browser to play, defaulting to a loopback host and port when `--out` is omitted + +### Requirement: cameraStream orchestrates full session lifecycle + +The `cameraStream` command SHALL orchestrate the complete camera streaming flow through Barton's resource API: +1. Execute `createSession` on the device's `ep/camera` endpoint +2. Execute `stream` and obtain the active protocol and entry-point URI from its `{ protocol, entryPoint }` result +3. Read the `negotiationRole` resource on `ep/webrtc` to learn whether this client is the `offerer` or the `answerer` +4. Create a local GStreamer `webrtcbin` peer connection using host candidates only (no STUN/TURN), configured for the negotiated role +5. Perform the role-appropriate SDP exchange via the `localSdp` resource on `ep/webrtc`: + - **Offerer**: generate a local SDP offer from webrtcbin, execute `localSdp` with it, then wait for a `remoteSdp` event (the camera's answer) and set it as the remote description + - **Answerer**: execute `localSdp` with empty input to open the flow, wait for a `remoteSdp` event (the camera's offer), set it as the remote description, generate a local SDP answer, and execute `localSdp` with the answer +6. Exchange ICE candidates (local → `localIceCandidates`, remote ← `remoteIceCandidates` events) +7. Wait for the peer connection to reach the connected state, subject to a bounded connectivity timeout +8. Route the received media to the destination selected by `--out`: serve it over the built-in HTTP server or record it to a file +9. On user interrupt (Ctrl+C) or a `webrtcError` event: execute `destroySession` and tear down the pipeline + +#### Scenario: Successful camera stream served to a browser +- **WHEN** a user executes `cameraStream ` (or with an `http://` `--out`) and the camera responds to signaling +- **THEN** the reference app SHALL serve the live stream over its built-in HTTP server for a browser to play and print status messages for each step, without opening a local display window + +#### Scenario: Successful camera stream to file +- **WHEN** a user executes `cameraStream --out file://recording.mp4` +- **THEN** the reference app SHALL record the video stream to the specified file path + +#### Scenario: User stops the stream +- **WHEN** a user presses Ctrl+C during an active stream +- **THEN** the reference app SHALL execute `destroySession`, stop the GStreamer pipeline, and return to the command prompt + +### Requirement: cameraStream uses only BCoreClient API for signaling + +The `cameraStream` command SHALL interact with Barton exclusively through `BCoreClient` APIs (`b_core_client_execute_resource`, event subscriptions). It SHALL NOT use Matter SDK APIs, link against Matter libraries, or reference Matter-specific types. + +#### Scenario: No Matter SDK dependency +- **WHEN** the reference app is compiled +- **THEN** the camera stream module SHALL compile without any Matter SDK headers in its include path + +### Requirement: cameraStream uses GStreamer webrtcbin for media + +The `cameraStream` command SHALL use GStreamer's `webrtcbin` element as its local WebRTC peer connection (host candidates only). The received H.264 SHALL be handled by a passthrough pipeline that neither decodes nor renders locally: `rtph264depay → h264parse → h264timestamper → capsfilter → mp4mux` (fragmented, streamable) `→ appsink`. The `h264timestamper` reconstructs the PTS/DTS the camera's RTP buffers lack (so `mp4mux` does not abort on a missing PTS), and the `capsfilter` forces AVC / `alignment=au` output so `mp4mux` can negotiate. The muxed fragmented-MP4 buffers SHALL be delivered either to the built-in HTTP media server or to a file, according to `--out`. + +#### Scenario: Serve mode pipeline +- **WHEN** `cameraStream` runs in serve mode (the default, or an `http://` `--out`) +- **THEN** the muxed fragmented-MP4 buffers SHALL be pushed to the built-in HTTP media server, which serves them to a browser that decodes and plays them via Media Source Extensions + +#### Scenario: Record mode pipeline +- **WHEN** `cameraStream` is invoked with `--out file://` +- **THEN** the muxed fragmented-MP4 buffers SHALL be written to the file at that path + +#### Scenario: GStreamer not available +- **WHEN** GStreamer libraries (with `webrtcbin`) are not available at runtime +- **THEN** the command SHALL print an error explaining the requirement and exit gracefully + +### Requirement: cameraStream subscribes to Barton events + +The `cameraStream` command SHALL subscribe to resource events on the device to receive signaling data asynchronously. Specifically: +- `remoteSdp` events on `ep/webrtc` (for the camera's remote SDP — an answer when the client is the offerer, or an offer when the client is the answerer) +- `remoteIceCandidates` events on `ep/webrtc` (for the camera's ICE candidates) +- `webrtcError` events on `ep/webrtc` (for asynchronous session termination and errors) + +The command SHALL NOT subscribe to any `sessionStatus` resource on `ep/camera`. It SHALL obtain the active protocol and entry-point URI from the `stream` execute result, and its negotiation role by reading the `negotiationRole` resource on `ep/webrtc`. + +#### Scenario: Remote SDP delivered via event +- **WHEN** the camera provides its remote SDP (an answer when the client offered, or an offer when the client is the answerer) +- **THEN** the reference app SHALL receive it as a `remoteSdp` event and feed it to webrtcbin as the remote description + +#### Scenario: Remote ICE candidates delivered via events +- **WHEN** the camera sends ICE candidates +- **THEN** the reference app SHALL receive them as `remoteIceCandidates` events and add each candidate to webrtcbin + +#### Scenario: Session error delivered via webrtcError event +- **WHEN** the driver emits a `webrtcError` event with an ended or failed value +- **THEN** the reference app SHALL treat it as a session-terminated signal and begin graceful teardown, reporting the reason from the event metadata + +### Requirement: cameraStream reports progress to user + +The command SHALL emit human-readable progress messages to stdout at each stage of the flow: +- Session created (sessionId) +- Streaming initiated (protocol, entryPoint) +- Local SDP sent (the offer, or the answer to the camera's offer) +- Remote SDP received (the camera's answer or offer) +- ICE candidates exchanged +- Media flowing / connected +- Stream ended (reason) + +#### Scenario: Progress output during successful stream +- **WHEN** `cameraStream` completes signaling and media begins flowing +- **THEN** the user SHALL see step-by-step status messages indicating progress through the flow + +### Requirement: cameraStream handles errors gracefully + +The command SHALL handle failures at any stage (session creation failure, signaling timeout, asynchronous `webrtcError` failure, and connectivity/ICE failure) by printing an error message, cleaning up any partial state (destroying the session if created), and returning to the command prompt. In particular, because Matter signaling cannot observe media-plane connectivity, the command SHALL enforce a client-side connectivity timeout: if the WebRTC peer connection does not reach a connected state within a bounded window after signaling completes — or transitions to a failed state — the command SHALL tear down and report the failure rather than wait indefinitely. + +#### Scenario: Device does not support camera streaming +- **WHEN** `cameraStream` is executed on a device without a `camera` endpoint +- **THEN** the command SHALL print an error and exit without crashing + +#### Scenario: Signaling timeout +- **WHEN** the camera does not respond to signaling within a reasonable timeout +- **THEN** the command SHALL print a timeout error, destroy the session, and exit + +#### Scenario: Connectivity never established +- **WHEN** signaling completes but the WebRTC peer connection does not reach a connected state within the connectivity timeout window +- **THEN** the command SHALL print a connectivity-failure message, destroy the session, and exit rather than hang on a blank window + +#### Scenario: Peer connection fails +- **WHEN** the WebRTC peer connection transitions to a failed state during or after ICE exchange +- **THEN** the command SHALL print a failure message, destroy the session, and exit + +### Requirement: Camera command category is gated by CMake flag + +The camera stream command and its GStreamer dependencies SHALL be gated behind a `BCORE_REFERENCE_CAMERA_SUPPORT` CMake option (default OFF). When disabled, the reference app builds without GStreamer dependencies and without the camera category. + +#### Scenario: Build without camera stream support +- **WHEN** `BCORE_REFERENCE_CAMERA_SUPPORT=OFF` (default) +- **THEN** the reference app SHALL build successfully without GStreamer development libraries + +#### Scenario: Build with camera stream support +- **WHEN** `BCORE_REFERENCE_CAMERA_SUPPORT=ON` +- **THEN** the reference app SHALL link against gstreamer-1.0, gstreamer-webrtc-1.0, gstreamer-sdp-1.0, gstreamer-app-1.0, and gio-2.0, and include the camera category + diff --git a/openspec/specs/sbmd-v4-runtime/spec.md b/openspec/specs/sbmd-v4-runtime/spec.md index 4f80b964..10ab1617 100644 --- a/openspec/specs/sbmd-v4-runtime/spec.md +++ b/openspec/specs/sbmd-v4-runtime/spec.md @@ -152,3 +152,19 @@ Aliases declared in the `aliases` section SHALL be resolved to cluster+ID pairs #### Scenario: Event alias prerequisite check - **WHEN** a resource has `prerequisites: ["lockOperation"]` and `lockOperation` is an event alias with `clusterId: 0x0101` - **THEN** the prerequisite is satisfied if cluster 0x0101 is present in the device's data cache + +### Requirement: volatile resource mode disables value caching + +The SBMD runtime SHALL support a resource mode `volatile`. A resource declared with the `volatile` mode SHALL be registered with `CACHING_POLICY_NEVER`, causing `updateResource` to emit a `resourceUpdated` event on every call (for a resource that emits events) regardless of whether the new value equals the currently stored value. The `volatile` mode SHALL NOT, by itself, add read, write, or execute access, and SHALL be accepted by mode-to-bitmask conversion without error. A resource is registered `CACHING_POLICY_NEVER` when it declares a read handler OR declares the `volatile` mode; otherwise it is registered `CACHING_POLICY_ALWAYS`. + +#### Scenario: Volatile resource emits on unchanged value +- **WHEN** a resource declared with `modes: ['volatile']` is updated twice with the same value +- **THEN** the runtime SHALL deliver two `resourceUpdated` events (no value-change suppression) + +#### Scenario: Volatile mode is accepted by the schema and runtime +- **WHEN** a driver declares a resource with `volatile` in its modes array +- **THEN** the spec SHALL validate against the SBMD schema AND the resource SHALL register successfully without adding read, write, or execute modes + +#### Scenario: Non-volatile resource without a read handler remains cached +- **WHEN** an event-only resource is declared without a read handler and without the `volatile` mode +- **THEN** the runtime SHALL register it with `CACHING_POLICY_ALWAYS` and suppress `resourceUpdated` events whose value is unchanged diff --git a/openspec/specs/webrtc-signaling-endpoint/spec.md b/openspec/specs/webrtc-signaling-endpoint/spec.md new file mode 100644 index 00000000..27d029fe --- /dev/null +++ b/openspec/specs/webrtc-signaling-endpoint/spec.md @@ -0,0 +1,172 @@ +# webrtc-signaling-endpoint Specification + +## Purpose +The camera SBMD driver's WebRTC protocol endpoint (`ep/webrtc`): the signaling resources (`localSdp`, `remoteSdp`, `localIceCandidates`, `remoteIceCandidates`) that relay SDP and ICE between a Barton client and a Matter camera's WebRTC Transport clusters, the `negotiationRole` resource that tells the client whether it is the offerer or answerer, plus the `webrtcError` event that surfaces asynchronous session termination and failures. The endpoint supports both Matter WebRTC negotiation flows — client-offers (`ProvideOffer`) and camera-offers (`SolicitOffer` + `ProvideAnswer`) — behind a single client-facing contract. All in-session state and error signaling for the WebRTC protocol lives here rather than on the abstract camera endpoint. +## Requirements +### Requirement: WebRTC endpoint declares signaling resources + +The camera SBMD driver SHALL declare an endpoint with id `"webrtc"` and profile `"webrtc"` containing six resources: + +| Resource | Type | Modes | Purpose | +|----------|------|-------|---------| +| `localSdp` | `function` | execute | Client posts its local SDP (offer or answer) to drive signaling | +| `negotiationRole` | `string` | [read] | Reports the negotiation role (`offerer` or `answerer`) the client must adopt | +| `remoteSdp` | `string` | [] (events only) | Delivers the camera's remote SDP (offer or answer) to client | +| `localIceCandidates` | `function` | execute | Client sends local ICE candidates | +| `remoteIceCandidates` | `string` | [] (events only) | Delivers remote ICE candidates to client | +| `webrtcError` | `string` | [volatile] (events only) | Delivers asynchronous session termination/error to client | + +The endpoint SHALL be declared within the same `camera.sbmd.js` file as the `ep/camera` endpoint. The `webrtcError` resource SHALL be declared with the `volatile` mode so that its events are emitted unconditionally (non-cached), independent of the previously emitted value. + +#### Scenario: Endpoint appears on commissioned camera device +- **WHEN** a Matter camera device (deviceType 0x0142) with WebRTCTransportProvider cluster (0x0553) is commissioned +- **THEN** the device SHALL have an endpoint with id `"webrtc"`, profile `"webrtc"`, and all six resources registered + +#### Scenario: Event-only resources are not readable +- **WHEN** a client attempts to read `remoteSdp`, `remoteIceCandidates`, or `webrtcError` +- **THEN** the read SHALL fail or return no value (modes list is empty — no read mode) + +### Requirement: negotiationRole read reports the client's role + +The `negotiationRole` read handler SHALL report whether the client is the `offerer` or the `answerer`, derived from the camera's advertised WebRTCTransportProvider `AcceptedCommandList`. When the camera accepts `SolicitOffer` the role SHALL be `answerer` (the camera generates the offer); otherwise, when the camera accepts `ProvideOffer`, the role SHALL be `offerer`. When the accepted-command list is unavailable, the handler SHALL default to `answerer`. The negotiation role is a WebRTC concept and lives on the `webrtc` endpoint, not in the abstract `stream` result. + +#### Scenario: Camera supporting SolicitOffer yields answerer +- **WHEN** a client reads `negotiationRole` and the camera's `AcceptedCommandList` includes `SolicitOffer` +- **THEN** the read SHALL return `answerer` + +#### Scenario: Camera supporting only ProvideOffer yields offerer +- **WHEN** a client reads `negotiationRole` and the camera's `AcceptedCommandList` includes `ProvideOffer` but not `SolicitOffer` +- **THEN** the read SHALL return `offerer` + +### Requirement: localSdp execute drives role-appropriate signaling + +The `localSdp` execute handler SHALL determine the client's negotiation role from the camera's advertised WebRTCTransportProvider commands (its `AcceptedCommandList`) and drive the corresponding Matter signaling. In every case that requires it, the handler SHALL first allocate a video stream via `VideoStreamAllocate` (cluster 0x0551, command 0x03) before the WebRTC-provider command, and SHALL pass the requestor's `originatingEndpointID` (the endpoint hosting the `WebRTCTransportRequestor` cluster) so the camera knows where to send its commands. + +- **Offerer flow** (camera accepts `ProvideOffer`): the execute input is the client's SDP offer. The handler SHALL allocate a video stream and then send a `ProvideOffer` command (ID 0x02) to the camera's `WebRTCTransportProvider` cluster (0x0553), carrying the SDP and the allocated `videoStreamID`. +- **Answerer flow** (camera accepts `SolicitOffer`): while no camera `webRTCSessionID` has been recorded yet, an execute (with empty input) SHALL allocate a video stream and then send a `SolicitOffer` command (ID 0x00) so the camera generates the offer. Once the camera's offer has arrived (its `webRTCSessionID` recorded), a subsequent execute SHALL carry the client's SDP answer and send a `ProvideAnswer` command (ID 0x04) with the SDP and the recorded `webRTCSessionID`. + +#### Scenario: Offerer posts an SDP offer +- **WHEN** the camera accepts `ProvideOffer` and a client executes `localSdp` with a valid SDP offer +- **THEN** the handler SHALL allocate a video stream and send a `ProvideOffer` command to the camera with the SDP and the allocated `videoStreamID` + +#### Scenario: Answerer opens the flow +- **WHEN** the camera accepts `SolicitOffer` and a client executes `localSdp` with empty input before any camera offer has arrived +- **THEN** the handler SHALL allocate a video stream and send a `SolicitOffer` command so the camera generates the offer + +#### Scenario: Answerer posts its SDP answer +- **WHEN** the camera has offered (its `webRTCSessionID` is recorded) and a client executes `localSdp` with an SDP answer +- **THEN** the handler SHALL send a `ProvideAnswer` command to the camera with the SDP and the recorded `webRTCSessionID` + +#### Scenario: No active session +- **WHEN** a client executes `localSdp` but no session is in `streaming` state +- **THEN** the handler SHALL return an error result + +### Requirement: localIceCandidates execute sends ProvideICECandidates to camera + +The `localIceCandidates` execute handler SHALL send a `ProvideICECandidates` command (ID 0x05) to the camera's `WebRTCTransportProvider` cluster (0x0553). The execute input is a JSON-encoded array of ICE candidate strings. + +#### Scenario: Client provides ICE candidates +- **WHEN** a client executes `localIceCandidates` with a JSON array of ICE candidate strings +- **THEN** the SBMD handler SHALL send a `ProvideICECandidates` command to the camera with the candidates in the `ICECandidates` field + +### Requirement: Incoming Offer and Answer commands emit remoteSdp event + +The SBMD driver SHALL register command handlers for both the `Offer` command (ID 0x00) and the `Answer` command (ID 0x01) on the `WebRTCTransportRequestor` cluster (0x0554). When either is received, the handler SHALL extract the SDP string, record the command's `webRTCSessionID` on the active session, and emit the SDP as an event on the `remoteSdp` resource of the `webrtc` endpoint. The `Offer` command carries the camera's offer (SolicitOffer flow); the `Answer` command carries the camera's answer (ProvideOffer flow). + +#### Scenario: Camera sends its offer (SolicitOffer flow) +- **WHEN** the camera sends an `Offer` command (cluster 0x0554, command 0x00) containing an SDP string +- **THEN** the SBMD handler SHALL record the `webRTCSessionID` AND call `updateResource('webrtc', 'remoteSdp', sdpString)` to emit an event to subscribed clients + +#### Scenario: Camera sends its answer (ProvideOffer flow) +- **WHEN** the camera sends an `Answer` command (cluster 0x0554, command 0x01) containing an SDP string +- **THEN** the SBMD handler SHALL record the `webRTCSessionID` AND call `updateResource('webrtc', 'remoteSdp', sdpString)` to emit an event to subscribed clients + +### Requirement: Incoming ICECandidates command emits remoteIceCandidates event + +The SBMD driver SHALL register a command handler for the `ICECandidates` command (ID 0x02) on the `WebRTCTransportRequestor` cluster (0x0554). When received, the handler SHALL extract the candidate list and emit it as a JSON-encoded array on the `remoteIceCandidates` resource. + +#### Scenario: Camera sends ICE candidates +- **WHEN** the camera sends an `ICECandidates` command (cluster 0x0554, command 0x02) containing ICE candidates +- **THEN** the SBMD handler SHALL call `updateResource('webrtc', 'remoteIceCandidates', jsonCandidates)` to emit an event to subscribed clients + +### Requirement: webrtcError resource emits every event unconditionally + +The `webrtcError` resource SHALL be declared with the `volatile` mode, which registers it with `CACHING_POLICY_NEVER` so that `updateResource` bypasses value-change detection and each emission delivers a `resourceUpdated` event to subscribers even when consecutive values are identical (including across sessions where a prior value persists). + +#### Scenario: Repeated identical values still emit +- **WHEN** the driver emits `webrtcError` twice in succession with the same value +- **THEN** the client SHALL receive two distinct `resourceUpdated` events (no no-change suppression) + +### Requirement: Incoming End command emits webrtcError event + +The SBMD driver SHALL register a command handler for the `End` command (ID 0x03) on the `WebRTCTransportRequestor` cluster (0x0554). When received, the handler SHALL clean up the associated session and emit a `webrtcError` event on the `webrtc` endpoint with a value indicating the session ended and metadata carrying the reason. + +#### Scenario: Camera ends session +- **WHEN** the camera sends an `End` command with a reason code +- **THEN** the SBMD handler SHALL call `updateResource('webrtc', 'webrtcError', , { "reason": "", "detail": "" })` AND remove the associated session from transient data + +### Requirement: Asynchronous signaling failures emit webrtcError event + +Each asynchronous WebRTC signaling failure that occurs after the originating execute has returned SHALL emit a `webrtcError` event so the client is notified rather than left to time out. This SHALL cover at least: a `VideoStreamAllocate` error, a `ProvideOffer` error (offerer flow), a `SolicitOffer` error (answerer flow), and a `requestCommand` overall-deadline timeout in the signaling flow. + +#### Scenario: VideoStreamAllocate rejected by camera +- **WHEN** the camera rejects the `VideoStreamAllocate` command during the `localSdp` flow +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and metadata describing the allocate error + +#### Scenario: ProvideOffer rejected by camera +- **WHEN** the camera rejects the `ProvideOffer` command during the offerer (`ProvideOffer`) flow +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and metadata describing the provide-offer error + +#### Scenario: SolicitOffer rejected by camera +- **WHEN** the camera rejects the `SolicitOffer` command during the answerer (`SolicitOffer`) flow +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and metadata describing the solicit-offer error + +#### Scenario: Signaling command times out +- **WHEN** a `requestCommand` in the `localSdp` flow exceeds its overall deadline +- **THEN** the SBMD handler SHALL emit a `webrtcError` event with a failure value and a timeout reason + +### Requirement: destroySession sends EndSession to camera + +When the camera session endpoint's `destroySession` is executed for a session that has progressed to WebRTC signaling, the handler SHALL send an `EndSession` command (ID 0x06) to the camera's `WebRTCTransportProvider` cluster (0x0553) before cleaning up local session state. + +#### Scenario: Client destroys active streaming session +- **WHEN** a client executes `destroySession` for a session in `streaming` state +- **THEN** the handler SHALL send `EndSession` to the camera AND remove the session from transient data + +#### Scenario: Client destroys session that never started streaming +- **WHEN** a client executes `destroySession` for a session in `created` state (never executed `stream`) +- **THEN** the handler SHALL only remove the session from transient data (no Matter command needed) + +### Requirement: WebRTC constants use correct Matter cluster and command IDs + +The SBMD driver SHALL define constants for all WebRTC cluster and command identifiers: + +| Constant | Value | Description | +|----------|-------|-------------| +| CL_WEBRTC_TRANSPORT_PROVIDER | 0x0553 | Camera's provider cluster | +| CL_WEBRTC_TRANSPORT_REQUESTOR | 0x0554 | Barton's requestor cluster | +| CL_CAMERA_AV_STREAM_MGMT | 0x0551 | Camera A/V stream management (video stream allocation) | +| CMD_VIDEO_STREAM_ALLOCATE | 0x03 | Allocate a video stream before offer/solicit | +| CMD_SOLICIT_OFFER | 0x00 | Ask the camera to generate the offer (answerer flow) | +| CMD_SOLICIT_OFFER_RESP | 0x01 | Camera's response to SolicitOffer | +| CMD_PROVIDE_OFFER | 0x02 | Send SDP offer to camera (offerer flow) | +| CMD_PROVIDE_ANSWER | 0x04 | Send SDP answer to camera (answerer flow) | +| CMD_PROVIDE_ICE | 0x05 | Send ICE candidates to camera | +| CMD_END_SESSION | 0x06 | End a WebRTC session | +| CMD_OFFER | 0x00 | Incoming offer from camera | +| CMD_ANSWER | 0x01 | Incoming answer from camera | +| CMD_ICE_CANDIDATES | 0x02 | Incoming ICE from camera | +| CMD_END | 0x03 | Incoming end from camera | + +#### Scenario: Constants match Matter specification +- **WHEN** the SBMD driver is loaded +- **THEN** all cluster and command ID constants SHALL match the values defined in the Matter 1.5 WebRTC Transport cluster specification + +### Requirement: WebRTC endpoint is separable by design + +The webrtc endpoint resources, constants, and handler functions SHALL be grouped together and access session state only through transient data supplements. No direct coupling between camera endpoint handlers and webrtc endpoint handlers beyond shared transient data keys. + +#### Scenario: Code organization supports extraction +- **WHEN** the webrtc endpoint code is reviewed +- **THEN** all webrtc-specific constants, resources, and handlers SHALL be identifiable as a cohesive group that could be moved to a separate file with only transient data key sharing as the interface +