From 736832aa912fe48f99873fb4abf83e69eef9417b Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Mon, 6 Jul 2026 19:39:16 -0400 Subject: [PATCH 01/10] docs: add deep sleep findings and implementation plan Co-Authored-By: Claude Fable 5 --- docs/DEEP_SLEEP_FINDINGS_2026-07-06.md | 172 ++++++++++ ...EP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md | 315 ++++++++++++++++++ 2 files changed, 487 insertions(+) create mode 100644 docs/DEEP_SLEEP_FINDINGS_2026-07-06.md create mode 100644 docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md diff --git a/docs/DEEP_SLEEP_FINDINGS_2026-07-06.md b/docs/DEEP_SLEEP_FINDINGS_2026-07-06.md new file mode 100644 index 0000000..a5f71e7 --- /dev/null +++ b/docs/DEEP_SLEEP_FINDINGS_2026-07-06.md @@ -0,0 +1,172 @@ +# OpenDisplay Deep Sleep — Cross-Stack Findings + +*Investigation of deep sleep behavior across Firmware (ESP32/nRF), Firmware_Silabs, py-opendisplay, and the Home Assistant integration — 2026-07-06* + +## TL;DR + +- **The ESP32 firmware's deep-sleep loop itself is sound** (timer wake, ~10 s advertising window, first-boot grace period, panel power-down). +- **Bug: `rebootFlag` is not persisted across deep sleep**, so every wake advertises "I rebooted". Combined with the HA integration's reload-on-reboot behavior, this can produce an intermittent **reload → connect → sleep → wake → reload churn loop** that drains battery and spams HA. +- **Gap: HA has no wake-window awareness.** Service calls connect immediately and fail if the device is mid-sleep; nothing waits for the next wake advertisement. Uploads to a sleeping device succeed only by luck. +- **Gap: a commanded reboot leaves almost no reconnect window** — the device re-enters deep sleep within ~1 s because the first-boot grace period is skipped. +- **Gap: py-opendisplay does not implement the deep-sleep command (0x0052)**, so neither HA nor the CLI can put a device to sleep, even though ESP32 and Silabs firmware both handle it. + +## How deep sleep works today + +### ESP32 (reference firmware) + +Deep sleep engages when `power_option.power_mode == 1` (BATTERY) **and** +`deep_sleep_time_seconds > 0`. + +- **Entry** — `enterDeepSleep()` (`Firmware/src/main.cpp:275`): stops advertising, deinits BLE, arms **timer wakeup only** (`esp_sleep_enable_timer_wakeup`, `main.cpp:301`), holds the power latch, calls `esp_deep_sleep_start()`. The loop enters it whenever `bleActive` is false (no connection, empty command/response queues, no EPD refresh, no WiFi LAN session). +- **Wake** — deep-sleep wake is a full CPU boot. `setup()` detects the wake cause and runs `minimalSetup()` (`main.cpp:245`): config init, IO init, BLE advertising only — no display init, no WiFi, no boot screen. +- **Wake window** — the device advertises for `sleep_timeout_ms` (default **10 s** when 0, `main.cpp:94-96`). If nothing connects, it returns to sleep. If a central connects, `fullSetupAfterConnection()` (`main.cpp:256`) brings up WiFi and the panel driver state, and queued BLE commands are processed on subsequent loop passes. +- **First boot** — a 2-minute grace period before the first deep sleep (`FIRST_BOOT_DEEP_SLEEP_DELAY_MS`, `main.cpp:180-196`) gives the user time to adopt/configure. Applies only when `deep_sleep_count == 0`. +- **State across sleep** — `woke_from_deep_sleep`, `deep_sleep_count`, `displayed_etag` are `RTC_DATA_ATTR` (`Firmware/src/main.h:285-294`) and survive. Ordinary globals (including `rebootFlag`) reset every wake. + +### nRF + +No deep sleep. `sleep_timeout_ms` is used as an idle-loop delay; the SoftDevice's +System-ON idle is the low-power state. The 0x0052 command replies +"not supported" (`Firmware/src/device_control.cpp:703`). + +### Silabs (Flex) + +Deep sleep is **command-driven only**: 0x0052 sets `s_pending_deep_sleep`; after the +BLE connection closes, the firmware powers off the panel, arms **EM4 wake on button +and NFC field-detect**, and calls `EMU_EnterEM4()` +(`Firmware_Silabs/opendisplay_ble.c:1921-1926`). There is **no timer wake** — +`deep_sleep_time_seconds` exists in the Silabs config struct but is unused. A +sleeping Flex device stays down until a button press or NFC field. + +### py-opendisplay + +Serializes/parses the power config fields (`power_mode`, `sleep_timeout_ms`, +`deep_sleep_time_seconds`, `deep_sleep_current_ua`) but **exposes no deep-sleep +command** — only `reboot()` (0x000F). Connections go through bleak-retry-connector +(4 attempts × 10 s timeout, plus one GATT-cache-clear retry, +`py-opendisplay/src/opendisplay/transport/connection.py`). + +### Home Assistant integration + +- Monitoring is fully passive: `OpenDisplayCoordinator` is a + `PassiveBluetoothDataUpdateCoordinator` fed by advertisements only. +- Connections happen for: entry setup (firmware/config interrogation, + `__init__.py:104`), service calls (`services.py:280` `_async_connect_and_run`), + and OTA (`update.py`). +- The coordinator watches the advertised **reboot flag** (bit 1 of the status + byte) and, on a False → True edge, **reloads the config entry** — which opens a + new BLE connection to re-read firmware + config + (`coordinator.py:129-150`, `__init__.py:180-195`). + +## Bug: deep-sleep wake is indistinguishable from a reboot + +`rebootFlag` is a plain global initialized to 1 (`Firmware/src/main.h:126`), +unlike its RTC-persisted neighbors. Since deep-sleep wake is a full boot, **every +wake re-advertises `reboot_flag = 1`**. + +The churn loop: + +1. HA connects (setup, service call, or a previous iteration of this loop) → + firmware clears the flag on connect (`Firmware/src/esp32_ble_callbacks.h:42`). +2. On disconnect, advertising restarts with flag = 0 + (`esp32_restart_ble_advertising` → `updatemsdata`, + `Firmware/src/ble_init.cpp:165`), but only for a sub-second burst before the + loop re-enters deep sleep. +3. If HA's scanner catches one of those flag-0 adverts, `_last_reboot_flag` + becomes False. +4. Next wake advertises flag = 1 → False → True edge → HA reloads the entry → + connects → interrogates → clears flag → device sleeps → **go to 2**. + +Each cycle also runs `initWiFi()` on the device (`fullSetupAfterConnection()`), +compounding the battery cost. Because step 3 depends on catching a brief +advertising burst, the loop is **intermittent** — it presents as unexplained +battery drain and periodic "Device rebooted since last connection" log lines at +the deep-sleep cadence. + +The coordinator's edge detection is otherwise well designed: None → True is +ignored (setup already synced) and a flag stuck at True self-guards. The defect +is purely that the firmware loses the flag across sleep. + +**Suggested fix (firmware, ~10 lines):** mirror `rebootFlag` into an +`RTC_DATA_ATTR` variable in `enterDeepSleep()` and restore it in `setup()` on a +deep-sleep wake. Power-on reset clears RTC RAM, so genuine cold boots still +advertise 1. The reboot command path (`esp_restart()` preserves RTC RAM) must +explicitly set the mirror back to 1 before restarting. + +## Gaps + +### 1. HA cannot reliably reach a sleeping device + +`_async_connect_and_run` (`services.py:289`) resolves the cached `BLEDevice` and +connects immediately. Against a device sleeping e.g. 5 minutes with a 10 s wake +window, an arbitrary service call has a few-percent success rate; failures raise +`upload_error` and the image is dropped (no queue, no retry-at-next-wake). +The integration already has everything needed to do better: the passive +coordinator sees every wake advertisement, and `entry.runtime_data.device_config` +contains the power settings. + +**Suggested fix (integration):** a "wait for next advertisement, then connect" +helper used by services, setup retry, and OTA when the device config indicates +battery + deep sleep. This is the standard ESL/e-ink hub pattern (queue content, +deliver at check-in). + +### 2. Commanded reboot → near-instant sleep + +`deep_sleep_count` survives `esp_restart()` (RTC RAM persists across software +resets), so after a reboot command the first-boot condition +(`main.cpp:180`, requires `deep_sleep_count == 0`) is false and the device +re-enters deep sleep within about a second. HA's reboot-triggered reload then +usually misses, lands in `ConfigEntryNotReady`, and the entities sit unavailable +until a timer-backoff retry happens to coincide with a wake window. + +**Suggested fix (firmware):** grant a short (e.g. 30–60 s) advertising grace +period after any normal boot, not just the very first one. + +### 3. Availability flapping on long sleep intervals + +Entity availability tracks advertisement freshness. If +`deep_sleep_time_seconds` exceeds HA Bluetooth's stale-advertisement timeout +(on the order of a few minutes), every sleep cycle marks all entities +unavailable and the next wake marks them available again — even though the +device is healthy. Setup retries are timer-based only; nothing retries +immediately when the device reappears. + +### 4. py-opendisplay lacks the deep-sleep command + +Firmware handles 0x0052 on ESP32 (`Firmware/src/device_control.cpp:691`) and +Silabs, but the library exposes only `reboot()`. HA therefore has no way to +command a device to sleep (relevant for Flex devices, where sleep is +command-driven). + +## Design notes / quirks + +- `sleep_timeout_ms` has two meanings: post-wake **advertising window** on ESP32 + (`main.cpp:94`), idle-loop delay on nRF. Default window is 10 s when the field + is 0. +- `connectionRequested` (bit 2 of the advertised status byte, + `Firmware/src/main.h:127`) is reserved and unused — a natural hook for a + future "content pending / stay awake" handshake between HA and the device. +- Wake-on-BLE from true deep sleep is **not possible on ESP32** — the radio is + off; wake sources are timer/GPIO/touch/ULP only. The near-equivalent (light + sleep + BT modem sleep, always connectable) costs ~0.5–2 mA average vs + ~10–20 µA in deep sleep. nRF52/EFR32 keep the radio scheduler alive at µA + levels in their idle states, which is why those targets don't need this + machinery. A cheap firmware improvement: also arm `esp_sleep_enable_ext1_wakeup` + on the button pin so a user can wake the tag on demand (the Silabs target + already wakes from EM4 on button/NFC). + +## What was verified to work + +- Wake window logic is correctly guarded; queued commands received during the + wake window are processed after `fullSetupAfterConnection()`. +- Panel power is managed per-refresh (`pwrmgm(false)` after refresh), so the + display rail is already off at sleep entry; external flash is parked + (CS high, CLK/MOSI low). +- First-boot 2-minute adoption grace period works (power-on resets clear + `deep_sleep_count`). +- The HA coordinator's reboot edge detection ignores the initial observation and + a permanently-set flag; the reload defers until any in-progress upload + finishes (`__init__.py:190-195`). +- Upload concurrency is last-writer-wins: a new upload cancels the in-flight one + (`services.py:396-401`) — appropriate for a display where only the latest + image matters. diff --git a/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md b/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md new file mode 100644 index 0000000..f07401e --- /dev/null +++ b/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md @@ -0,0 +1,315 @@ +# OpenDisplay Deep Sleep — Architecture & Implementation Plan + +*2026-07-06 — Companion to [DEEP_SLEEP_FINDINGS_2026-07-06.md](DEEP_SLEEP_FINDINGS_2026-07-06.md) (cross-stack behavioral findings). This document is the forward-looking design: current state, architectural considerations, component factoring, and a detailed implementation plan. Scope: ESP32 firmware variant, with changes focused on the Home Assistant integration; py-opendisplay changes where necessary; firmware changes minimized and listed separately.* + +--- + +## 1. Current state of the integration and components + +### 1.1 The device side (ESP32 firmware) — what the integration must accommodate + +Validated against `Firmware/src` at HEAD: + +| Behavior | Detail | Source | +|---|---|---| +| Sleep entry condition | `power_mode == 1` (BATTERY) **and** `deep_sleep_time_seconds > 0`; entered whenever BLE is idle | `main.cpp:180-198` | +| Sleep mechanics | Advertising stopped, BLE deinitialized, **timer wakeup only**, power latch held, `esp_deep_sleep_start()` | `main.cpp:275-307` | +| Wake behavior | Full CPU boot → `minimalSetup()` (config + IO + BLE advertising; no display init, no WiFi) | `main.cpp:46-51, 245` | +| Wake window | Advertises for `sleep_timeout_ms` (default **10 s** when 0); returns to sleep if nothing connects | `main.cpp:85-101` | +| On connect | `fullSetupAfterConnection()` brings up WiFi + panel driver; device stays awake while a central is connected (`bleActive`) | `main.cpp:85-90, 256` | +| First boot | 2-minute grace period before first sleep (`deep_sleep_count == 0` only) — the adoption window | `main.cpp:180-196` | +| State across sleep | `woke_from_deep_sleep`, `deep_sleep_count`, `displayed_etag` are `RTC_DATA_ATTR` and survive; `rebootFlag` does **not** (known bug — every wake advertises "rebooted") | `main.h:126, 286-295` | +| Sleep interval range | `deep_sleep_time_seconds` is uint16 → 1 s to ~18.2 h | `models/config.py:207` | +| Command 0x0052 | "Deep sleep now" handled by ESP32 and Silabs firmware; in the official protocol spec | `device_control.cpp:692`, opendisplay.org `protocol/ble-flow.html` | + +Key consequence: **a sleeping ESP32 device is completely dark** — no radio, not connectable, not scannable. The only contact opportunity is the ~10 s advertising window after each timer wake. Once a central connects inside that window, the device stays awake for the whole session, so a connection established at wake time can run arbitrarily long work (uploads, OTA). + +### 1.2 py-opendisplay + +- **Connection layer**: `bleak-retry-connector` with `max_attempts=4`, `timeout=10 s`, plus one GATT-cache-clear retry (`transport/connection.py:33-117`). Both knobs are already exposed on `OpenDisplayDevice(timeout=…, max_attempts=…)` (`device.py:390-392`). +- **Power config**: parses/serializes `power_mode`, `sleep_timeout_ms`, `deep_sleep_time_seconds`, `deep_sleep_current_ua` (`models/config.py:196-256`). No convenience "is deep sleep enabled" predicate. +- **Config serialization**: `config_to_json()` / `config_from_json()` round-trip a full `GlobalConfig` (`models/config_json.py:68, 434`) — ready to use for caching device config in the HA config entry. +- **Advertisement model**: parses the status byte, exposing `reboot_flag` (bit 1) and the reserved `connection_requested` (bit 2) (`models/advertisement.py:384-394`). +- **Gap**: no deep-sleep command — only `reboot()` (0x000F, `device.py:912`). 0x0052 is unimplemented. + +### 1.3 Home Assistant integration (`custom_components/opendisplay`) + +- **Monitoring is fully passive.** `OpenDisplayCoordinator` is a `PassiveBluetoothDataUpdateCoordinator` fed only by advertisements (`coordinator.py:40-51`). Sensors (battery, temperature, RSSI, last-seen) never connect. +- **Connections happen at exactly three points:** + 1. **Entry setup** — `async_setup_entry` requires a connectable `BLEDevice` *and* a successful connect to read firmware + config; otherwise raises `ConfigEntryNotReady` (`__init__.py:97-128`). + 2. **Service calls** — `upload_image`, `drawcustom`, `activate_led`, `activate_buzzer` all go through `_async_connect_and_run`, which resolves the cached `BLEDevice` and connects immediately (`services.py:283-346`). Failure raises `upload_error` and **the content is dropped** — no queue, no retry. + 3. **OTA** — `update.py` connects directly for version checks and flashing. +- **Concurrency primitives already in place** (reused by this design): + - Per-entry `ble_lock` serializing all BLE access to one tag (`__init__.py:58`). + - Latest-wins upload semantics — a new upload cancels the in-flight one (`services.py:444-450`). + - `partial_state` tracking the last-uploaded frame for differential partial refresh (`__init__.py:62`); the panel-side `displayed_etag` survives deep sleep, so partial refresh remains valid across sleep cycles. +- **Reboot handling**: coordinator detects a False→True edge on the advertised reboot flag and reloads the entire config entry, which reconnects (`coordinator.py:129-150`, `__init__.py:190-210`). Combined with the firmware's unpersisted `rebootFlag`, this is the churn-loop hazard documented in the findings doc. +- **Image entity** (`image.py`) shows the last *successfully delivered* frame, updated via dispatcher signal after upload completes (`services.py:409`). This is the natural place to represent queued-but-unsent content. +- **Config flow**: bluetooth discovery + user flow + encryption key + reauth. **No options flow exists** (`config_flow.py`). +- **`const.py`** holds only `DOMAIN`, `CONF_ENCRYPTION_KEY`, and one dispatcher signal — no option constants yet. + +### 1.4 What HA core already provides (validated against `core` checkout, 2026.6 dev) + +These built-in mechanisms shape the design — some help, some actively interfere: + +1. **Setup retry with exponential backoff.** `ConfigEntryNotReady` → `SETUP_RETRY` state, retried at `min(2^tries × 5 s, 600 s)` (`config_entries.py:843`). Blind timer retries against a device sleeping N minutes with a 10 s window have a per-try success probability of roughly `10/N·60` — a few percent. **Timer-based retry alone is not a solution.** +2. **Advertisement-triggered reload of entries in SETUP_RETRY.** When a Bluetooth *discovery flow* fires for an already-configured entry in `SETUP_RETRY`, HA schedules an immediate reload (`config_entries.py:3158-3169`). This is the built-in "retry when the device reappears" path — **but** it only fires if the integration matcher re-triggers discovery, which only happens after the address *disappears* from the scanner history (`bluetooth/manager.py:174-176`) — i.e. after the device has been unavailable long enough to be expired. It is timing-dependent and races the 10 s wake window (reload → connect must land inside the window; scheduled reload usually does, since it fires on the wake advertisement itself). It works *sometimes* today; it is not something to build on, and extending the availability window (below) deliberately disables it. +3. **Availability tracking with a configurable horizon.** Devices are marked unavailable when advertisements go stale (fallback ~5 min, `UNAVAILABLE_TRACK_SECONDS = 300`). Critically, HA exposes **`async_set_fallback_availability_interval(hass, address, seconds)`** (`bluetooth/api.py:297`) — a per-address override of the staleness horizon. This is the sanctioned lever to keep a deep-sleeping device "available" between wakes. (The *learned* advertising-interval mechanism needs many consecutive adverts and will never learn a sleep cycle; the fallback interval is the right tool.) +4. **Advertisement callbacks.** `async_register_callback` (per-address advertisement callback) and `async_process_advertisements` (await-next-matching-advert with timeout) (`bluetooth/api.py:138, 165`). The coordinator already receives every advertisement via `_async_handle_bluetooth_event` — no new registration is needed for the wake trigger; a hook on the existing coordinator suffices. +5. **Service response data** (`SupportsResponse.OPTIONAL`) — lets `upload_image`/`drawcustom` report "delivered" vs "queued" to automations without blocking. + +--- + +## 2. Architectural considerations + +### 2.1 Design principles + +**P1 — Asleep is a state, not an error.** For a device configured for deep sleep, unreachability is its normal operating condition ~99% of the time. Every code path that today treats "can't connect" as failure must, in sleep mode, treat it as "defer". + +**P2 — The advertisement is the only reliable rendezvous.** All communication with a sleeping device must be initiated within seconds of seeing a wake advertisement. Everything else (setup, delivery, OTA) is architected as *work queued until the next rendezvous*. + +**P3 — Never connect without a reason.** Each connection forces the device through `fullSetupAfterConnection()` (including WiFi init) and holds it awake — battery cost is dominated by connected time, not advertising. The integration must connect only when it has pending work, and must batch all pending work into a single connection per wake. + +**P4 — Transport-agnostic trigger.** The rendezvous trigger is "device seen on any transport". Today that is a BLE advertisement; the future WiFi implementation (see [WIFI_ARCHITECTURE_2026-07-06.md](WIFI_ARCHITECTURE_2026-07-06.md)) delivers the same trigger from mDNS. The delivery machinery must depend on a *device-seen event*, not on BLE specifics. + +**P5 — Minimize firmware changes.** Everything below works against today's firmware. Two small firmware fixes are recommended (§5.4) but nothing depends on them. + +### 2.2 Key design decisions + +#### D1 — How sleep mode is determined + +Auto-detect from the device's own config, already held in `entry.runtime_data.device_config`: + +``` +sleepy = power_option.power_mode == PowerMode.BATTERY (1) + and power_option.deep_sleep_time_seconds > 0 +``` + +This exactly mirrors the firmware's own sleep-entry condition (`main.cpp:198`), so integration and device can never disagree. An options-flow override (`auto` / `force on` / `force off`, default `auto`) covers edge cases (e.g. Silabs Flex devices whose sleep is command-driven, or a user who wants strict-failure semantics). + +#### D2 — Setup from cache: the entry must load without the device + +**Problem (user scenario 1):** device asleep at HA startup → `async_setup_entry` can't connect → `ConfigEntryNotReady` → entities gone, timer retries rarely coincide with a wake window, entry effectively dead until luck strikes. + +**Decision:** cache everything setup needs — serialized `GlobalConfig` (via `config_to_json`), firmware version, `is_flex` — in `entry.data` after every successful interrogation. On subsequent setups, when the device is not immediately reachable **and** the cache says it is a sleepy device, **set up entirely from cache**: build runtime data, register device info, start the passive coordinator, forward platforms — no connection at all. Mark a `config_resync_pending` flag; the delivery manager (D4) re-reads firmware/config opportunistically at the next wake and updates the cache. + +Rationale for caching over smarter retries: +- It removes the race entirely instead of trying to win it. HA startup, HA restarts, and reloads all become instant and deterministic. +- The built-in rediscovery-reload path (§1.4-2) is disabled anyway once we extend the availability interval (D3): the address never expires, the matcher never resets, discovery never re-fires. Caching replaces a mechanism this design would otherwise silently break. +- It matches how HA treats other sleepy-device ecosystems (Zigbee ESLs, ESPHome deep-sleep nodes): entities exist and hold state; freshness is communicated via availability and `last_seen`. + +First-time adoption still requires a live connection — acceptable, because the config flow runs during the firmware's 2-minute first-boot grace period, and a user adopting a device has it awake by definition. If setup finds no cache and cannot connect, `ConfigEntryNotReady` remains correct. + +Cache invalidation: rewritten after every successful config read (setup, post-reboot resync, post-wake resync). Reauth clears nothing (the key lives separately in `entry.data`). + +#### D3 — Availability policy: entities stay available across sleep + +**Problem (user scenario 1a):** with the default ~5 min staleness horizon, any sleep interval > ~4 min flaps every entity unavailable/available once per cycle. + +**Decision:** when sleep mode is active, call +`async_set_fallback_availability_interval(hass, address, deep_sleep_time_seconds × missed_cycles + wake_window + 60 s)` +at setup and whenever the config changes. `missed_cycles` is an options-flow setting (default **3**): the device is marked unavailable only after missing three consecutive expected wakes. Examples: 5 min sleep → unavailable after ~16 min of silence; 12 h sleep → ~36 h. This directly implements the user requirement "leave the entry alive until some defined period of unavailability (hours or days)" — with the period derived from the device's own cadence rather than a wall-clock guess, and clamped by an absolute options override for users who want a fixed horizon. + +The `last_seen` sensor already reports true freshness, so nothing is hidden from the user. Availability now means "checking in on schedule" instead of "advertising right now" — the correct semantic for a sleepy device. + +#### D4 — Wake-triggered delivery: queue work, deliver at the rendezvous + +**Problem (user scenario 2):** sending to a sleeping device fails after ~40 s of bleak retries and drops the content. + +**Decision:** introduce a per-entry **delivery manager** owning *pending work slots*, triggered by the coordinator's advertisement handler: + +- **Pending slots, latest-wins per type** (consistent with existing upload semantics): + - `pending_upload`: the *prepared* image (post dither/encode/compress — CPU work done at queue time, once), refresh mode, partial state reference, plus the preview JPEG, `created_at`, `expires_at`, `attempts`. + - `pending_config_resync`: flag — re-read firmware + config, refresh cache (set by D2 cache-setup and D6 reboot handling). + - `pending_ota`: deferred to the OTA flow itself (§4, Phase 3); the slot exists so a wake can resume a user-requested update. +- **Trigger:** `OpenDisplayCoordinator._async_handle_bluetooth_event` already fires on every advertisement. Add a `async_subscribe_device_seen` hook (mirroring the existing `async_subscribe_reboot` pattern, `coordinator.py:62-74`). The delivery manager subscribes; on device-seen with pending work and no delivery in flight, it starts a delivery task **immediately** — every millisecond of the 10 s window counts. +- **Single connection per wake (P3):** the delivery task acquires `ble_lock`, opens one `OpenDisplayDevice` session, and drains *all* pending slots in priority order: upload first (user-visible), then config resync (cheap reads on the already-open link), then OTA. The device stays awake while connected, so the window only constrains connection establishment, not the work. +- **Failure handling:** if a delivery attempt fails (device slept mid-connect, interference), the work stays queued for the next wake; `attempts` increments. A **deadline timer** (options: `queue_timeout`, default **24 h** — safely above the 18 h max sleep interval) expires the slot: fire `opendisplay_content_expired` event, log a warning, clear the pending flag. This is the user's "backup timer / failure code". +- **Transport-agnostic (P4):** the manager's entry point is `async_device_seen(source: str)`. The BLE coordinator calls it with `"ble"`; a future WiFi presence tracker calls it with `"mdns"`. Nothing else changes. + +#### D5 — Service-call semantics on a sleeping device + +`upload_image` / `drawcustom` flow becomes: + +1. Render + `prepare_image` immediately (unchanged — CPU work is front-loaded and reused on every retry). +2. **Freshness gate:** if sleep mode is active and the last advertisement is older than `sleep_timeout_ms + slack (~5 s)`, the device is provably asleep — **skip the doomed ~40 s bleak retry cycle** and queue directly. If the device was seen within the window (possibly still awake), attempt an immediate send as today. +3. On immediate-send success → done, exactly as today. +4. On `BLEConnectionError`/`BLETimeoutError` in sleep mode → queue into `pending_upload` instead of raising. Non-sleep devices keep today's strict failure. +5. The service returns response data (`SupportsResponse.OPTIONAL`): `{"status": "delivered" | "queued", "expires_at": …}` so automations can branch. It does **not** block until delivery (sleep intervals can be hours; blocking would time out the service call and jam automation queues). + +This is precisely the tuning the user asked for regarding bleak dynamics: *when we do connect, it is triggered by the wake advertisement itself*, so the first attempt starts ~0.1–2 s into the 10 s window and the default 10 s/attempt budget is ample; and *we never burn 40 s of retries against a device we know is dark*. No py-opendisplay retry-parameter changes are required (though the delivery task will bound each wake attempt with an overall deadline of ~30 s so one bad wake can't overlap the next). + +LED/buzzer services stay immediate-only (a notification that fires hours late is worse than an error); in sleep mode with the device dark they fail fast with a clear "device is sleeping" error. + +#### D6 — Representing queued content: the image entity + a pending sensor + +Per the user's instinct, the existing Display Content image entity is the queue's visible face: + +- On **queue**: update the image entity immediately with the rendered frame (it now shows *intended* content) and set entity attribute `pending: true` plus `queued_at`. A new **binary sensor "Update pending"** exposes the same state for automations/dashboards (attributes: `queued_at`, `expires_at`, `attempts`). +- On **delivery**: clear `pending`; image already matches. +- On **expiry**: fire `opendisplay_content_expired`, set the binary sensor off, and revert the image entity attribute to `pending: false` with `last_error: expired` (the panel still shows the old frame; the image entity keeps the intended frame as the record of what was attempted — with the attribute making the mismatch explicit). + +The queue itself is **memory-only in v1**: an HA restart drops a pending upload (documented limitation; the binary sensor goes off honestly). Persisting prepared frames (hundreds of KB) to a `Store` is a v2 option if real usage demands it — the delivery manager's API is designed so persistence can be added without touching callers. + +#### D7 — Reboot-edge handling in sleep mode + +Today's behavior — full entry reload on the advertised reboot edge — is wrong for sleepy devices twice over: the firmware's unpersisted `rebootFlag` makes every wake look like a reboot (churn loop, findings doc §Bug), and the reload's reconnect races the wake window. **Decision:** in sleep mode, the reboot edge sets `pending_config_resync` on the delivery manager instead of reloading. The resync rides the next delivery connection (or triggers one on the current wake, which is exactly when the edge is observed). Non-sleep devices keep the reload behavior. This makes the integration robust against the firmware bug while remaining correct once the firmware persists the flag. + +#### D8 — OTA on sleepy devices + +`update.py` connects directly today. In sleep mode: a user-initiated install registers `pending_ota` and reports progress state "waiting for device wake"; the next device-seen event starts the flash over the wake connection (device stays awake once connected — OTA duration is not window-constrained). Version *checks* remain opportunistic (piggyback on any delivery connection rather than connecting on a timer). + +#### D9 — py-opendisplay: add the deep-sleep command (0x0052) + +Not strictly required for ESP32 timer-driven sleep, but added for completeness and the Flex/Silabs story (where sleep is command-only): `OpenDisplayDevice.deep_sleep()` sending 0x0052, plus a `PowerConfig.deep_sleep_enabled` convenience property so the integration's D1 predicate lives next to the fields it reads. Also enables a future "sleep immediately after upload" optimization (save the remainder of a wake window after delivery). + +### 2.3 Timing analysis — why the rendezvous works + +With defaults (wake window W = 10 s; bleak timeout 10 s × 4 attempts): + +| Step | Budget | +|---|---| +| Device wakes, first advertisement out | ~0.1–1 s into window (fast adv interval in `minimalSetup`) | +| Scanner → coordinator callback (local adapter) | < 0.5 s; ESPHome BLE proxy adds ~0.5–1.5 s | +| Delivery task start → `establish_connection` first attempt | < 0.1 s (already-running event loop task, `BLEDevice` fresh from this very advertisement) | +| Connection establishment | typically 1–3 s; up to 10 s budget fits inside remaining ~8 s window | +| After connect | device exits window logic and stays awake (`main.cpp:85-90`) — unlimited work time | + +Failure modes are all recoverable: a missed window (proxy latency spike, connection slot exhaustion) simply waits `deep_sleep_time_seconds` for the next one; the deadline timer bounds total wait. Recommended user guidance (docs): keep `sleep_timeout_ms` ≥ 5000; typical `deep_sleep_time_seconds` 300–3600 gives content latency of at most one sleep interval. + +### 2.4 Scenario walkthroughs (target behavior) + +1. **HA restarts at 03:00; device sleeps 30 min.** Entry loads instantly from cache; all entities restored and available; `config_resync_pending` set. At the next wake (≤ 03:30) the delivery manager connects once, re-reads config, updates cache. No `ConfigEntryNotReady`, no flapping. +2. **Automation pushes a new image at 09:00; device sleeping until 09:25.** Image is rendered/prepared at 09:00; freshness gate sees stale adverts → queued instantly (no 40 s stall); service returns `queued`; image entity shows new frame with `pending: true`; binary sensor on. At 09:25 the wake advert triggers delivery; panel refreshes; `pending` clears; automations see `opendisplay_content_delivered`. +3. **Device battery dies / removed.** No wakes observed; after `missed_cycles × interval` entities go unavailable (honest signal); a queued upload expires after `queue_timeout` with `opendisplay_content_expired`. +4. **Five uploads queued while asleep.** Latest-wins: only the newest survives (existing semantics extended to the queue); each superseded call had returned `queued` and the final delivery event carries the last `queued_at`. +5. **Reboot flag seen on wake (firmware bug or real reboot).** No reload storm; config resync piggybacks the next delivery connection. +6. **New device adoption.** Unchanged — happens in the 2-minute first-boot window; first setup connects live and seeds the cache. + +--- + +## 3. Component factoring + +Proposed decomposition, smallest-surface-first. New modules in **bold**. + +``` +custom_components/opendisplay/ +├── const.py + option keys, defaults, event names, new dispatcher signals +├── config_flow.py + OpenDisplayOptionsFlow (sleep_mode, missed_cycles, queue_timeout) +├── __init__.py + config cache read/write; setup-from-cache branch; +│ availability-interval registration; DeliveryManager wiring; +│ runtime_data: delivery manager + sleep profile +├── coordinator.py + async_subscribe_device_seen (mirror of async_subscribe_reboot) +├── **sleep.py** SleepProfile: resolves options + device config → is_sleepy, +│ availability_interval, freshness gate ("probably_asleep(now)") +├── **delivery.py** DeliveryManager: pending slots (upload / config_resync / ota), +│ device-seen handler, single-connection drain, deadline timers, +│ delivered/expired events, state for sensors & diagnostics +├── services.py + freshness gate + queue-on-failure via DeliveryManager; +│ SupportsResponse.OPTIONAL with delivered/queued payload +├── image.py + pending/queued_at attributes; shows queued frame at queue time +├── **binary_sensor.py** "Update pending" entity backed by DeliveryManager state +├── update.py + sleep-mode gating: pending_ota slot, "waiting for wake" state +├── diagnostics.py + sleep profile + pending-slot state (redacted image bytes) +└── strings.json, translations/, icons.json — options flow + new entities + new errors + +py-opendisplay/ +├── protocol/commands.py + DEEP_SLEEP = 0x0052 (+ encoder) +├── device.py + async deep_sleep() +└── models/config.py + PowerConfig.deep_sleep_enabled property + +Firmware/ (recommended, decoupled — §5.4) +├── rebootFlag RTC persistence across deep sleep +└── post-reboot advertising grace window +``` + +Dependency direction: `services.py`, `update.py`, `image.py`, `binary_sensor.py` → `delivery.py` → `sleep.py` + `coordinator.py`. `delivery.py` owns all queue state; entities and diagnostics only read it; services only submit to it. The coordinator knows nothing about queues — it just announces "device seen". + +--- + +## 4. Implementation plan + +### Phase 0 — py-opendisplay groundwork *(small; independent release)* + +1. `PowerConfig.deep_sleep_enabled` property (`models/config.py`): `power_mode == PowerMode.BATTERY and deep_sleep_time_seconds > 0`. Unit tests over parse/serialize round-trips. +2. `Command.DEEP_SLEEP = 0x0052` + `OpenDisplayDevice.deep_sleep()` (`device.py`, modeled on `reboot()` at `device.py:912`; fire-and-forget semantics — the ESP32 drops the link on entry, so tolerate a disconnect instead of awaiting an ACK; verify exact response behavior against `device_control.cpp:692` during implementation). +3. CLI: `opendisplay sleep ` subcommand for testing. +4. Release (7.12.0) and bump `manifest.json` requirement in the integration. + +*Acceptance: CLI can put an ESP32 dev board to sleep; config round-trip tests pass.* + +### Phase 1 — Sleep awareness, availability, setup-from-cache *(the "entry survives" milestone)* + +1. **`const.py`**: `CONF_SLEEP_MODE` (`auto|on|off`), `CONF_MISSED_CYCLES` (default 3), `CONF_QUEUE_TIMEOUT_HOURS` (default 24), event name constants, `CONF_CACHED_STATE` key. +2. **`sleep.py`**: `SleepProfile.from_entry(entry, config)` — resolves option override + `deep_sleep_enabled`; computes `availability_interval = interval × missed_cycles + wake_window_s + 60`; exposes `probably_asleep(last_seen)` using `sleep_timeout_ms + 5 s` slack. Pure functions, fully unit-testable. +3. **`__init__.py` — cache write**: after every successful interrogation, store `{"config": config_to_json(cfg), "firmware": fw, "is_flex": …, "cached_at": …}` under `entry.data[CONF_CACHED_STATE]` via `async_update_entry`. +4. **`__init__.py` — setup-from-cache branch**: if no connectable device *or* connect fails, and cached state exists with `deep_sleep_enabled` (or option forced on): rebuild `GlobalConfig` via `config_from_json`, construct runtime data without connecting, set `pending_config_resync` (consumed in Phase 2; in Phase 1 it may simply resync on next reboot-edge/reload), and proceed. Otherwise preserve today's `ConfigEntryNotReady`/`ConfigEntryAuthFailed` behavior exactly. +5. **`__init__.py` — availability interval**: when the profile is sleepy, call `async_set_fallback_availability_interval(hass, address, profile.availability_interval)` before starting the coordinator; re-apply on reload. +6. **`config_flow.py`**: add `OpenDisplayOptionsFlow` with the three options; options changes trigger entry reload (standard listener). +7. **strings/translations** for the options form. + +*Acceptance: with a device configured for 10 min sleep — restart HA mid-sleep: entry loads, entities available and populated at next advert; no unavailable flap across three sleep cycles; options visible and effective after reload. Regression: non-sleepy device setup behavior unchanged (existing tests).* + +### Phase 2 — Delivery manager and queued uploads *(the core feature)* + +1. **`coordinator.py`**: `async_subscribe_device_seen(cb)` invoked from `_async_handle_bluetooth_event` after parsing (same pattern as `async_subscribe_reboot`). +2. **`delivery.py`**: `DeliveryManager(hass, entry, profile)`: + - Slots as in D4; `submit_upload(prepared, params, preview_jpeg) -> PendingReceipt`; `request_config_resync()`; internal `_deliver()` task guarded by an asyncio flag + `ble_lock`, bounded by a ~30 s per-wake deadline; drain order upload → resync → ota. + - Deadline timers via `async_call_later`; on expiry fire `opendisplay_content_expired` (payload: device_id, queued_at, attempts) and notify state listeners (dispatcher signal for entities). + - On delivery: send existing `SIGNAL_IMAGE_UPDATED` (unchanged image pipeline), fire `opendisplay_content_delivered`, persist refreshed config to cache when a resync ran. + - Teardown on unload: cancel timers + in-flight task (extend `async_unload_entry` alongside the existing upload-task cancel, `__init__.py:221-227`). +3. **`services.py`**: implement D5 — freshness gate; queue-on-connect-failure for `upload_image`/`drawcustom`; `SupportsResponse.OPTIONAL` returning `{"status", "expires_at"}`; LED/buzzer get the fast "device_sleeping" error when provably asleep. The existing latest-wins cancel logic remains for concurrent *live* uploads; the manager applies the same rule to the slot. +4. **`image.py`**: `pending`/`queued_at` attributes; listen to the manager's state signal; show queued frame at queue time (D6). +5. **`binary_sensor.py`**: "Update pending" backed by manager state; add platform to `_BASE_PLATFORMS`/`_FLEX_PLATFORMS`. +6. **`__init__.py`**: instantiate the manager in runtime data; setup-from-cache path now registers `pending_config_resync` with it, making the Phase 1 flag fully functional. +7. **strings/translations/icons** for the sensor, events, and the new error key. + +*Acceptance: end-to-end on hardware — call `drawcustom` mid-sleep: returns `queued` in < 2 s, sensor on, image entity shows frame with `pending: true`; panel refreshes on next wake ≤ interval; sensor off; `opendisplay_content_delivered` observed. Kill the device (pull battery) with queued content: expiry event at deadline. Five rapid queued calls → one delivery of the last frame.* + +### Phase 3 — Hardening and edges + +1. **Reboot-edge rework (D7)**: in `__init__.py`, when profile is sleepy route the reboot subscription to `manager.request_config_resync()` instead of `_async_reload_after_reboot`. +2. **OTA gating (D8)** in `update.py`: `pending_ota` slot, `in_progress`/extra state "waiting for device wake", resume-on-wake. +3. **`diagnostics.py`**: sleep profile, availability interval, slot states (timestamps/attempts only — no image payloads). +4. **Encrypted devices**: queued delivery reuses the stored key; on `AuthenticationFailedError` during delivery → keep slot paused, trigger reauth flow (existing pattern in `_async_connect_and_run`), resume after successful reauth. +5. **Test suite**: unit tests for `sleep.py` and `delivery.py` (mock coordinator/device); integration-style tests with `inject_bluetooth_service_info` simulating wake cadences: setup-from-cache, flap-free availability, queue→deliver, queue→expire, reboot-edge resync, unload-with-pending. +6. **Docs**: user-facing page (docs/) covering options, latency expectations table (interval vs. worst-case content delay), and the memory-only queue limitation. + +### Phase 4 — Firmware coordination *(separate repo; recommended, not blocking)* + +1. Persist `rebootFlag` across deep sleep via `RTC_DATA_ATTR` mirror (findings doc, ~10 lines) — kills the churn loop at the source; D7 already defuses it HA-side. +2. Post-reboot advertising grace window (30–60 s after *any* boot where `woke_from_deep_sleep` is false) — makes commanded reboots recoverable. +3. Optional: `esp_sleep_enable_ext1_wakeup` on the button pin — user-initiated wake ("press button to sync now"), which composes perfectly with the delivery manager (button wake → advert → immediate delivery). + +### Sequencing and risk + +- Phases 0→1→2 are strictly ordered; 3 and 4 can proceed in parallel after 2. +- Riskiest assumption: wake-window connect reliability through ESPHome proxies (extra advert latency + connection-slot contention). Mitigation is inherent — a missed window costs one sleep interval, and Phase 2 acceptance is run against both a local adapter and a proxy. +- Backward compatibility: all changes are additive; non-sleepy devices follow existing code paths verbatim; new options default to today's behavior for them (`auto` resolves to off). +- HA quality-scale note: setup-from-cache plus passive-only polling keeps the integration aligned with bluetooth-integration guidance (no connections outside user intent or discovery). + +--- + +## Appendix A — Options summary + +| Option | Default | Meaning | +|---|---|---| +| `sleep_mode` | `auto` | `auto`: follow device power config; `on`/`off`: force | +| `missed_cycles` | 3 | Wake cycles missed before entities go unavailable | +| `queue_timeout` | 24 h | Pending content expires with `opendisplay_content_expired` | + +## Appendix B — New events / signals / entities + +| Surface | Name | Payload | +|---|---|---| +| Bus event | `opendisplay_content_delivered` | device_id, queued_at, attempts | +| Bus event | `opendisplay_content_expired` | device_id, queued_at, attempts | +| Service response | `upload_image` / `drawcustom` | `{status: delivered\|queued, expires_at}` | +| Entity | `binary_sensor._update_pending` | attrs: queued_at, expires_at, attempts | +| Entity attrs | image `display_content` | `pending`, `queued_at` | + +## Appendix C — Open questions (decide during implementation) + +1. Should a delivery connection also refresh the `last_seen`-adjacent sensors by reading live values (battery under load), or stay advert-only? (Lean: advert-only; keep connections short.) +2. `deep_sleep()` post-upload to return the device to sleep immediately after delivery (saves the rest of the wake window) — worth it once 0x0052 ships? (Lean: yes, guarded by an option, after measuring real window costs.) +3. v2 queue persistence across HA restarts via `helpers.storage.Store` — wait for user demand. From 48e4d8b581382f988f79d17e9dc2f60d849f48e3 Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Mon, 6 Jul 2026 20:16:47 -0400 Subject: [PATCH 02/10] feat: phase 1 - sleep awareness, availability, setup-from-cache Introduce the deep-sleep model that lets a battery-powered display's entry survive while the device is dark: - const.py: sleep_mode/missed_cycles/queue_timeout options, cached-state key, content delivered/expired event names, and the pending-state dispatcher signal. - sleep.py: SleepProfile resolves the options override plus the device power config into is_sleepy, availability_interval and probably_asleep(); pure and unit-tested. deep_sleep_enabled is computed locally (power_mode == BATTERY and deep_sleep_time_seconds > 0) with no dependency on unreleased library API. - __init__.py: cache every successful interrogation into entry.data; on a dark, sleepy device set up entirely from cache without connecting; apply async_set_fallback_availability_interval so entities stay available across sleep cycles. All non-cache paths keep the original ConfigEntryNotReady/ConfigEntryAuthFailed behavior. - config_flow.py: OptionsFlowWithReload options flow (sleep_mode, missed_cycles, queue_timeout) that reloads the entry on change. - strings/translations/icons: options form, pending sensor, and new error keys (grouped here for all phases to avoid churn). - tests: pure unit tests for SleepProfile. Co-Authored-By: Claude Fable 5 --- custom_components/opendisplay/__init__.py | 202 ++++++++++++++---- custom_components/opendisplay/config_flow.py | 100 ++++++++- custom_components/opendisplay/const.py | 33 +++ custom_components/opendisplay/icons.json | 8 + custom_components/opendisplay/sleep.py | 153 +++++++++++++ custom_components/opendisplay/strings.json | 34 +++ .../opendisplay/translations/en.json | 34 +++ tests/conftest.py | 6 + tests/test_sleep.py | 100 +++++++++ 9 files changed, 628 insertions(+), 42 deletions(-) create mode 100644 custom_components/opendisplay/sleep.py create mode 100644 tests/conftest.py create mode 100644 tests/test_sleep.py diff --git a/custom_components/opendisplay/__init__.py b/custom_components/opendisplay/__init__.py index e024a8f..3d8e8f1 100644 --- a/custom_components/opendisplay/__init__.py +++ b/custom_components/opendisplay/__init__.py @@ -3,7 +3,8 @@ import asyncio import contextlib from dataclasses import dataclass, field -from typing import TYPE_CHECKING +import time +from typing import TYPE_CHECKING, Any from opendisplay import ( AuthenticationFailedError, @@ -15,11 +16,13 @@ OpenDisplayError, PartialState, ) +from opendisplay.models.config_json import config_from_json, config_to_json from homeassistant.components.bluetooth import ( BluetoothReachabilityIntent, async_address_reachability_diagnostics, async_ble_device_from_address, + async_set_fallback_availability_interval, ) from homeassistant.config_entries import ConfigEntry from homeassistant.const import Platform @@ -29,12 +32,13 @@ from homeassistant.helpers.device_registry import CONNECTION_BLUETOOTH from homeassistant.helpers.typing import ConfigType -if TYPE_CHECKING: - from opendisplay.models import FirmwareVersion - -from .const import CONF_ENCRYPTION_KEY, DOMAIN +from .const import CONF_CACHED_STATE, CONF_ENCRYPTION_KEY, DOMAIN from .coordinator import OpenDisplayCoordinator from .services import async_setup_services +from .sleep import SleepProfile + +if TYPE_CHECKING: + from opendisplay.models import FirmwareVersion CONFIG_SCHEMA = cv.config_entry_only_config_schema(DOMAIN) @@ -50,6 +54,8 @@ class OpenDisplayRuntimeData: firmware: FirmwareVersion device_config: GlobalConfig is_flex: bool + # Resolved deep-sleep behavior (options + device power config). + sleep_profile: SleepProfile upload_task: asyncio.Task | None = None # Serializes every BLE connection to this tag (one config entry == one MAC). # The device exposes a single BLE link with no per-address lock in the @@ -60,11 +66,86 @@ class OpenDisplayRuntimeData: # (0x76). Replaced with a fresh instance on every full/fast refresh so the # next partial diffs against the frame actually on the panel. partial_state: PartialState = field(default_factory=PartialState) + # Set when the entry was set up from cache without connecting; consumed in + # Phase 2 by the delivery manager, which re-reads firmware/config at the + # next wake. In Phase 1 the resync rides the reboot-edge reload. + config_resync_pending: bool = False type OpenDisplayConfigEntry = ConfigEntry[OpenDisplayRuntimeData] +@dataclass +class _CachedState: + """Device state restored from ``entry.data`` for setup-without-connect.""" + + firmware: FirmwareVersion + is_flex: bool + device_config: GlobalConfig + landing_url: str | None + + +def _load_cache(entry: OpenDisplayConfigEntry) -> _CachedState | None: + """Rebuild cached device state from the config entry, or None if absent.""" + raw = entry.data.get(CONF_CACHED_STATE) + if not raw: + return None + try: + device_config = config_from_json(raw["config"]) + return _CachedState( + firmware=raw["firmware"], + is_flex=raw["is_flex"], + device_config=device_config, + landing_url=raw.get("landing_url"), + ) + except (KeyError, ValueError, TypeError): + return None + + +def _write_cache( + hass: HomeAssistant, + entry: OpenDisplayConfigEntry, + device_config: GlobalConfig, + firmware: FirmwareVersion, + is_flex: bool, + landing_url: str | None, +) -> None: + """Persist the device state needed to set up the entry without connecting. + + Called after every successful interrogation (setup and, later, wake-time + resync). Writes only when the meaningful contents changed so we don't churn + the config-entry store on every reload. + """ + payload: dict[str, Any] = { + "config": config_to_json(device_config), + "firmware": dict(firmware), + "is_flex": is_flex, + "landing_url": landing_url, + } + existing = entry.data.get(CONF_CACHED_STATE) + if existing is not None and all( + existing.get(key) == value for key, value in payload.items() + ): + return + hass.config_entries.async_update_entry( + entry, + data={**entry.data, CONF_CACHED_STATE: {**payload, "cached_at": time.time()}}, + ) + + +def _cache_setup_if_sleepy( + entry: OpenDisplayConfigEntry, +) -> _CachedState | None: + """Return cached state iff it exists and resolves to a sleepy device.""" + cached = _load_cache(entry) + if cached is None: + return None + profile = SleepProfile.from_entry(entry, cached.device_config) + if not profile.is_sleepy: + return None + return cached + + def _get_encryption_key(entry: OpenDisplayConfigEntry) -> bytes | None: """Return the encryption key bytes from entry data, or None.""" raw = entry.data.get(CONF_ENCRYPTION_KEY) @@ -89,47 +170,73 @@ async def async_setup(hass: HomeAssistant, config: ConfigType) -> bool: async def async_setup_entry(hass: HomeAssistant, entry: OpenDisplayConfigEntry) -> bool: - """Set up OpenDisplay from a config entry.""" + """Set up OpenDisplay from a config entry. + + A reachable device is interrogated live (firmware + config) and the result + cached. When the device is dark and the cache says it is a deep-sleeping + tag, the entry is set up entirely from cache without connecting; the + delivery manager re-reads it at the next wake. Any other unreachable case + preserves the original ConfigEntryNotReady/ConfigEntryAuthFailed behavior. + """ address = entry.unique_id if TYPE_CHECKING: assert address is not None ble_device = async_ble_device_from_address(hass, address, connectable=True) - if ble_device is None: - raise ConfigEntryNotReady( - translation_domain=DOMAIN, - translation_key="device_not_found", - translation_placeholders={ - "address": address, - "reason": async_address_reachability_diagnostics( - hass, - address.upper(), - BluetoothReachabilityIntent.CONNECTION, - ), - }, - ) - encryption_key = _get_encryption_key(entry) - - try: - async with OpenDisplayDevice( - mac_address=address, ble_device=ble_device, encryption_key=encryption_key - ) as device: - fw = await device.read_firmware_version() - is_flex = device.is_flex - # Capture while connected: landing_url() reads the advertised name. - landing_url = device.landing_url() - except (AuthenticationFailedError, AuthenticationRequiredError) as err: - raise ConfigEntryAuthFailed( - f"Encryption key rejected by OpenDisplay device: {err}" - ) from err - except (BLEConnectionError, BLETimeoutError, OpenDisplayError) as err: - raise ConfigEntryNotReady( - f"Failed to connect to OpenDisplay device: {err}" - ) from err - device_config = device.config - if TYPE_CHECKING: - assert device_config is not None + from_cache = False + if ble_device is None: + cached = _cache_setup_if_sleepy(entry) + if cached is None: + raise ConfigEntryNotReady( + translation_domain=DOMAIN, + translation_key="device_not_found", + translation_placeholders={ + "address": address, + "reason": async_address_reachability_diagnostics( + hass, + address.upper(), + BluetoothReachabilityIntent.CONNECTION, + ), + }, + ) + fw = cached.firmware + is_flex = cached.is_flex + device_config = cached.device_config + landing_url = cached.landing_url + from_cache = True + else: + encryption_key = _get_encryption_key(entry) + try: + async with OpenDisplayDevice( + mac_address=address, + ble_device=ble_device, + encryption_key=encryption_key, + ) as device: + fw = await device.read_firmware_version() + is_flex = device.is_flex + # Capture while connected: landing_url() reads the advertised name. + landing_url = device.landing_url() + device_config = device.config + if TYPE_CHECKING: + assert device_config is not None + except (AuthenticationFailedError, AuthenticationRequiredError) as err: + raise ConfigEntryAuthFailed( + f"Encryption key rejected by OpenDisplay device: {err}" + ) from err + except (BLEConnectionError, BLETimeoutError, OpenDisplayError) as err: + cached = _cache_setup_if_sleepy(entry) + if cached is None: + raise ConfigEntryNotReady( + f"Failed to connect to OpenDisplay device: {err}" + ) from err + fw = cached.firmware + is_flex = cached.is_flex + device_config = cached.device_config + landing_url = cached.landing_url + from_cache = True + + profile = SleepProfile.from_entry(entry, device_config) coordinator = OpenDisplayCoordinator(hass, address) manufacturer = device_config.manufacturer @@ -162,8 +269,23 @@ async def async_setup_entry(hass: HomeAssistant, entry: OpenDisplayConfigEntry) firmware=fw, device_config=device_config, is_flex=is_flex, + sleep_profile=profile, + config_resync_pending=from_cache, ) + # Persist a fresh interrogation so the next dark startup can set up from + # cache; skip when we just loaded from cache (nothing new to store). + if not from_cache: + _write_cache(hass, entry, device_config, fw, is_flex, landing_url) + + # Keep entities available across sleep cycles: without this, any sleep + # interval longer than the ~5 min staleness horizon flaps everything + # unavailable once per cycle. + if profile.is_sleepy: + async_set_fallback_availability_interval( + hass, address, profile.availability_interval + ) + await hass.config_entries.async_forward_entry_setups( entry, _get_platforms(entry.runtime_data) ) diff --git a/custom_components/opendisplay/config_flow.py b/custom_components/opendisplay/config_flow.py index 622723e..696b89d 100644 --- a/custom_components/opendisplay/config_flow.py +++ b/custom_components/opendisplay/config_flow.py @@ -19,14 +19,80 @@ async_ble_device_from_address, async_discovered_service_info, ) -from homeassistant.config_entries import ConfigFlow, ConfigFlowResult +from homeassistant.config_entries import ( + ConfigEntry, + ConfigFlow, + ConfigFlowResult, + OptionsFlow, + OptionsFlowWithReload, +) from homeassistant.const import CONF_ADDRESS +from homeassistant.core import callback +from homeassistant.helpers.selector import ( + NumberSelector, + NumberSelectorConfig, + NumberSelectorMode, + SelectSelector, + SelectSelectorConfig, + SelectSelectorMode, +) -from .const import CONF_ENCRYPTION_KEY, DOMAIN +from .const import ( + CONF_ENCRYPTION_KEY, + CONF_MISSED_CYCLES, + CONF_QUEUE_TIMEOUT_HOURS, + CONF_SLEEP_MODE, + DEFAULT_MISSED_CYCLES, + DEFAULT_QUEUE_TIMEOUT_HOURS, + DEFAULT_SLEEP_MODE, + DOMAIN, + SLEEP_MODE_AUTO, + SLEEP_MODE_OFF, + SLEEP_MODE_ON, +) _LOGGER = logging.getLogger(__name__) +def _options_schema() -> vol.Schema: + """Return the options-flow schema.""" + return vol.Schema( + { + vol.Required( + CONF_SLEEP_MODE, default=DEFAULT_SLEEP_MODE + ): SelectSelector( + SelectSelectorConfig( + options=[SLEEP_MODE_AUTO, SLEEP_MODE_ON, SLEEP_MODE_OFF], + translation_key="sleep_mode", + mode=SelectSelectorMode.DROPDOWN, + ) + ), + vol.Required(CONF_MISSED_CYCLES, default=DEFAULT_MISSED_CYCLES): vol.All( + NumberSelector( + NumberSelectorConfig( + min=1, max=100, step=1, mode=NumberSelectorMode.BOX + ) + ), + vol.Coerce(int), + ), + vol.Required( + CONF_QUEUE_TIMEOUT_HOURS, default=DEFAULT_QUEUE_TIMEOUT_HOURS + ): vol.All( + NumberSelector( + NumberSelectorConfig( + min=1, + max=168, + step=1, + mode=NumberSelectorMode.BOX, + unit_of_measurement="h", + ) + ), + vol.Coerce(int), + ), + } + ) + + _ENCRYPTION_KEY_VALIDATOR = vol.All(str.strip, str.lower, vol.Match(r"^[0-9a-f]{32}$")) @@ -38,6 +104,12 @@ def __init__(self) -> None: self._discovery_info: BluetoothServiceInfoBleak | None = None self._discovered_devices: dict[str, BluetoothServiceInfoBleak] = {} + @staticmethod + @callback + def async_get_options_flow(config_entry: ConfigEntry) -> OptionsFlow: + """Return the options flow handler.""" + return OpenDisplayOptionsFlow() + async def _async_test_connection( self, address: str, encryption_key: bytes | None = None ) -> None: @@ -241,3 +313,27 @@ async def async_step_reauth_confirm( description_placeholders={"name": reauth_entry.title}, errors=errors, ) + + +class OpenDisplayOptionsFlow(OptionsFlowWithReload): + """Handle deep-sleep options for an OpenDisplay device. + + Extends ``OptionsFlowWithReload`` so saving changed options automatically + reloads the entry, re-resolving the sleep profile and re-applying the + availability interval. No manual update listener is registered (mixing the + two is disallowed). + """ + + async def async_step_init( + self, user_input: dict[str, Any] | None = None + ) -> ConfigFlowResult: + """Manage the deep-sleep options.""" + if user_input is not None: + return self.async_create_entry(data=user_input) + + return self.async_show_form( + step_id="init", + data_schema=self.add_suggested_values_to_schema( + _options_schema(), self.config_entry.options + ), + ) diff --git a/custom_components/opendisplay/const.py b/custom_components/opendisplay/const.py index 779b74c..92cb76d 100644 --- a/custom_components/opendisplay/const.py +++ b/custom_components/opendisplay/const.py @@ -2,4 +2,37 @@ DOMAIN = "opendisplay" CONF_ENCRYPTION_KEY = "encryption_key" + +# Dispatcher signals (suffixed with the device address at send/subscribe time). +# Carries the JPEG preview of the frame shown by the image entity. SIGNAL_IMAGE_UPDATED = f"{DOMAIN}_image_updated" +# Carries a DeliverySnapshot describing the delivery manager's pending state +# (see delivery.py). Consumed by the image entity and the "update pending" +# binary sensor. +SIGNAL_PENDING_STATE = f"{DOMAIN}_pending_state" + +# --- Options (options flow) ------------------------------------------------- +# Deep-sleep handling mode: follow the device power config, or force on/off. +CONF_SLEEP_MODE = "sleep_mode" +SLEEP_MODE_AUTO = "auto" +SLEEP_MODE_ON = "on" +SLEEP_MODE_OFF = "off" +DEFAULT_SLEEP_MODE = SLEEP_MODE_AUTO + +# Consecutive expected wakes that may be missed before entities go unavailable. +CONF_MISSED_CYCLES = "missed_cycles" +DEFAULT_MISSED_CYCLES = 3 + +# Hours a queued upload survives before expiring (safely above the 18 h max +# firmware sleep interval). +CONF_QUEUE_TIMEOUT_HOURS = "queue_timeout_hours" +DEFAULT_QUEUE_TIMEOUT_HOURS = 24 + +# --- Cache ------------------------------------------------------------------ +# entry.data key holding the serialized device state used to set up the entry +# without connecting when a sleepy device is dark at startup. +CONF_CACHED_STATE = "cached_state" + +# --- Bus events ------------------------------------------------------------- +EVENT_CONTENT_DELIVERED = f"{DOMAIN}_content_delivered" +EVENT_CONTENT_EXPIRED = f"{DOMAIN}_content_expired" diff --git a/custom_components/opendisplay/icons.json b/custom_components/opendisplay/icons.json index 6150e04..da60480 100644 --- a/custom_components/opendisplay/icons.json +++ b/custom_components/opendisplay/icons.json @@ -1,5 +1,13 @@ { "entity": { + "binary_sensor": { + "update_pending": { + "default": "mdi:image-sync", + "state": { + "off": "mdi:image-check" + } + } + }, "update": { "firmware": { "default": "mdi:chip" diff --git a/custom_components/opendisplay/sleep.py b/custom_components/opendisplay/sleep.py new file mode 100644 index 0000000..e4e1276 --- /dev/null +++ b/custom_components/opendisplay/sleep.py @@ -0,0 +1,153 @@ +"""Deep-sleep awareness for OpenDisplay devices. + +`SleepProfile` resolves the integration options and the device's own power +configuration into the three facts the rest of the integration needs: + +* ``is_sleepy`` — should this device be treated as a deep-sleeping tag (dark + most of the time, reachable only during a short post-wake advertising + window)? +* ``availability_interval`` — how long HA should keep entities available + between wakes before flagging the device as gone. +* ``probably_asleep(last_seen)`` — given the last advertisement time, is the + device almost certainly back asleep right now? + +The predicate for "sleepy" mirrors the firmware's own deep-sleep entry +condition (``power_mode == BATTERY and deep_sleep_time_seconds > 0``, see +``Firmware/src/main.cpp``) so the integration and the device can never disagree, +with an options override for edge cases. All computation here is pure and +side-effect free; the values are consumed by ``__init__``, ``services`` and the +delivery manager. +""" + +from __future__ import annotations + +from dataclasses import dataclass +import time +from typing import TYPE_CHECKING + +from opendisplay.models.enums import PowerMode + +from .const import ( + CONF_MISSED_CYCLES, + CONF_QUEUE_TIMEOUT_HOURS, + CONF_SLEEP_MODE, + DEFAULT_MISSED_CYCLES, + DEFAULT_QUEUE_TIMEOUT_HOURS, + DEFAULT_SLEEP_MODE, + SLEEP_MODE_OFF, + SLEEP_MODE_ON, +) + +if TYPE_CHECKING: + from opendisplay import GlobalConfig + + from . import OpenDisplayConfigEntry + +# Firmware default post-wake advertising window when sleep_timeout_ms is 0 +# (Firmware/src/main.cpp:94-96). Expressed in milliseconds to match the config. +DEFAULT_WAKE_WINDOW_MS = 10_000 +# Extra slack added on top of the wake window when deciding the device is back +# asleep, to absorb scanner/proxy advertisement latency. +FRESHNESS_SLACK_S = 5.0 +# Fixed slack added to the availability horizon so a marginally late wake does +# not immediately flap entities unavailable. +AVAILABILITY_SLACK_S = 60.0 + + +@dataclass(frozen=True) +class SleepProfile: + """Resolved deep-sleep behavior for one config entry.""" + + is_sleepy: bool + deep_sleep_enabled: bool + deep_sleep_time_seconds: int + sleep_timeout_ms: int + missed_cycles: int + queue_timeout_hours: int + + @property + def wake_window_s(self) -> float: + """Post-wake advertising window in seconds (firmware default 10 s).""" + return (self.sleep_timeout_ms or DEFAULT_WAKE_WINDOW_MS) / 1000.0 + + @property + def availability_interval(self) -> float: + """Seconds of silence tolerated before entities go unavailable. + + ``deep_sleep_time_seconds * missed_cycles`` (the device may skip a few + expected wakes) plus one wake window and a fixed slack. + """ + return ( + self.deep_sleep_time_seconds * self.missed_cycles + + self.wake_window_s + + AVAILABILITY_SLACK_S + ) + + @property + def queue_timeout_s(self) -> float: + """Seconds a queued upload survives before expiring.""" + return self.queue_timeout_hours * 3600 + + def probably_asleep(self, last_seen: float | None, now: float | None = None) -> bool: + """Return True if the device is almost certainly back asleep. + + The device advertises only during its wake window. If the most recent + advertisement is older than one wake window plus slack, it has returned + to deep sleep and connecting would just burn a doomed retry cycle. A + device never seen (``last_seen is None``) is treated as asleep. + + This is a pure freshness test independent of ``is_sleepy``; callers + combine it with ``is_sleepy`` where the distinction matters. + """ + if last_seen is None: + return True + current = time.time() if now is None else now + return (current - last_seen) > (self.wake_window_s + FRESHNESS_SLACK_S) + + @classmethod + def create( + cls, + *, + sleep_mode: str, + power_mode: int, + sleep_timeout_ms: int, + deep_sleep_time_seconds: int, + missed_cycles: int, + queue_timeout_hours: int, + ) -> SleepProfile: + """Build a profile from primitive values (pure; unit-testable).""" + deep_sleep_enabled = ( + power_mode == PowerMode.BATTERY and deep_sleep_time_seconds > 0 + ) + if sleep_mode == SLEEP_MODE_ON: + is_sleepy = True + elif sleep_mode == SLEEP_MODE_OFF: + is_sleepy = False + else: # SLEEP_MODE_AUTO + is_sleepy = deep_sleep_enabled + return cls( + is_sleepy=is_sleepy, + deep_sleep_enabled=deep_sleep_enabled, + deep_sleep_time_seconds=deep_sleep_time_seconds, + sleep_timeout_ms=sleep_timeout_ms, + missed_cycles=missed_cycles, + queue_timeout_hours=queue_timeout_hours, + ) + + @classmethod + def from_entry( + cls, entry: OpenDisplayConfigEntry, config: GlobalConfig + ) -> SleepProfile: + """Build a profile from a config entry's options and device config.""" + options = entry.options + power = config.power + return cls.create( + sleep_mode=options.get(CONF_SLEEP_MODE, DEFAULT_SLEEP_MODE), + power_mode=power.power_mode, + sleep_timeout_ms=power.sleep_timeout_ms, + deep_sleep_time_seconds=power.deep_sleep_time_seconds, + missed_cycles=options.get(CONF_MISSED_CYCLES, DEFAULT_MISSED_CYCLES), + queue_timeout_hours=options.get( + CONF_QUEUE_TIMEOUT_HOURS, DEFAULT_QUEUE_TIMEOUT_HOURS + ), + ) diff --git a/custom_components/opendisplay/strings.json b/custom_components/opendisplay/strings.json index 582c954..3602612 100644 --- a/custom_components/opendisplay/strings.json +++ b/custom_components/opendisplay/strings.json @@ -50,7 +50,28 @@ } } }, + "options": { + "step": { + "init": { + "data": { + "sleep_mode": "Deep sleep handling", + "missed_cycles": "Missed wake cycles before unavailable", + "queue_timeout_hours": "Queued content timeout (hours)" + }, + "data_description": { + "sleep_mode": "Automatic follows the device's own power configuration (battery power with deep sleep enabled). Force on or off to override.", + "missed_cycles": "How many expected wake-ups the device may miss before its entities are marked unavailable.", + "queue_timeout_hours": "How long content queued for a sleeping device waits for a wake before it expires." + } + } + } + }, "entity": { + "binary_sensor": { + "update_pending": { + "name": "Update pending" + } + }, "image": { "content": { "name": "Display content" @@ -108,6 +129,12 @@ "device_not_found": { "message": "Could not find Bluetooth device with address `{address}`. Reason: {reason}" }, + "device_sleeping": { + "message": "Device `{device_id}` is asleep and cannot be reached immediately. Wake it and try again." + }, + "device_sleeping_ota": { + "message": "The device is asleep and cannot be updated right now. Wake it (press its button or wait for its next scheduled check-in) and start the update again." + }, "invalid_device_id": { "message": "Device `{device_id}` is not a valid OpenDisplay device." }, @@ -161,6 +188,13 @@ "full": "Full", "partial": "Partial" } + }, + "sleep_mode": { + "options": { + "auto": "Automatic (follow device)", + "off": "Force off", + "on": "Force on" + } } }, "services": { diff --git a/custom_components/opendisplay/translations/en.json b/custom_components/opendisplay/translations/en.json index b14c078..57f4615 100644 --- a/custom_components/opendisplay/translations/en.json +++ b/custom_components/opendisplay/translations/en.json @@ -50,7 +50,28 @@ } } }, + "options": { + "step": { + "init": { + "data": { + "sleep_mode": "Deep sleep handling", + "missed_cycles": "Missed wake cycles before unavailable", + "queue_timeout_hours": "Queued content timeout (hours)" + }, + "data_description": { + "sleep_mode": "Automatic follows the device's own power configuration (battery power with deep sleep enabled). Force on or off to override.", + "missed_cycles": "How many expected wake-ups the device may miss before its entities are marked unavailable.", + "queue_timeout_hours": "How long content queued for a sleeping device waits for a wake before it expires." + } + } + } + }, "entity": { + "binary_sensor": { + "update_pending": { + "name": "Update pending" + } + }, "image": { "content": { "name": "Display content" @@ -105,6 +126,12 @@ "device_not_found": { "message": "Could not find Bluetooth device with address `{address}`." }, + "device_sleeping": { + "message": "Device `{device_id}` is asleep and cannot be reached immediately. Wake it and try again." + }, + "device_sleeping_ota": { + "message": "The device is asleep and cannot be updated right now. Wake it (press its button or wait for its next scheduled check-in) and start the update again." + }, "invalid_device_id": { "message": "Device `{device_id}` is not a valid OpenDisplay device." }, @@ -158,6 +185,13 @@ "full": "Full", "partial": "Partial" } + }, + "sleep_mode": { + "options": { + "auto": "Automatic (follow device)", + "off": "Force off", + "on": "Force on" + } } }, "services": { diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..67f149e --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,6 @@ +"""Test bootstrap: make ``custom_components`` importable from the repo root.""" + +import pathlib +import sys + +sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent.parent)) diff --git a/tests/test_sleep.py b/tests/test_sleep.py new file mode 100644 index 0000000..0a344de --- /dev/null +++ b/tests/test_sleep.py @@ -0,0 +1,100 @@ +"""Unit tests for the pure SleepProfile logic.""" + +import pytest + +from custom_components.opendisplay.sleep import ( + AVAILABILITY_SLACK_S, + DEFAULT_WAKE_WINDOW_MS, + FRESHNESS_SLACK_S, + SleepProfile, +) + + +def _profile(**overrides): + """Build a SleepProfile from sensible defaults overridden per test.""" + params = { + "sleep_mode": "auto", + "power_mode": 1, # BATTERY + "sleep_timeout_ms": 0, + "deep_sleep_time_seconds": 300, + "missed_cycles": 3, + "queue_timeout_hours": 24, + } + params.update(overrides) + return SleepProfile.create(**params) + + +def test_deep_sleep_enabled_requires_battery_and_interval(): + assert _profile(power_mode=1, deep_sleep_time_seconds=300).deep_sleep_enabled + # USB power → not a deep-sleeper even with an interval set. + assert not _profile(power_mode=2, deep_sleep_time_seconds=300).deep_sleep_enabled + # Battery but no interval → not a deep-sleeper. + assert not _profile(power_mode=1, deep_sleep_time_seconds=0).deep_sleep_enabled + + +def test_sleep_mode_auto_follows_device(): + assert _profile(sleep_mode="auto", power_mode=1, deep_sleep_time_seconds=300).is_sleepy + assert not _profile( + sleep_mode="auto", power_mode=2, deep_sleep_time_seconds=300 + ).is_sleepy + + +def test_sleep_mode_force_on_and_off_override_device(): + # Force on even for a USB device with no deep sleep. + assert _profile(sleep_mode="on", power_mode=2, deep_sleep_time_seconds=0).is_sleepy + # Force off even for a battery device configured for deep sleep. + forced_off = _profile(sleep_mode="off", power_mode=1, deep_sleep_time_seconds=300) + assert not forced_off.is_sleepy + # deep_sleep_enabled still reflects the device, independent of the override. + assert forced_off.deep_sleep_enabled + + +def test_wake_window_uses_firmware_default_when_zero(): + assert _profile(sleep_timeout_ms=0).wake_window_s == DEFAULT_WAKE_WINDOW_MS / 1000.0 + assert _profile(sleep_timeout_ms=5000).wake_window_s == 5.0 + + +def test_availability_interval_formula(): + profile = _profile( + sleep_timeout_ms=0, deep_sleep_time_seconds=300, missed_cycles=3 + ) + # 300 * 3 + 10 (default window) + 60 slack = 970 + expected = 300 * 3 + DEFAULT_WAKE_WINDOW_MS / 1000.0 + AVAILABILITY_SLACK_S + assert profile.availability_interval == expected == 970.0 + + +def test_queue_timeout_seconds(): + assert _profile(queue_timeout_hours=24).queue_timeout_s == 86400 + assert _profile(queue_timeout_hours=1).queue_timeout_s == 3600 + + +def test_probably_asleep_never_seen_is_true(): + assert _profile().probably_asleep(None) is True + + +def test_probably_asleep_recent_advert_is_false(): + profile = _profile(sleep_timeout_ms=10000) # 10 s window + now = 1_000_000.0 + # Seen 1 s ago: still inside the wake window -> may be awake. + assert profile.probably_asleep(now - 1.0, now=now) is False + + +def test_probably_asleep_stale_advert_is_true(): + profile = _profile(sleep_timeout_ms=10000) + now = 1_000_000.0 + # Seen well beyond window + slack -> back asleep. + stale = now - (10.0 + FRESHNESS_SLACK_S + 1.0) + assert profile.probably_asleep(stale, now=now) is True + + +def test_probably_asleep_boundary(): + profile = _profile(sleep_timeout_ms=10000) + now = 1_000_000.0 + threshold = 10.0 + FRESHNESS_SLACK_S + # Exactly at the threshold is not yet "asleep" (strict greater-than). + assert profile.probably_asleep(now - threshold, now=now) is False + assert profile.probably_asleep(now - threshold - 0.01, now=now) is True + + +if __name__ == "__main__": + raise SystemExit(pytest.main([__file__, "-v"])) From 3ff006bb3a500d43f3c2acfde2f6d950cb6cb241 Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Mon, 6 Jul 2026 20:17:31 -0400 Subject: [PATCH 03/10] feat: phase 2 - delivery manager and queued uploads Deliver content to a sleeping display at its next wake instead of dropping it: - coordinator.py: async_subscribe_device_seen fires on every parsed advertisement (the wake rendezvous), mirroring async_subscribe_reboot. - delivery.py: DeliveryManager owns pending slots (latest-wins upload + config-resync), drains them over a single BLE session on device-seen bounded by a ~30 s per-wake deadline, arms per-upload deadline timers that fire opendisplay_content_expired, fires opendisplay_content_delivered on success, and refreshes the entry cache after a resync. - services.py: upload_image/drawcustom become SupportsResponse.OPTIONAL returning {status, expires_at}. A freshness gate queues directly when the tag is provably asleep; a live BLE connect/timeout failure on a sleepy device queues instead of raising. LED/buzzer fail fast with a device_sleeping error when provably asleep. Non-sleepy devices follow the original code paths. - __init__.py: instantiate the manager, keep entities available across sleep, route the reboot edge to a config resync for sleepy devices (D7), and tear the manager down on unload. Add the binary_sensor platform. - image.py: show the queued frame immediately with pending/queued_at attributes. - binary_sensor.py: "Update pending" entity backed by the manager state. - tests: DeliveryManager unit tests with mocked hass/coordinator/device. Co-Authored-By: Claude Fable 5 --- custom_components/opendisplay/__init__.py | 50 ++- .../opendisplay/binary_sensor.py | 93 ++++ custom_components/opendisplay/coordinator.py | 21 + custom_components/opendisplay/delivery.py | 421 ++++++++++++++++++ custom_components/opendisplay/image.py | 54 ++- custom_components/opendisplay/services.py | 179 +++++++- tests/test_delivery.py | 305 +++++++++++++ 7 files changed, 1090 insertions(+), 33 deletions(-) create mode 100644 custom_components/opendisplay/binary_sensor.py create mode 100644 custom_components/opendisplay/delivery.py create mode 100644 tests/test_delivery.py diff --git a/custom_components/opendisplay/__init__.py b/custom_components/opendisplay/__init__.py index 3d8e8f1..4059761 100644 --- a/custom_components/opendisplay/__init__.py +++ b/custom_components/opendisplay/__init__.py @@ -34,6 +34,7 @@ from .const import CONF_CACHED_STATE, CONF_ENCRYPTION_KEY, DOMAIN from .coordinator import OpenDisplayCoordinator +from .delivery import DeliveryManager from .services import async_setup_services from .sleep import SleepProfile @@ -42,8 +43,18 @@ CONFIG_SCHEMA = cv.config_entry_only_config_schema(DOMAIN) -_BASE_PLATFORMS: list[Platform] = [Platform.IMAGE, Platform.SENSOR] -_FLEX_PLATFORMS = [Platform.EVENT, Platform.IMAGE, Platform.SENSOR, Platform.UPDATE] +_BASE_PLATFORMS: list[Platform] = [ + Platform.BINARY_SENSOR, + Platform.IMAGE, + Platform.SENSOR, +] +_FLEX_PLATFORMS = [ + Platform.BINARY_SENSOR, + Platform.EVENT, + Platform.IMAGE, + Platform.SENSOR, + Platform.UPDATE, +] @dataclass @@ -66,10 +77,11 @@ class OpenDisplayRuntimeData: # (0x76). Replaced with a fresh instance on every full/fast refresh so the # next partial diffs against the frame actually on the panel. partial_state: PartialState = field(default_factory=PartialState) - # Set when the entry was set up from cache without connecting; consumed in - # Phase 2 by the delivery manager, which re-reads firmware/config at the - # next wake. In Phase 1 the resync rides the reboot-edge reload. + # Set when the entry was set up from cache without connecting; the delivery + # manager re-reads firmware/config at the next wake and refreshes the cache. config_resync_pending: bool = False + # Owns queued work delivered at the next wake (set in async_setup_entry). + delivery: DeliveryManager | None = None type OpenDisplayConfigEntry = ConfigEntry[OpenDisplayRuntimeData] @@ -286,15 +298,32 @@ async def async_setup_entry(hass: HomeAssistant, entry: OpenDisplayConfigEntry) hass, address, profile.availability_interval ) + manager = DeliveryManager(hass, entry) + entry.runtime_data.delivery = manager + await hass.config_entries.async_forward_entry_setups( entry, _get_platforms(entry.runtime_data) ) entry.async_on_unload(coordinator.async_start()) + manager.async_start() + + if from_cache: + # Re-read firmware/config on the next wake and refresh the cache. + manager.request_config_resync() @callback def _schedule_reboot_reload() -> None: - """Re-read firmware/config after the device signals a reboot.""" - hass.async_create_task(_async_reload_after_reboot(hass, entry)) + """React to the device's advertised reboot edge. + + For sleepy devices the firmware's unpersisted reboot flag makes every + wake look like a reboot, and a reconnecting reload races the wake + window; route to an opportunistic config resync instead. Non-sleepy + devices keep the reload behavior. + """ + if profile.is_sleepy: + manager.request_config_resync() + else: + hass.async_create_task(_async_reload_after_reboot(hass, entry)) entry.async_on_unload(coordinator.async_subscribe_reboot(_schedule_reboot_reload)) @@ -337,9 +366,12 @@ async def async_unload_entry( ) -> bool: """Unload a config entry.""" runtime = entry.runtime_data - # Abort an in-flight image upload quickly so unload is not blocked on a long - # BLE transfer, then wait for any other in-flight BLE op (LED, buzzer, OTA, + # Cancel pending deadline timers and any in-flight delivery task, then abort + # an in-flight image upload quickly so unload is not blocked on a long BLE + # transfer, then wait for any other in-flight BLE op (LED, buzzer, OTA, # drawcustom) to release the link before tearing the entry down. + if runtime.delivery is not None: + await runtime.delivery.async_shutdown() if (task := runtime.upload_task) and not task.done(): task.cancel() with contextlib.suppress(asyncio.CancelledError): diff --git a/custom_components/opendisplay/binary_sensor.py b/custom_components/opendisplay/binary_sensor.py new file mode 100644 index 0000000..2678fee --- /dev/null +++ b/custom_components/opendisplay/binary_sensor.py @@ -0,0 +1,93 @@ +"""Binary sensor platform for OpenDisplay devices.""" + +from datetime import datetime, timezone +from typing import Any + +from homeassistant.components.binary_sensor import BinarySensorEntity +from homeassistant.core import HomeAssistant, callback +from homeassistant.helpers.device_registry import CONNECTION_BLUETOOTH, DeviceInfo +from homeassistant.helpers.dispatcher import async_dispatcher_connect +from homeassistant.helpers.entity_platform import AddConfigEntryEntitiesCallback + +from . import OpenDisplayConfigEntry +from .const import SIGNAL_PENDING_STATE +from .delivery import DeliveryManager, DeliverySnapshot + +PARALLEL_UPDATES = 0 + + +async def async_setup_entry( + hass: HomeAssistant, + entry: OpenDisplayConfigEntry, + async_add_entities: AddConfigEntryEntitiesCallback, +) -> None: + """Set up the OpenDisplay update-pending binary sensor.""" + manager = entry.runtime_data.delivery + if manager is None: + return + async_add_entities( + [OpenDisplayUpdatePendingSensor(entry.unique_id or "", manager)] + ) + + +def _to_iso(epoch: float | None) -> str | None: + """Convert an epoch timestamp to an ISO string, or None.""" + if epoch is None: + return None + return datetime.fromtimestamp(epoch, tz=timezone.utc).isoformat() + + +class OpenDisplayUpdatePendingSensor(BinarySensorEntity): + """On while content is queued for delivery at the next wake. + + Backed entirely by the delivery manager's state, so it stays available and + meaningful even while the device is dark. + """ + + _attr_has_entity_name = True + _attr_translation_key = "update_pending" + + def __init__(self, address: str, manager: DeliveryManager) -> None: + """Initialize the binary sensor from the manager's current state.""" + self._address = address + self._attr_unique_id = f"{address}-update_pending" + self._attr_device_info = DeviceInfo( + connections={(CONNECTION_BLUETOOTH, address)}, + ) + self._apply(manager.state) + + @callback + def _apply(self, snapshot: DeliverySnapshot) -> None: + """Store the latest delivery snapshot on the entity.""" + self._attr_is_on = snapshot.pending + self._queued_at = snapshot.queued_at + self._expires_at = snapshot.expires_at + self._attempts = snapshot.attempts + self._last_error = snapshot.last_error + + @property + def extra_state_attributes(self) -> dict[str, Any]: + """Expose queue timing details for automations/dashboards.""" + return { + "queued_at": _to_iso(self._queued_at), + "expires_at": _to_iso(self._expires_at), + "attempts": self._attempts, + "last_error": self._last_error, + } + + async def async_added_to_hass(self) -> None: + """Subscribe to delivery state updates.""" + await super().async_added_to_hass() + self.async_on_remove( + async_dispatcher_connect( + self.hass, + f"{SIGNAL_PENDING_STATE}_{self._address}", + self._handle_pending_state, + ) + ) + + @callback + def _handle_pending_state(self, snapshot: DeliverySnapshot) -> None: + """Handle a delivery-state change.""" + self._apply(snapshot) + self.async_write_ha_state() diff --git a/custom_components/opendisplay/coordinator.py b/custom_components/opendisplay/coordinator.py index ec743a0..2858ff4 100644 --- a/custom_components/opendisplay/coordinator.py +++ b/custom_components/opendisplay/coordinator.py @@ -55,6 +55,10 @@ def __init__(self, hass: HomeAssistant, address: str) -> None: # Subscribers notified once when the advertised reboot flag goes # False -> True (the device rebooted since we last talked to it). self._reboot_callbacks: set[CALLBACK_TYPE] = set() + # Subscribers notified on every parsed advertisement (the device is + # awake and reachable right now). The delivery manager uses this as the + # wake rendezvous to drain queued work. + self._device_seen_callbacks: set[CALLBACK_TYPE] = set() # Reboot-flag edge detection: the device sets the advertised reboot flag # on boot and clears it on first connect. None until the first v1 advert. self._last_reboot_flag: bool | None = None @@ -73,6 +77,21 @@ def _unsubscribe() -> None: return _unsubscribe + @callback + def async_subscribe_device_seen(self, callback_: CALLBACK_TYPE) -> CALLBACK_TYPE: + """Subscribe to "device seen" events (every parsed advertisement). + + Fired after each successfully parsed advertisement, i.e. whenever the + device is awake and reachable. Returns an unsubscribe callback. + """ + self._device_seen_callbacks.add(callback_) + + @callback + def _unsubscribe() -> None: + self._device_seen_callbacks.discard(callback_) + + return _unsubscribe + @callback def _async_handle_unavailable( self, service_info: BluetoothServiceInfoBleak @@ -123,6 +142,8 @@ def _async_handle_bluetooth_event( button_events=button_events, touch_events=touch_events, ) + for device_seen_callback in list(self._device_seen_callbacks): + device_seen_callback() super()._async_handle_bluetooth_event(service_info, change) diff --git a/custom_components/opendisplay/delivery.py b/custom_components/opendisplay/delivery.py new file mode 100644 index 0000000..da79a43 --- /dev/null +++ b/custom_components/opendisplay/delivery.py @@ -0,0 +1,421 @@ +"""Wake-triggered delivery of queued work to deep-sleeping OpenDisplay devices. + +A sleeping ESP32 tag is dark most of the time and reachable only during a short +advertising window after each timer wake. The `DeliveryManager` owns the work +that could not be delivered immediately (a prepared image, a config resync) and +drains it over a single BLE connection the moment the device is seen advertising +again. + +Design (see docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md, D4): + +* Latest-wins per work type — only the newest image survives, matching the + existing live-upload semantics. +* One connection per wake — the device stays awake while connected, so all + pending work drains over a single session in priority order (upload first). +* Transport-agnostic trigger — `notify_device_seen(source)` is the entry point; + today the BLE coordinator calls it, a future WiFi presence tracker could too. +* Memory-only in v1 — a HA restart drops a pending upload (documented). +""" + +from __future__ import annotations + +import asyncio +from dataclasses import dataclass +import logging +import time +from typing import TYPE_CHECKING, Any + +from opendisplay import ( + AuthenticationFailedError, + AuthenticationRequiredError, + BLEConnectionError, + BLETimeoutError, + OpenDisplayDevice, + OpenDisplayError, + PartialState, + RefreshMode, +) + +from homeassistant.components.bluetooth import async_ble_device_from_address +from homeassistant.core import CALLBACK_TYPE, HomeAssistant, callback +from homeassistant.helpers import device_registry as dr +from homeassistant.helpers.device_registry import CONNECTION_BLUETOOTH +from homeassistant.helpers.dispatcher import async_dispatcher_send +from homeassistant.helpers.event import async_call_later + +from .const import ( + CONF_ENCRYPTION_KEY, + EVENT_CONTENT_DELIVERED, + EVENT_CONTENT_EXPIRED, + SIGNAL_IMAGE_UPDATED, + SIGNAL_PENDING_STATE, +) + +if TYPE_CHECKING: + from . import OpenDisplayConfigEntry + +_LOGGER = logging.getLogger(__name__) + +# Upper bound on a single wake's delivery attempt: connection establishment plus +# the queued work. Bounds one bad wake so it cannot overlap the next. +DELIVERY_DEADLINE_S = 30.0 + +# Sentinel distinguishing "no key" (None) from "malformed key" during resolve. +_KEY_INVALID = object() + + +@dataclass +class PendingUpload: + """A prepared image queued for delivery at the next wake.""" + + # (uncompressed_data, compressed_data or None, processed_image) from + # prepare_image() — the heavy CPU work is done once, at queue time. + prepared: tuple[bytes, bytes | None, Any] + refresh_mode: RefreshMode + partial_state: PartialState + use_measured_palettes: bool + preview_jpeg: bytes + device_id: str | None + queued_at: float + expires_at: float + attempts: int = 0 + # Set when a delivery hit an auth failure; suppresses further doomed + # attempts until the entry reloads after reauth. + paused: bool = False + cancel_deadline: CALLBACK_TYPE | None = None + + +@dataclass(frozen=True) +class DeliveryReceipt: + """Result of submitting content: whether it was delivered or queued.""" + + status: str # "delivered" | "queued" + expires_at: float | None + + +@dataclass(frozen=True) +class DeliverySnapshot: + """Public delivery state consumed by entities (image, binary sensor).""" + + pending: bool + queued_at: float | None + expires_at: float | None + attempts: int + last_error: str | None + + +class DeliveryManager: + """Owns queued work for one config entry and delivers it at the next wake.""" + + def __init__(self, hass: HomeAssistant, entry: OpenDisplayConfigEntry) -> None: + """Initialize the delivery manager.""" + self._hass = hass + self._entry = entry + runtime = entry.runtime_data + self._coordinator = runtime.coordinator + self._profile = runtime.sleep_profile + self._address: str = entry.unique_id or "" + + self._pending_upload: PendingUpload | None = None + self._pending_config_resync: bool = False + self._last_error: str | None = None + + self._delivering: bool = False + self._delivery_task: asyncio.Task[None] | None = None + self._unsub_device_seen: CALLBACK_TYPE | None = None + + # -- lifecycle ---------------------------------------------------------- + + @callback + def async_start(self) -> None: + """Subscribe to wake advertisements.""" + self._unsub_device_seen = self._coordinator.async_subscribe_device_seen( + self._on_device_seen + ) + + async def async_shutdown(self) -> None: + """Cancel timers and any in-flight delivery, and unsubscribe.""" + if self._unsub_device_seen is not None: + self._unsub_device_seen() + self._unsub_device_seen = None + if self._pending_upload is not None and self._pending_upload.cancel_deadline: + self._pending_upload.cancel_deadline() + self._pending_upload.cancel_deadline = None + if self._delivery_task is not None and not self._delivery_task.done(): + self._delivery_task.cancel() + try: + await self._delivery_task + except asyncio.CancelledError: + pass + except Exception: # noqa: BLE001 - shutdown must not raise + _LOGGER.debug("Delivery task raised during shutdown", exc_info=True) + self._delivery_task = None + self._delivering = False + + # -- public state ------------------------------------------------------- + + @property + def state(self) -> DeliverySnapshot: + """Return the current pending-delivery state for entities.""" + upload = self._pending_upload + return DeliverySnapshot( + pending=upload is not None, + queued_at=upload.queued_at if upload else None, + expires_at=upload.expires_at if upload else None, + attempts=upload.attempts if upload else 0, + last_error=self._last_error, + ) + + # -- submission --------------------------------------------------------- + + @callback + def submit_upload( + self, + *, + prepared: tuple[bytes, bytes | None, Any], + refresh_mode: RefreshMode, + partial_state: PartialState, + use_measured_palettes: bool, + preview_jpeg: bytes, + device_id: str | None, + ) -> DeliveryReceipt: + """Queue a prepared image for delivery at the next wake (latest-wins).""" + now = time.time() + # Latest-wins: drop any previously queued image and its deadline timer. + if self._pending_upload is not None and self._pending_upload.cancel_deadline: + self._pending_upload.cancel_deadline() + expires_at = now + self._profile.queue_timeout_s + slot = PendingUpload( + prepared=prepared, + refresh_mode=refresh_mode, + partial_state=partial_state, + use_measured_palettes=use_measured_palettes, + preview_jpeg=preview_jpeg, + device_id=device_id, + queued_at=now, + expires_at=expires_at, + ) + self._schedule_expiry(slot) + self._pending_upload = slot + self._last_error = None + # Show the intended frame immediately (the image entity now reflects + # what will be delivered, not what is on the panel). + async_dispatcher_send( + self._hass, f"{SIGNAL_IMAGE_UPDATED}_{self._address}", preview_jpeg + ) + self._notify_state() + return DeliveryReceipt(status="queued", expires_at=expires_at) + + @callback + def request_config_resync(self) -> None: + """Request a firmware/config re-read at the next wake.""" + self._pending_config_resync = True + self._entry.runtime_data.config_resync_pending = True + + # -- wake handling ------------------------------------------------------ + + @callback + def _on_device_seen(self) -> None: + """Coordinator callback: the device advertised (it is awake now).""" + self.notify_device_seen("ble") + + @callback + def notify_device_seen(self, source: str = "ble") -> None: + """Start a delivery drain if there is pending work and none in flight.""" + if self._delivering or not self._has_pending_work(): + return + _LOGGER.debug("%s: device seen (%s); starting delivery", self._address, source) + self._delivering = True + self._delivery_task = self._entry.async_create_background_task( + self._hass, self._deliver(), f"opendisplay_delivery_{self._address}" + ) + + def _has_pending_work(self) -> bool: + """Return True if there is deliverable work queued.""" + upload_ready = self._pending_upload is not None and not self._pending_upload.paused + return upload_ready or self._pending_config_resync + + # -- delivery drain ----------------------------------------------------- + + async def _deliver(self) -> None: + """Open one connection and drain all pending slots in priority order.""" + try: + await self._drain_once() + except asyncio.CancelledError: + raise + except (BLEConnectionError, BLETimeoutError, TimeoutError) as err: + # Device slept mid-connect or the wake window was missed; keep the + # work queued and try again on the next wake. + self._register_attempt_failure(str(err)) + except (AuthenticationFailedError, AuthenticationRequiredError): + # Bad/rotated key: pause the upload and prompt the user to reauth. + _LOGGER.warning("%s: delivery auth failed; starting reauth", self._address) + if self._pending_upload is not None: + self._pending_upload.paused = True + self._last_error = "auth" + self._entry.async_start_reauth(self._hass) + self._notify_state() + except OpenDisplayError as err: + _LOGGER.warning("%s: delivery failed: %s", self._address, err) + self._register_attempt_failure(str(err)) + finally: + self._delivering = False + self._delivery_task = None + + async def _drain_once(self) -> None: + """The connection + drain body (see `_deliver` for error handling).""" + key = self._resolve_key() + if key is _KEY_INVALID: + # Malformed stored key; reauth was already started. + return + + runtime = self._entry.runtime_data + async with runtime.ble_lock: + ble_device = async_ble_device_from_address( + self._hass, self._address, connectable=True + ) + if ble_device is None: + # The advertisement that woke us has already aged out; retry next. + self._register_attempt_failure("device not connectable") + return + + upload = self._pending_upload + use_measured = upload.use_measured_palettes if upload else True + async with ( + asyncio.timeout(DELIVERY_DEADLINE_S), + OpenDisplayDevice( + mac_address=self._address, + ble_device=ble_device, + config=runtime.device_config, + use_measured_palettes=use_measured, + encryption_key=key, # type: ignore[arg-type] + ) as device, + ): + if upload is not None and not upload.paused: + await self._drain_upload(device, upload) + if self._pending_config_resync: + await self._drain_resync(device) + + async def _drain_upload( + self, device: OpenDisplayDevice, upload: PendingUpload + ) -> None: + """Send the queued prepared image and fire the delivered event.""" + await device.upload_prepared_image( + upload.prepared, + refresh_mode=upload.refresh_mode, + state=upload.partial_state, + ) + if upload.cancel_deadline: + upload.cancel_deadline() + self._pending_upload = None + self._last_error = None + # Bump the image entity's timestamp to when the frame actually landed. + async_dispatcher_send( + self._hass, f"{SIGNAL_IMAGE_UPDATED}_{self._address}", upload.preview_jpeg + ) + self._fire_content_event(EVENT_CONTENT_DELIVERED, upload) + self._notify_state() + _LOGGER.info("%s: queued content delivered", self._address) + + async def _drain_resync(self, device: OpenDisplayDevice) -> None: + """Re-read firmware/config over the open link and refresh the cache.""" + # Local import avoids an import cycle (__init__ imports this module). + from . import _write_cache # noqa: PLC0415 + + fw = await device.read_firmware_version() + is_flex = device.is_flex + landing_url = device.landing_url() + device_config = device.config + if device_config is None: + return + runtime = self._entry.runtime_data + runtime.firmware = fw + runtime.device_config = device_config + runtime.is_flex = is_flex + runtime.config_resync_pending = False + self._pending_config_resync = False + _write_cache(self._hass, self._entry, device_config, fw, is_flex, landing_url) + _LOGGER.debug("%s: config resync complete", self._address) + + # -- expiry ------------------------------------------------------------- + + @callback + def _schedule_expiry(self, slot: PendingUpload) -> None: + """Arm the deadline timer that expires this queued upload.""" + + @callback + def _expired(_now: Any) -> None: + # Guard against a stale timer firing after latest-wins replaced it. + if self._pending_upload is slot: + self._expire_upload(slot) + + slot.cancel_deadline = async_call_later( + self._hass, self._profile.queue_timeout_s, _expired + ) + + @callback + def _expire_upload(self, slot: PendingUpload) -> None: + """Drop an expired queued upload and fire the expired event.""" + self._pending_upload = None + self._last_error = "expired" + _LOGGER.warning( + "%s: queued content expired after %s h without a wake", + self._address, + self._profile.queue_timeout_hours, + ) + self._fire_content_event(EVENT_CONTENT_EXPIRED, slot) + self._notify_state() + + # -- helpers ------------------------------------------------------------ + + def _register_attempt_failure(self, reason: str) -> None: + """Record a failed wake attempt and keep the work queued.""" + if self._pending_upload is not None: + self._pending_upload.attempts += 1 + self._last_error = reason + _LOGGER.debug( + "%s: delivery attempt failed (%s); will retry next wake", + self._address, + reason, + ) + self._notify_state() + + def _resolve_key(self) -> bytes | None | object: + """Return the encryption key bytes, None, or a sentinel on bad format.""" + raw = self._entry.data.get(CONF_ENCRYPTION_KEY) + if raw is None: + return None + if len(raw) != 32: + self._entry.async_start_reauth(self._hass) + return _KEY_INVALID + try: + return bytes.fromhex(raw) + except ValueError: + self._entry.async_start_reauth(self._hass) + return _KEY_INVALID + + @callback + def _notify_state(self) -> None: + """Push the current pending state to entities.""" + async_dispatcher_send( + self._hass, f"{SIGNAL_PENDING_STATE}_{self._address}", self.state + ) + + def _fire_content_event(self, event: str, slot: PendingUpload) -> None: + """Fire a content delivered/expired bus event for automations.""" + device_id = slot.device_id or self._device_id() + self._hass.bus.async_fire( + event, + { + "device_id": device_id, + "queued_at": slot.queued_at, + "attempts": slot.attempts, + }, + ) + + def _device_id(self) -> str | None: + """Resolve this entry's device registry id, if any.""" + device = dr.async_get(self._hass).async_get_device( + connections={(CONNECTION_BLUETOOTH, self._address)} + ) + return device.id if device else None diff --git a/custom_components/opendisplay/image.py b/custom_components/opendisplay/image.py index 3d32703..fbb0428 100644 --- a/custom_components/opendisplay/image.py +++ b/custom_components/opendisplay/image.py @@ -1,5 +1,8 @@ """Image entity for OpenDisplay devices.""" +from datetime import datetime, timezone +from typing import Any + from homeassistant.components.image import ImageEntity from homeassistant.core import HomeAssistant, callback from homeassistant.helpers.device_registry import CONNECTION_BLUETOOTH, DeviceInfo @@ -8,8 +11,9 @@ import homeassistant.util.dt as dt_util from . import OpenDisplayConfigEntry -from .const import SIGNAL_IMAGE_UPDATED +from .const import SIGNAL_IMAGE_UPDATED, SIGNAL_PENDING_STATE from .coordinator import OpenDisplayCoordinator +from .delivery import DeliverySnapshot PARALLEL_UPDATES = 0 @@ -20,11 +24,20 @@ async def async_setup_entry( async_add_entities: AddConfigEntryEntitiesCallback, ) -> None: """Set up the OpenDisplay image entity.""" - async_add_entities([OpenDisplayImageEntity(hass, entry.runtime_data.coordinator)]) + async_add_entities( + [OpenDisplayImageEntity(hass, entry.runtime_data.coordinator)] + ) + + +def _to_iso(epoch: float | None) -> str | None: + """Convert an epoch timestamp to an ISO string, or None.""" + if epoch is None: + return None + return datetime.fromtimestamp(epoch, tz=timezone.utc).isoformat() class OpenDisplayImageEntity(ImageEntity): - """Shows the last image sent to an OpenDisplay device.""" + """Shows the last image sent to (or queued for) an OpenDisplay device.""" _attr_has_entity_name = True _attr_translation_key = "content" @@ -39,13 +52,27 @@ def __init__(self, hass: HomeAssistant, coordinator: OpenDisplayCoordinator) -> connections={(CONNECTION_BLUETOOTH, coordinator.address)}, ) self._image_bytes: bytes | None = None + # When True the shown frame is queued for the next wake, not yet on the + # panel (D6). + self._pending: bool = False + self._queued_at: float | None = None + self._last_error: str | None = None + + @property + def extra_state_attributes(self) -> dict[str, Any]: + """Expose whether the shown frame is still waiting to be delivered.""" + return { + "pending": self._pending, + "queued_at": _to_iso(self._queued_at), + "last_error": self._last_error, + } async def async_image(self) -> bytes | None: - """Return the last uploaded image bytes.""" + """Return the last uploaded (or queued) image bytes.""" return self._image_bytes async def async_added_to_hass(self) -> None: - """Subscribe to image update signals.""" + """Subscribe to image and pending-state update signals.""" await super().async_added_to_hass() self.async_on_remove( async_dispatcher_connect( @@ -54,10 +81,25 @@ async def async_added_to_hass(self) -> None: self._handle_image_update, ) ) + self.async_on_remove( + async_dispatcher_connect( + self.hass, + f"{SIGNAL_PENDING_STATE}_{self._coordinator.address}", + self._handle_pending_state, + ) + ) @callback def _handle_image_update(self, image_bytes: bytes) -> None: - """Handle a new image from a completed upload.""" + """Handle a new image from a completed or queued upload.""" self._image_bytes = image_bytes self._attr_image_last_updated = dt_util.utcnow() self.async_write_ha_state() + + @callback + def _handle_pending_state(self, snapshot: DeliverySnapshot) -> None: + """Reflect the delivery manager's pending state on the entity.""" + self._pending = snapshot.pending + self._queued_at = snapshot.queued_at + self._last_error = snapshot.last_error + self.async_write_ha_state() diff --git a/custom_components/opendisplay/services.py b/custom_components/opendisplay/services.py index f347997..4dfc4c3 100644 --- a/custom_components/opendisplay/services.py +++ b/custom_components/opendisplay/services.py @@ -3,7 +3,7 @@ import asyncio from collections.abc import Awaitable, Callable import contextlib -from datetime import timedelta +from datetime import datetime, timedelta, timezone from enum import IntEnum import functools import io @@ -16,6 +16,8 @@ from opendisplay import ( AuthenticationFailedError, AuthenticationRequiredError, + BLEConnectionError, + BLETimeoutError, ColorScheme, DitherMode, FitMode, @@ -41,7 +43,13 @@ from homeassistant.components.media_source import async_resolve_media from homeassistant.config_entries import ConfigEntryState from homeassistant.const import ATTR_DEVICE_ID -from homeassistant.core import HomeAssistant, ServiceCall, callback +from homeassistant.core import ( + HomeAssistant, + ServiceCall, + ServiceResponse, + SupportsResponse, + callback, +) from homeassistant.exceptions import HomeAssistantError, ServiceValidationError from homeassistant.helpers import config_validation as cv, device_registry as dr from homeassistant.helpers.aiohttp_client import async_get_clientsession @@ -54,6 +62,7 @@ from . import OpenDisplayConfigEntry from .const import CONF_ENCRYPTION_KEY, DOMAIN, SIGNAL_IMAGE_UPDATED +from .delivery import DeliveryReceipt ATTR_IMAGE = "image" ATTR_ROTATION = "rotation" @@ -280,17 +289,34 @@ async def _async_download_image(hass: HomeAssistant, url: str) -> PILImage.Image return await hass.async_add_executor_job(_load_image_from_bytes, data) +class _DeviceUnavailable(Exception): + """Signals the tag was dark or dropped the link during a live send. + + Raised only when ``reraise_ble=True`` so the caller can queue the content + for the next wake instead of surfacing an ``upload_error``. + """ + + async def _async_connect_and_run( hass: HomeAssistant, entry: "OpenDisplayConfigEntry", action: Callable[[OpenDisplayDevice], Awaitable[None]], use_measured_palettes: bool = False, + reraise_ble: bool = False, ) -> None: - """Resolve BLE device, open a connection, run action, handle auth errors.""" + """Resolve BLE device, open a connection, run action, handle auth errors. + + When ``reraise_ble`` is set, a missing connectable device or a BLE + connect/timeout failure raises ``_DeviceUnavailable`` instead of a + translated ``device_not_found``/``upload_error``, so the caller can defer + the work to the delivery queue. All other behavior is unchanged. + """ address = entry.unique_id assert address is not None ble_device = async_ble_device_from_address(hass, address, connectable=True) if ble_device is None: + if reraise_ble: + raise _DeviceUnavailable raise HomeAssistantError( translation_domain=DOMAIN, translation_key="device_not_found", @@ -338,6 +364,14 @@ async def _async_connect_and_run( raise HomeAssistantError( translation_domain=DOMAIN, translation_key="authentication_error" ) from err + except (BLEConnectionError, BLETimeoutError) as err: + if reraise_ble: + raise _DeviceUnavailable from err + raise HomeAssistantError( + translation_domain=DOMAIN, + translation_key="upload_error", + translation_placeholders={"error": str(err)}, + ) from err except OpenDisplayError as err: raise HomeAssistantError( translation_domain=DOMAIN, @@ -351,14 +385,20 @@ async def _async_send_image( entry: "OpenDisplayConfigEntry", img: PILImage.Image, *, + device_id: str | None = None, dither_mode: DitherMode, refresh_mode: RefreshMode, fit: FitMode = FitMode.CONTAIN, tone: float | str = "auto", rotate: Rotation = Rotation.ROTATE_0, use_measured_palettes: bool = False, -) -> None: - """Upload a PIL image to the device.""" +) -> DeliveryReceipt: + """Upload a PIL image, delivering live or queuing it for the next wake. + + Returns a receipt describing whether the frame was delivered immediately or + queued (for a sleeping device). Non-sleepy devices always deliver live and + keep the original strict-failure behavior. + """ # Split the upload into its heavy CPU half and its BLE-I/O half. The CPU # work (rotate + fit + dither + encode + zlib on a full frame) is offloaded # to an executor thread so it never blocks the event loop; only the BLE @@ -399,20 +439,66 @@ async def _async_send_image( runtime.partial_state = PartialState() state = runtime.partial_state + # The preview JPEG is built up-front so a queued frame can be shown on the + # image entity immediately (D6), not only after a successful delivery. + jpeg = await hass.async_add_executor_job(_pil_to_jpeg, img) + + profile = runtime.sleep_profile + manager = runtime.delivery + sleepy = manager is not None and profile.is_sleepy + + def _queue() -> DeliveryReceipt: + assert manager is not None + return manager.submit_upload( + prepared=prepared, + refresh_mode=refresh_mode, + partial_state=state, + use_measured_palettes=use_measured_palettes, + preview_jpeg=jpeg, + device_id=device_id, + ) + + # Freshness gate: a provably-asleep tag will not answer a live connect, so + # skip the doomed ~40 s retry cycle and queue directly for the next wake. + if sleepy: + last_seen = runtime.coordinator.data.last_seen if runtime.coordinator.data else None + if profile.probably_asleep(last_seen): + return _queue() + async def _upload(device: OpenDisplayDevice) -> None: await device.upload_prepared_image(prepared, refresh_mode=refresh_mode, state=state) - await _async_connect_and_run( - hass, entry, _upload, use_measured_palettes=use_measured_palettes - ) - jpeg = await hass.async_add_executor_job(_pil_to_jpeg, img) + try: + await _async_connect_and_run( + hass, + entry, + _upload, + use_measured_palettes=use_measured_palettes, + reraise_ble=sleepy, + ) + except _DeviceUnavailable: + # The device was seen recently but dropped/refused the link; defer. + return _queue() + async_dispatcher_send(hass, f"{SIGNAL_IMAGE_UPDATED}_{entry.unique_id}", jpeg) + return DeliveryReceipt(status="delivered", expires_at=None) -async def _async_upload_image(call: ServiceCall) -> None: +def _receipt_response(receipt: DeliveryReceipt) -> ServiceResponse: + """Build the service response payload from a delivery receipt.""" + expires_at = ( + datetime.fromtimestamp(receipt.expires_at, tz=timezone.utc).isoformat() + if receipt.expires_at is not None + else None + ) + return {"status": receipt.status, "expires_at": expires_at} + + +async def _async_upload_image(call: ServiceCall) -> ServiceResponse: """Handle the upload_image service call.""" entry = _get_entry_for_device(call) + device_id: str = call.data[ATTR_DEVICE_ID] image_data: dict[str, Any] | str = call.data[ATTR_IMAGE] rotation: Rotation = call.data[ATTR_ROTATION] dither_mode: DitherMode = call.data[ATTR_DITHER_MODE] @@ -463,10 +549,11 @@ async def _async_upload_image(call: ServiceCall) -> None: else: pil_image = await _async_download_image(call.hass, media.url) - await _async_send_image( + receipt = await _async_send_image( call.hass, entry, pil_image, + device_id=device_id, dither_mode=dither_mode, refresh_mode=refresh_mode, fit=fit_mode, @@ -474,8 +561,11 @@ async def _async_upload_image(call: ServiceCall) -> None: rotate=rotation, use_measured_palettes=use_measured_palettes, ) + return _receipt_response(receipt) except asyncio.CancelledError: - return + # Superseded by a newer upload (latest-wins); report it honestly rather + # than surfacing the cancellation. + return {"status": "superseded", "expires_at": None} finally: if entry.runtime_data.upload_task is current: entry.runtime_data.upload_task = None @@ -577,7 +667,7 @@ async def _get_device_ids_from_area(hass: HomeAssistant, area_id: str) -> list[s ] -async def _async_drawcustom(call: ServiceCall) -> None: +async def _async_drawcustom(call: ServiceCall) -> ServiceResponse: """Handle the drawcustom service call.""" hass = call.hass @@ -596,11 +686,16 @@ async def _async_drawcustom(call: ServiceCall) -> None: ) errors: list[str] = [] + receipts: list[DeliveryReceipt] = [] + results: dict[str, Any] = {} for device_id in unique_ids: try: - await _drawcustom_for_device(hass, device_id, call) + receipt = await _drawcustom_for_device(hass, device_id, call) except (HomeAssistantError, ServiceValidationError) as err: errors.append(f"{device_id}: {err}") + else: + receipts.append(receipt) + results[device_id] = _receipt_response(receipt) if errors: raise ServiceValidationError( translation_domain=DOMAIN, @@ -608,6 +703,25 @@ async def _async_drawcustom(call: ServiceCall) -> None: translation_placeholders={"errors": "\n".join(errors)}, ) + # Summarize across all targets: any queued device makes the batch "queued" + # (with the soonest expiry); otherwise delivered (or dry_run). + queued = [r for r in receipts if r.status == "queued" and r.expires_at is not None] + if queued: + status = "queued" + expires_epoch: float | None = min(r.expires_at for r in queued) # type: ignore[type-var] + elif receipts and all(r.status == "dry_run" for r in receipts): + status = "dry_run" + expires_epoch = None + else: + status = "delivered" + expires_epoch = None + expires_at = ( + datetime.fromtimestamp(expires_epoch, tz=timezone.utc).isoformat() + if expires_epoch is not None + else None + ) + return {"status": status, "expires_at": expires_at, "results": results} + def _font_search_dirs(hass: HomeAssistant) -> list[str]: """Return font search directories in priority order.""" @@ -621,7 +735,7 @@ def _font_search_dirs(hass: HomeAssistant) -> list[str]: async def _drawcustom_for_device( hass: HomeAssistant, device_id: str, call: ServiceCall -) -> None: +) -> DeliveryReceipt: entry = _get_entry_for_device_id(hass, device_id) display = entry.runtime_data.device_config.displays[0] cs = display.color_scheme_enum @@ -656,7 +770,7 @@ async def _drawcustom_for_device( _LOGGER.info("Drawcustom dry run for device %s", device_id) jpeg = await hass.async_add_executor_job(_pil_to_jpeg, img) async_dispatcher_send(hass, f"{SIGNAL_IMAGE_UPDATED}_{entry.unique_id}", jpeg) - return + return DeliveryReceipt(status="dry_run", expires_at=None) dither_mode: DitherMode = call.data["dither"] refresh_mode: RefreshMode = call.data["refresh_type"] @@ -666,10 +780,11 @@ async def _drawcustom_for_device( ) use_measured_palettes: bool = call.data[ATTR_USE_MEASURED_PALETTES] - await _async_send_image( + return await _async_send_image( hass, entry, img, + device_id=device_id, dither_mode=dither_mode, refresh_mode=refresh_mode, tone=tone_compression, @@ -678,6 +793,25 @@ async def _drawcustom_for_device( ) +def _raise_if_sleeping(entry: "OpenDisplayConfigEntry", device_id: str) -> None: + """Fail fast for an immediate-only action when the tag is provably asleep. + + LED/buzzer notifications that fire hours late are worse than an error, so a + sleeping device rejects them immediately rather than queuing. + """ + runtime = entry.runtime_data + profile = runtime.sleep_profile + if not profile.is_sleepy: + return + last_seen = runtime.coordinator.data.last_seen if runtime.coordinator.data else None + if profile.probably_asleep(last_seen): + raise HomeAssistantError( + translation_domain=DOMAIN, + translation_key="device_sleeping", + translation_placeholders={"device_id": device_id}, + ) + + async def _async_activate_led(call: ServiceCall) -> None: """Handle the activate_led service call.""" entry = _get_entry_for_device(call) @@ -687,6 +821,7 @@ async def _async_activate_led(call: ServiceCall) -> None: translation_key="no_leds", translation_placeholders={"device_id": call.data[ATTR_DEVICE_ID]}, ) + _raise_if_sleeping(entry, call.data[ATTR_DEVICE_ID]) repeats: int = call.data["repeats"] flash_config = LedFlashConfig( mode=1, @@ -728,6 +863,7 @@ async def _async_activate_buzzer(call: ServiceCall) -> None: translation_key="no_buzzers", translation_placeholders={"device_id": call.data[ATTR_DEVICE_ID]}, ) + _raise_if_sleeping(entry, call.data[ATTR_DEVICE_ID]) buzz_config = BuzzerActivateConfig.single_tone( frequency_hz=call.data["frequency_hz"], duration_ms=call.data["duration_ms"], @@ -749,7 +885,14 @@ def async_setup_services(hass: HomeAssistant) -> None: "upload_image", _async_upload_image, schema=SCHEMA_UPLOAD_IMAGE, + supports_response=SupportsResponse.OPTIONAL, + ) + hass.services.async_register( + DOMAIN, + "drawcustom", + _async_drawcustom, + schema=SCHEMA_DRAWCUSTOM, + supports_response=SupportsResponse.OPTIONAL, ) - hass.services.async_register(DOMAIN, "drawcustom", _async_drawcustom, schema=SCHEMA_DRAWCUSTOM) hass.services.async_register(DOMAIN, "activate_led", _async_activate_led, schema=SCHEMA_ACTIVATE_LED) hass.services.async_register(DOMAIN, "activate_buzzer", _async_activate_buzzer, schema=SCHEMA_ACTIVATE_BUZZER) \ No newline at end of file diff --git a/tests/test_delivery.py b/tests/test_delivery.py new file mode 100644 index 0000000..a572694 --- /dev/null +++ b/tests/test_delivery.py @@ -0,0 +1,305 @@ +"""Unit tests for DeliveryManager with mocked hass/coordinator/device. + +These avoid the full Home Assistant test harness: the manager's HA touchpoints +(``async_call_later``, ``async_dispatcher_send``, ``async_ble_device_from_address`` +and ``OpenDisplayDevice``) are patched in the delivery module namespace. +""" + +import asyncio +from types import SimpleNamespace +from unittest.mock import AsyncMock, MagicMock, patch + +from opendisplay import AuthenticationFailedError, BLEConnectionError, RefreshMode +import pytest + +from custom_components.opendisplay import delivery as delivery_mod +from custom_components.opendisplay.const import ( + EVENT_CONTENT_DELIVERED, + EVENT_CONTENT_EXPIRED, +) +from custom_components.opendisplay.delivery import DeliveryManager +from custom_components.opendisplay.sleep import SleepProfile + +ADDRESS = "AA:BB:CC:DD:EE:FF" + + +def _profile(**overrides): + params = { + "sleep_mode": "on", + "power_mode": 1, + "sleep_timeout_ms": 0, + "deep_sleep_time_seconds": 300, + "missed_cycles": 3, + "queue_timeout_hours": 24, + } + params.update(overrides) + return SleepProfile.create(**params) + + +def _make_env(profile=None, entry_data=None, last_seen=None): + """Return (hass, entry, coordinator) with a fake runtime for the manager.""" + profile = profile or _profile() + coordinator = MagicMock() + coordinator.data = SimpleNamespace(last_seen=last_seen) + coordinator.async_subscribe_device_seen = MagicMock(return_value=MagicMock()) + runtime = SimpleNamespace( + coordinator=coordinator, + sleep_profile=profile, + device_config=MagicMock(), + ble_lock=asyncio.Lock(), + config_resync_pending=False, + firmware=None, + is_flex=False, + ) + entry = MagicMock() + entry.unique_id = ADDRESS + entry.runtime_data = runtime + entry.data = entry_data if entry_data is not None else {} + entry.async_start_reauth = MagicMock() + hass = MagicMock() + return hass, entry, coordinator + + +def _submit(mgr, device_id="dev1"): + return mgr.submit_upload( + prepared=(b"img", None, object()), + refresh_mode=RefreshMode.FULL, + partial_state=MagicMock(), + use_measured_palettes=False, + preview_jpeg=b"jpeg", + device_id=device_id, + ) + + +def _fake_device_ctx(device): + """Return a factory producing an async-context-manager wrapping device.""" + + class _Ctx: + async def __aenter__(self): + return device + + async def __aexit__(self, *exc): + return False + + return lambda **kwargs: _Ctx() + + +def _raising_device_ctx(exc): + class _Ctx: + async def __aenter__(self): + raise exc + + async def __aexit__(self, *exc_info): + return False + + return lambda **kwargs: _Ctx() + + +def test_submit_upload_queues_and_reports(): + hass, entry, _ = _make_env() + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()) as later, + patch.object(delivery_mod, "async_dispatcher_send") as dispatch, + ): + mgr = DeliveryManager(hass, entry) + receipt = _submit(mgr) + + assert receipt.status == "queued" + assert receipt.expires_at is not None + snap = mgr.state + assert snap.pending is True + assert snap.queued_at is not None + assert snap.attempts == 0 + later.assert_called_once() # deadline armed + # Both the image preview and the pending-state signals were dispatched. + assert dispatch.call_count == 2 + + +def test_latest_wins_cancels_previous_deadline(): + hass, entry, _ = _make_env() + cancels = [MagicMock(), MagicMock()] + with ( + patch.object(delivery_mod, "async_call_later", side_effect=cancels), + patch.object(delivery_mod, "async_dispatcher_send"), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr, device_id="a") + _submit(mgr, device_id="b") + + cancels[0].assert_called_once() # the first deadline timer was cancelled + assert mgr.state.pending is True + + +def test_expiry_clears_slot_and_fires_event(): + hass, entry, _ = _make_env() + captured = {} + + def _fake_later(_hass, _delay, callback): + captured["cb"] = callback + return MagicMock() + + with ( + patch.object(delivery_mod, "async_call_later", side_effect=_fake_later), + patch.object(delivery_mod, "async_dispatcher_send"), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr, device_id="dev1") + captured["cb"](None) # simulate the deadline firing + + assert mgr.state.pending is False + assert mgr.state.last_error == "expired" + hass.bus.async_fire.assert_called_once() + event, payload = hass.bus.async_fire.call_args[0] + assert event == EVENT_CONTENT_EXPIRED + assert payload["device_id"] == "dev1" + assert "queued_at" in payload + + +def test_request_config_resync_sets_flags(): + hass, entry, _ = _make_env() + mgr = DeliveryManager(hass, entry) + assert mgr._has_pending_work() is False + mgr.request_config_resync() + assert entry.runtime_data.config_resync_pending is True + assert mgr._has_pending_work() is True + + +def test_notify_device_seen_starts_one_delivery(): + hass, entry, _ = _make_env() + + def _capture(_hass, coro, _name): + coro.close() # don't actually run the drain + return MagicMock() + + entry.async_create_background_task = MagicMock(side_effect=_capture) + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()), + patch.object(delivery_mod, "async_dispatcher_send"), + ): + mgr = DeliveryManager(hass, entry) + mgr.notify_device_seen() # nothing queued -> no delivery + entry.async_create_background_task.assert_not_called() + + _submit(mgr) + mgr.notify_device_seen() # work queued -> one delivery + entry.async_create_background_task.assert_called_once() + + mgr.notify_device_seen() # already delivering -> still one + entry.async_create_background_task.assert_called_once() + + +@pytest.mark.asyncio +async def test_drain_delivers_upload(): + hass, entry, _ = _make_env() + device = MagicMock() + device.upload_prepared_image = AsyncMock() + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()), + patch.object(delivery_mod, "async_dispatcher_send"), + patch.object(delivery_mod, "async_ble_device_from_address", return_value=MagicMock()), + patch.object(delivery_mod, "OpenDisplayDevice", side_effect=_fake_device_ctx(device)), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr, device_id="dev1") + await mgr._deliver() + + device.upload_prepared_image.assert_awaited_once() + assert mgr.state.pending is False + fired = [call.args[0] for call in hass.bus.async_fire.call_args_list] + assert EVENT_CONTENT_DELIVERED in fired + + +@pytest.mark.asyncio +async def test_drain_ble_failure_keeps_slot_and_counts_attempt(): + hass, entry, _ = _make_env() + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()), + patch.object(delivery_mod, "async_dispatcher_send"), + patch.object(delivery_mod, "async_ble_device_from_address", return_value=MagicMock()), + patch.object( + delivery_mod, + "OpenDisplayDevice", + side_effect=_raising_device_ctx(BLEConnectionError("boom")), + ), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr) + await mgr._deliver() + + assert mgr.state.pending is True + assert mgr.state.attempts == 1 + + +@pytest.mark.asyncio +async def test_drain_auth_failure_pauses_and_starts_reauth(): + hass, entry, _ = _make_env() + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()), + patch.object(delivery_mod, "async_dispatcher_send"), + patch.object(delivery_mod, "async_ble_device_from_address", return_value=MagicMock()), + patch.object( + delivery_mod, + "OpenDisplayDevice", + side_effect=_raising_device_ctx(AuthenticationFailedError("bad key")), + ), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr) + await mgr._deliver() + + assert mgr._pending_upload is not None + assert mgr._pending_upload.paused is True + assert mgr.state.last_error == "auth" + entry.async_start_reauth.assert_called_once() + # A paused slot is not retried on the next wake. + assert mgr._has_pending_work() is False + + +@pytest.mark.asyncio +async def test_drain_config_resync_updates_runtime_and_cache(): + hass, entry, _ = _make_env() + device = MagicMock() + device.read_firmware_version = AsyncMock( + return_value={"major": 1, "minor": 2, "sha": "abc"} + ) + device.is_flex = False + device.landing_url = MagicMock(return_value="http://landing") + device.config = MagicMock() + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()), + patch.object(delivery_mod, "async_dispatcher_send"), + patch.object(delivery_mod, "async_ble_device_from_address", return_value=MagicMock()), + patch.object(delivery_mod, "OpenDisplayDevice", side_effect=_fake_device_ctx(device)), + patch("custom_components.opendisplay._write_cache") as write_cache, + ): + mgr = DeliveryManager(hass, entry) + mgr.request_config_resync() + await mgr._deliver() + + assert entry.runtime_data.config_resync_pending is False + assert mgr._pending_config_resync is False + assert entry.runtime_data.firmware == {"major": 1, "minor": 2, "sha": "abc"} + write_cache.assert_called_once() + + +@pytest.mark.asyncio +async def test_shutdown_unsubscribes_and_cancels_deadline(): + hass, entry, coordinator = _make_env() + unsub = MagicMock() + coordinator.async_subscribe_device_seen = MagicMock(return_value=unsub) + deadline_cancel = MagicMock() + with ( + patch.object(delivery_mod, "async_call_later", return_value=deadline_cancel), + patch.object(delivery_mod, "async_dispatcher_send"), + ): + mgr = DeliveryManager(hass, entry) + mgr.async_start() + _submit(mgr) + await mgr.async_shutdown() + + unsub.assert_called_once() + deadline_cancel.assert_called_once() + + +if __name__ == "__main__": + raise SystemExit(pytest.main([__file__, "-v"])) From 91daf421d292f208654c182cc9ade028004e816e Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Mon, 6 Jul 2026 20:17:41 -0400 Subject: [PATCH 04/10] feat: phase 3 - OTA gating and sleep diagnostics - update.py: an install requested while the tag is provably asleep fails fast with a translated device_sleeping_ota error telling the user to wake it and retry. The multi-connection AppLoader flash cannot be driven reliably inside a wake window, so this is the minimal-correct gate rather than queuing the OTA into the delivery drain (documented limitation). - diagnostics.py: report the resolved sleep profile, availability interval, config-resync flag, and delivery slot state (timestamps/attempts only, never queued image bytes). Reboot-edge rework (D7) and encrypted-device reauth-on-delivery (Phase 3) are implemented in __init__.py and delivery.py respectively, committed with Phase 2 to keep those modules import-consistent. Co-Authored-By: Claude Fable 5 --- custom_components/opendisplay/diagnostics.py | 24 ++++++++++++++++++++ custom_components/opendisplay/update.py | 19 ++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/custom_components/opendisplay/diagnostics.py b/custom_components/opendisplay/diagnostics.py index 0fd89c3..14a6b57 100644 --- a/custom_components/opendisplay/diagnostics.py +++ b/custom_components/opendisplay/diagnostics.py @@ -28,6 +28,29 @@ async def async_get_config_entry_diagnostics( """Return diagnostics for a config entry.""" runtime = entry.runtime_data fw = runtime.firmware + profile = runtime.sleep_profile + + sleep_diag: dict[str, Any] = { + "is_sleepy": profile.is_sleepy, + "deep_sleep_enabled": profile.deep_sleep_enabled, + "deep_sleep_time_seconds": profile.deep_sleep_time_seconds, + "sleep_timeout_ms": profile.sleep_timeout_ms, + "missed_cycles": profile.missed_cycles, + "queue_timeout_hours": profile.queue_timeout_hours, + "availability_interval_s": profile.availability_interval, + "config_resync_pending": runtime.config_resync_pending, + } + + # Delivery slot state only: timestamps/attempts, never queued image bytes. + if runtime.delivery is not None: + state = runtime.delivery.state + sleep_diag["delivery"] = { + "pending": state.pending, + "queued_at": state.queued_at, + "expires_at": state.expires_at, + "attempts": state.attempts, + "last_error": state.last_error, + } return { "firmware": { @@ -36,5 +59,6 @@ async def async_get_config_entry_diagnostics( "sha": fw["sha"], }, "is_flex": runtime.is_flex, + "sleep": sleep_diag, "device_config": async_redact_data(_asdict(runtime.device_config), TO_REDACT), } diff --git a/custom_components/opendisplay/update.py b/custom_components/opendisplay/update.py index daa5f92..e603132 100644 --- a/custom_components/opendisplay/update.py +++ b/custom_components/opendisplay/update.py @@ -26,6 +26,7 @@ from homeassistant.helpers.entity_platform import AddConfigEntryEntitiesCallback from . import OpenDisplayConfigEntry, _get_encryption_key +from .const import DOMAIN from .entity import OpenDisplayEntity _LOGGER = logging.getLogger(__name__) @@ -166,6 +167,24 @@ async def async_install(self, version: str | None, backup: bool, **kwargs: Any) if not tag: raise HomeAssistantError("No firmware version available to install") + # A deep-sleeping tag is dark most of the time and the multi-connection + # AppLoader flash (DFU trigger -> reconnect -> OTA) cannot be driven + # reliably inside a ~10 s wake window. Rather than start an install that + # will strand mid-flash, fail fast with clear guidance to wake it first. + runtime = self._entry.runtime_data + profile = runtime.sleep_profile + if profile.is_sleepy: + last_seen = ( + runtime.coordinator.data.last_seen + if runtime.coordinator.data + else None + ) + if profile.probably_asleep(last_seen): + raise HomeAssistantError( + translation_domain=DOMAIN, + translation_key="device_sleeping_ota", + ) + asset_name = firmware_ota_asset(self._ic_type, tag) if asset_name is None: raise HomeAssistantError( From 2f6c0be210acb8faa6c95fc4577d3464debad37c Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Wed, 8 Jul 2026 20:24:45 -0400 Subject: [PATCH 05/10] chore: pin py-opendisplay to fork feat/ble-speed for WNR testing Temporarily install py-opendisplay from the davelee98 fork's feat/ble-speed branch so the BLE write-without-response change for 0x71 image data can be tested before a PyPI release. Revert to a versioned pin (==7.12.0) once that release is published. Co-Authored-By: Claude Opus 4.8 (1M context) --- custom_components/opendisplay/manifest.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/custom_components/opendisplay/manifest.json b/custom_components/opendisplay/manifest.json index ade3dee..1445405 100644 --- a/custom_components/opendisplay/manifest.json +++ b/custom_components/opendisplay/manifest.json @@ -15,6 +15,6 @@ "iot_class": "local_push", "issue_tracker": "https://github.com/OpenDisplay/Home_Assistant_Integration/issues", "loggers": ["opendisplay"], - "requirements": ["py-opendisplay[silabs-ota]==7.11.1", "odl-renderer==0.5.10"], + "requirements": ["py-opendisplay[silabs-ota] @ git+https://github.com/davelee98/py-opendisplay.git@feat/ble-speed", "odl-renderer==0.5.10"], "version": "3.0.0-beta.7" } From 655ac16cdfae007c86c3dbb8d57467a7228041d5 Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Wed, 8 Jul 2026 21:21:29 -0400 Subject: [PATCH 06/10] feat: raise delivery deadline to 600s, fail loudly, cap retries at 5 The wake-drain deadline of 30s was too short for a large image over a slow BLE link (the device stays awake while connected, so it was HA's own timeout aborting mid-transfer). Raise DELIVERY_DEADLINE_S to 600s. On deadline timeout, split TimeoutError out of the quiet BLE-retry path: record the attempt, log at ERROR, and raise HomeAssistantError so it is not silently retried. py-opendisplay wraps its own read/connect timeouts as BLETimeoutError, so a bare TimeoutError here can only be the drain deadline firing. Add MAX_DELIVERY_ATTEMPTS=5: after five failed wakes, _give_up_upload drops the slot, cancels the expiry timer, sets last_error="failed", and fires content_expired (attempts==5 distinguishes it from time expiry) so an undeliverable frame stops retrying every wake until queue_timeout. Tests cover the deadline raise-and-count path and the max-attempts giveup. Co-Authored-By: Claude Opus 4.8 (1M context) --- custom_components/opendisplay/delivery.py | 81 +++++++++++++++++++++-- tests/test_delivery.py | 69 +++++++++++++++++++ 2 files changed, 143 insertions(+), 7 deletions(-) diff --git a/custom_components/opendisplay/delivery.py b/custom_components/opendisplay/delivery.py index da79a43..b7ffb62 100644 --- a/custom_components/opendisplay/delivery.py +++ b/custom_components/opendisplay/delivery.py @@ -38,6 +38,7 @@ from homeassistant.components.bluetooth import async_ble_device_from_address from homeassistant.core import CALLBACK_TYPE, HomeAssistant, callback +from homeassistant.exceptions import HomeAssistantError from homeassistant.helpers import device_registry as dr from homeassistant.helpers.device_registry import CONNECTION_BLUETOOTH from homeassistant.helpers.dispatcher import async_dispatcher_send @@ -57,8 +58,18 @@ _LOGGER = logging.getLogger(__name__) # Upper bound on a single wake's delivery attempt: connection establishment plus -# the queued work. Bounds one bad wake so it cannot overlap the next. -DELIVERY_DEADLINE_S = 30.0 +# the queued work. Bounds one bad wake so it cannot overlap the next. The device +# stays awake while the BLE link is held, so this is sized to let a full image +# stream over a slow link within one wake rather than to cap a healthy transfer. +# Exceeding it is therefore treated as a genuine failure (payload too large / link +# too slow) and surfaced loudly by ``_deliver`` instead of silently retried. +DELIVERY_DEADLINE_S = 600.0 + +# Give up a queued upload after this many failed wake attempts. Without this cap a +# frame that can never be delivered (unsupported payload, link too slow) would +# retry on every wake until ``queue_timeout`` hours later; instead it is dropped +# with a ``content_expired`` event so the failure surfaces promptly. +MAX_DELIVERY_ATTEMPTS = 5 # Sentinel distinguishing "no key" (None) from "malformed key" during resolve. _KEY_INVALID = object() @@ -243,7 +254,29 @@ async def _deliver(self) -> None: await self._drain_once() except asyncio.CancelledError: raise - except (BLEConnectionError, BLETimeoutError, TimeoutError) as err: + except TimeoutError as err: + # The whole drain exceeded DELIVERY_DEADLINE_S. Because the device + # stays awake while connected, hitting this ceiling is not a missed + # wake but a genuine failure: the payload is too large or the link too + # slow to complete in one wake. Record it, log loudly, and raise so it + # is not swallowed as a routine retry. (py-opendisplay wraps its own + # read/connect timeouts as BLETimeoutError, so a bare TimeoutError here + # can only be the asyncio.timeout(DELIVERY_DEADLINE_S) firing.) + self._register_attempt_failure( + f"delivery deadline exceeded ({DELIVERY_DEADLINE_S:.0f}s)" + ) + _LOGGER.error( + "%s: delivery aborted after exceeding the %.0f s deadline; the " + "image is likely too large to stream within a single wake", + self._address, + DELIVERY_DEADLINE_S, + ) + raise HomeAssistantError( + f"OpenDisplay {self._address}: delivery timed out after " + f"{DELIVERY_DEADLINE_S:.0f}s (image too large or BLE link too slow " + "to finish in one wake)" + ) from err + except (BLEConnectionError, BLETimeoutError) as err: # Device slept mid-connect or the wake window was missed; keep the # work queued and try again on the next wake. self._register_attempt_failure(str(err)) @@ -366,16 +399,50 @@ def _expire_upload(self, slot: PendingUpload) -> None: self._fire_content_event(EVENT_CONTENT_EXPIRED, slot) self._notify_state() + @callback + def _give_up_upload(self, slot: PendingUpload, reason: str) -> None: + """Drop an upload that has exhausted ``MAX_DELIVERY_ATTEMPTS``. + + Cancels the pending expiry timer, clears the slot, and fires the same + ``content_expired`` event as time-based expiry — the payload's + ``attempts == MAX_DELIVERY_ATTEMPTS`` distinguishes this cause, and the + entity's ``last_error`` reads ``"failed"`` rather than ``"expired"``. + """ + if slot.cancel_deadline: + slot.cancel_deadline() + slot.cancel_deadline = None + self._pending_upload = None + self._last_error = "failed" + _LOGGER.error( + "%s: giving up queued content after %s failed delivery attempts (%s)", + self._address, + slot.attempts, + reason, + ) + self._fire_content_event(EVENT_CONTENT_EXPIRED, slot) + self._notify_state() + # -- helpers ------------------------------------------------------------ def _register_attempt_failure(self, reason: str) -> None: - """Record a failed wake attempt and keep the work queued.""" - if self._pending_upload is not None: - self._pending_upload.attempts += 1 + """Record a failed wake attempt and keep the work queued. + + After ``MAX_DELIVERY_ATTEMPTS`` failures the upload is given up and + dropped (``_give_up_upload``) so a frame that can never be delivered does + not retry on every wake until ``queue_timeout``. + """ + upload = self._pending_upload + if upload is not None: + upload.attempts += 1 + if upload.attempts >= MAX_DELIVERY_ATTEMPTS: + self._give_up_upload(upload, reason) + return self._last_error = reason _LOGGER.debug( - "%s: delivery attempt failed (%s); will retry next wake", + "%s: delivery attempt %s/%s failed (%s); will retry next wake", self._address, + upload.attempts if upload is not None else 0, + MAX_DELIVERY_ATTEMPTS, reason, ) self._notify_state() diff --git a/tests/test_delivery.py b/tests/test_delivery.py index a572694..7d7437d 100644 --- a/tests/test_delivery.py +++ b/tests/test_delivery.py @@ -12,6 +12,8 @@ from opendisplay import AuthenticationFailedError, BLEConnectionError, RefreshMode import pytest +from homeassistant.exceptions import HomeAssistantError + from custom_components.opendisplay import delivery as delivery_mod from custom_components.opendisplay.const import ( EVENT_CONTENT_DELIVERED, @@ -230,6 +232,73 @@ async def test_drain_ble_failure_keeps_slot_and_counts_attempt(): assert mgr.state.attempts == 1 +@pytest.mark.asyncio +async def test_drain_deadline_timeout_raises_and_counts_attempt(): + """Exceeding DELIVERY_DEADLINE_S raises loudly but keeps the slot queued.""" + hass, entry, _ = _make_env() + with ( + patch.object(delivery_mod, "async_call_later", return_value=MagicMock()), + patch.object(delivery_mod, "async_dispatcher_send"), + patch.object(delivery_mod, "async_ble_device_from_address", return_value=MagicMock()), + patch.object( + delivery_mod, + "OpenDisplayDevice", + side_effect=_raising_device_ctx(TimeoutError()), + ), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr) + with pytest.raises(HomeAssistantError): + await mgr._deliver() + + # The slot is retained for the next wake and the failed attempt is recorded. + assert mgr.state.pending is True + assert mgr.state.attempts == 1 + assert "deadline exceeded" in (mgr.state.last_error or "") + # Background-task bookkeeping is still cleaned up despite the raise. + assert mgr._delivering is False + assert mgr._delivery_task is None + + +@pytest.mark.asyncio +async def test_drain_gives_up_after_max_attempts(): + """After MAX_DELIVERY_ATTEMPTS failures the slot is dropped with an expired event.""" + hass, entry, _ = _make_env() + deadline_cancel = MagicMock() + with ( + patch.object(delivery_mod, "async_call_later", return_value=deadline_cancel), + patch.object(delivery_mod, "async_dispatcher_send"), + patch.object(delivery_mod, "async_ble_device_from_address", return_value=MagicMock()), + patch.object( + delivery_mod, + "OpenDisplayDevice", + side_effect=_raising_device_ctx(BLEConnectionError("boom")), + ), + ): + mgr = DeliveryManager(hass, entry) + _submit(mgr) + for attempt in range(1, delivery_mod.MAX_DELIVERY_ATTEMPTS + 1): + await mgr._deliver() + if attempt < delivery_mod.MAX_DELIVERY_ATTEMPTS: + # Still queued and retriable until the cap is reached. + assert mgr.state.pending is True + assert mgr.state.attempts == attempt + + # On the capped attempt the slot is dropped and the expiry timer cancelled. + assert mgr.state.pending is False + assert mgr.state.last_error == "failed" + deadline_cancel.assert_called_once() + + # A content_expired event fired once, carrying attempts == the cap. + expired = [ + c.args[1] + for c in hass.bus.async_fire.call_args_list + if c.args[0] == EVENT_CONTENT_EXPIRED + ] + assert len(expired) == 1 + assert expired[0]["attempts"] == delivery_mod.MAX_DELIVERY_ATTEMPTS + + @pytest.mark.asyncio async def test_drain_auth_failure_pauses_and_starts_reauth(): hass, entry, _ = _make_env() From 8dd7b85eaae422b5addd8aed075d0b23b97b272d Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Thu, 9 Jul 2026 10:41:24 -0400 Subject: [PATCH 07/10] fix(sensor): source last_seen from the bluetooth stack, not the gated callback last_seen read coordinator.data.last_seen, a time.time() snapshot written only inside the de-dup/connectable-gated advertisement callback, so it froze while the device was still being seen. Read async_last_service_info(..., connectable=False).time instead -- the same _all_history source the Bluetooth advertisement monitor uses -- converted from the monotonic clock to wall time. Sensor-only; the coordinator OpenDisplayUpdate.last_seen field is unchanged (still used by services/update). Co-Authored-By: Claude Opus 4.8 (1M context) --- custom_components/opendisplay/sensor.py | 35 ++++++++++--- tests/test_last_seen.py | 65 +++++++++++++++++++++++++ 2 files changed, 94 insertions(+), 6 deletions(-) create mode 100644 tests/test_last_seen.py diff --git a/custom_components/opendisplay/sensor.py b/custom_components/opendisplay/sensor.py index 335f0a0..8107ed0 100644 --- a/custom_components/opendisplay/sensor.py +++ b/custom_components/opendisplay/sensor.py @@ -3,10 +3,12 @@ from collections.abc import Callable from dataclasses import dataclass from datetime import datetime, timezone +import time from opendisplay import voltage_to_percent from opendisplay.models.enums import CapacityEstimator, PowerMode +from homeassistant.components.bluetooth import async_last_service_info from homeassistant.components.sensor import ( SensorDeviceClass, SensorEntity, @@ -77,11 +79,9 @@ class OpenDisplaySensorEntityDescription(SensorEntityDescription): device_class=SensorDeviceClass.TIMESTAMP, entity_category=EntityCategory.DIAGNOSTIC, entity_registry_enabled_default=False, - value_fn=lambda upd: ( - datetime.fromtimestamp(upd.last_seen, tz=timezone.utc) - if upd.last_seen is not None - else None - ), + # native_value is overridden by OpenDisplayLastSeenSensor, so this value_fn + # is dead code; value_fn is a required field, hence the no-op. + value_fn=lambda _upd: None, ) @@ -116,7 +116,11 @@ async def async_setup_entry( ] async_add_entities( - OpenDisplaySensorEntity(coordinator, description) + ( + OpenDisplayLastSeenSensor(coordinator, description) + if description.key == "last_seen" + else OpenDisplaySensorEntity(coordinator, description) + ) for description in descriptions ) @@ -132,3 +136,22 @@ def native_value(self) -> float | int | str | datetime | None: if self.coordinator.data is None: return None return self.entity_description.value_fn(self.coordinator.data) + + +class OpenDisplayLastSeenSensor(OpenDisplaySensorEntity): + """last_seen sourced from the bluetooth stack, not the gated callback.""" + + @property + def native_value(self) -> datetime | None: + # connectable=False matches the Bluetooth advertisement monitor's + # "updated" field: _all_history, refreshed on every received advert + # before core's connectable/de-dup gates. + info = async_last_service_info( + self.hass, self.coordinator.address, connectable=False + ) + if info is None: + return None + # info.time is a monotonic clock (monotonic_time_coarse); convert to + # wall time with the same offset the advertisement monitor uses. + wall = info.time + (time.time() - time.monotonic()) + return datetime.fromtimestamp(wall, tz=timezone.utc) diff --git a/tests/test_last_seen.py b/tests/test_last_seen.py new file mode 100644 index 0000000..94af071 --- /dev/null +++ b/tests/test_last_seen.py @@ -0,0 +1,65 @@ +"""Unit tests for the last_seen sensor with a mocked bluetooth stack. + +These avoid the full Home Assistant test harness: ``async_last_service_info`` +(the sensor's only HA touchpoint) is patched in the sensor module namespace and +the entity's ``native_value`` property is asserted directly. +""" + +import time +from datetime import datetime, timezone +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +from custom_components.opendisplay import sensor as sensor_mod +from custom_components.opendisplay.sensor import ( + _LAST_SEEN_DESCRIPTION, + OpenDisplayLastSeenSensor, +) + +ADDRESS = "AA:BB:CC:DD:EE:FF" + + +def _make_sensor(): + """Return an OpenDisplayLastSeenSensor wired to mock coordinator/hass.""" + coordinator = MagicMock() + coordinator.address = ADDRESS + entity = OpenDisplayLastSeenSensor(coordinator, _LAST_SEEN_DESCRIPTION) + entity.hass = MagicMock() + return entity + + +def test_native_value_converts_monotonic_to_wall_time(): + entity = _make_sensor() + # info.time is a monotonic-clock reading, like habluetooth's _all_history. + mono = time.monotonic() + with patch.object( + sensor_mod, + "async_last_service_info", + return_value=SimpleNamespace(time=mono), + ) as mock_info: + value = entity.native_value + + # A monotonic "now" maps to wall-clock "now"; must be tz-aware for TIMESTAMP. + assert isinstance(value, datetime) + assert value.tzinfo is not None + expected = datetime.now(tz=timezone.utc) + assert abs((value - expected).total_seconds()) < 2 + + # The stack is queried with connectable=False to match the advertisement + # monitor's "updated" column (pre-gate _all_history), and for the coordinator + # address. + mock_info.assert_called_once() + assert mock_info.call_args.kwargs["connectable"] is False + assert ADDRESS in mock_info.call_args.args + + +def test_native_value_none_when_no_service_info(): + entity = _make_sensor() + with patch.object(sensor_mod, "async_last_service_info", return_value=None): + assert entity.native_value is None + + +if __name__ == "__main__": + import pytest + + raise SystemExit(pytest.main([__file__, "-v"])) From aa108bc7e6221da32f7bc4d58a40d5ac50edc6f9 Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Tue, 7 Jul 2026 20:26:22 -0400 Subject: [PATCH 08/10] feat: probe before queue for image sends to probably-asleep devices The freshness gate treated probably_asleep as authoritative and queued image sends without any connect attempt. That over-triggers: BLE adverts are lossy (a busy scanner/proxy can miss an entire 10 s wake window), and Silabs tags advertise continuously in EM2 while their power config reads as sleepy. Meanwhile HA retains a connectable BLEDevice ~3-5 min after the last advert, so a connect attempt stays possible long after the gate flips at ~15 s. Image sends now spend one short connect attempt (max_attempts=1, timeout=5 s) before queuing. A dark ESP32 (radio fully off in deep sleep) costs at most ~5 s vs the old doomed ~40 s budget the gate was built to avoid, and pays zero device battery; a probe that lands holds the tag awake and delivers live. A never-seen/long-gone tag has no connectable BLEDevice and short-circuits to the queue at near-zero cost. - New probe_before_queue option (default on) in the options flow, resolved via SleepProfile - _async_connect_and_run gains optional connect_timeout/max_attempts, threaded to OpenDisplayDevice; all other call sites keep defaults - Post-probe freshness re-check kicks notify_device_seen("post-probe") to close the advert-during-probe drain race - LED/buzzer and OTA gates unchanged (still fail fast) - New tests/test_services.py covering the send gate; D5 revision note Co-Authored-By: Claude Opus 4.8 (1M context) --- custom_components/opendisplay/config_flow.py | 6 + custom_components/opendisplay/const.py | 7 + custom_components/opendisplay/services.py | 77 +++++- custom_components/opendisplay/sleep.py | 8 + custom_components/opendisplay/strings.json | 6 +- .../opendisplay/translations/en.json | 6 +- ...EP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md | 4 + tests/test_services.py | 254 ++++++++++++++++++ tests/test_sleep.py | 9 + 9 files changed, 363 insertions(+), 14 deletions(-) create mode 100644 tests/test_services.py diff --git a/custom_components/opendisplay/config_flow.py b/custom_components/opendisplay/config_flow.py index 696b89d..cdfe9cb 100644 --- a/custom_components/opendisplay/config_flow.py +++ b/custom_components/opendisplay/config_flow.py @@ -29,6 +29,7 @@ from homeassistant.const import CONF_ADDRESS from homeassistant.core import callback from homeassistant.helpers.selector import ( + BooleanSelector, NumberSelector, NumberSelectorConfig, NumberSelectorMode, @@ -40,9 +41,11 @@ from .const import ( CONF_ENCRYPTION_KEY, CONF_MISSED_CYCLES, + CONF_PROBE_BEFORE_QUEUE, CONF_QUEUE_TIMEOUT_HOURS, CONF_SLEEP_MODE, DEFAULT_MISSED_CYCLES, + DEFAULT_PROBE_BEFORE_QUEUE, DEFAULT_QUEUE_TIMEOUT_HOURS, DEFAULT_SLEEP_MODE, DOMAIN, @@ -89,6 +92,9 @@ def _options_schema() -> vol.Schema: ), vol.Coerce(int), ), + vol.Required( + CONF_PROBE_BEFORE_QUEUE, default=DEFAULT_PROBE_BEFORE_QUEUE + ): BooleanSelector(), } ) diff --git a/custom_components/opendisplay/const.py b/custom_components/opendisplay/const.py index 92cb76d..ee9143f 100644 --- a/custom_components/opendisplay/const.py +++ b/custom_components/opendisplay/const.py @@ -28,6 +28,13 @@ CONF_QUEUE_TIMEOUT_HOURS = "queue_timeout_hours" DEFAULT_QUEUE_TIMEOUT_HOURS = 24 +# Probe a probably-asleep device with one short connect attempt before queuing +# an image send. Catches wake adverts the scanner missed and Silabs tags that +# advertise continuously despite a battery+deep-sleep power config. Bounded +# cost: one attempt with a short timeout (see services.PROBE_CONNECT_TIMEOUT_S). +CONF_PROBE_BEFORE_QUEUE = "probe_before_queue" +DEFAULT_PROBE_BEFORE_QUEUE = True + # --- Cache ------------------------------------------------------------------ # entry.data key holding the serialized device state used to set up the entry # without connecting when a sleepy device is dark at startup. diff --git a/custom_components/opendisplay/services.py b/custom_components/opendisplay/services.py index 4dfc4c3..d2dfcca 100644 --- a/custom_components/opendisplay/services.py +++ b/custom_components/opendisplay/services.py @@ -297,19 +297,37 @@ class _DeviceUnavailable(Exception): """ +# Probe budget for a probably-asleep tag: one connect attempt, short timeout. +# 5 s is >2x the observed 1-3 s connect-during-window latency (plus proxy +# slack), so a genuinely awake tag connects comfortably, while a dark ESP32 +# (radio fully off in timer deep sleep) costs at most ~5 s before queuing — +# vs the ~40 s default budget (4 attempts x 10 s). Deliberately below both the +# 10 s wake window and the 15 s freshness horizon (wake_window_s + +# FRESHNESS_SLACK_S), so a probe triggered by a just-missed advert still lands +# inside the window it is betting on. +PROBE_CONNECT_TIMEOUT_S = 5.0 +PROBE_MAX_ATTEMPTS = 1 + + async def _async_connect_and_run( hass: HomeAssistant, entry: "OpenDisplayConfigEntry", action: Callable[[OpenDisplayDevice], Awaitable[None]], use_measured_palettes: bool = False, reraise_ble: bool = False, + *, + connect_timeout: float | None = None, + max_attempts: int | None = None, ) -> None: """Resolve BLE device, open a connection, run action, handle auth errors. When ``reraise_ble`` is set, a missing connectable device or a BLE connect/timeout failure raises ``_DeviceUnavailable`` instead of a translated ``device_not_found``/``upload_error``, so the caller can defer - the work to the delivery queue. All other behavior is unchanged. + the work to the delivery queue. ``connect_timeout``/``max_attempts`` + override the library's connect budget (10 s x 4 attempts) — used by the + probe path to bound a likely-doomed attempt; ``None`` keeps the library + defaults. All other behavior is unchanged. """ address = entry.unique_id assert address is not None @@ -344,6 +362,12 @@ async def _async_connect_and_run( translation_domain=DOMAIN, translation_key="authentication_error" ) from err + device_kwargs: dict[str, Any] = {} + if connect_timeout is not None: + device_kwargs["timeout"] = connect_timeout + if max_attempts is not None: + device_kwargs["max_attempts"] = max_attempts + try: # Serialize all BLE access to this tag: the device has a single BLE link # and the library has no per-address lock, so overlapping connections @@ -357,6 +381,7 @@ async def _async_connect_and_run( config=entry.runtime_data.device_config, use_measured_palettes=use_measured_palettes, encryption_key=encryption_key, + **device_kwargs, ) as device: await action(device) except (AuthenticationFailedError, AuthenticationRequiredError) as err: @@ -458,15 +483,34 @@ def _queue() -> DeliveryReceipt: device_id=device_id, ) - # Freshness gate: a provably-asleep tag will not answer a live connect, so - # skip the doomed ~40 s retry cycle and queue directly for the next wake. + async def _upload(device: OpenDisplayDevice) -> None: + await device.upload_prepared_image( + prepared, refresh_mode=refresh_mode, state=state + ) + + # Freshness gate: a probably-asleep tag will not usually answer a live + # connect. Instead of queuing blind, spend one short connect attempt (the + # "probe") in case the tag is actually awake: its wake adverts may have + # been missed by the scanner, and Silabs tags advertise continuously even + # when their power config looks sleepy. A dark ESP32 costs at most + # ~PROBE_CONNECT_TIMEOUT_S before falling back to the queue. HA drops the + # connectable BLEDevice ~3-5 min after the last advert, so a long-dark or + # never-seen tag short-circuits to the queue at near-zero cost (no + # BLEDevice -> _DeviceUnavailable without any radio traffic). + probing = False + connect_kwargs: dict[str, Any] = {} if sleepy: - last_seen = runtime.coordinator.data.last_seen if runtime.coordinator.data else None + last_seen = ( + runtime.coordinator.data.last_seen if runtime.coordinator.data else None + ) if profile.probably_asleep(last_seen): - return _queue() - - async def _upload(device: OpenDisplayDevice) -> None: - await device.upload_prepared_image(prepared, refresh_mode=refresh_mode, state=state) + if not profile.probe_before_queue: + return _queue() + probing = True + connect_kwargs = { + "connect_timeout": PROBE_CONNECT_TIMEOUT_S, + "max_attempts": PROBE_MAX_ATTEMPTS, + } try: await _async_connect_and_run( @@ -475,10 +519,23 @@ async def _upload(device: OpenDisplayDevice) -> None: _upload, use_measured_palettes=use_measured_palettes, reraise_ble=sleepy, + **connect_kwargs, ) except _DeviceUnavailable: - # The device was seen recently but dropped/refused the link; defer. - return _queue() + # The device was dark, dropped, or refused the link; defer. + receipt = _queue() + if probing: + # Race: a wake advert arriving DURING the failed probe found no + # pending work (nothing queued yet), so no drain started. If the + # tag now looks fresh, kick a drain instead of waiting out a full + # sleep cycle. notify_device_seen is a no-op while one is running. + last_seen = ( + runtime.coordinator.data.last_seen if runtime.coordinator.data else None + ) + if not profile.probably_asleep(last_seen): + assert manager is not None + manager.notify_device_seen("post-probe") + return receipt async_dispatcher_send(hass, f"{SIGNAL_IMAGE_UPDATED}_{entry.unique_id}", jpeg) return DeliveryReceipt(status="delivered", expires_at=None) diff --git a/custom_components/opendisplay/sleep.py b/custom_components/opendisplay/sleep.py index e4e1276..81628d0 100644 --- a/custom_components/opendisplay/sleep.py +++ b/custom_components/opendisplay/sleep.py @@ -29,9 +29,11 @@ from .const import ( CONF_MISSED_CYCLES, + CONF_PROBE_BEFORE_QUEUE, CONF_QUEUE_TIMEOUT_HOURS, CONF_SLEEP_MODE, DEFAULT_MISSED_CYCLES, + DEFAULT_PROBE_BEFORE_QUEUE, DEFAULT_QUEUE_TIMEOUT_HOURS, DEFAULT_SLEEP_MODE, SLEEP_MODE_OFF, @@ -64,6 +66,7 @@ class SleepProfile: sleep_timeout_ms: int missed_cycles: int queue_timeout_hours: int + probe_before_queue: bool @property def wake_window_s(self) -> float: @@ -114,6 +117,7 @@ def create( deep_sleep_time_seconds: int, missed_cycles: int, queue_timeout_hours: int, + probe_before_queue: bool = DEFAULT_PROBE_BEFORE_QUEUE, ) -> SleepProfile: """Build a profile from primitive values (pure; unit-testable).""" deep_sleep_enabled = ( @@ -132,6 +136,7 @@ def create( sleep_timeout_ms=sleep_timeout_ms, missed_cycles=missed_cycles, queue_timeout_hours=queue_timeout_hours, + probe_before_queue=probe_before_queue, ) @classmethod @@ -150,4 +155,7 @@ def from_entry( queue_timeout_hours=options.get( CONF_QUEUE_TIMEOUT_HOURS, DEFAULT_QUEUE_TIMEOUT_HOURS ), + probe_before_queue=options.get( + CONF_PROBE_BEFORE_QUEUE, DEFAULT_PROBE_BEFORE_QUEUE + ), ) diff --git a/custom_components/opendisplay/strings.json b/custom_components/opendisplay/strings.json index 3602612..5d5fe25 100644 --- a/custom_components/opendisplay/strings.json +++ b/custom_components/opendisplay/strings.json @@ -56,12 +56,14 @@ "data": { "sleep_mode": "Deep sleep handling", "missed_cycles": "Missed wake cycles before unavailable", - "queue_timeout_hours": "Queued content timeout (hours)" + "queue_timeout_hours": "Queued content timeout (hours)", + "probe_before_queue": "Try connecting before queueing" }, "data_description": { "sleep_mode": "Automatic follows the device's own power configuration (battery power with deep sleep enabled). Force on or off to override.", "missed_cycles": "How many expected wake-ups the device may miss before its entities are marked unavailable.", - "queue_timeout_hours": "How long content queued for a sleeping device waits for a wake before it expires." + "queue_timeout_hours": "How long content queued for a sleeping device waits for a wake before it expires.", + "probe_before_queue": "Before queueing an image for a sleeping device, make one quick connection attempt in case it is actually awake (missed advertisements, always-on radios). Adds at most a few seconds when the device really is asleep." } } } diff --git a/custom_components/opendisplay/translations/en.json b/custom_components/opendisplay/translations/en.json index 57f4615..9243cbf 100644 --- a/custom_components/opendisplay/translations/en.json +++ b/custom_components/opendisplay/translations/en.json @@ -56,12 +56,14 @@ "data": { "sleep_mode": "Deep sleep handling", "missed_cycles": "Missed wake cycles before unavailable", - "queue_timeout_hours": "Queued content timeout (hours)" + "queue_timeout_hours": "Queued content timeout (hours)", + "probe_before_queue": "Try connecting before queueing" }, "data_description": { "sleep_mode": "Automatic follows the device's own power configuration (battery power with deep sleep enabled). Force on or off to override.", "missed_cycles": "How many expected wake-ups the device may miss before its entities are marked unavailable.", - "queue_timeout_hours": "How long content queued for a sleeping device waits for a wake before it expires." + "queue_timeout_hours": "How long content queued for a sleeping device waits for a wake before it expires.", + "probe_before_queue": "Before queueing an image for a sleeping device, make one quick connection attempt in case it is actually awake (missed advertisements, always-on radios). Adds at most a few seconds when the device really is asleep." } } } diff --git a/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md b/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md index f07401e..dfd51a6 100644 --- a/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md +++ b/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md @@ -141,6 +141,10 @@ This is precisely the tuning the user asked for regarding bleak dynamics: *when LED/buzzer services stay immediate-only (a notification that fires hours late is worse than an error); in sleep mode with the device dark they fail fast with a clear "device is sleeping" error. +**Revision 2026-07-07 — probe before queue.** Field review showed the freshness gate over-triggers: it treats `probably_asleep` as authoritative, but (a) BLE adverts are lossy — a busy scanner/proxy can miss an entire 10 s wake window, leaving `last_seen` stale while the tag is awake; (b) the Silabs firmware advertises continuously in EM2 and only enters true deep sleep (EM4) on explicit command, yet its power config (`power_mode==BATTERY`, `deep_sleep_time_seconds>0`) reads as sleepy, so such tags were queued while almost always reachable. Meanwhile HA retains a *connectable* `BLEDevice` for ~3–5 min after the last advert (habluetooth connectable-history pruning: scanner staleness ~195 s + 300 s sweep), so a connect attempt remains *possible* long after the gate flips at ~15 s. + +Image sends now spend **one short connect attempt** (the "probe": `max_attempts=1`, `timeout=5 s` — `services.PROBE_CONNECT_TIMEOUT_S`) before queuing, instead of queuing blind. Cost analysis: a dark ESP32 has its radio fully off in timer deep sleep (`Firmware/src/main.cpp:365-383`), so a doomed probe costs the *device* zero battery and HA at most ~5 s — vs the old doomed 4×10 s ≈ 40 s budget this gate was built to avoid. A probe that lands cancels the wake-window timer and holds the tag awake (`main.cpp:138-170, 233-244`), converting into a normal full-quality delivery. A long-dark or never-seen tag has no connectable `BLEDevice`, so the probe short-circuits to the queue at near-zero cost. The probe is opt-out via the `probe_before_queue` option (default on). A post-probe freshness re-check closes the race where a wake advert arriving *during* the failed probe found no pending work and started no drain: if the tag looks fresh after queuing, `notify_device_seen("post-probe")` kicks an immediate drain. The ESP32 wake window is a fixed 10 s from wake (only a *connection* extends wakefulness), which is why the probe timeout sits deliberately under it. LED/buzzer and OTA gates are unchanged (still fail-fast). + #### D6 — Representing queued content: the image entity + a pending sensor Per the user's instinct, the existing Display Content image entity is the queue's visible face: diff --git a/tests/test_services.py b/tests/test_services.py new file mode 100644 index 0000000..2947838 --- /dev/null +++ b/tests/test_services.py @@ -0,0 +1,254 @@ +"""Unit tests for the _async_send_image sleep gate and probe-before-queue. + +Mirrors the test_delivery.py approach: no Home Assistant test harness; the +service module's HA/BLE touchpoints (``prepare_image``, ``_pil_to_jpeg``, +``async_dispatcher_send``, ``async_ble_device_from_address`` and +``OpenDisplayDevice``) are patched in the services module namespace. +""" + +import asyncio +import time +from types import SimpleNamespace +from unittest.mock import AsyncMock, MagicMock, patch + +from opendisplay import BLEConnectionError, RefreshMode, DitherMode +from PIL import Image as PILImage +import pytest + +from custom_components.opendisplay import services as services_mod +from custom_components.opendisplay.delivery import DeliveryReceipt +from custom_components.opendisplay.services import ( + PROBE_CONNECT_TIMEOUT_S, + PROBE_MAX_ATTEMPTS, + _async_send_image, +) +from custom_components.opendisplay.sleep import SleepProfile + +ADDRESS = "AA:BB:CC:DD:EE:FF" + + +def _profile(**overrides): + params = { + "sleep_mode": "on", + "power_mode": 1, + "sleep_timeout_ms": 0, + "deep_sleep_time_seconds": 300, + "missed_cycles": 3, + "queue_timeout_hours": 24, + } + params.update(overrides) + return SleepProfile.create(**params) + + +def _make_env(profile=None, last_seen=None): + """Return (hass, entry, coordinator, manager) with a fake runtime.""" + profile = profile or _profile() + coordinator = MagicMock() + coordinator.data = SimpleNamespace(last_seen=last_seen) + manager = MagicMock() + manager.submit_upload = MagicMock( + return_value=DeliveryReceipt(status="queued", expires_at=123.0) + ) + manager.notify_device_seen = MagicMock() + runtime = SimpleNamespace( + coordinator=coordinator, + sleep_profile=profile, + delivery=manager, + device_config=None, + ble_lock=asyncio.Lock(), + partial_state=MagicMock(), + ) + entry = MagicMock() + entry.unique_id = ADDRESS + entry.runtime_data = runtime + entry.data = {} # no encryption key + hass = MagicMock() + + async def _executor_job(fn, *args): + return fn(*args) + + hass.async_add_executor_job = _executor_job + return hass, entry, coordinator, manager + + +def _device_ctx_factory(device=None, exc=None, on_enter=None): + """MagicMock construct-recording factory for OpenDisplayDevice. + + Produces an async context manager that yields ``device``, or raises + ``exc`` from ``__aenter__`` (after calling ``on_enter`` if given). + """ + + class _Ctx: + async def __aenter__(self): + if on_enter is not None: + on_enter() + if exc is not None: + raise exc + return device + + async def __aexit__(self, *exc_info): + return False + + return MagicMock(side_effect=lambda **kwargs: _Ctx()) + + +def _send(hass, entry): + img = PILImage.new("RGB", (1, 1)) + return _async_send_image( + hass, + entry, + img, + dither_mode=DitherMode.NONE, + refresh_mode=RefreshMode.FULL, + ) + + +def _patches(od_factory, ble_device="ble-device"): + """Common patch set for a _async_send_image drive.""" + return ( + patch.object( + services_mod, "prepare_image", return_value=(b"img", None, object()) + ), + patch.object(services_mod, "_pil_to_jpeg", return_value=b"jpeg"), + patch.object(services_mod, "async_dispatcher_send"), + patch.object( + services_mod, "async_ble_device_from_address", return_value=ble_device + ), + patch.object(services_mod, "OpenDisplayDevice", od_factory), + ) + + +@pytest.mark.asyncio +async def test_probe_success_delivers(): + """Stale device + probe on: one reduced-budget attempt that succeeds.""" + hass, entry, _, manager = _make_env(last_seen=None) # never seen -> asleep + device = MagicMock() + device.upload_prepared_image = AsyncMock() + od = _device_ctx_factory(device=device) + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "delivered" + od.assert_called_once() + kwargs = od.call_args.kwargs + assert kwargs["timeout"] == PROBE_CONNECT_TIMEOUT_S + assert kwargs["max_attempts"] == PROBE_MAX_ATTEMPTS + manager.submit_upload.assert_not_called() + + +@pytest.mark.asyncio +async def test_probe_failure_queues(): + """Stale device + probe on: the failed attempt falls back to the queue.""" + hass, entry, _, manager = _make_env(last_seen=None) + od = _device_ctx_factory(exc=BLEConnectionError("dark")) + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "queued" + od.assert_called_once() + manager.submit_upload.assert_called_once() + + +@pytest.mark.asyncio +async def test_probe_disabled_queues_immediately(): + """probe_before_queue=False restores the old queue-without-connect gate.""" + hass, entry, _, manager = _make_env( + profile=_profile(probe_before_queue=False), last_seen=None + ) + od = _device_ctx_factory() + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4 as ble_lookup, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "queued" + ble_lookup.assert_not_called() + od.assert_not_called() + manager.submit_upload.assert_called_once() + + +@pytest.mark.asyncio +async def test_not_sleepy_full_budget(): + """Non-sleepy devices keep the library's default connect budget.""" + hass, entry, _, manager = _make_env( + profile=_profile(sleep_mode="off"), last_seen=None + ) + device = MagicMock() + device.upload_prepared_image = AsyncMock() + od = _device_ctx_factory(device=device) + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "delivered" + kwargs = od.call_args.kwargs + assert "timeout" not in kwargs + assert "max_attempts" not in kwargs + manager.submit_upload.assert_not_called() + + +@pytest.mark.asyncio +async def test_probe_no_ble_device_queues_cheaply(): + """No connectable BLEDevice: the probe short-circuits to the queue.""" + hass, entry, _, manager = _make_env(last_seen=None) + od = _device_ctx_factory() + p1, p2, p3, p4, p5 = _patches(od, ble_device=None) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "queued" + od.assert_not_called() + manager.submit_upload.assert_called_once() + + +@pytest.mark.asyncio +async def test_post_probe_fresh_advert_triggers_drain(): + """A wake advert landing during the failed probe kicks an immediate drain.""" + hass, entry, coordinator, manager = _make_env(last_seen=None) + + def _advert_arrives(): + coordinator.data = SimpleNamespace(last_seen=time.time()) + + od = _device_ctx_factory( + exc=BLEConnectionError("dropped"), on_enter=_advert_arrives + ) + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "queued" + manager.notify_device_seen.assert_called_once_with("post-probe") + + +@pytest.mark.asyncio +async def test_post_probe_still_stale_no_drain_kick(): + """No advert during the failed probe: wait for the natural next wake.""" + hass, entry, _, manager = _make_env(last_seen=None) + od = _device_ctx_factory(exc=BLEConnectionError("dark")) + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "queued" + manager.notify_device_seen.assert_not_called() + + +@pytest.mark.asyncio +async def test_fresh_device_failure_no_probe_kwargs(): + """Sleepy but recently seen: full budget, queue on failure, no drain kick.""" + hass, entry, _, manager = _make_env(last_seen=time.time()) + od = _device_ctx_factory(exc=BLEConnectionError("dropped")) + p1, p2, p3, p4, p5 = _patches(od) + with p1, p2, p3, p4, p5: + receipt = await _send(hass, entry) + + assert receipt.status == "queued" + kwargs = od.call_args.kwargs + assert "timeout" not in kwargs + assert "max_attempts" not in kwargs + manager.notify_device_seen.assert_not_called() + + +if __name__ == "__main__": + raise SystemExit(pytest.main([__file__, "-v"])) diff --git a/tests/test_sleep.py b/tests/test_sleep.py index 0a344de..91dc28c 100644 --- a/tests/test_sleep.py +++ b/tests/test_sleep.py @@ -87,6 +87,15 @@ def test_probably_asleep_stale_advert_is_true(): assert profile.probably_asleep(stale, now=now) is True +def test_probe_before_queue_defaults_true(): + # Default also proves create() keeps working for callers that omit it. + assert _profile().probe_before_queue is True + + +def test_probe_before_queue_override(): + assert _profile(probe_before_queue=False).probe_before_queue is False + + def test_probably_asleep_boundary(): profile = _profile(sleep_timeout_ms=10000) now = 1_000_000.0 From ebfc7918fb0dc94be0b2d903ab2c09058668d396 Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Thu, 9 Jul 2026 22:14:55 -0400 Subject: [PATCH 09/10] fix: remove space from py-opendisplay requirement for hassfest Co-Authored-By: Claude Opus 4.8 (1M context) --- custom_components/opendisplay/manifest.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/custom_components/opendisplay/manifest.json b/custom_components/opendisplay/manifest.json index 1445405..ebfe322 100644 --- a/custom_components/opendisplay/manifest.json +++ b/custom_components/opendisplay/manifest.json @@ -15,6 +15,6 @@ "iot_class": "local_push", "issue_tracker": "https://github.com/OpenDisplay/Home_Assistant_Integration/issues", "loggers": ["opendisplay"], - "requirements": ["py-opendisplay[silabs-ota] @ git+https://github.com/davelee98/py-opendisplay.git@feat/ble-speed", "odl-renderer==0.5.10"], + "requirements": ["py-opendisplay[silabs-ota]@git+https://github.com/davelee98/py-opendisplay.git@feat/ble-speed", "odl-renderer==0.5.10"], "version": "3.0.0-beta.7" } From 1b7605ec6d56d54af364f03eda0cdcc13e04725b Mon Sep 17 00:00:00 2001 From: David Lee <247393336+davelee98@users.noreply.github.com> Date: Sat, 11 Jul 2026 01:07:50 -0400 Subject: [PATCH 10/10] docs: drop deep-sleep investigative notes from PR These were internal investigation/planning notes, not user-facing docs. Co-Authored-By: Claude Fable 5 --- docs/DEEP_SLEEP_FINDINGS_2026-07-06.md | 172 ---------- ...EP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md | 319 ------------------ 2 files changed, 491 deletions(-) delete mode 100644 docs/DEEP_SLEEP_FINDINGS_2026-07-06.md delete mode 100644 docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md diff --git a/docs/DEEP_SLEEP_FINDINGS_2026-07-06.md b/docs/DEEP_SLEEP_FINDINGS_2026-07-06.md deleted file mode 100644 index a5f71e7..0000000 --- a/docs/DEEP_SLEEP_FINDINGS_2026-07-06.md +++ /dev/null @@ -1,172 +0,0 @@ -# OpenDisplay Deep Sleep — Cross-Stack Findings - -*Investigation of deep sleep behavior across Firmware (ESP32/nRF), Firmware_Silabs, py-opendisplay, and the Home Assistant integration — 2026-07-06* - -## TL;DR - -- **The ESP32 firmware's deep-sleep loop itself is sound** (timer wake, ~10 s advertising window, first-boot grace period, panel power-down). -- **Bug: `rebootFlag` is not persisted across deep sleep**, so every wake advertises "I rebooted". Combined with the HA integration's reload-on-reboot behavior, this can produce an intermittent **reload → connect → sleep → wake → reload churn loop** that drains battery and spams HA. -- **Gap: HA has no wake-window awareness.** Service calls connect immediately and fail if the device is mid-sleep; nothing waits for the next wake advertisement. Uploads to a sleeping device succeed only by luck. -- **Gap: a commanded reboot leaves almost no reconnect window** — the device re-enters deep sleep within ~1 s because the first-boot grace period is skipped. -- **Gap: py-opendisplay does not implement the deep-sleep command (0x0052)**, so neither HA nor the CLI can put a device to sleep, even though ESP32 and Silabs firmware both handle it. - -## How deep sleep works today - -### ESP32 (reference firmware) - -Deep sleep engages when `power_option.power_mode == 1` (BATTERY) **and** -`deep_sleep_time_seconds > 0`. - -- **Entry** — `enterDeepSleep()` (`Firmware/src/main.cpp:275`): stops advertising, deinits BLE, arms **timer wakeup only** (`esp_sleep_enable_timer_wakeup`, `main.cpp:301`), holds the power latch, calls `esp_deep_sleep_start()`. The loop enters it whenever `bleActive` is false (no connection, empty command/response queues, no EPD refresh, no WiFi LAN session). -- **Wake** — deep-sleep wake is a full CPU boot. `setup()` detects the wake cause and runs `minimalSetup()` (`main.cpp:245`): config init, IO init, BLE advertising only — no display init, no WiFi, no boot screen. -- **Wake window** — the device advertises for `sleep_timeout_ms` (default **10 s** when 0, `main.cpp:94-96`). If nothing connects, it returns to sleep. If a central connects, `fullSetupAfterConnection()` (`main.cpp:256`) brings up WiFi and the panel driver state, and queued BLE commands are processed on subsequent loop passes. -- **First boot** — a 2-minute grace period before the first deep sleep (`FIRST_BOOT_DEEP_SLEEP_DELAY_MS`, `main.cpp:180-196`) gives the user time to adopt/configure. Applies only when `deep_sleep_count == 0`. -- **State across sleep** — `woke_from_deep_sleep`, `deep_sleep_count`, `displayed_etag` are `RTC_DATA_ATTR` (`Firmware/src/main.h:285-294`) and survive. Ordinary globals (including `rebootFlag`) reset every wake. - -### nRF - -No deep sleep. `sleep_timeout_ms` is used as an idle-loop delay; the SoftDevice's -System-ON idle is the low-power state. The 0x0052 command replies -"not supported" (`Firmware/src/device_control.cpp:703`). - -### Silabs (Flex) - -Deep sleep is **command-driven only**: 0x0052 sets `s_pending_deep_sleep`; after the -BLE connection closes, the firmware powers off the panel, arms **EM4 wake on button -and NFC field-detect**, and calls `EMU_EnterEM4()` -(`Firmware_Silabs/opendisplay_ble.c:1921-1926`). There is **no timer wake** — -`deep_sleep_time_seconds` exists in the Silabs config struct but is unused. A -sleeping Flex device stays down until a button press or NFC field. - -### py-opendisplay - -Serializes/parses the power config fields (`power_mode`, `sleep_timeout_ms`, -`deep_sleep_time_seconds`, `deep_sleep_current_ua`) but **exposes no deep-sleep -command** — only `reboot()` (0x000F). Connections go through bleak-retry-connector -(4 attempts × 10 s timeout, plus one GATT-cache-clear retry, -`py-opendisplay/src/opendisplay/transport/connection.py`). - -### Home Assistant integration - -- Monitoring is fully passive: `OpenDisplayCoordinator` is a - `PassiveBluetoothDataUpdateCoordinator` fed by advertisements only. -- Connections happen for: entry setup (firmware/config interrogation, - `__init__.py:104`), service calls (`services.py:280` `_async_connect_and_run`), - and OTA (`update.py`). -- The coordinator watches the advertised **reboot flag** (bit 1 of the status - byte) and, on a False → True edge, **reloads the config entry** — which opens a - new BLE connection to re-read firmware + config - (`coordinator.py:129-150`, `__init__.py:180-195`). - -## Bug: deep-sleep wake is indistinguishable from a reboot - -`rebootFlag` is a plain global initialized to 1 (`Firmware/src/main.h:126`), -unlike its RTC-persisted neighbors. Since deep-sleep wake is a full boot, **every -wake re-advertises `reboot_flag = 1`**. - -The churn loop: - -1. HA connects (setup, service call, or a previous iteration of this loop) → - firmware clears the flag on connect (`Firmware/src/esp32_ble_callbacks.h:42`). -2. On disconnect, advertising restarts with flag = 0 - (`esp32_restart_ble_advertising` → `updatemsdata`, - `Firmware/src/ble_init.cpp:165`), but only for a sub-second burst before the - loop re-enters deep sleep. -3. If HA's scanner catches one of those flag-0 adverts, `_last_reboot_flag` - becomes False. -4. Next wake advertises flag = 1 → False → True edge → HA reloads the entry → - connects → interrogates → clears flag → device sleeps → **go to 2**. - -Each cycle also runs `initWiFi()` on the device (`fullSetupAfterConnection()`), -compounding the battery cost. Because step 3 depends on catching a brief -advertising burst, the loop is **intermittent** — it presents as unexplained -battery drain and periodic "Device rebooted since last connection" log lines at -the deep-sleep cadence. - -The coordinator's edge detection is otherwise well designed: None → True is -ignored (setup already synced) and a flag stuck at True self-guards. The defect -is purely that the firmware loses the flag across sleep. - -**Suggested fix (firmware, ~10 lines):** mirror `rebootFlag` into an -`RTC_DATA_ATTR` variable in `enterDeepSleep()` and restore it in `setup()` on a -deep-sleep wake. Power-on reset clears RTC RAM, so genuine cold boots still -advertise 1. The reboot command path (`esp_restart()` preserves RTC RAM) must -explicitly set the mirror back to 1 before restarting. - -## Gaps - -### 1. HA cannot reliably reach a sleeping device - -`_async_connect_and_run` (`services.py:289`) resolves the cached `BLEDevice` and -connects immediately. Against a device sleeping e.g. 5 minutes with a 10 s wake -window, an arbitrary service call has a few-percent success rate; failures raise -`upload_error` and the image is dropped (no queue, no retry-at-next-wake). -The integration already has everything needed to do better: the passive -coordinator sees every wake advertisement, and `entry.runtime_data.device_config` -contains the power settings. - -**Suggested fix (integration):** a "wait for next advertisement, then connect" -helper used by services, setup retry, and OTA when the device config indicates -battery + deep sleep. This is the standard ESL/e-ink hub pattern (queue content, -deliver at check-in). - -### 2. Commanded reboot → near-instant sleep - -`deep_sleep_count` survives `esp_restart()` (RTC RAM persists across software -resets), so after a reboot command the first-boot condition -(`main.cpp:180`, requires `deep_sleep_count == 0`) is false and the device -re-enters deep sleep within about a second. HA's reboot-triggered reload then -usually misses, lands in `ConfigEntryNotReady`, and the entities sit unavailable -until a timer-backoff retry happens to coincide with a wake window. - -**Suggested fix (firmware):** grant a short (e.g. 30–60 s) advertising grace -period after any normal boot, not just the very first one. - -### 3. Availability flapping on long sleep intervals - -Entity availability tracks advertisement freshness. If -`deep_sleep_time_seconds` exceeds HA Bluetooth's stale-advertisement timeout -(on the order of a few minutes), every sleep cycle marks all entities -unavailable and the next wake marks them available again — even though the -device is healthy. Setup retries are timer-based only; nothing retries -immediately when the device reappears. - -### 4. py-opendisplay lacks the deep-sleep command - -Firmware handles 0x0052 on ESP32 (`Firmware/src/device_control.cpp:691`) and -Silabs, but the library exposes only `reboot()`. HA therefore has no way to -command a device to sleep (relevant for Flex devices, where sleep is -command-driven). - -## Design notes / quirks - -- `sleep_timeout_ms` has two meanings: post-wake **advertising window** on ESP32 - (`main.cpp:94`), idle-loop delay on nRF. Default window is 10 s when the field - is 0. -- `connectionRequested` (bit 2 of the advertised status byte, - `Firmware/src/main.h:127`) is reserved and unused — a natural hook for a - future "content pending / stay awake" handshake between HA and the device. -- Wake-on-BLE from true deep sleep is **not possible on ESP32** — the radio is - off; wake sources are timer/GPIO/touch/ULP only. The near-equivalent (light - sleep + BT modem sleep, always connectable) costs ~0.5–2 mA average vs - ~10–20 µA in deep sleep. nRF52/EFR32 keep the radio scheduler alive at µA - levels in their idle states, which is why those targets don't need this - machinery. A cheap firmware improvement: also arm `esp_sleep_enable_ext1_wakeup` - on the button pin so a user can wake the tag on demand (the Silabs target - already wakes from EM4 on button/NFC). - -## What was verified to work - -- Wake window logic is correctly guarded; queued commands received during the - wake window are processed after `fullSetupAfterConnection()`. -- Panel power is managed per-refresh (`pwrmgm(false)` after refresh), so the - display rail is already off at sleep entry; external flash is parked - (CS high, CLK/MOSI low). -- First-boot 2-minute adoption grace period works (power-on resets clear - `deep_sleep_count`). -- The HA coordinator's reboot edge detection ignores the initial observation and - a permanently-set flag; the reload defers until any in-progress upload - finishes (`__init__.py:190-195`). -- Upload concurrency is last-writer-wins: a new upload cancels the in-flight one - (`services.py:396-401`) — appropriate for a display where only the latest - image matters. diff --git a/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md b/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md deleted file mode 100644 index dfd51a6..0000000 --- a/docs/DEEP_SLEEP_IMPLEMENTATION_PLAN_2026-07-06.md +++ /dev/null @@ -1,319 +0,0 @@ -# OpenDisplay Deep Sleep — Architecture & Implementation Plan - -*2026-07-06 — Companion to [DEEP_SLEEP_FINDINGS_2026-07-06.md](DEEP_SLEEP_FINDINGS_2026-07-06.md) (cross-stack behavioral findings). This document is the forward-looking design: current state, architectural considerations, component factoring, and a detailed implementation plan. Scope: ESP32 firmware variant, with changes focused on the Home Assistant integration; py-opendisplay changes where necessary; firmware changes minimized and listed separately.* - ---- - -## 1. Current state of the integration and components - -### 1.1 The device side (ESP32 firmware) — what the integration must accommodate - -Validated against `Firmware/src` at HEAD: - -| Behavior | Detail | Source | -|---|---|---| -| Sleep entry condition | `power_mode == 1` (BATTERY) **and** `deep_sleep_time_seconds > 0`; entered whenever BLE is idle | `main.cpp:180-198` | -| Sleep mechanics | Advertising stopped, BLE deinitialized, **timer wakeup only**, power latch held, `esp_deep_sleep_start()` | `main.cpp:275-307` | -| Wake behavior | Full CPU boot → `minimalSetup()` (config + IO + BLE advertising; no display init, no WiFi) | `main.cpp:46-51, 245` | -| Wake window | Advertises for `sleep_timeout_ms` (default **10 s** when 0); returns to sleep if nothing connects | `main.cpp:85-101` | -| On connect | `fullSetupAfterConnection()` brings up WiFi + panel driver; device stays awake while a central is connected (`bleActive`) | `main.cpp:85-90, 256` | -| First boot | 2-minute grace period before first sleep (`deep_sleep_count == 0` only) — the adoption window | `main.cpp:180-196` | -| State across sleep | `woke_from_deep_sleep`, `deep_sleep_count`, `displayed_etag` are `RTC_DATA_ATTR` and survive; `rebootFlag` does **not** (known bug — every wake advertises "rebooted") | `main.h:126, 286-295` | -| Sleep interval range | `deep_sleep_time_seconds` is uint16 → 1 s to ~18.2 h | `models/config.py:207` | -| Command 0x0052 | "Deep sleep now" handled by ESP32 and Silabs firmware; in the official protocol spec | `device_control.cpp:692`, opendisplay.org `protocol/ble-flow.html` | - -Key consequence: **a sleeping ESP32 device is completely dark** — no radio, not connectable, not scannable. The only contact opportunity is the ~10 s advertising window after each timer wake. Once a central connects inside that window, the device stays awake for the whole session, so a connection established at wake time can run arbitrarily long work (uploads, OTA). - -### 1.2 py-opendisplay - -- **Connection layer**: `bleak-retry-connector` with `max_attempts=4`, `timeout=10 s`, plus one GATT-cache-clear retry (`transport/connection.py:33-117`). Both knobs are already exposed on `OpenDisplayDevice(timeout=…, max_attempts=…)` (`device.py:390-392`). -- **Power config**: parses/serializes `power_mode`, `sleep_timeout_ms`, `deep_sleep_time_seconds`, `deep_sleep_current_ua` (`models/config.py:196-256`). No convenience "is deep sleep enabled" predicate. -- **Config serialization**: `config_to_json()` / `config_from_json()` round-trip a full `GlobalConfig` (`models/config_json.py:68, 434`) — ready to use for caching device config in the HA config entry. -- **Advertisement model**: parses the status byte, exposing `reboot_flag` (bit 1) and the reserved `connection_requested` (bit 2) (`models/advertisement.py:384-394`). -- **Gap**: no deep-sleep command — only `reboot()` (0x000F, `device.py:912`). 0x0052 is unimplemented. - -### 1.3 Home Assistant integration (`custom_components/opendisplay`) - -- **Monitoring is fully passive.** `OpenDisplayCoordinator` is a `PassiveBluetoothDataUpdateCoordinator` fed only by advertisements (`coordinator.py:40-51`). Sensors (battery, temperature, RSSI, last-seen) never connect. -- **Connections happen at exactly three points:** - 1. **Entry setup** — `async_setup_entry` requires a connectable `BLEDevice` *and* a successful connect to read firmware + config; otherwise raises `ConfigEntryNotReady` (`__init__.py:97-128`). - 2. **Service calls** — `upload_image`, `drawcustom`, `activate_led`, `activate_buzzer` all go through `_async_connect_and_run`, which resolves the cached `BLEDevice` and connects immediately (`services.py:283-346`). Failure raises `upload_error` and **the content is dropped** — no queue, no retry. - 3. **OTA** — `update.py` connects directly for version checks and flashing. -- **Concurrency primitives already in place** (reused by this design): - - Per-entry `ble_lock` serializing all BLE access to one tag (`__init__.py:58`). - - Latest-wins upload semantics — a new upload cancels the in-flight one (`services.py:444-450`). - - `partial_state` tracking the last-uploaded frame for differential partial refresh (`__init__.py:62`); the panel-side `displayed_etag` survives deep sleep, so partial refresh remains valid across sleep cycles. -- **Reboot handling**: coordinator detects a False→True edge on the advertised reboot flag and reloads the entire config entry, which reconnects (`coordinator.py:129-150`, `__init__.py:190-210`). Combined with the firmware's unpersisted `rebootFlag`, this is the churn-loop hazard documented in the findings doc. -- **Image entity** (`image.py`) shows the last *successfully delivered* frame, updated via dispatcher signal after upload completes (`services.py:409`). This is the natural place to represent queued-but-unsent content. -- **Config flow**: bluetooth discovery + user flow + encryption key + reauth. **No options flow exists** (`config_flow.py`). -- **`const.py`** holds only `DOMAIN`, `CONF_ENCRYPTION_KEY`, and one dispatcher signal — no option constants yet. - -### 1.4 What HA core already provides (validated against `core` checkout, 2026.6 dev) - -These built-in mechanisms shape the design — some help, some actively interfere: - -1. **Setup retry with exponential backoff.** `ConfigEntryNotReady` → `SETUP_RETRY` state, retried at `min(2^tries × 5 s, 600 s)` (`config_entries.py:843`). Blind timer retries against a device sleeping N minutes with a 10 s window have a per-try success probability of roughly `10/N·60` — a few percent. **Timer-based retry alone is not a solution.** -2. **Advertisement-triggered reload of entries in SETUP_RETRY.** When a Bluetooth *discovery flow* fires for an already-configured entry in `SETUP_RETRY`, HA schedules an immediate reload (`config_entries.py:3158-3169`). This is the built-in "retry when the device reappears" path — **but** it only fires if the integration matcher re-triggers discovery, which only happens after the address *disappears* from the scanner history (`bluetooth/manager.py:174-176`) — i.e. after the device has been unavailable long enough to be expired. It is timing-dependent and races the 10 s wake window (reload → connect must land inside the window; scheduled reload usually does, since it fires on the wake advertisement itself). It works *sometimes* today; it is not something to build on, and extending the availability window (below) deliberately disables it. -3. **Availability tracking with a configurable horizon.** Devices are marked unavailable when advertisements go stale (fallback ~5 min, `UNAVAILABLE_TRACK_SECONDS = 300`). Critically, HA exposes **`async_set_fallback_availability_interval(hass, address, seconds)`** (`bluetooth/api.py:297`) — a per-address override of the staleness horizon. This is the sanctioned lever to keep a deep-sleeping device "available" between wakes. (The *learned* advertising-interval mechanism needs many consecutive adverts and will never learn a sleep cycle; the fallback interval is the right tool.) -4. **Advertisement callbacks.** `async_register_callback` (per-address advertisement callback) and `async_process_advertisements` (await-next-matching-advert with timeout) (`bluetooth/api.py:138, 165`). The coordinator already receives every advertisement via `_async_handle_bluetooth_event` — no new registration is needed for the wake trigger; a hook on the existing coordinator suffices. -5. **Service response data** (`SupportsResponse.OPTIONAL`) — lets `upload_image`/`drawcustom` report "delivered" vs "queued" to automations without blocking. - ---- - -## 2. Architectural considerations - -### 2.1 Design principles - -**P1 — Asleep is a state, not an error.** For a device configured for deep sleep, unreachability is its normal operating condition ~99% of the time. Every code path that today treats "can't connect" as failure must, in sleep mode, treat it as "defer". - -**P2 — The advertisement is the only reliable rendezvous.** All communication with a sleeping device must be initiated within seconds of seeing a wake advertisement. Everything else (setup, delivery, OTA) is architected as *work queued until the next rendezvous*. - -**P3 — Never connect without a reason.** Each connection forces the device through `fullSetupAfterConnection()` (including WiFi init) and holds it awake — battery cost is dominated by connected time, not advertising. The integration must connect only when it has pending work, and must batch all pending work into a single connection per wake. - -**P4 — Transport-agnostic trigger.** The rendezvous trigger is "device seen on any transport". Today that is a BLE advertisement; the future WiFi implementation (see [WIFI_ARCHITECTURE_2026-07-06.md](WIFI_ARCHITECTURE_2026-07-06.md)) delivers the same trigger from mDNS. The delivery machinery must depend on a *device-seen event*, not on BLE specifics. - -**P5 — Minimize firmware changes.** Everything below works against today's firmware. Two small firmware fixes are recommended (§5.4) but nothing depends on them. - -### 2.2 Key design decisions - -#### D1 — How sleep mode is determined - -Auto-detect from the device's own config, already held in `entry.runtime_data.device_config`: - -``` -sleepy = power_option.power_mode == PowerMode.BATTERY (1) - and power_option.deep_sleep_time_seconds > 0 -``` - -This exactly mirrors the firmware's own sleep-entry condition (`main.cpp:198`), so integration and device can never disagree. An options-flow override (`auto` / `force on` / `force off`, default `auto`) covers edge cases (e.g. Silabs Flex devices whose sleep is command-driven, or a user who wants strict-failure semantics). - -#### D2 — Setup from cache: the entry must load without the device - -**Problem (user scenario 1):** device asleep at HA startup → `async_setup_entry` can't connect → `ConfigEntryNotReady` → entities gone, timer retries rarely coincide with a wake window, entry effectively dead until luck strikes. - -**Decision:** cache everything setup needs — serialized `GlobalConfig` (via `config_to_json`), firmware version, `is_flex` — in `entry.data` after every successful interrogation. On subsequent setups, when the device is not immediately reachable **and** the cache says it is a sleepy device, **set up entirely from cache**: build runtime data, register device info, start the passive coordinator, forward platforms — no connection at all. Mark a `config_resync_pending` flag; the delivery manager (D4) re-reads firmware/config opportunistically at the next wake and updates the cache. - -Rationale for caching over smarter retries: -- It removes the race entirely instead of trying to win it. HA startup, HA restarts, and reloads all become instant and deterministic. -- The built-in rediscovery-reload path (§1.4-2) is disabled anyway once we extend the availability interval (D3): the address never expires, the matcher never resets, discovery never re-fires. Caching replaces a mechanism this design would otherwise silently break. -- It matches how HA treats other sleepy-device ecosystems (Zigbee ESLs, ESPHome deep-sleep nodes): entities exist and hold state; freshness is communicated via availability and `last_seen`. - -First-time adoption still requires a live connection — acceptable, because the config flow runs during the firmware's 2-minute first-boot grace period, and a user adopting a device has it awake by definition. If setup finds no cache and cannot connect, `ConfigEntryNotReady` remains correct. - -Cache invalidation: rewritten after every successful config read (setup, post-reboot resync, post-wake resync). Reauth clears nothing (the key lives separately in `entry.data`). - -#### D3 — Availability policy: entities stay available across sleep - -**Problem (user scenario 1a):** with the default ~5 min staleness horizon, any sleep interval > ~4 min flaps every entity unavailable/available once per cycle. - -**Decision:** when sleep mode is active, call -`async_set_fallback_availability_interval(hass, address, deep_sleep_time_seconds × missed_cycles + wake_window + 60 s)` -at setup and whenever the config changes. `missed_cycles` is an options-flow setting (default **3**): the device is marked unavailable only after missing three consecutive expected wakes. Examples: 5 min sleep → unavailable after ~16 min of silence; 12 h sleep → ~36 h. This directly implements the user requirement "leave the entry alive until some defined period of unavailability (hours or days)" — with the period derived from the device's own cadence rather than a wall-clock guess, and clamped by an absolute options override for users who want a fixed horizon. - -The `last_seen` sensor already reports true freshness, so nothing is hidden from the user. Availability now means "checking in on schedule" instead of "advertising right now" — the correct semantic for a sleepy device. - -#### D4 — Wake-triggered delivery: queue work, deliver at the rendezvous - -**Problem (user scenario 2):** sending to a sleeping device fails after ~40 s of bleak retries and drops the content. - -**Decision:** introduce a per-entry **delivery manager** owning *pending work slots*, triggered by the coordinator's advertisement handler: - -- **Pending slots, latest-wins per type** (consistent with existing upload semantics): - - `pending_upload`: the *prepared* image (post dither/encode/compress — CPU work done at queue time, once), refresh mode, partial state reference, plus the preview JPEG, `created_at`, `expires_at`, `attempts`. - - `pending_config_resync`: flag — re-read firmware + config, refresh cache (set by D2 cache-setup and D6 reboot handling). - - `pending_ota`: deferred to the OTA flow itself (§4, Phase 3); the slot exists so a wake can resume a user-requested update. -- **Trigger:** `OpenDisplayCoordinator._async_handle_bluetooth_event` already fires on every advertisement. Add a `async_subscribe_device_seen` hook (mirroring the existing `async_subscribe_reboot` pattern, `coordinator.py:62-74`). The delivery manager subscribes; on device-seen with pending work and no delivery in flight, it starts a delivery task **immediately** — every millisecond of the 10 s window counts. -- **Single connection per wake (P3):** the delivery task acquires `ble_lock`, opens one `OpenDisplayDevice` session, and drains *all* pending slots in priority order: upload first (user-visible), then config resync (cheap reads on the already-open link), then OTA. The device stays awake while connected, so the window only constrains connection establishment, not the work. -- **Failure handling:** if a delivery attempt fails (device slept mid-connect, interference), the work stays queued for the next wake; `attempts` increments. A **deadline timer** (options: `queue_timeout`, default **24 h** — safely above the 18 h max sleep interval) expires the slot: fire `opendisplay_content_expired` event, log a warning, clear the pending flag. This is the user's "backup timer / failure code". -- **Transport-agnostic (P4):** the manager's entry point is `async_device_seen(source: str)`. The BLE coordinator calls it with `"ble"`; a future WiFi presence tracker calls it with `"mdns"`. Nothing else changes. - -#### D5 — Service-call semantics on a sleeping device - -`upload_image` / `drawcustom` flow becomes: - -1. Render + `prepare_image` immediately (unchanged — CPU work is front-loaded and reused on every retry). -2. **Freshness gate:** if sleep mode is active and the last advertisement is older than `sleep_timeout_ms + slack (~5 s)`, the device is provably asleep — **skip the doomed ~40 s bleak retry cycle** and queue directly. If the device was seen within the window (possibly still awake), attempt an immediate send as today. -3. On immediate-send success → done, exactly as today. -4. On `BLEConnectionError`/`BLETimeoutError` in sleep mode → queue into `pending_upload` instead of raising. Non-sleep devices keep today's strict failure. -5. The service returns response data (`SupportsResponse.OPTIONAL`): `{"status": "delivered" | "queued", "expires_at": …}` so automations can branch. It does **not** block until delivery (sleep intervals can be hours; blocking would time out the service call and jam automation queues). - -This is precisely the tuning the user asked for regarding bleak dynamics: *when we do connect, it is triggered by the wake advertisement itself*, so the first attempt starts ~0.1–2 s into the 10 s window and the default 10 s/attempt budget is ample; and *we never burn 40 s of retries against a device we know is dark*. No py-opendisplay retry-parameter changes are required (though the delivery task will bound each wake attempt with an overall deadline of ~30 s so one bad wake can't overlap the next). - -LED/buzzer services stay immediate-only (a notification that fires hours late is worse than an error); in sleep mode with the device dark they fail fast with a clear "device is sleeping" error. - -**Revision 2026-07-07 — probe before queue.** Field review showed the freshness gate over-triggers: it treats `probably_asleep` as authoritative, but (a) BLE adverts are lossy — a busy scanner/proxy can miss an entire 10 s wake window, leaving `last_seen` stale while the tag is awake; (b) the Silabs firmware advertises continuously in EM2 and only enters true deep sleep (EM4) on explicit command, yet its power config (`power_mode==BATTERY`, `deep_sleep_time_seconds>0`) reads as sleepy, so such tags were queued while almost always reachable. Meanwhile HA retains a *connectable* `BLEDevice` for ~3–5 min after the last advert (habluetooth connectable-history pruning: scanner staleness ~195 s + 300 s sweep), so a connect attempt remains *possible* long after the gate flips at ~15 s. - -Image sends now spend **one short connect attempt** (the "probe": `max_attempts=1`, `timeout=5 s` — `services.PROBE_CONNECT_TIMEOUT_S`) before queuing, instead of queuing blind. Cost analysis: a dark ESP32 has its radio fully off in timer deep sleep (`Firmware/src/main.cpp:365-383`), so a doomed probe costs the *device* zero battery and HA at most ~5 s — vs the old doomed 4×10 s ≈ 40 s budget this gate was built to avoid. A probe that lands cancels the wake-window timer and holds the tag awake (`main.cpp:138-170, 233-244`), converting into a normal full-quality delivery. A long-dark or never-seen tag has no connectable `BLEDevice`, so the probe short-circuits to the queue at near-zero cost. The probe is opt-out via the `probe_before_queue` option (default on). A post-probe freshness re-check closes the race where a wake advert arriving *during* the failed probe found no pending work and started no drain: if the tag looks fresh after queuing, `notify_device_seen("post-probe")` kicks an immediate drain. The ESP32 wake window is a fixed 10 s from wake (only a *connection* extends wakefulness), which is why the probe timeout sits deliberately under it. LED/buzzer and OTA gates are unchanged (still fail-fast). - -#### D6 — Representing queued content: the image entity + a pending sensor - -Per the user's instinct, the existing Display Content image entity is the queue's visible face: - -- On **queue**: update the image entity immediately with the rendered frame (it now shows *intended* content) and set entity attribute `pending: true` plus `queued_at`. A new **binary sensor "Update pending"** exposes the same state for automations/dashboards (attributes: `queued_at`, `expires_at`, `attempts`). -- On **delivery**: clear `pending`; image already matches. -- On **expiry**: fire `opendisplay_content_expired`, set the binary sensor off, and revert the image entity attribute to `pending: false` with `last_error: expired` (the panel still shows the old frame; the image entity keeps the intended frame as the record of what was attempted — with the attribute making the mismatch explicit). - -The queue itself is **memory-only in v1**: an HA restart drops a pending upload (documented limitation; the binary sensor goes off honestly). Persisting prepared frames (hundreds of KB) to a `Store` is a v2 option if real usage demands it — the delivery manager's API is designed so persistence can be added without touching callers. - -#### D7 — Reboot-edge handling in sleep mode - -Today's behavior — full entry reload on the advertised reboot edge — is wrong for sleepy devices twice over: the firmware's unpersisted `rebootFlag` makes every wake look like a reboot (churn loop, findings doc §Bug), and the reload's reconnect races the wake window. **Decision:** in sleep mode, the reboot edge sets `pending_config_resync` on the delivery manager instead of reloading. The resync rides the next delivery connection (or triggers one on the current wake, which is exactly when the edge is observed). Non-sleep devices keep the reload behavior. This makes the integration robust against the firmware bug while remaining correct once the firmware persists the flag. - -#### D8 — OTA on sleepy devices - -`update.py` connects directly today. In sleep mode: a user-initiated install registers `pending_ota` and reports progress state "waiting for device wake"; the next device-seen event starts the flash over the wake connection (device stays awake once connected — OTA duration is not window-constrained). Version *checks* remain opportunistic (piggyback on any delivery connection rather than connecting on a timer). - -#### D9 — py-opendisplay: add the deep-sleep command (0x0052) - -Not strictly required for ESP32 timer-driven sleep, but added for completeness and the Flex/Silabs story (where sleep is command-only): `OpenDisplayDevice.deep_sleep()` sending 0x0052, plus a `PowerConfig.deep_sleep_enabled` convenience property so the integration's D1 predicate lives next to the fields it reads. Also enables a future "sleep immediately after upload" optimization (save the remainder of a wake window after delivery). - -### 2.3 Timing analysis — why the rendezvous works - -With defaults (wake window W = 10 s; bleak timeout 10 s × 4 attempts): - -| Step | Budget | -|---|---| -| Device wakes, first advertisement out | ~0.1–1 s into window (fast adv interval in `minimalSetup`) | -| Scanner → coordinator callback (local adapter) | < 0.5 s; ESPHome BLE proxy adds ~0.5–1.5 s | -| Delivery task start → `establish_connection` first attempt | < 0.1 s (already-running event loop task, `BLEDevice` fresh from this very advertisement) | -| Connection establishment | typically 1–3 s; up to 10 s budget fits inside remaining ~8 s window | -| After connect | device exits window logic and stays awake (`main.cpp:85-90`) — unlimited work time | - -Failure modes are all recoverable: a missed window (proxy latency spike, connection slot exhaustion) simply waits `deep_sleep_time_seconds` for the next one; the deadline timer bounds total wait. Recommended user guidance (docs): keep `sleep_timeout_ms` ≥ 5000; typical `deep_sleep_time_seconds` 300–3600 gives content latency of at most one sleep interval. - -### 2.4 Scenario walkthroughs (target behavior) - -1. **HA restarts at 03:00; device sleeps 30 min.** Entry loads instantly from cache; all entities restored and available; `config_resync_pending` set. At the next wake (≤ 03:30) the delivery manager connects once, re-reads config, updates cache. No `ConfigEntryNotReady`, no flapping. -2. **Automation pushes a new image at 09:00; device sleeping until 09:25.** Image is rendered/prepared at 09:00; freshness gate sees stale adverts → queued instantly (no 40 s stall); service returns `queued`; image entity shows new frame with `pending: true`; binary sensor on. At 09:25 the wake advert triggers delivery; panel refreshes; `pending` clears; automations see `opendisplay_content_delivered`. -3. **Device battery dies / removed.** No wakes observed; after `missed_cycles × interval` entities go unavailable (honest signal); a queued upload expires after `queue_timeout` with `opendisplay_content_expired`. -4. **Five uploads queued while asleep.** Latest-wins: only the newest survives (existing semantics extended to the queue); each superseded call had returned `queued` and the final delivery event carries the last `queued_at`. -5. **Reboot flag seen on wake (firmware bug or real reboot).** No reload storm; config resync piggybacks the next delivery connection. -6. **New device adoption.** Unchanged — happens in the 2-minute first-boot window; first setup connects live and seeds the cache. - ---- - -## 3. Component factoring - -Proposed decomposition, smallest-surface-first. New modules in **bold**. - -``` -custom_components/opendisplay/ -├── const.py + option keys, defaults, event names, new dispatcher signals -├── config_flow.py + OpenDisplayOptionsFlow (sleep_mode, missed_cycles, queue_timeout) -├── __init__.py + config cache read/write; setup-from-cache branch; -│ availability-interval registration; DeliveryManager wiring; -│ runtime_data: delivery manager + sleep profile -├── coordinator.py + async_subscribe_device_seen (mirror of async_subscribe_reboot) -├── **sleep.py** SleepProfile: resolves options + device config → is_sleepy, -│ availability_interval, freshness gate ("probably_asleep(now)") -├── **delivery.py** DeliveryManager: pending slots (upload / config_resync / ota), -│ device-seen handler, single-connection drain, deadline timers, -│ delivered/expired events, state for sensors & diagnostics -├── services.py + freshness gate + queue-on-failure via DeliveryManager; -│ SupportsResponse.OPTIONAL with delivered/queued payload -├── image.py + pending/queued_at attributes; shows queued frame at queue time -├── **binary_sensor.py** "Update pending" entity backed by DeliveryManager state -├── update.py + sleep-mode gating: pending_ota slot, "waiting for wake" state -├── diagnostics.py + sleep profile + pending-slot state (redacted image bytes) -└── strings.json, translations/, icons.json — options flow + new entities + new errors - -py-opendisplay/ -├── protocol/commands.py + DEEP_SLEEP = 0x0052 (+ encoder) -├── device.py + async deep_sleep() -└── models/config.py + PowerConfig.deep_sleep_enabled property - -Firmware/ (recommended, decoupled — §5.4) -├── rebootFlag RTC persistence across deep sleep -└── post-reboot advertising grace window -``` - -Dependency direction: `services.py`, `update.py`, `image.py`, `binary_sensor.py` → `delivery.py` → `sleep.py` + `coordinator.py`. `delivery.py` owns all queue state; entities and diagnostics only read it; services only submit to it. The coordinator knows nothing about queues — it just announces "device seen". - ---- - -## 4. Implementation plan - -### Phase 0 — py-opendisplay groundwork *(small; independent release)* - -1. `PowerConfig.deep_sleep_enabled` property (`models/config.py`): `power_mode == PowerMode.BATTERY and deep_sleep_time_seconds > 0`. Unit tests over parse/serialize round-trips. -2. `Command.DEEP_SLEEP = 0x0052` + `OpenDisplayDevice.deep_sleep()` (`device.py`, modeled on `reboot()` at `device.py:912`; fire-and-forget semantics — the ESP32 drops the link on entry, so tolerate a disconnect instead of awaiting an ACK; verify exact response behavior against `device_control.cpp:692` during implementation). -3. CLI: `opendisplay sleep ` subcommand for testing. -4. Release (7.12.0) and bump `manifest.json` requirement in the integration. - -*Acceptance: CLI can put an ESP32 dev board to sleep; config round-trip tests pass.* - -### Phase 1 — Sleep awareness, availability, setup-from-cache *(the "entry survives" milestone)* - -1. **`const.py`**: `CONF_SLEEP_MODE` (`auto|on|off`), `CONF_MISSED_CYCLES` (default 3), `CONF_QUEUE_TIMEOUT_HOURS` (default 24), event name constants, `CONF_CACHED_STATE` key. -2. **`sleep.py`**: `SleepProfile.from_entry(entry, config)` — resolves option override + `deep_sleep_enabled`; computes `availability_interval = interval × missed_cycles + wake_window_s + 60`; exposes `probably_asleep(last_seen)` using `sleep_timeout_ms + 5 s` slack. Pure functions, fully unit-testable. -3. **`__init__.py` — cache write**: after every successful interrogation, store `{"config": config_to_json(cfg), "firmware": fw, "is_flex": …, "cached_at": …}` under `entry.data[CONF_CACHED_STATE]` via `async_update_entry`. -4. **`__init__.py` — setup-from-cache branch**: if no connectable device *or* connect fails, and cached state exists with `deep_sleep_enabled` (or option forced on): rebuild `GlobalConfig` via `config_from_json`, construct runtime data without connecting, set `pending_config_resync` (consumed in Phase 2; in Phase 1 it may simply resync on next reboot-edge/reload), and proceed. Otherwise preserve today's `ConfigEntryNotReady`/`ConfigEntryAuthFailed` behavior exactly. -5. **`__init__.py` — availability interval**: when the profile is sleepy, call `async_set_fallback_availability_interval(hass, address, profile.availability_interval)` before starting the coordinator; re-apply on reload. -6. **`config_flow.py`**: add `OpenDisplayOptionsFlow` with the three options; options changes trigger entry reload (standard listener). -7. **strings/translations** for the options form. - -*Acceptance: with a device configured for 10 min sleep — restart HA mid-sleep: entry loads, entities available and populated at next advert; no unavailable flap across three sleep cycles; options visible and effective after reload. Regression: non-sleepy device setup behavior unchanged (existing tests).* - -### Phase 2 — Delivery manager and queued uploads *(the core feature)* - -1. **`coordinator.py`**: `async_subscribe_device_seen(cb)` invoked from `_async_handle_bluetooth_event` after parsing (same pattern as `async_subscribe_reboot`). -2. **`delivery.py`**: `DeliveryManager(hass, entry, profile)`: - - Slots as in D4; `submit_upload(prepared, params, preview_jpeg) -> PendingReceipt`; `request_config_resync()`; internal `_deliver()` task guarded by an asyncio flag + `ble_lock`, bounded by a ~30 s per-wake deadline; drain order upload → resync → ota. - - Deadline timers via `async_call_later`; on expiry fire `opendisplay_content_expired` (payload: device_id, queued_at, attempts) and notify state listeners (dispatcher signal for entities). - - On delivery: send existing `SIGNAL_IMAGE_UPDATED` (unchanged image pipeline), fire `opendisplay_content_delivered`, persist refreshed config to cache when a resync ran. - - Teardown on unload: cancel timers + in-flight task (extend `async_unload_entry` alongside the existing upload-task cancel, `__init__.py:221-227`). -3. **`services.py`**: implement D5 — freshness gate; queue-on-connect-failure for `upload_image`/`drawcustom`; `SupportsResponse.OPTIONAL` returning `{"status", "expires_at"}`; LED/buzzer get the fast "device_sleeping" error when provably asleep. The existing latest-wins cancel logic remains for concurrent *live* uploads; the manager applies the same rule to the slot. -4. **`image.py`**: `pending`/`queued_at` attributes; listen to the manager's state signal; show queued frame at queue time (D6). -5. **`binary_sensor.py`**: "Update pending" backed by manager state; add platform to `_BASE_PLATFORMS`/`_FLEX_PLATFORMS`. -6. **`__init__.py`**: instantiate the manager in runtime data; setup-from-cache path now registers `pending_config_resync` with it, making the Phase 1 flag fully functional. -7. **strings/translations/icons** for the sensor, events, and the new error key. - -*Acceptance: end-to-end on hardware — call `drawcustom` mid-sleep: returns `queued` in < 2 s, sensor on, image entity shows frame with `pending: true`; panel refreshes on next wake ≤ interval; sensor off; `opendisplay_content_delivered` observed. Kill the device (pull battery) with queued content: expiry event at deadline. Five rapid queued calls → one delivery of the last frame.* - -### Phase 3 — Hardening and edges - -1. **Reboot-edge rework (D7)**: in `__init__.py`, when profile is sleepy route the reboot subscription to `manager.request_config_resync()` instead of `_async_reload_after_reboot`. -2. **OTA gating (D8)** in `update.py`: `pending_ota` slot, `in_progress`/extra state "waiting for device wake", resume-on-wake. -3. **`diagnostics.py`**: sleep profile, availability interval, slot states (timestamps/attempts only — no image payloads). -4. **Encrypted devices**: queued delivery reuses the stored key; on `AuthenticationFailedError` during delivery → keep slot paused, trigger reauth flow (existing pattern in `_async_connect_and_run`), resume after successful reauth. -5. **Test suite**: unit tests for `sleep.py` and `delivery.py` (mock coordinator/device); integration-style tests with `inject_bluetooth_service_info` simulating wake cadences: setup-from-cache, flap-free availability, queue→deliver, queue→expire, reboot-edge resync, unload-with-pending. -6. **Docs**: user-facing page (docs/) covering options, latency expectations table (interval vs. worst-case content delay), and the memory-only queue limitation. - -### Phase 4 — Firmware coordination *(separate repo; recommended, not blocking)* - -1. Persist `rebootFlag` across deep sleep via `RTC_DATA_ATTR` mirror (findings doc, ~10 lines) — kills the churn loop at the source; D7 already defuses it HA-side. -2. Post-reboot advertising grace window (30–60 s after *any* boot where `woke_from_deep_sleep` is false) — makes commanded reboots recoverable. -3. Optional: `esp_sleep_enable_ext1_wakeup` on the button pin — user-initiated wake ("press button to sync now"), which composes perfectly with the delivery manager (button wake → advert → immediate delivery). - -### Sequencing and risk - -- Phases 0→1→2 are strictly ordered; 3 and 4 can proceed in parallel after 2. -- Riskiest assumption: wake-window connect reliability through ESPHome proxies (extra advert latency + connection-slot contention). Mitigation is inherent — a missed window costs one sleep interval, and Phase 2 acceptance is run against both a local adapter and a proxy. -- Backward compatibility: all changes are additive; non-sleepy devices follow existing code paths verbatim; new options default to today's behavior for them (`auto` resolves to off). -- HA quality-scale note: setup-from-cache plus passive-only polling keeps the integration aligned with bluetooth-integration guidance (no connections outside user intent or discovery). - ---- - -## Appendix A — Options summary - -| Option | Default | Meaning | -|---|---|---| -| `sleep_mode` | `auto` | `auto`: follow device power config; `on`/`off`: force | -| `missed_cycles` | 3 | Wake cycles missed before entities go unavailable | -| `queue_timeout` | 24 h | Pending content expires with `opendisplay_content_expired` | - -## Appendix B — New events / signals / entities - -| Surface | Name | Payload | -|---|---|---| -| Bus event | `opendisplay_content_delivered` | device_id, queued_at, attempts | -| Bus event | `opendisplay_content_expired` | device_id, queued_at, attempts | -| Service response | `upload_image` / `drawcustom` | `{status: delivered\|queued, expires_at}` | -| Entity | `binary_sensor._update_pending` | attrs: queued_at, expires_at, attempts | -| Entity attrs | image `display_content` | `pending`, `queued_at` | - -## Appendix C — Open questions (decide during implementation) - -1. Should a delivery connection also refresh the `last_seen`-adjacent sensors by reading live values (battery under load), or stay advert-only? (Lean: advert-only; keep connections short.) -2. `deep_sleep()` post-upload to return the device to sleep immediately after delivery (saves the rest of the wake window) — worth it once 0x0052 ships? (Lean: yes, guarded by an option, after measuring real window costs.) -3. v2 queue persistence across HA restarts via `helpers.storage.Store` — wait for user demand.