Skip to content

Fix realtime resubscribe using an expired access token + bounded teardown - #435

Merged
Danielhiversen merged 2 commits into
Danielhiversen:masterfrom
honzup:pr/realtime-token-refresh
Jul 27, 2026
Merged

Fix realtime resubscribe using an expired access token + bounded teardown#435
Danielhiversen merged 2 commits into
Danielhiversen:masterfrom
honzup:pr/realtime-token-refresh

Conversation

@honzup

@honzup honzup commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes two related realtime bugs reported against Home Assistant (home-assistant/core#176268, home-assistant/core#168480), plus teardown hangs that wedge HA config-entry unloads into FAILED_UNLOAD.

Bug 1: delayed resubscribe uses the cached (expired) token

After a TransportError, TibberHome._delayed_resubscribe called rt_resubscribe(), which re-authenticates with the cached access token. HA's OAuth tokens live ~1 hour and usually expire while the websocket is up, so the resubscribe failed with InvalidLoginError: "exp" claim timestamp check failed — and was logged once and abandoned (no retry, and rt_unsubscribe() had already torn the listener down). This is the endless Watchdog: Connection is down loop in core#176268.

Fix: _delayed_resubscribe now goes through TibberRT.reconnect() — the only token-refreshing path — with bounded retries (5 attempts, exponential backoff). It detaches itself before reconnecting so the connection-wide resubscribe can't cancel its own task mid-flight, and early-returns if another path already restored the subscription.

Bug 2: reconnect() silently reuses an expired token

reconnect() only rebuilt the sub_manager when the refreshed token string differed from the cached one — but the token is baked into the transport's init_payload at construction, so an expired-but-identical token was reused forever. The rebuild now runs unconditionally after a refresh.

Teardown hardening

  • TibberWebsocketsTransport.close(): wait_closed() bounded to 10 s (a half-dead socket previously blocked forever — this is what deadlocks HA's config-entry unload into FAILED_UNLOAD).
  • sub_manager.close_async() bounded to 15 s in _reset_connection_locked.
  • The connect lock is now per-instance instead of module-global, so one wedged connection can't block another instance's disconnect.

Testing

  • 6 new unit tests (written to fail against the unpatched code): reconnect-rebuilds-transport-on-unchanged-token, per-instance lock, both close timeouts, and the retry/backoff behaviour of the delayed resubscribe (asserts attempt counts and delays).
  • Full suite passes (32 passed), ruff check/ruff format clean, mypy clean.
  • Production testing: running on a Home Assistant OS instance for 48+ hours, through ~48 hourly OAuth token-rotation reconnect cycles and forced TCP socket kills, with zero stuck connections (previously wedged within a day).

Two related realtime bugs, both reported against Home Assistant
(home-assistant/core#176268, home-assistant/core#168480):

1. The delayed resubscribe scheduled after a TransportError called
rt_resubscribe(), which re-authenticates with the cached access token.
When the token had expired in the meantime (Home Assistant OAuth tokens
live ~1 hour and typically expire while the websocket is up), the
resubscribe failed with InvalidLoginError ('"exp" claim timestamp
check failed'), was logged once and abandoned - leaving the home
permanently unsubscribed. It now goes through TibberRT.reconnect() -
the token-refreshing path - with bounded retries and exponential
backoff, and detaches itself before reconnecting so the connection-wide
resubscribe cannot cancel its own task mid-flight.

2. reconnect() only rebuilt the subscription manager when the refreshed
token string differed from the cached one. The token is baked into the
transport's init_payload at construction, so an expired-but-identical
token was silently reused and the reconnect loop never recovered. The
rebuild now happens unconditionally after a token refresh.

Also hardens teardown, which could block forever on a half-dead socket
(wedging Home Assistant config-entry unloads into FAILED_UNLOAD):
close paths are bounded (10 s transport close, 15 s sub_manager close)
and the connect lock is per-instance instead of module-global.

Verified in production on a Home Assistant OS instance for 48+ hours,
including ~48 hourly token-rotation reconnect cycles and forced TCP
socket kills, with no stuck connections.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DERobfZZh3npFRbwpdxzz2
@honzup
honzup marked this pull request as ready for review July 16, 2026 07:56
Comment thread test/test_realtime.py Fixed
Comment thread test/test_tibber.py Fixed
@Danielhiversen
Danielhiversen merged commit 9d670d3 into Danielhiversen:master Jul 27, 2026
4 checks passed
@ralftar

ralftar commented Jul 28, 2026

Copy link
Copy Markdown

Field confirmation from production (pyTibber 0.37.6 via Home Assistant on NixOS), 2026-07-28:

  • 16:01:52 UTC — short network blip (~1–2 min); last realtime data, then TransportServerError: No pong received after 15.0 seconds
  • 16:02:57 — Error in delayed rt_resubscribe: the one-shot resubscribe hit the still-recovering network, swallowed the exception and was never retried (bug 1 here)
  • 16:03–17:54 — the realtime watchdog retried 21+ times (backoff capped at 300 s) within the same process; every attempt failed during the websocket handshake with ConnectionClosedError in _wait_ack, consistent with the stale access token baked into the transport init_payload at build time (the reuse this PR eliminates)
  • No self-recovery in 1 h 53 min — the loop was only broken by a Home Assistant restart, after which a fresh setup connected immediately

This PR looks like it fixes precisely the behavior above. A tagged release would be much appreciated so it can reach Home Assistant and downstream distros.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants