Release v3.2.1 (MQTT crash fixes + diagnostics)#28
Merged
Merged
Conversation
A coredump is raw addresses; /api/coredump/summary's backtrace only becomes function names + line numbers with the EXACT ELF that built the running image. Releases attached firmware.bin but not the ELF, so a crash on a released build couldn't be fully symbolised without rebuilding and hoping the layout matched. Now every release attaches wallbox-gateway-<tag>-esp32s3.elf, and every build retains firmware.elf as a 90-day artifact. Debug symbols only — never flashed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The /logs Download button linked to /api/logs, firing a second ~16 KB request that raced the page's 3s auto-refresh poll for the async server's connection slots. Under the heap pressure of overlapping 16 KB String responses some came back empty and the page hung — the 'stuck on export' report. Download now builds the file client-side (a Blob from the log the page already displays), so it makes no server request, can't contend, and is instant. /api/logs is unchanged for external monitors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the v3.2.0 release tag into feature/3.2's ancestry so git-describe on the dev branch anchors to v3.2.0 (not the older rc.4), keeping dev-build version strings monotonic — a build off feature/3.2 now reports 'v3.2.0-N-g<hash>' (> 3.2.0) instead of 'v3.2.0-rc.4-N-...' (< 3.2.0), so the HA update entity never mistakes newer dev code for a downgrade target. Content is unchanged (main == the PR #27 merge of this branch); this only records the ancestry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The core dump pins #168 to a core-0 task-watchdog with our wb_ble task priority-boosted (holding a BT-stack lock during writeValue), but a single partial dump isn't enough to fix the carefully-built mutex code confidently. Add lightweight instrumentation to catch the stalling op on the next crash: - Time each ATT write and the full write+response round-trip; log a warning when a write exceeds 300ms (normal < 50ms) or a round-trip exceeds 2s. - Track worst-case ble_write_max_ms / ble_rt_max_ms on /api/diag/runtime, so the peak survives the in-RAM log ring wrapping. - Mark the RTC-NOINIT crash breadcrumb 'w:<met>' during the write and 'q:<met>' during the response-wait. The log ring is wiped on reboot and the running BLE task's coredump backtrace is often unwind-corrupt, but the breadcrumb survives — so a watchdog reset mid-write is identifiable post-reboot via /api/boot/history. Instrumentation only — no change to the write/mutex flow, so it can't reintroduce the write-race panics the mutexes were added to prevent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coredump from the box (task=loopTask, LoadProhibited read @0x14, clean backtrace) pinned a #168 crash exactly: loop -> WallboxMQTT::loop -> _connect -> PubSubClient::connect -> readPacket -> readByte -> WiFiClient::read() <- NULL deref MQTT reconnect had no WiFi-up guard, and PubSubClient::connect() skips its own TCP connect when wifiClient.connected() returns true. After a WiFi blip that read is STALE (connected() true while the socket handle is NULL), so connect() jumps straight to reading the CONNACK and WiFiClient::read() dereferences the NULL handle -> LoadProhibited panic in loopTask. Two guards: - loop() skips _connect() unless WiFi.status()==WL_CONNECTED (don't even try while WiFi is down). - _connect() calls wifiClient.stop() first, forcing connect() to establish a fresh socket instead of trusting a stale connected()==true. Found via the coredump + breadcrumb instrumentation added this cycle (the surviving breadcrumb was 'q:g_psh'). Note: an earlier #168 dump had a different signature (task-watchdog / IDLE0 with the BLE task priority-boosted) — that may be a separate issue; keep the instrumentation and watch for it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second coredump (task=wb_ble, breadcrumb q:r_lse, clean bt) symbolized to: WallboxMQTT::loop -> PubSubClient::loop -> readPacket -> WiFiClient::read -> lwip_recv_tcp -> pbuf_free() <- lwip assertion (abort) The BLE yield callback (set in setupServices) pumped wallboxMQTT.loop() + ArduinoOTA.handle() during BLE response-waits. That callback was written when sendCommand() ran on the web/main task, but BLE now runs on its own pinned task (wb_ble) — so those calls fired CONCURRENTLY with the main loop's own wallboxMQTT.loop() (main.cpp:512) and ArduinoOTA.handle() (main.cpp:493). PubSubClient / WiFiClient / lwip are not thread-safe; two tasks reading the same socket corrupted an lwip pbuf -> pbuf_free assert -> crash. This is the same root as the earlier loopTask MQTT-connect NULL-deref (1a3e1f5): MQTT touched from two contexts. That commit's WiFi-up guard + wifiClient.stop() stay as complementary defense for the main loop's own reconnect; this removes the cross-task access entirely. Fix: drop the yield callback. Its keepalive-starvation rationale is moot now — BLE waits run off the main loop, so the main loop services MQTT (60s keepalive) + OTA continuously on its own. The BLE wait just delay(1)-yields to the scheduler. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stability release on top of v3.2.0. Fixes the recurring MQTT-related crash/reboot (root-caused from on-device coredumps), the
/logsDownload hang, and ships the crash-diagnostics tooling that made the fixes possible.Fixed
MQTT.loop()concurrently with the main loop (cross-task access to non-thread-safe PubSubClient/lwip → pbuf corruption). Yield callback removed.WL_CONNECTED+ fresh socket./logsDownload hang — now builds the file client-side (no second server request racing the poll).Added
firmware.elfattached to releases (coredump symbolisation)./api/diag/runtime+ crash-breadcrumb phase.Known issues
Full detail in the
[3.2.1]section ofCHANGELOG.md.🤖 Generated with Claude Code