Route fountain-decoder allocations to PSRAM on ESP32#9
Closed
kdmukAI-bot wants to merge 1 commit into
Closed
Conversation
On an ESP32 with external SPIRAM, ESP-IDF's malloc() forces every allocation at or below CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL (16 KB by default) into scarce internal RAM. The fountain decoder's working set is many small, variably-sized allocations — up to MAX_MIXED_PARTS part buffers plus per-part fragments, allocated and freed as parts arrive and are XOR-reduced. Kept in internal RAM they steadily grow the footprint and, worse, fragment the heap: the largest contiguous free block shrinks well below total free. The damaging symptom is not the decode itself running out of memory, but a later allocation that needs a contiguous internal block failing even with ample free RAM — e.g. being unable to re-initialize another internal-RAM subsystem after a scan (re-launching the camera, whose driver needs a contiguous internal buffer/stack the fragmented heap can no longer supply). Route the shared utility allocators to PSRAM: - safe_malloc, safe_malloc_uninit, safe_realloc allocate via heap_caps_malloc / heap_caps_realloc(MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT), falling back to internal RAM when no PSRAM is present. - safe_strdup inherits PSRAM through safe_malloc. - All guarded by ESP_PLATFORM; host/desktop builds keep plain malloc/realloc. Moving the churn off the internal heap removes both the internal consumption and the fragmentation it caused. These are plain CPU-accessed byte buffers, so cached PSRAM is appropriate and free() continues to work on heap_caps allocations. Co-Authored-By: kdmukai <934746+kdmukai@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kdmukAI-bot
force-pushed
the
feat/esp-psram-allocator
branch
from
d3a58e1 to
8758b00
Compare
kdmukai
added a commit
to kdmukai/seedsigner-micropython-builder
that referenced
this pull request
Jul 11, 2026
…ix + board_common Wire the P4-35 partition-mode camera preview through camera_scanner / camera_entropy (opt-in BOARD_CAMERA_PARTITION_MODE; live gutter progress scaffolding). Bump board_common (partition preview + zero-copy SPI stripe blit) and pin deps/cUR to the PSRAM-allocator branch: the fountain decoder's many small allocations are routed to PSRAM (upstream odudex/cUR#9) so a long animated-UR scan no longer grows/fragments internal RAM and blocks a later contiguous internal alloc (e.g. re-launching the camera). Adds docs/knowledge notes for the SPI stripe DMA-bounce fragmentation root cause and the camera-QR internal-RAM starvation follow-on. Co-Authored-By: kdmukai <934746+kdmukai@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Owner
|
The commit from this PR was cherry-picked. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On an ESP32 with external SPIRAM, ESP-IDF's
malloc()forces every allocationat or below
CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL(16 KB by default) intointernal RAM. The fountain decoder's working set is many small, variably-sized
allocations — up to
MAX_MIXED_PARTSpart buffers plus per-part fragments,allocated and freed as parts arrive and are XOR-reduced. Kept in internal RAM
they have two compounding effects:
sized small allocations, so the largest contiguous free block shrinks well
below total free.
Fragmentation is the more damaging effect. The failure we hit was not the decode
itself running out of memory, but a later allocation that needs a contiguous
internal block failing even with ample free RAM — specifically being unable to
re-initialize another internal-RAM subsystem after a scan (re-launching the
camera, whose driver needs a contiguous internal buffer/stack that the
fragmented heap could no longer supply).
This routes the shared utility allocators to PSRAM:
safe_malloc,safe_malloc_uninit,safe_reallocallocate viaheap_caps_malloc/heap_caps_realloc(MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT),falling back to internal RAM when PSRAM is absent.
safe_strdupinherits PSRAM throughsafe_malloc.#ifdef ESP_PLATFORM; host/desktop builds are unchanged.Moving the churn off the internal heap removes both the internal consumption and
the fragmentation it caused. These are plain CPU-accessed byte buffers, so cached
PSRAM is appropriate; the fountain XOR/copy work over them is sequential and
cache-friendly.
free()continues to work onheap_capsallocations.Verified on an ESP32-P4: a multi-part UR round-trips byte-exact and the fountain
working set no longer draws down or fragments internal RAM.