Document the threaded @batch per-block allocation and its root cause (#198)

src/downconvert_and_correlate_cpu.jl

@@ -65,13 +65,53 @@ CPUDownconvertAndCorrelator() = CPUDownconvertAndCorrelator(ScratchBuffers())
 """
 $(SIGNATURES)
 
-Multi-threaded CPU downconvert and correlate. Holds one
-`ScratchBuffers` per thread, indexed by `Threads.threadid()`
-inside `@batch` (which pins each iteration to a fixed thread). Buffers
-grow lazily on first use and are reused thereafter, so a hoisted
-instance has near-zero allocations per `track!` call in steady state
-(Polyester's `@batch` keeps a small irreducible per-call closure
-allocation).
+Multi-threaded CPU downconvert and correlate, parallelized over the
+satellites (PRNs) of each group. Holds one `ScratchBuffers` per thread,
+indexed by `Threads.threadid()` inside `@batch` (which pins each iteration
+to a fixed thread). Buffers grow lazily on first use and are reused
+thereafter, so a hoisted instance's scratch is allocation-free in steady
+state.
+
+One `@batch` is launched per code block (once per `downconvert_and_correlate!`
+call inside `track!`'s inner loop). When the process runs with more than one
+thread *and* the group has more than one satellite, each launch keeps a small
+Polyester allocation (~64 B), so the total scales with the number of completed
+code blocks in the chunk rather than staying flat per `track!` call. With a
+single thread, a single satellite, or the single-threaded
+`CPUDownconvertAndCorrelator`, there is no such residual.
+
+Why it allocates at all — Polyester's `@batch` roots only *bare* `Array`s and
+`isbits` values into its worker tasks for free (that is why a `Vector{Float64}`
+kernel allocates nothing). It pays a small per-launch allocation to root any
+**non-`isbits` struct** referenced inside the region — and it is the struct-ness
+that costs, not the contents: capturing even a plain struct whose only fields
+are `Matrix`es and isbits scalars measures the same ~64 B/launch, whereas
+capturing those same bare arrays measures 0. The culprit here is the GNSS
+**signal**: `GPSL1CA`, for instance, is not `isbits` — it wraps a `Matrix{Int16}`
+code table and a (also non-`isbits`) `SignalLUT`. Each satellite's code-replica
+generation needs its signal, so every `@batch` launch touches that non-`isbits`
+object once. (The per-satellite `TrackedSat` is likewise non-`isbits` — it
+carries the signal plus the CN0/bit buffers — and the loop reaches the signal
+through it, so iterating `Vector{TrackedSat}` is not free the way iterating a
+`Vector{Float64}` is.)
+
+Note that merely *hoisting* the `SignalLUT` out of the signal does not help — a
+`SignalLUT` is itself a non-`isbits` struct (it wraps `Matrix{Int8}` fields), so
+capturing it costs the same as capturing the whole `GPSL1CA`. What removes the
+cost entirely is a code-generation entry point that takes the LUT's bare arrays
+and isbits fields as *separate arguments* (`padded`, `secondary`,
+`subchip_factor`, …) rather than a struct: the `@batch` closure then captures
+only bare `Array`s and isbits values, which Polyester roots for free. A
+prototype of exactly this — generating from the raw `padded` matrix, done
+*inside* the parallel loop — is bit-faithful and measures **0 B/launch**, and
+because generation stays in the `@batch` region it keeps the full parallel
+throughput. Realizing it needs a GNSSSignals-side API that threads the bare
+arrays down to the resample kernel without re-wrapping them in a struct inside
+the region (reconstructing one there sends the generator dynamic and allocates
+far more). Absent that, the only in-tree way to reach 0 is a serial code-gen
+pre-pass, which is allocation-free but serializes ~30 % of the work and slows
+the threaded pipeline — the inferior fallback. Neither is currently worth
+~64 B/block, which is bounded per call and dwarfed by the caller's input buffer.
 
 For real-time loops, construct the correlator **once outside** the
 `track!` loop and pass it via the `downconvert_and_correlator` keyword

src/track.jl

@@ -122,9 +122,33 @@ In-place version of [`track`](@ref). Mutates `track_state` by overwriting the
 immutable wrappers. Returns the same `track_state` object.
 
 After one warmup call (which seats each satellite's `filtered_prompts`
-buffer capacity), the single-threaded path is fully allocation-free; the
-threaded path keeps an irreducible Polyester `@batch` closure-capture
-allocation of about 160 B per GNSS system per call.
+buffer capacity), the single-threaded path is fully allocation-free.
+
+The threaded path (`CPUThreadedDownconvertAndCorrelator`) keeps a small
+residual — about 64 B per **completed code-block integration** per group —
+but only when the process runs with more than one thread *and* the group
+holds more than one satellite (so Polyester's `@batch` actually distributes
+work). `track!` launches one `@batch` per code block, so this residual
+scales with the chunk length rather than staying flat per call; for a
+real-time loop with fixed-size chunks it is bounded per call and, in
+practice, dwarfed by the input sample buffer the caller allocates each
+chunk. The single-threaded backend has none of it.
+
+The root cause is not the per-satellite state per se — it is that the
+GNSS **signal** object is not an `isbits` type: `GPSL1CA`, for example, holds
+a `Matrix{Int16}` code table and a (also non-`isbits`) `SignalLUT`. Polyester
+roots bare `Array`s and `isbits` values into its worker tasks for free (that
+is why a `Vector{Float64}` kernel is allocation-free), but it pays a small
+per-launch allocation to root any *other* non-`isbits` object touched inside
+the `@batch` region. Each satellite's `downconvert_and_correlate` reaches its
+signal (for code-replica generation), so the parallel loop touches that
+non-`isbits` object once per launch. It could be removed by generating the code
+from the LUT's *bare arrays* (a prototype doing so inside the loop measures 0 B
+at full parallel throughput) — which needs a GNSSSignals-side bare-array
+`gen_code!` — or, in-tree, by a serial code-gen pre-pass (allocation-free but
+slower, since it serializes ~30 % of the work). Neither is currently worth
+~64 B/block. See [`CPUThreadedDownconvertAndCorrelator`](@ref) for the full
+analysis.
 
 For real-time loops, **construct the correlator once outside the loop**
 and pass it via the `downconvert_and_correlator` keyword argument:

test/track_in_place.jl

@@ -151,7 +151,13 @@ end
 # per-code-block bit-edge search (`_buffer_find_bit`) once per code block. A
 # regression there — e.g. a closure-capture `Core.Box` + boxed `SyncResult`
 # (~80 B/block) — makes `track!` allocate in proportion to the signal length
-# instead of staying allocation-free after the first (buffer-seating) call. The
+# instead of staying allocation-free after the first (buffer-seating) call.
+# This is exactly the "allocates per completed integration, scales with signal
+# length" symptom reported in #198: measuring at two chunk lengths and asserting
+# both are 0 pins the allocation flat across completion counts (a per-block leak
+# would grow 10× from 2 to 20 blocks). The box shows up at any runtime thread
+# count on the single-threaded backend, so this single-threaded `== 0` catches
+# it on CI without needing a multi-threaded run. The
 # per-stage test above does NOT catch it: it tracks a real signal that reaches
 # bit sync within the warmup, so by the time it measures it exercises only the
 # post-sync path, where `_buffer_find_bit` is no longer called.