feat(track): external correlator-output ingest for FPGA offload (#203)

docs/src/correlator.md

@@ -124,10 +124,14 @@ get_num_ants(::AbstractCorrelator)
 Each completed integration within a processing chunk is recorded as a
 `CorrelatorOutput`, collected per signal and consumed by the Doppler estimator
 after the chunk (see [Chunked Doppler updates](track.md#Chunked-Doppler-updates)).
+An external correlator producer (e.g. an FPGA) can build these itself and feed
+them straight to the estimator — see
+[External correlator producers](track.md#External-correlator-producers).
 
 ```@docs
 CorrelatorOutput
 get_correlator_outputs
+append_correlator_output!
 ```
 
 ## Sample shifts

docs/src/custom_doppler_estimator.md

@@ -96,11 +96,22 @@ per-sat fields directly and rewraps `doppler_estimator_state` unchanged.
    value that generated the chunk), not an intermediate per-output estimate.
 
    The matching mutating method
-   `estimate_dopplers_and_filter_prompt!(track_state, measurements, prefer)`
+   `estimate_dopplers_and_filter_prompt!(track_state, measurements)`
    is what [`track!`](@ref) calls. To support real-time loops, define
    both — the mutating version walks each group's
    `satellites.values::Vector{TrackedSat}` and reassigns slots in place.
 
+   The estimator only needs the per-band **sampling frequency** out of
+   `measurements` (to turn each output's `integrated_samples` into an
+   integration time and to normalize the DLL discriminator). The shipped
+   `ConventionalPLLAndDLL` therefore also accepts a bare per-band
+   sampling-frequency source in place of `measurements` — a `NamedTuple`/`Dict`
+   keyed by `get_band_id` — so an external correlator producer can run the
+   estimator with no sample buffer (see
+   [External correlator producers](track.md#External-correlator-producers)).
+   A custom estimator that likewise reads only the rate is encouraged to offer
+   the same overload.
+
 ## Skeleton
 
 The skeleton below is the smallest possible working estimator: every
@@ -188,4 +199,6 @@ estimator the `TrackState` was built with.
 AbstractDopplerEstimator
 init_estimator_state
 update_estimator_on_handoff
+Tracking.estimate_dopplers_and_filter_prompt
+Tracking.estimate_dopplers_and_filter_prompt!
 ```

docs/src/track.md

@@ -1,5 +1,9 @@
 # Track
 
+```@meta
+CurrentModule = Tracking
+```
+
 [`track`](@ref) is the main entry point: it takes an incoming measurement and a [`TrackState`](@ref), runs downconversion + correlation for every tracked satellite, drives the Doppler estimator, and returns the updated `TrackState`. [`track!`](@ref) is the in-place counterpart for hard real-time loops where GC pauses must be avoided — after one warmup call (to seat the `filtered_prompts` buffer's capacity), the single-threaded `track!` path is **fully allocation-free**; the threaded path keeps a small irreducible per-call residual from Polyester's `@batch` closure capture (160 B per GNSS system).
 
 ```@docs
@@ -55,6 +59,41 @@ Both [`CPUDownconvertAndCorrelator`](@ref) and [`CPUThreadedDownconvertAndCorrel
 
 `track!` writes back into the existing `Vector{TrackedSat}` slots of each per-group dictionary, so the tracking loop runs without GC pressure once the sat set is steady. The first `track!` call may grow each signal's `filtered_prompts` buffer via `push!`; from the second call onwards the capacity is settled.
 
+## External correlator producers
+
+The correlate phase and the Doppler estimator are decoupled: the estimator consumes each signal's `correlator_outputs` buffer and never touches the sample buffer directly. That lets an **external producer** — e.g. an FPGA/hardware correlator that streams completed correlator dumps — supply the outputs and run only the loop filters on the host. The FPGA does downconversion + correlation; the host folds the dumps through the estimator and streams the resulting NCO Dopplers back. (The DMA transport, wire format and NCO marshaling live in [GNSSReceiver.jl](https://github.com/JuliaGNSS/GNSSReceiver.jl); this section is only the Tracking.jl-side contract.)
+
+The offload loop, per processing chunk (epoch):
+
+1. **Ingest.** For each satellite/signal, build a [`CorrelatorOutput`](@ref) from the producer's raw accumulator and append it with [`append_correlator_output!`](@ref) — appended **per signal in `sample_index` order**:
+
+   ```julia
+   append_correlator_output!(track_state, output, group, prn, sig)
+   ```
+
+   Prefer this over mutating the vector [`get_correlator_outputs`](@ref) returns: it documents intent and type-checks the correlator against the signal's.
+
+2. **Estimate.** Fold the batch and update every satellite's NCO once by calling [`estimate_dopplers_and_filter_prompt!`](@ref) with a **per-band sampling-frequency source** instead of a sample buffer — a `NamedTuple`/`Dict` keyed by the band's `get_band_id` (e.g. `:L1`, `:L5`):
+
+   ```julia
+   estimate_dopplers_and_filter_prompt!(track_state, (L1 = 25e6Hz, L5 = 25e6Hz))
+   # or: estimate_dopplers_and_filter_prompt!(track_state, Dict(:L1 => 25e6Hz))
+   ```
+
+   This skips `downconvert_and_correlate!` entirely. The rate must be **per band** (the estimator walks groups that may be on different bands); passing the original [`BandMeasurements`](@ref) works too and reads the rate off each band's `BandMeasurement`, so the CPU `track!` path is unchanged. The estimator **consumes and clears** each signal's `correlator_outputs` as part of this call — it is empty again afterwards, ready for the next chunk. This is the public contract external producers rely on.
+
+3. **Marshal** the updated `code_doppler`/`carrier_doppler` (read with [`get_code_doppler`](@ref)/[`get_carrier_doppler`](@ref)) back to the hardware NCOs.
+
+### Caller contract
+
+- `CorrelatorOutput.correlator` — the **raw** accumulator (sum-of-products over `integrated_samples`, matching what `normalize` expects). Reusing [`EarlyPromptLateCorrelator`](@ref) / `update_accumulator` on the producer side satisfies this by construction.
+- `CorrelatorOutput.integrated_samples` — the producer's true sample count for that integration.
+- `CorrelatorOutput.sample_index` — the chunk-relative end sample. The software path writes it buffer-relative (`signal_start_sample` returns to 1 each `track!`); a producer with a free-running **global** sample counter must subtract the current chunk/epoch origin so every satellite reads a consistent per-chunk time grid (the estimator itself does not read it — it is preserved for downstream vector/Kalman tracking).
+
+### Transport delay
+
+The DLL discriminator uses the satellite's **pre-update** `code_doppler` as the value in effect during the integration — correct for a one-epoch feedback pipeline (the NCO written from this chunk's estimate takes effect on the next chunk). If your hardware pipeline has deeper feedback delay, that is the caller's responsibility: tag outputs with their epoch and schedule each NCO update to a known future epoch so the loop stays consistent with when the Doppler actually reaches the correlator.
+
 ## BandMeasurement
 
 One band's incoming sample buffer plus the front-end metadata needed to process it. Bundles `samples` with `sampling_frequency` and `intermediate_frequency` — these are inseparable in practice, and the bundle removes the chance of mismatched parallel NamedTuples in a multi-band `track` call.

src/Tracking.jl

@@ -41,6 +41,7 @@ export get_early,
     get_last_fully_integrated_filtered_prompt,
     get_filtered_prompts,
     get_correlator_outputs,
+    append_correlator_output!,
     CorrelatorOutput,
     get_bit_buffer,
     get_bits,

src/conventional_pll_and_dll.jl

@@ -621,10 +621,19 @@ post correlation filter.
 In the case that the that the correlation hasn't reached the end, e.g. in the case
 the incoming signal did not provide enough samples, it will return struct with
 zeroed values.
+
+The sampling-frequency argument may be either a [`BandMeasurements`](@ref)
+NamedTuple (the `track` path, from which the per-band rate is read) or a bare
+per-band sampling-frequency source — a `NamedTuple`/`Dict` keyed by
+`get_band_id` mapping each band to its sampling frequency. The latter is
+the entry point for an **external correlator producer** (e.g. an FPGA): it needs
+no sample buffer, only the per-signal `correlator_outputs` and the rate that
+maps `integrated_samples` to an integration time. See
+[External correlator producers](@ref).
 """
 function estimate_dopplers_and_filter_prompt(
     track_state::TrackState{<:SignalGroups,<:ConventionalPLLAndDLL},
-    measurements::BandMeasurements,
+    sampling_frequencies::Union{BandMeasurements,NamedTuple,AbstractDict},
 )
     # Detach the slot *values* from the input (sharing the key set), then
     # delegate to the in-place form. This step never changes the key set, so
@@ -635,37 +644,59 @@ function estimate_dopplers_and_filter_prompt(
     # ownership differs.
     new_track_state =
         TrackState(track_state; groups = _copy_groups_slot_vectors(track_state.groups))
-    estimate_dopplers_and_filter_prompt!(new_track_state, measurements)
+    estimate_dopplers_and_filter_prompt!(new_track_state, sampling_frequencies)
 end
 
-"""
-$(SIGNATURES)
+# Per-band sampling frequency for a group, from either a `BandMeasurements`
+# NamedTuple (read the rate off the band's `BandMeasurement`) or a bare
+# per-band rate source keyed by `get_band_id` (a `NamedTuple`/`Dict`). Both are
+# looked up by the group's band id, so the estimator stays per-band and is
+# never handed a scalar (groups may sit on different bands).
+@inline _band_sampling_frequency(m::BandMeasurements, key) = m[key].sampling_frequency
+@inline _band_sampling_frequency(fs::NamedTuple, key) = fs[key]
+@inline _band_sampling_frequency(fs::AbstractDict, key) = fs[key]
 
-In-place version of [`estimate_dopplers_and_filter_prompt`](@ref). Walks each
-group's `Vector{TrackedSat}` backing storage and overwrites slots with the
-new immutable `TrackedSat` value. Returns the same `track_state` object —
-allocation-free in steady state when [`track!`](@ref)'s preconditions are met.
-"""
 # Per-group body for the doppler estimator. Pulled out so
 # `_foreach_group!` can call it without boxing when the groups tuple
 # is heterogeneous (e.g. GPS L1 + Galileo E1B). The per-signal type
 # is recovered from each signal inside `_update_tracked_sat_doppler`.
-# Routes to this group's band's `BandMeasurement` for sampling frequency.
-@inline function _est_one_group!(g::SignalGroup, measurements::BandMeasurements)
+# Routes to this group's band's sampling frequency (see
+# `_band_sampling_frequency`).
+@inline function _est_one_group!(
+    g::SignalGroup,
+    sampling_frequencies::Union{BandMeasurements,NamedTuple,AbstractDict},
+)
     vals = g.satellites.values
     isempty(vals) && return nothing
-    sampling_frequency = measurements[get_band_id(g.band)].sampling_frequency
+    sampling_frequency = _band_sampling_frequency(sampling_frequencies, get_band_id(g.band))
     @inbounds for i in eachindex(vals)
         vals[i] = _update_tracked_sat_doppler(vals[i], sampling_frequency)
     end
     return nothing
 end
 
+"""
+$(SIGNATURES)
+
+In-place version of [`estimate_dopplers_and_filter_prompt`](@ref). Walks each
+group's `Vector{TrackedSat}` backing storage and overwrites slots with the
+new immutable `TrackedSat` value. Returns the same `track_state` object —
+allocation-free in steady state when [`track!`](@ref)'s preconditions are met.
+
+As with the immutable form, the second argument is either a
+[`BandMeasurements`](@ref) NamedTuple or a bare per-band sampling-frequency
+source keyed by `get_band_id`. The estimator only reads the per-band
+sampling frequency (to turn each output's `integrated_samples` into an
+integration time and to normalize the DLL discriminator); it consumes each
+signal's `correlator_outputs` and **clears** them, whether they were produced
+by the software correlate phase or appended by an external producer via
+[`append_correlator_output!`](@ref).
+"""
 function estimate_dopplers_and_filter_prompt!(
     track_state::TrackState{<:SignalGroups,<:ConventionalPLLAndDLL},
-    measurements::BandMeasurements,
+    sampling_frequencies::Union{BandMeasurements,NamedTuple,AbstractDict},
 )
-    _foreach_group!(_est_one_group!, track_state.groups, measurements)
+    _foreach_group!(_est_one_group!, track_state.groups, sampling_frequencies)
     return track_state
 end
 

src/correlators/correlator.jl

@@ -14,13 +14,30 @@ then folds over those records after the chunk. Storing the raw correlator plus
 `integrated_samples` lets the estimator normalize it (dividing by the sample
 count) and matches `last_fully_integrated_correlator`.
 
+An **external correlator producer** (e.g. an FPGA) can build these itself and
+feed them straight to the estimator with [`append_correlator_output!`](@ref);
+see [External correlator producers](@ref).
+
 Fields:
 
   - `correlator`: the raw accumulated correlator at completion (not normalized).
+    Fill it with the accumulator scaling `normalize` expects — the raw
+    sum-of-products over `integrated_samples` — which an external producer gets
+    for free by reusing [`EarlyPromptLateCorrelator`](@ref) / `update_accumulator`.
   - `integrated_samples`: samples integrated into this output (for `normalize`,
-    the loop-filter `integration_time`, and the bit-buffer block count).
-  - `sample_index`: buffer-relative sample index at which this integration ended
-    (the epoch of the measurement; used by vector tracking).
+    the loop-filter `integration_time`, and the bit-buffer block count). For an
+    external producer this is the true sample count of that integration.
+  - `sample_index`: sample index at which this integration ended, on the time
+    grid the Doppler estimator and vector tracking read. The software correlate
+    phase writes it **buffer-relative** — the end sample within the current
+    `track!` measurement (`signal_start_sample` returns to 1 at the top of every
+    `track!` call). An external producer with a free-running **global** sample
+    counter must therefore map its global timestamp onto the same per-chunk
+    origin before storing it here: subtract the sample index of the current
+    chunk/epoch origin so the value is relative to the chunk the estimator is
+    folding, keeping every satellite on one consistent time grid. The estimator
+    itself does not read `sample_index` (the loop filters key off
+    `integrated_samples`); it is preserved for downstream vector/Kalman tracking.
 """
 struct CorrelatorOutput{C<:AbstractCorrelator}
     correlator::C

src/sat_state.jl

@@ -188,6 +188,30 @@ cleared by the Doppler estimator after each chunk, so it is empty between
 `downconvert_and_correlate!`. See [Chunked Doppler updates](@ref).
 """
 get_correlator_outputs(t::TrackedSignal) = t.correlator_outputs
+
+"""
+$(SIGNATURES)
+
+Append an externally built [`CorrelatorOutput`](@ref) to `signal`'s
+per-chunk `correlator_outputs` buffer and return `signal`.
+
+This is the blessed ingest path for an **external correlator producer** (e.g.
+an FPGA streaming correlator dumps): build a [`CorrelatorOutput`](@ref) from the
+producer's raw accumulator, sample count and chunk-relative end index, append it
+here per signal in `sample_index` order, then run
+[`estimate_dopplers_and_filter_prompt!`](@ref) with a per-band sampling-frequency
+source to fold the batch and update the NCOs — no sample buffer or
+`downconvert_and_correlate!` needed. The estimator consumes and **clears** the
+buffer as part of that call, so between chunks it is empty again (same contract
+as the software correlate phase). Prefer this over mutating the vector returned
+by [`get_correlator_outputs`](@ref) directly — it documents intent and is
+type-checked (the output's correlator type must match the signal's).
+
+See [External correlator producers](@ref) for the full offload contract.
+"""
+append_correlator_output!(t::TrackedSignal, output::CorrelatorOutput) =
+    (push!(t.correlator_outputs, output); t)
+
 get_post_corr_filter(t::TrackedSignal) = t.post_corr_filter
 get_cn0_estimator(t::TrackedSignal) = t.cn0_estimator
 get_bit_buffer(t::TrackedSignal) = t.bit_buffer
@@ -709,9 +733,17 @@ has_bit_or_secondary_code_been_found(s::TrackedSat, sel...) =
     has_bit_or_secondary_code_been_found(_find_signal(s.signals, sel...))
 get_integrated_samples(s::TrackedSat, sel...) =
     get_integrated_samples(_find_signal(s.signals, sel...))
+get_correlator_outputs(s::TrackedSat, sel...) =
+    get_correlator_outputs(_find_signal(s.signals, sel...))
 get_preferred_num_code_blocks_to_integrate(s::TrackedSat, sel...) =
     get_preferred_num_code_blocks_to_integrate(_find_signal(s.signals, sel...))
 
+# Append an external `CorrelatorOutput` to one signal of a sat. `output` comes
+# first so an optional trailing signal selector (integer index / signal type)
+# disambiguates a multi-signal sat, matching the per-signal accessor ladder.
+append_correlator_output!(s::TrackedSat, output::CorrelatorOutput, sel...) =
+    (append_correlator_output!(_find_signal(s.signals, sel...), output); s)
+
 # Reset the satellite's signal-start sample and per-signal bit buffer between
 # `track` calls. Per-signal `filtered_prompts` vectors are emptied in place;
 # the per-signal `bit_buffer` is reset to its no-sync state; the

src/tracking_state.jl

@@ -789,6 +789,7 @@ const _SignalSelector = Union{Integer,Type{<:AbstractGNSSSignal}}
 for fn in (
     :get_integrated_samples,
     :get_correlator,
+    :get_correlator_outputs,
     :get_last_fully_integrated_correlator,
     :get_last_fully_integrated_filtered_prompt,
     :get_filtered_prompts,
@@ -813,6 +814,32 @@ for fn in (
     end
 end
 
+"""
+$(SIGNATURES)
+
+Append an externally built [`CorrelatorOutput`](@ref) to the buffer of the
+addressed signal and return `track_state`. The `output` is followed by the same
+satellite/signal addressing forms as the per-signal accessors (e.g.
+[`get_correlator_outputs`](@ref)):
+
+  - `append_correlator_output!(track_state, output)` — one group, one sat, one signal.
+  - `append_correlator_output!(track_state, output, prn)` — one group, one signal.
+  - `append_correlator_output!(track_state, output, group, prn)` — multi-group, one signal.
+  - `append_correlator_output!(track_state, output, group, prn, sig)` — per-signal.
+
+The signal-level method (`append_correlator_output!(::TrackedSignal, output)`)
+carries the contract; see it and [External correlator producers](@ref).
+"""
+append_correlator_output!(s::TrackState, output::CorrelatorOutput, id...) =
+    (append_correlator_output!(get_sat_state(s, id...), output); s)
+append_correlator_output!(
+    s::TrackState{<:SignalGroups},
+    output::CorrelatorOutput,
+    group::Union{Symbol,Integer,Val},
+    sat_id,
+    sig::_SignalSelector,
+) = (append_correlator_output!(get_sat_state(s, group, sat_id), output, sig); s)
+
 # Resolve the index of the addressed signal within a sat's signals tuple.
 # Config-time only (not the hot path), so plain control flow is fine.
 _signal_index(signals::Tuple) = length(signals) == 1 ? 1 : _throw_needs_signal_selector()