gsc_pgo: online and offline PGO #2587
Open
jeff-hykin wants to merge 24 commits into
Open
Conversation
Convert the memory2 Recorder from thread/disposable rx subscriptions to manual async callbacks via process_observable, and let pose_setter_for methods be async (awaited in _resolve_pose). Update the fastlio and go2 recorders accordingly.
Raise TypeError at decoration time if a non-async function is decorated, and always await the setter in _resolve_pose.
…imos into jeff/fix/pose_setter_for
…kitti, voxel_map, module_loading)
jeff-hykin
requested review from
leshy,
mustafab0,
paul-nechifor and
spomichter
as code owners
process_observable gains an optional on_drop callback fired once per message dropped by the dispatcher's single-slot LATEST mailbox. The Recorder uses it to count dropped frames per stream and log a throttled warning, so a slow sink no longer loses data silently.
Contributor
Greptile SummaryThis PR ports the PGO / loop-closure stack into the
Confidence Score: 5/5Safe to merge — all previously-flagged concerns (SQL injection, private-method access, missing atexit cleanup, thread-safety) have been addressed in this revision. The core additions (PGO NativeModule, offline post-processing pipeline, memory2 tf feature) are well-structured with proper table-name guards, double-checked locking, and a public lookup API. The eval harness adds a minor cross-class coupling and temp-file collision edge case, but neither affects production correctness — the eval scripts run in isolated dev sessions. eval.py — temp-file naming and cross-class RateReplay dependency worth a follow-up cleanup; post_process.py — unconditional camera_intrinsics.json read should be deferred behind the RAW_STREAM guard for better UX on lidar-only recordings. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant LR as LockstepReplay
participant PGO as gsc_pgo (C++ binary)
participant GC as GraphCapture
participant Eval as evaluate()
Eval->>LR: start replay (odom + lidar)
loop per scan
LR->>PGO: odometry (fire-and-forget burst)
LR->>PGO: lidar scan
PGO-->>LR: corrected_odometry (ack)
LR->>LR: advance to next scan
end
PGO-->>GC: pose_graph / loop_closure_event
GC->>GC: write JSON to temp file
Eval->>Eval: read JSON → score tag agreement + voxel agreement
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant LR as LockstepReplay
participant PGO as gsc_pgo (C++ binary)
participant GC as GraphCapture
participant Eval as evaluate()
Eval->>LR: start replay (odom + lidar)
loop per scan
LR->>PGO: odometry (fire-and-forget burst)
LR->>PGO: lidar scan
PGO-->>LR: corrected_odometry (ack)
LR->>LR: advance to next scan
end
PGO-->>GC: pose_graph / loop_closure_event
GC->>GC: write JSON to temp file
Eval->>Eval: read JSON → score tag agreement + voxel agreement
Reviews (6): Last reviewed commit: "rename DEFAULT_MARKER_LENGTH_M -> DEFAUL..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## main #2587 +/- ##
==========================================
+ Coverage 69.61% 70.88% +1.27%
==========================================
Files 878 876 -2
Lines 79326 77819 -1507
Branches 7126 6917 -209
==========================================
- Hits 55220 55164 -56
+ Misses 22301 20851 -1450
+ Partials 1805 1804 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 16 files with indirect coverage changes 🚀 New features to boost your workflow:
|
…eff/feat/jnav_pgo
…ose cached stores
…ms) so add_april imports resolve
jeff-hykin
changed the title
github-actions
Bot
added
the
ready-to-merge
github-actions
Bot
added
ready-to-merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gsc_pgo, Ports the PGO / loop-closure stack into the new
jnavlayout, plus a tf-tree feature formemory2stores.