Implementation of custom bytetrack graph [GSoC'26]#131
Open
Vishwa2684 wants to merge 10 commits into
Open
Conversation
Vishwa2684
changed the title
Author
|
@atobiszei To test the example in real time, I'm currently using IP Webcam app in my mobile. Which starts a server. And I pass the camera ip address as |
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
3 times, most recently
from
bb74456 to
11cacc4
Compare
Author
|
@dtrawins Could you please review |
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
2 times, most recently
from
bc86735 to
b5b4cd4
Compare
Collaborator
|
Hi Vishwa, In the attached video you can see that the person in yellow vest, is standing almost still |
atobiszei
reviewed
May 26, 2026
There was a problem hiding this comment.
Pull request overview
This PR adds a MediaPipe-based multi-object tracking demo pipeline built around a custom C++ implementation of the ByteTrack algorithm, with graphs for both TFLite (CPU) and OpenVINO Model Server (OVMS) inference backends and optional visualization utilities.
Changes:
- Added ByteTrack tracker core (track objects, Kalman filter, association/matching utilities) and a
ByteTrackCalculator. - Added YOLOX decoding calculators (TFLite + OpenVINO) and example graphs/demos wiring detection → tracking → rendering.
- Added visualization support (color-by-track-id) plus build/Docker integration for OVMS demo builds.
Reviewed changes
Copilot reviewed 39 out of 40 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| mediapipe/models/BUILD | Adds model/label filegroups for the demos. |
| mediapipe/graphs/bytetrack/test_ovms.pbtxt | Minimal OVMS test graph wiring flow limiting, detection, tracking, rendering. |
| mediapipe/graphs/bytetrack/test_cpu.pbtxt | Minimal CPU test graph wiring flow limiting, detection, tracking, rendering. |
| mediapipe/graphs/bytetrack/subgraphs/renderer_cpu.pbtxt | Rendering subgraph (detections → RenderData → overlay). |
| mediapipe/graphs/bytetrack/subgraphs/object_tracking_cpu.pbtxt | Tracking subgraph wrapping ByteTrack usage. |
| mediapipe/graphs/bytetrack/subgraphs/object_detection_ovms.pbtxt | OVMS detection subgraph definition. |
| mediapipe/graphs/bytetrack/subgraphs/object_detection_cpu.pbtxt | CPU (TFLite) detection subgraph definition. |
| mediapipe/graphs/bytetrack/subgraphs/BUILD | Bazel targets to register bytetrack subgraphs. |
| mediapipe/graphs/bytetrack/calculators/strack.h | Adds the per-track state container (STrack). |
| mediapipe/graphs/bytetrack/calculators/strack.cc | Implements STrack prediction/update/activation logic. |
| mediapipe/graphs/bytetrack/calculators/render_data_passthrough_calculator.cc | Debug calculator to log/forward RenderData. |
| mediapipe/graphs/bytetrack/calculators/matching_utils.h | Matching utilities (IoU cost, score fusion, assignment). |
| mediapipe/graphs/bytetrack/calculators/kalman_matrices.h | Kalman model matrices/constants helpers. |
| mediapipe/graphs/bytetrack/calculators/kalman_filter.h | KalmanFilter API for tracker state estimation. |
| mediapipe/graphs/bytetrack/calculators/kalman_filter.cc | KalmanFilter implementation (init/predict/update/multipredict). |
| mediapipe/graphs/bytetrack/calculators/bytetrack_calculator.proto | ByteTrack calculator options proto. |
| mediapipe/graphs/bytetrack/calculators/bytetrack_calculator.h | ByteTrackCalculator interface and state. |
| mediapipe/graphs/bytetrack/calculators/bytetrack_calculator.cc | Main ByteTrack algorithm calculator implementation. |
| mediapipe/graphs/bytetrack/calculators/BUILD | Bazel build targets for bytetrack calculators/libs. |
| mediapipe/graphs/bytetrack/calculators/basetrack.h | Base track abstraction and track-state enum. |
| mediapipe/graphs/bytetrack/calculators/basetrack.cc | BaseTrack static id counter definition. |
| mediapipe/graphs/bytetrack/bytetrack_ovms.pbtxt | Full OVMS demo graph (YOLOXn via OVMS → ByteTrack → overlay). |
| mediapipe/graphs/bytetrack/bytetrack_cpu.pbtxt | Full CPU demo graph (YOLOX TFLite → ByteTrack → overlay). |
| mediapipe/graphs/bytetrack/BUILD | Bazel targets collecting calculator deps for bytetrack graphs. |
| mediapipe/examples/desktop/bytetrack/README.md | Demo documentation and graph diagrams. |
| mediapipe/examples/desktop/bytetrack/BUILD | Bazel binaries for bytetrack desktop demos. |
| mediapipe/calculators/util/detection_color_by_id_calculator.proto | Options proto for color-by-id visualization. |
| mediapipe/calculators/util/detection_color_by_id_calculator.cc | Calculator producing RenderData with per-id colors and labels. |
| mediapipe/calculators/util/BUILD | Registers the new visualization calculator + proto targets. |
| mediapipe/calculators/tflite/yolox_tensors_to_detections_calculator.proto | Options proto for YOLOX TFLite decode calculator. |
| mediapipe/calculators/tflite/yolox_tensors_to_detections_calculator.cc | YOLOX TFLite output decoding into Detection protos. |
| mediapipe/calculators/tflite/BUILD | Registers YOLOX TFLite decode calculator + proto targets. |
| mediapipe/calculators/ovms/config.json | Adds OVMS model config entry used by bytetrack OVMS demo. |
| mediapipe/calculators/ovms/BUILD | Wires OpenVINO YOLOX decode calculator into OVMS deps. |
| mediapipe/calculators/openvino/openvino_yolox_tensors_to_detections_calculator.proto | Options proto for YOLOX OpenVINO decode calculator. |
| mediapipe/calculators/openvino/openvino_yolox_tensors_to_detections_calculator.cc | YOLOX OpenVINO output decoding into Detection protos. |
| mediapipe/calculators/openvino/BUILD | Registers YOLOX OpenVINO decode calculator + proto targets. |
| Dockerfile.openvino | Downloads extra YOLOX assets and stages OVMS model layout in Docker build. |
| build_desktop_examples.sh | Special-cases bytetrack example target selection during build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
3 times, most recently
from
63aca34 to
6fd2a27
Compare
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
from
6fd2a27 to
12e8d51
Compare
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
from
297de9d to
e13fe8a
Compare
Author
|
@atobiszei could you review the following calculators?
And I'm using the yolox model from this repo. I've downloaded the nano weights from the model zoo in this repository, then i converted them to ONNX following the model guidelines and then converted the ONNX weights to tflite. |
atobiszei
reviewed
May 27, 2026
Comment on lines
+1
to
+10
| #include <vector> | ||
| #include <algorithm> | ||
| #include <numeric> | ||
| #include "mediapipe/framework/calculator_framework.h" | ||
| #include "mediapipe/framework/formats/detection.pb.h" | ||
| #include "mediapipe/framework/formats/location_data.pb.h" | ||
| #include "mediapipe/framework/port/ret_check.h" | ||
| #include "mediapipe/framework/port/status.h" | ||
| #include <cmath> | ||
| #include "mediapipe/calculators/openvino/openvino_yolox_tensors_to_detections_calculator.pb.h" |
Collaborator
There was a problem hiding this comment.
Follow standard Google C++ Style guide with includes. Eg:
#include "mediapipe/calculators/openvino/openvino_yolox_tensors_to_detections_calculator.pb.h"
#include
#include
#include
#include
#include <openvino/openvino.hpp>
#include "mediapipe/framework/calculator_framework.h"
#include "mediapipe/framework/formats/detection.pb.h"
#include "mediapipe/framework/formats/location_data.pb.h"
#include "mediapipe/framework/port/ret_check.h"
#include "mediapipe/framework/port/status.h"
atobiszei
reviewed
May 27, 2026
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
from
2aa15b6 to
a8ed205
Compare
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
from
efd6ee2 to
b98e792
Compare
Vishwa2684
force-pushed
the
custom_bytetrack_graph
branch
from
b98e792 to
4b28af1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello @atobiszei , @dtrawins
This is the implementation of my bytetrack algorithm as stated in my proposal. To fix the issues in my previous approach , I did the following things:
filter_detections_by_confidence_calculatorwhich was stated in the proposal.This is the output video generated by my graph. I'm looking forward to any suggestions to improve my approach and correct any mistakes which are done on my side.
out_bt_ovms.mp4
out_bt_ovms_desk.mp4
Regards,
Konkala Vishwa Teja