add roadmap#65
Conversation
niksirbi
left a comment
niksirbi
left a comment
There was a problem hiding this comment.
Thanks @themattinthehatt! Nice to see you've followed movement's roadmap structure.
I've left some comments with my initial impressions. Happy to further discuss in the meeting tomorrow and do another round of review.
| ideas. | ||
| We will take community feedback into account when planning future releases. | ||
|
|
||
| ## Long-term vision |
There was a problem hiding this comment.
It feels like point tracking is not as prominent in this long-term vision. Could we integrate it more into the first two bullet points (which are currently very pose estimation-centric)? The other four bullet points are general by nature.
There was a problem hiding this comment.
I'm just flagging this because it's somewhat inconsistent with the Mission statement and the overview figure (both of which give equal weight to pose estimation and point tracking), so we have to decide what gives (do we adapt the roadmap to the mission or vice versa?).
There was a problem hiding this comment.
More generally, I was thinking how your 6 long-term vision points map onto the existing 3 Mission points:
- The first 3 vision points are mostly in service of the 1st mission point
- The 4th vision point partly addresses the 2nd mission point
- The 5th vision point is in service of the 3rd mission point
- The 6th vision point is more general.
Just wanted to highlight this, no concrete suggestions yet.
There was a problem hiding this comment.
I drafted this based on the proposal, but you're right that it should include more point tracking verbiage - this is still certainly an important part of the roadmap in my opinion. I added CoTracker as our first point tracker (under 2026), with more to be added in 2027 (with TAPIR as an example).
| - **Support multi-animal and multi-camera datasets.** The schema and tooling | ||
| should extend naturally beyond single-animal, single-camera use cases. | ||
|
|
||
| ## Focus areas for 2026 |
There was a problem hiding this comment.
Similar to the above comment, point tracking is completely missing from focus areas.
There is also no mention of evaluation metrics for comparing the outputs across models (which is mentioned in the Mission). In general, the roadmap is very thin on the evaluation metrics side.
There was a problem hiding this comment.
Added both - CoTracker inference in 2026/TAPIR inference in 2027; define standardized eval metrics in 2026.
| - [ ] Define and document a stable programmatic interface for **training** | ||
| Lightning Pose models via PoseInterface. |
There was a problem hiding this comment.
What about a programmatic interface for running point tracking inference? Have we abandoned that front?
| schema, plus submission utilities for contributing datasets to the benchmark | ||
| corpus. |
There was a problem hiding this comment.
I feel like the submission utilities are already well underway, given our poseinterface.io module, the 2 gallery examples, and our ongoing work to enable proofreading in movement. So perhaps that bit should be in 2025? Unless you meant something else.
There was a problem hiding this comment.
good point; it's a little weird with the proposal timing, since it technically doesn't start until Dec 2026, but I'd rather align the roadmap with what we've actually done/are doing rather than tying it too closely to the proposal (at least in terms of dates).
| standardized schema, sharing code with the Lightning Pose converter where | ||
| possible. | ||
| - Define and document a stable, agent-callable programmatic interface for | ||
| running Lightning Pose **inference** on raw video via PoseInterface. |
There was a problem hiding this comment.
Lines 53 and 73 spell the project as PoseInterface, but in most other place we refer to it as poseinterface (monospace and lowercase)
There was a problem hiding this comment.
Aslo, what's the reason for doing tacking LP training before tackling LP inference? I'm curious about your thought process on this.
There was a problem hiding this comment.
fixed both - yes we definitely want to do LP inference first
2 participants
Before submitting a pull request (PR), please read the contributing guide.
Please fill out as much of this template as you can, but if you have any problems or questions, just leave a comment and we will help out :)
Description
What is this PR
Why is this PR needed?
Addition of roadmap for the documentation website.
What does this PR do?
Add a single roadmap document that lays out development plans.
References
None
How has this PR been tested?
No new code.
Is this a breaking change?
No.
Does this PR require an update to the documentation?
Documentation update only.
Checklist: