Feature Request
Add support for SAM2 (Segment Anything Model 2) segmentation masks.
Background
CoreMLPlayer currently supports:
- Object Detection (bounding boxes)
- Image Classification (labels)
SAM2 outputs segmentation masks (pixel-level), which would be a new visualization type for the app.
Requirements
- Handle mask outputs - Parse SAM2's multi-array segmentation data
- Render masks - Create SwiftUI overlay views for semi-transparent masks
- Handle SAM2 inputs - Support point/box prompts for interactive mode
- Mask visualization settings - Color, opacity, blend mode controls
Existing Infrastructure to Leverage
- ✅ Video/image pipeline
- ✅ CoreML loading (.mlpackage support already added)
- ✅ Coordinate transformation
- ✅ Overlay rendering system (DetectionRect)
- ✅ Model configuration UI
Technical Notes
- SAM2 outputs multi-array masks
- Need new SwiftUI view for mask overlay (similar to DetectionRect)
- Vision framework doesn't have built-in SAM2 support, so direct CoreML usage needed
- Mask data likely needs conversion from multi-array to visual format
Feature Request
Add support for SAM2 (Segment Anything Model 2) segmentation masks.
Background
CoreMLPlayer currently supports:
SAM2 outputs segmentation masks (pixel-level), which would be a new visualization type for the app.
Requirements
Existing Infrastructure to Leverage
Technical Notes