AR-drums

Virtual MIDI drum kit played in physical space with AR visuals and stick-based haptic feedback, driving any DAW. MVP target: FL Studio on macOS (Apple Silicon).

North star

Play a perceptually-convincing drum kit in mid-air or on a flat surface with:

• Strike-to-DAW-sound latency competitive with electronic kits (Roland TD-17 baseline ≈ 3 ms strike→MIDI, ≈ 10 ms strike→audio at low buffer). Target: < 15 ms end-to-end, stretch < 10 ms.
• AR overlay showing kit layout, hit zones, and visual feedback aligned to real-world striking surface or hand position.
• Tactile feedback in the stick that simulates the impact of striking a real drum head/cymbal.

MVP scope (Phase 0)

• 1 user, seated.
• 4 pads minimum: kick (foot or stick), snare, hi-hat, one cymbal.
• macOS Sonoma+ on MacBook Pro M1 acting as DAW host (FL Studio 21+).
• iPhone 12 as AR display and/or sensor hub.
• MIDI delivered to FL Studio via Core MIDI (USB or Network MIDI).
• Velocity-sensitive strikes. No advanced articulations (rim, choke) yet.

Out of scope for MVP

• Multi-user / networked play.
• Foot pedal hardware beyond MIDI footswitch.
• Cymbal choke, rim shots, brush detection.
• Windows / Linux / iPad-only modes.
• Other DAWs (Ableton, Logic) — design must not block them, but no testing in MVP.

Available resources

• MacBook Pro M1 — primary compute, DAW host, build machine.
• iPhone 12 — ARKit, Core MIDI, Bluetooth LE central, Wi-Fi 6 (802.11ax — iPhone 12 supports Wi-Fi 6; macOS M1 also).
• Network: local Wi-Fi, BLE 5.0 on both devices, USB-C / Lightning.
• No commitment to specific drumstick hardware yet — we will evaluate off-the-shelf (Aerodrums, Senstroke, Freedrum), bespoke ESP32-based sticks, and hybrid optical/IMU approaches. See docs/HARDWARE.md.

Parallel agent workflow

Two coding agents work this repo in parallel:

• Claude Code (Anthropic) — host app, AR scene, macOS/iOS integration, MIDI plumbing, system perf.
• OpenAI Codex — firmware, embedded signal processing, hardware prototyping, custom wireless link.

Both follow AGENTS.md (logging, ownership, handoff). Read it first.

Repository map

.
├── README.md                       Project intent and MVP scope
├── AGENTS.md                       Instructions for Claude Code and Codex (READ FIRST)
├── docs/
│   ├── ARCHITECTURE.md             Components, data flow, interfaces
│   ├── RESEARCH.md                 Questions to answer BEFORE coding
│   ├── HARDWARE.md                 Drumstick options, custom hardware ideas
│   ├── AR_DISPLAY.md               AR display options compared
│   ├── LATENCY.md                  Targets, budget, e-drum reference
│   ├── DAW_INTEGRATION.md          MIDI, Core MIDI, FL Studio specifics
│   ├── MCP_SKILLS.md               MCP servers and skills to accelerate work
│   ├── RISKS.md                    Risk register
│   └── ROADMAP.md                  Phased plan
└── logs/
    ├── SESSION_LOG.md              Terse timestamped findings (handoff doc)
    └── prompts/                    Daily-rotated verbatim prompt log
        ├── README.md               Rotation policy
        └── prompts-2026-05-17.md   Today's prompts

Status

Phase: Pre-development research. No code written. See docs/RESEARCH.md for the open question set.