Skip to content

feat: Offline (batch) capture mode — store BLE audio locally, transcribe on demand#7981

Draft
mdmohsin7 wants to merge 45 commits into
mainfrom
feat/batch-offline-capture
Draft

feat: Offline (batch) capture mode — store BLE audio locally, transcribe on demand#7981
mdmohsin7 wants to merge 45 commits into
mainfrom
feat/batch-offline-capture

Conversation

@mdmohsin7

@mdmohsin7 mdmohsin7 commented Jun 16, 2026

Copy link
Copy Markdown
Member

Summary

Adds an Offline (batch) capture mode — the opposite of realtime transcription. When enabled, the app keeps receiving BLE audio from the device but does not open the transcription websocket. Instead the native layer (Android + iOS) stores the audio to local .bin files with timestamps; the recordings appear inline in the conversations list, and the user transcribes any one on demand — the existing /v2/sync-local-files backend turns it into a conversation.

Doing the storage natively means the Flutter engine does no per-packet work while the app is minimized/closed — lower battery/CPU/network when idle.

Architecture — recordings are their own subsystem (not WALs)

Local recordings are deliberately decoupled from the offline-sync WAL stack. The recordings directory is the queue and the .bin files are the single source of truth:

  • LocalRecording (model) + LocalRecordingsProvider (lib/providers/local_recordings_provider.dart) scan the recordings dir → list; nothing is stored in the WAL _wals list.
  • Only the stateless leaf helpers are reused: uploadLocalFilesV2 (the /v2/sync-local-files call), fetchSyncJobStatus (reconcile), and AudioPlayerUtils/WaveformUtils for on-device opus playback (fed a throwaway Wal, never stored).
  • In-flight upload job_ids persist in a small sidecar (a SharedPreferences map), reconciled by a lightweight 15s poll; the local file is deleted only when the job reports completed, and a completed job's conversation is inserted via ConversationProvider.upsertConversation. App-kill mid-processing recovers on next launch.

This keeps the Sync page (device SD/flash offline-sync) completely untouchedsync_provider.dart / local_wal_sync.dart are unchanged. Recordings and device-sync files no longer leak into each other's screens.

What it does

Native writers (Android OmiBatchAudioWriter.kt, iOS BatchAudioWriter.swift)

  • Append each opus frame as [4-byte LE length][frame] into audio_omibatch_{codec}_{sr}_{ch}_fs{frameSize}_{startSec}.bin (the exact format /v2/sync-local-files decodes). The omibatch device-segment marker distinguishes batch recordings from offline-sync WAL files that share the same directory + naming (the backend treats that segment as a free-form label).
  • Rotate by size (~32 MB) / duration (~30 min); start a new file after a BLE silence gap; finalize on disconnect.
  • Durability: write to a .bin.part file and atomically rename to .bin only once sealed; recover a stale .bin.part left by a crashed process on next start; periodic fsync; a free-space guard that pauses + flags rather than crashing.

Wiring

  • Android foreground service routes each packet to the writer (mutually exclusive with background streaming via prefs); iOS OmiBleManager.didUpdateValueFor hands audio to the writer in batch mode and skips the Dart forward; background capture rides the existing bluetooth-central mode + state restoration.
  • Capture provider: in batch mode the transcription socket never opens; it writes the native config (batchAudioDir, etc.) for the writer. No WAL ingestion.
  • LocalRecordingsProvider is refreshed on BLE disconnect, app resume, and conversations-page load.
  • Settings → Offline Mode toggle (Android + iOS) with a low-storage warning.

Recordings in the conversations list

  • Unsynced local recordings appear inline on the conversations page, grouped into the same date buckets and time-sorted alongside conversations.
  • A recording row has no title/icon (not processed yet) — it shows time + duration, state, and an inline play/pause button that decodes and plays the local audio on device; swipe to delete; tap opens a dedicated recording detail page (waveform + playback + Transcribe + share + delete + info), where Transcribe uploads that single recording and turns it into a conversation.

Verification

  • flutter analyze clean on changed Dart; filename-parser unit tests pass; iOS BatchAudioWriter.swift typechecks.
  • On-device testing in progress (iOS). Full app build + the failure matrix (crash/kill, storage-full, BLE drop, device off) still to do.

Remaining (follow-ups)

  • Translate the offlineMode* settings strings to non-English locales (the recordings UI reuses existing localized strings — no new keys added).
  • Full end-to-end device testing across the failure matrix.

🤖 Generated with Claude Code

Appends incoming BLE audio frames to length-prefixed .bin files in the app documents directory (the format the offline sync pipeline accepts) instead of streaming to the transcription socket. Includes size/duration rotation, gap-based finalize, periodic fsync, a journal so partially written files are never ingested, and a free-space guard.
… batch mode

Dispatches each audio packet to the batch writer alongside the background streamer (the two self-gate via prefs), subscribes to the audio characteristic when the engine is gone, finalizes the file on destroy, and keeps the foreground service sticky whenever batch mode is on.
…cordings

In batch mode the transcription websocket is never opened and the Dart WAL writer stays off (native owns writing). Finalized .bin files are registered into the existing WAL upload/reconcile pipeline via addExternalWal.
@ThomsenDrake

Copy link
Copy Markdown
Collaborator

ClawSweeper local pilot review

Recommendation: keep open, but not merge-ready.

What it found:

  • The offline/batch capture direction is useful and worth maintainer review.
  • The hardware path lacks real-device proof.
  • New localized strings appear to ship as English in non-English locales.
  • Audio/transcription flow docs were not updated, and the interrupted-recording recovery story needs to be clear.

Suggested next step: add real-device proof, translate the new l10n keys, update the audio/transcription pipeline docs, and document recovery behavior for interrupted journaled recordings.

Posted from a local report-only ClawSweeper pilot by request; no labels, closes, repairs, or merges were performed.

mdmohsin7 added 22 commits June 17, 2026 15:56
Pure, testable parser for batch recording filenames (codec, frame size, timestamp) mirroring how the offline-sync backend interprets the name.
Active files are now *.bin.part (excluded by the .bin scan), so the journal-based skip is no longer needed.
…ch writer

Write to .bin.part and rename to .bin only when sealed (rotation/gap/stop) so a half-written file is never ingested; promote stale .bin.part from a crashed process on next start. Replaces the journal.
Mirrors the Android writer: length-prefixed .bin files, size/duration rotation, silence-gap finalize, periodic fsync, atomic .bin.part rename, stale-part recovery, and a free-space guard.
In didUpdateValueFor, hand audio packets to BatchAudioWriter when batch mode is on and skip forwarding to Dart so the Flutter engine stays idle.
A conversations-list row for a local batch recording: time + duration, sync status, swipe-to-delete, and an inline play/pause button that decodes and plays the local audio on device. Tapping opens the WAL detail page.
ConversationsGroupWidget now merges conversations and unsynced recordings into one time-sorted list under each date header.
Group pendingWals by date, union with conversation dates, and render them in the same list (default view only); empty-state guards account for recordings.
Filter pendingWals to the batch marker so device SD-card/flash sync WALs and realtime offline buffers no longer appear here (also avoids their duplicate-id collisions).
mdmohsin7 added 16 commits June 17, 2026 18:38
…hange)

The previous trigger lived in onConnectionStateChanged, which is wired to network connectivity — so a device disconnect never re-scanned for the finalized recording. Moved to device_provider.onDeviceDisconnected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants