Skip to content

Audio file support: no AudioExtractor registered; audio files scan but yield no metadata #175

Description

@utof

Problem

CompositeExtractor::default_extractors() (in crates/media/src/extractor.rs) only registers ImageExtractor and VideoExtractor. There is no AudioExtractor. As a result, audio-only files (.mp3, .m4a, .wav, .flac, .ogg, .opus) that get picked up by a scan flow through the queue but their MetadataExtractor dispatch produces no match — duration, bitrate, codec, and the rest of MediaMetadata stay None for audio files.

Behavior today, end to end:

  • Scanner walks a directory, hashes each file, inserts a row into files.
  • mime_guess will tag .mp3audio/mpeg, .m4aaudio/mp4, .wavaudio/wav, etc.
  • MetadataQueue worker calls composite.extract(hash, path, mime).
  • No registered extractor accepts audio/*, so no MediaMetadata row is written.
  • File appears in the table by name + path + size + hash but is otherwise opaque.

Impact

  • Search by audio metadata (duration, codec, bitrate) impossible.
  • File-detail sidebar (File-detail sidebar — surface extracted metadata + future preview/transcription/tags #153) cannot surface anything beyond filesystem-level fields for audio.
  • Anything downstream that wants to operate on audio files first (e.g. a future speech-to-text pipeline that needs to know "this is audio, length N, codec X" before deciding whether to transcribe) will have to re-derive the basics.
  • User-facing: a music or podcast directory looks broken — files scan in, but nothing about them is indexed.

Suggested approach

New AudioExtractor impl living next to VideoExtractor in crates/media/src/extractor.rs. accepts(mime) returns true for audio/*. extract reads container metadata only (duration, bitrate, sample rate, codec, channel count) — no thumbnail, no waveform image in v1.

Container/codec coverage worth weighing:

  • symphonia (pure Rust, MPL-2.0) — handles MP3, FLAC, WAV, OGG/Vorbis, OGG/Opus, MP4/AAC, MKV, WebM. No system deps. Aligns with the existing "no libavcodec in bundle" rule.
  • mp4parse (already a workspace dep) — already covers audio/mp4 (m4a) container; could extract duration + codec for those without adding a crate.
  • lofty (Rust, MIT/Apache-2.0) — focused on audio tag/metadata reading (ID3, Vorbis comments, MP4 atoms). Handles every common audio container and is cheaper than full decode if all we want is metadata. Worth comparing against symphonia for this specific use case.

Single-extractor approach is preferred over per-format extractors — keep the MetadataExtractor registry small.

Out of scope for this issue

  • Speech-to-text / transcription of the extracted audio.
  • Waveform thumbnail generation.
  • Audio playback in the desktop UI.
  • Tag/lyric metadata (artist / album / title from ID3 etc.) — separate concern; could be a follow-up once basic extraction lands.

Definition of done

  • AudioExtractor registered in CompositeExtractor::default_extractors().
  • MediaMetadata rows populated for audio/* files with at least duration_ms, codec, bitrate_bps where the source provides them.
  • Integration tests in crates/media/tests/integration.rs for at least one mp3, one m4a, and one wav fixture (synthesized via test helpers, following the existing convention in crates/media/tests/common/mod.rs).
  • File-detail sidebar (File-detail sidebar — surface extracted metadata + future preview/transcription/tags #153 follow-up) gets non-empty metadata for audio files.

Adjacent

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority/mediumDefault priority — address in normal flowstatus/needs-researchDeep multi-source research required before committing to an approachtype/featureNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions