Skip to content

MOSS-Music stage extraction: expose valid-frame metadata in stable C ABI #344

Description

@jajmangold

Context

The moss_audio backend already exposes encoder, DeepStack tap, and adapter stage helpers in src/moss_audio.h / src/moss_audio.cpp. A downstream MOSS-Music-8B-Thinking feature pipeline needs a stable way to obtain the final 1280-D encoder stream, taps 8/16/24, and the 4096-D adapted stream together with observed valid-frame metadata, without the 3000-frame mel padding being mistaken for real content.

This request is only for a stable source-level C ABI. Downstream HTTP, CAS publishing, authentication, and rights policy remain outside CrispASR.

Acceptance criteria

  • Add new exported C symbols without changing existing moss_audio_* signatures. A metadata-returning encoder entrypoint returns the existing tensor and out_T_enc / out_d, plus caller-allocated per-chunk valid-frame counts, out_num_chunks, the pre-pad out_T_mel_actual, and out_total_valid.
  • Reuse the existing encoder chunk loop and its computed valid_lens; do not derive valid frame counts from a global floor formula.
  • Preserve the existing final 1280-D encoder output and DeepStack taps 8/16/24. Report the adapter output dimension from GGUF metadata/weights rather than hardcoding it.
  • Correct the stale header comment: moss_audio.llm.hidden_size is 2560 for MOSS-Audio-4B and 4096 for MOSS-Music-8B-Thinking.
  • Fail closed at load time if llm_hidden disagrees with the adapter output-row shape; do not silently accept a 2560 fallback for a 4096-D checkpoint.
  • Document ownership and threading: one context has one in-flight stage run; callbacks, if added, run synchronously on the caller thread and must not re-enter the same context; caller owns returned allocations/copies.
  • Preserve cached-encoder-graph invalidation/rebuild behavior on the new entrypoint.
  • Add a differential test proving the sum of per-chunk valid frames equals out_total_valid and that a sub-30-second input does not expose zero-content frames created only by the 3000-frame mel pad.

Suggested smallest patch

  • src/moss_audio.h: declare a metadata-returning encoder API and document dimensions, ownership, and single-inflight behavior.
  • src/moss_audio.cpp: expose the existing per-chunk valid_lens, retain the pre-pad mel length, validate adapter/LLM dimensions, and keep existing symbols untouched.

An optional chunk-local callback may follow the existing moss_audio_process_cb pattern, but it is not required if the metadata-returning entrypoint can return final/tap/adapter tensors without duplicate model execution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions