Expose CEF audio stream callbacks through the Dullahan API#32
Open
t-noami wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This is my first time opening a pull request to this project, so please let me know if I should adjust the format, scope, or explanation.
Thank you for maintaining Dullahan and the technical foundation that keeps MOAP media playback working in Second Life viewers. Dullahan is also an important part of my own creative work in Second Life, where I perform music as "Tia Rungray", so I appreciate the long-term maintenance of this project and the systems it supports.
Summary
This PR exposes CEF audio stream callbacks through the Dullahan public API.
Embedding applications can now receive browser audio as planar float PCM instead of relying only on the native OS audio output selected by CEF. Dullahan forwards the CEF audio stream events to the embedder and does not resample, mix, buffer, or play the audio itself.
The new public callbacks are:
setOnAudioStreamStartedCallback(...)setOnAudioStreamPacketCallback(...)setOnAudioStreamStoppedCallback(...)setOnAudioStreamErrorCallback(...)Motivation
AYAstorm is a downstream Firestorm / Second Life viewer that embeds Dullahan for web media:
https://github.com/mayatonton/phoenix-firestorm.git
In that viewer, media and MOAP audio need to be routed through the viewer audio system instead of bypassing it through CEF's native audio output. This lets the viewer apply its own media volume, mute, buffering, routing, and spatial-audio policy.
AYAstorm also has a 3D Stream spatial audio system built on FMOD APIs. For that use case, browser media audio has to be available to the viewer as decoded PCM so the downstream application can route it into FMOD channel groups, 3D channels, speaker placement, rolloff, occlusion, binaural / HRTF processing, and related DSP.
Dullahan should not own that playback policy. This PR only adds the API boundary needed for embedders to receive the audio stream and make their own routing decisions.
Audio Format Rationale
GetAudioParameters()currently requests this CEF audio format:The values are intended to provide a stable decoded PCM contract for downstream audio routing, not to make Dullahan implement 7.1 playback itself.
CEF_CHANNEL_LAYOUT_7_1is used as a surround-capable callback bus. The main downstream target is Ogg Opus 6-channel / 5.1 streaming, but Ogg Opus mapping family 1 defines conventional surround layouts from 1 to 8 channels, including 5.1 and 7.1. Requesting a 7.1 callback layout preserves channel identity for surround sources before the embedder maps the PCM into its own audio engine.48000Hz matches Opus fullband audio and the 48 kHz timing model used by Ogg Opus. It also matches the downstream AYAstorm / FMOD 3D Stream processing path.1024frames is the packet granularity used by the downstream media ring / FMOD callback path, avoiding an extra repacketization step in the embedder.Chromium supports Opus as a default audio decoder type, so Ogg Opus surround streams that Chromium / CEF can play as web media can be exposed to the embedder as decoded PCM through this callback API.
References:
Implementation Notes
dullahan_browser_clientnow implementsCefAudioHandler.GetAudioHandler()returns the browser client so CEF can deliver audio stream events.OnAudioStreamStarted()reports sample rate, channel count, channel layout, and frames per buffer throughdullahan_audio_stream_info.OnAudioStreamPacket()forwards CEF'sconst float** data,frames, andptsto the callback manager.Compatibility
This is intended to be source-compatible for existing Dullahan users:
The main behavior change is that
dullahan_browser_clientnow provides aCefAudioHandlerand requests the audio format shown above.Testing
Validated in the AYAstorm downstream viewer:
Review Focus
const float** datawith callback-lifetime validity acceptable for this API?GetAudioHandler()always returnthis, or only when audio callbacks are registered?CEF_CHANNEL_LAYOUT_7_1/48000/1024audio format acceptable as the default callback contract?