diff --git a/docs/README.md b/docs/README.md index dcaadd86..d4a3496d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,8 +1,9 @@ # Light SDK Docs Topics: + - [Navigating this repository](repo) - [Overview of Light primitives (tool building blocks)](../sdk/client) - [Declaring tool metadata (name, version, etc.)](tool_metadata) - [Using the LightOS Emulator](system_app) - +- [Why parts of the SDK are built the way they are](design_decisions) diff --git a/docs/design_decisions/README.md b/docs/design_decisions/README.md index e69de29b..f122851c 100644 --- a/docs/design_decisions/README.md +++ b/docs/design_decisions/README.md @@ -0,0 +1,4 @@ +# Design decisions + +- [Detached audio](detached_audio.md) — service-owned playback that survives a + tool leaving the foreground. diff --git a/docs/design_decisions/detached_audio.md b/docs/design_decisions/detached_audio.md new file mode 100644 index 00000000..58b436ce --- /dev/null +++ b/docs/design_decisions/detached_audio.md @@ -0,0 +1,192 @@ +# Detached audio + +Why detached playback is built the way it is. For how to use it, see the +[audio section of the SDK README](../../sdk/client/README.md#audio). + +## Attachment modes + +`LightAudioPlayer` supports two attachment modes: + +- `Attached` owns an in-process player. Releasing the tool's handle stops and releases playback. +- `Detached` controls a player owned by an SDK `MediaSessionService`. Releasing the handle disconnects the tool while playback and its queue may remain live. Tools using this method opt in by adding `detached-audio` to their `capabilities` in `lighttool.toml`. + +The attached/detached (ownership-based) terminology is preferred over background/foreground (visibility-based) because it describes the relationship between the tool and the player object, not whether audio is currently playing or whether the tool is visible. +Android implements detached playback with a _foreground_ service, so visibility-based naming (background/foreground) would make SDK terminology confusing. + +Here's a basic architecture diagram: + +```text +LightAudioPlayer +├── Attached ── ExoPlayer in the tool screen lifecycle +└── Detached ── media3 MediaController ── MediaSession + └── LightAudioService + (MediaSessionService) + └── ExoPlayer +``` + +## Architecture + +This section zooms in on the components and their relationships: + +```text +┌─ TOOL PROCESS (com.example.mytool) ─────────────────────────────┐ +│ │ +│ PlayerViewModel │ +│ │ │ +│ ▼ │ +│ DefaultLightAudio : LightAudio │ +│ wraps SealedLightActivity │ +│ │ .newPlayer(usage, playback) │ +│ │ │ +│ ├── checks ──▶ CAPABILITY_DETACHED_AUDIO marker │ +│ ├── opens ───▶ DetachedSessionState │ +│ │ └── also holds the live session's usage │ +│ ▼ │ +│ LightAudioPlayer │ +│ │ wraps a media3 Player implementation │ +│ │ │ +│ ├─ Attached ──▶ ExoPlayer ─────────────────────────────┐ │ +│ │ (owned by the player, dies with it) │ │ +│ │ │ │ +│ └─ Detached ──▶ MediaController ─┐ │ │ +│ + connectionHints│ │ │ +│ │ │ │ +│ binder (loopback, same proc) │ │ +│ │ │ │ +│ ┌─ LightAudioService : MediaSessionService ─────────────┐ │ │ +│ │ plugin-generated; no android:process │ │ │ +│ │ foregroundServiceType="mediaPlayback" │ │ │ +│ │ │ │ │ +│ │ MediaSession ◀───── SessionCallback: onConnect, │ │ │ +│ │ │ onPostConnect, onDisconnected │ │ │ +│ │ ▼ │ │ │ +│ │ ExoPlayer ─────────────────────────────────────────┼───┤ │ +│ │ setAudioAttributes(usage, handleAudioFocus=true) │ │ │ +│ └───┬───────────────────────────────────────────────────┘ │ │ +└──────┼───────────────────────────────────────────────────────┼──┘ + │ ▲ │ + │ │ other controllers of the same MediaSession: │ + │ ├── Android system media controls │ + │ ├── Bluetooth / headset controls │ + │ └── media3 notification controller │ + │ │ + │ publishes platform session │ + ▼ │ +┌───────────────────────── ANDROID SYSTEM ─────────────────────┼──┐ +│ MediaSessionManagerService AudioManager │ │ +└──────────────────────────────────────────────────────────────┼──┘ + ▲ query active sessions AUDIOFOCUS_GAIN │ + └───────────────────────────┐ arbitration │ +┌─ LIGHTOS PROCESS ─────────────────┼──────────────────────────┼──┐ +│ (uid.system, the launcher) │ │ │ +│ │ │ │ +│ MediaSessionManager.getActiveSessions() │ │ +│ └── android.media.session.MediaController │ │ +│ └── Now-playing: LockScreen / Toolbox │ │ +│ │ │ +│ LightOSAudioPlayerService │ │ +│ │ survives as uid.system persistent launcher │ │ +│ ├── LightOSAudioPlayerAudioFocus ◀──────────────────────┘ │ +│ │ requests AUDIOFOCUS_GAIN, │ +│ │ OnAudioFocusChangeListener already pauses │ +│ │ LightOS music when a tool plays │ +│ └── LightOSAudioPlayerState │ +└─────────────────────────────────────────────────────────────────┘ +``` + +Detached audio wraps media3's `MediaSessionService`, described in the [official Android documentation](https://developer.android.com/media/media3/session/background-playback). `LightAudioService` extends that service, owns one `ExoPlayer`, and publishes the player through one `MediaSession`. + +`LightAudioService` is declared by the Gradle plugin, in the manifest of each tool that opted in, without `android:process` — so it runs inside the tool process. The service asserts this process relationship when it starts, since it's a crucial detail for the architecture. +Being in the same process allows the tool code and the service to access the same `DetachedSessionState` instance, which is responsible for: + +- enforcing one detached handle +- recording the live session's audio usage +- telling the service whether a tool still holds the detached handle. + +The public player API is backed by media3's `Player` interface. +Attached mode uses `ExoPlayer` and detached mode uses `MediaController`. +Queue, transport, position, metadata, and playback errors therefore have the same SDK surface in both modes. + +The SDK maps its audio API onto the media3 components as follows: + +- In detached mode, `LightAudioPlayer` creates a `MediaController` for that session instead of creating its own `ExoPlayer`. +- `LightAudioItem` becomes a media3 `MediaItem`. Its `LightMediaMetadata` becomes `MediaMetadata`. The session can then expose the same queue and metadata to Android, media buttons, and future LightOS controls. +- Player state received by the controller is mirrored into the `LightAudioPlayer` state flows. +- media3 `PlaybackException` values are mapped into SDK-owned `LightAudioError` values so attached and detached players have the same error surface. +- Releasing `LightAudioPlayer` closes its controller and handle without releasing the service-owned player. + +The controller identifies itself as a tool controller and sends its `LightAudioUsage` through connection hints. Other platform controllers, such as Bluetooth, may connect to the session, but they do not own the SDK's detached handle or select its audio usage. + +## Foreground-service notification on LP3 + +Android requires a media-playback foreground service to publish a notification. +On stock Android, SystemUI renders that notification. LP3 uses LightOS, which +does not render a notification shade and does not listen for notifications from +tool processes. The media3 notification therefore satisfies the Android +foreground-service requirement but is not visible on LP3. + +The notification is not the LightOS now-playing surface. LightOS discovers the +platform `MediaSession` separately and renders its own controls. + +## Lifetime and idle stop + +Playing detached audio keeps the service alive after the tool releases its handle. +A paused service must not retain its player and process forever, so the service starts a 60-second timer when both conditions hold: + +1. playback is not playing +2. no tool holds the detached handle. + +Playback resuming or a handle opening cancels the timer. When it fires, the service stops itself and releases the session and player. +The handle, rather than the number of connected controllers, is the liveness signal because it covers controller connection gaps and gives ownership one source of truth. +The timeout governs abandoned paused playback, not active playback. A paused tool that still holds its player can resume after 60 seconds. + +## Reconnecting + +A new detached player connects to one of two states: + +- **Live:** the service still owns its queue, index, position, and playback state. +- **Fresh:** the service was never started or has stopped, so its queue is empty. + +Before inspecting the queue, a tool waits for `availability` to become `Ready`, normally through `awaitReady()`. Re-initializing an already active queue results in replacing playback that survived from the previous screen. + +`release()` only disconnects a detached handle. A tool that intends to end detached playback calls `stop()` first. Once the idle rule has stopped the service, restoring queue and position is the tool's responsibility. + +A live session also retains its `LightAudioUsage`. Reconnecting with a different usage throws synchronously instead of silently changing or ignoring the live session's audio attributes. + +## Playback errors + +Both modes report failures as `LightAudioError` (SDK-owned type) rather than exposing media3's `PlaybackException`. +This keeps media3 out of the public API and gives attached and detached modes a single error interface. + +An in-process `ExoPlayer` throws `ExoPlaybackException`, which adds additional fields to the base class that do not survive the controller boundary. `PlaybackException` itself is serializable and reaches a `MediaController`, but its subclass detail does not. + +Anything the SDK derived from the exception type would therefore be richer in attached mode than in detached mode, and the two would diverge exactly where the rest of this design keeps them identical. + +`errorCode` crosses the controller/session boundary. The SDK maps it into four categories a tool can act on: `Source`, `Unsupported`, `Output` and `Unknown`. It carries the stable media3 error-code name as a diagnostic string for logs. + +media3 stops the queue when an item fails; it does not skip to the next one. The SDK preserves that. Automatically advancing would be a playback policy rather than a player behavior, and unplayable content would advance in a loop. +Recovering is the tool's decision. + +## Tool opt-in + +Tools opt into detached playback through `lighttool.toml`: + +```toml +[tool] +capabilities = ["detached-audio"] +``` + +The Light SDK Gradle plugin translates this capability into the `FOREGROUND_SERVICE` and `FOREGROUND_SERVICE_MEDIA_PLAYBACK` manifest +permissions, the capability marker, and the `` declaration. + +The SDK factory rejects detached construction without the capability and reports the required entry. +Attached playback needs no additional opt-in. + +## LightOS contract + +The integration boundary is the platform `MediaSession`: + +- Audio-focus arbitration with LightOS already uses Android `AudioManager`. +- Detached tools publish queue and metadata through their session. +- LightOS publishes metadata for its own player and resolves media-button arbitration between sessions. +- A unified now-playing surface can discover active sessions with `MEDIA_CONTENT_CONTROL` and render metadata plus transport from an `android.media.session.MediaController`. diff --git a/docs/tool_metadata/README.md b/docs/tool_metadata/README.md index a895aa23..6987e962 100644 --- a/docs/tool_metadata/README.md +++ b/docs/tool_metadata/README.md @@ -15,8 +15,9 @@ id = "com.example.mytool" # Java package id, dotted, lowerc label = "My Tool" # Your tool's display name versionCode = 1 # monotonically-increasing integer versionName = "1.0.0" # ^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$ -permissions = ["android.permission.CAMERA"] # allowlisted permissions only -orientation = "portrait" # optional; omit for no orientation lock +permissions = ["android.permission.CAMERA"] # allowlisted permissions only +capabilities = [] # allowlisted SDK features +orientation = "portrait" # optional; omit for no orientation lock ``` ## Fields @@ -37,6 +38,19 @@ pre-release (`1.2.3-rc.1`), and no build metadata (`1.2.3+build`). This will be ### `permissions` — Android permissions your tool needs An array of permission strings, each one from the allowlist below. Anything not on the list will fail the build. Each entry becomes a `` element in the generated manifest. +### `capabilities` — optional SDK features + +An array of capability names from the SDK allowlist. Omit it when the tool needs none. A capability may generate the permissions, manifest components, and runtime marker required by that feature. + +Detached playback uses: + +```toml +[tool] +capabilities = ["detached-audio"] +``` + +`detached-audio` generates the `FOREGROUND_SERVICE` and `FOREGROUND_SERVICE_MEDIA_PLAYBACK` permissions, the detached audio service, and the SDK marker checked by `LightAudio.newPlayer`. None of those permissions can be listed under `permissions`. The capability owns them, and the build fails with an error naming the capability to declare instead. + ### `orientation` — optional screen orientation lock Set to `"portrait"` to keep the tool in portrait orientation. Omit this field to let the system choose the orientation. diff --git a/examples/audio-demo/lighttool.toml b/examples/audio-demo/lighttool.toml index a18ad32f..04a61491 100644 --- a/examples/audio-demo/lighttool.toml +++ b/examples/audio-demo/lighttool.toml @@ -4,6 +4,7 @@ label = "Light Audio Demo" versionCode = 1 versionName = "0.0.1" permissions = ["android.permission.INTERNET", "android.permission.RECORD_AUDIO"] +capabilities = ["detached-audio"] # change if you run this on an LP3! serverPackage = "com.lightos" orientation = "portrait" diff --git a/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/AudioLibraryRepository.kt b/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/AudioLibraryRepository.kt index c1eb964d..11be0d0b 100644 --- a/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/AudioLibraryRepository.kt +++ b/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/AudioLibraryRepository.kt @@ -75,6 +75,14 @@ object SampleAudioCatalog { durationMs = 0L, formatLabel = "STREAM", ), + AudioClip( + source = AudioClipSource.UrlSource("http://127.0.0.1:1/missing.mp3"), + displayName = "Broken source (error demo)", + usage = LightAudioUsage.Music, + kind = AudioContentKind.Music, + durationMs = 0L, + formatLabel = "BROKEN URL", + ), ) } diff --git a/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/PlayerScreen.kt b/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/PlayerScreen.kt index d5cbd4f0..f0528e4f 100644 --- a/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/PlayerScreen.kt +++ b/examples/audio-demo/src/main/kotlin/com/thelightphone/audiodemo/PlayerScreen.kt @@ -11,13 +11,19 @@ import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue import androidx.compose.ui.Modifier import androidx.compose.ui.text.style.TextAlign +import androidx.datastore.core.DataStore +import androidx.datastore.preferences.core.Preferences +import androidx.datastore.preferences.core.booleanPreferencesKey +import androidx.datastore.preferences.core.edit import androidx.lifecycle.viewModelScope import com.thelightphone.sdk.LightScreen import com.thelightphone.sdk.LightViewModel import com.thelightphone.sdk.SealedLightActivity import com.thelightphone.sdk.audio.DefaultLightAudio import com.thelightphone.sdk.audio.LightAudio +import com.thelightphone.sdk.audio.LightAudioError import com.thelightphone.sdk.audio.LightAudioItem +import com.thelightphone.sdk.audio.LightAudioPlayback import com.thelightphone.sdk.audio.LightAudioPlayer import com.thelightphone.sdk.audio.LightAudioSource import com.thelightphone.sdk.audio.LightMediaMetadata @@ -35,66 +41,158 @@ import com.thelightphone.sdk.ui.LightTopBarCenter import com.thelightphone.sdk.ui.gridUnitsAsDp import com.thelightphone.sdk.ui.lightClickable import java.io.File +import kotlinx.coroutines.ExperimentalCoroutinesApi +import kotlinx.coroutines.launch import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.SharingStarted import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.flatMapLatest import kotlinx.coroutines.flow.map +import kotlinx.coroutines.flow.first import kotlinx.coroutines.flow.stateIn +interface PlayerModeStore { + suspend fun getPlayback(): LightAudioPlayback + suspend fun setPlayback(playback: LightAudioPlayback) +} + +// In order to get the detached audio player demo working, we need to +// be able to get in and out of the tool while audio keeps playing. +// For that, the detached mode toggle UI state should be persistent. +internal class DataStorePlayerModeStore( + private val dataStore: DataStore, +) : PlayerModeStore { + override suspend fun getPlayback(): LightAudioPlayback = + if (dataStore.data.first()[DETACHED_MODE_KEY] == true) { + LightAudioPlayback.Detached + } else { + LightAudioPlayback.Attached + } + + override suspend fun setPlayback(playback: LightAudioPlayback) { + dataStore.edit { preferences -> + preferences[DETACHED_MODE_KEY] = playback == LightAudioPlayback.Detached + } + } + + private companion object { + val DETACHED_MODE_KEY = booleanPreferencesKey("player_detached") + } +} + +internal class PlayerDemoMode( + private val store: PlayerModeStore, + initialPlayback: LightAudioPlayback = LightAudioPlayback.Attached, +) { + val playback = MutableStateFlow(initialPlayback) + + suspend fun load(): LightAudioPlayback = store.getPlayback().also { + playback.value = it + } + + suspend fun set(playback: LightAudioPlayback) { + store.setPlayback(playback) + this.playback.value = playback + } +} + +internal fun nextPlaybackMode(playback: LightAudioPlayback): LightAudioPlayback = when (playback) { + LightAudioPlayback.Attached -> LightAudioPlayback.Detached + LightAudioPlayback.Detached -> LightAudioPlayback.Attached +} + +@OptIn(ExperimentalCoroutinesApi::class) class PlayerViewModel( filesDir: File, - audio: LightAudio + private val audio: LightAudio, + modeStore: PlayerModeStore, ) : LightViewModel() { - private val player: LightAudioPlayer = audio.newPlayer() + private val mode = PlayerDemoMode(modeStore) + // The playback toggle swaps the player instance, so screen state follows + // the flow rather than one player's own flows, which would go stale. + private val playerFlow = MutableStateFlow(audio.newPlayer(playback = mode.playback.value)) + private val currentPlayer: LightAudioPlayer get() = playerFlow.value val clips = AudioLibraryRepository(filesDir).list() - val currentClip: StateFlow = player.currentMediaItemIndex + val currentClip: StateFlow = playerFlow + .flatMapLatest { it.currentMediaItemIndex } .map(clips::getOrNull) - .stateIn(viewModelScope, SharingStarted.Eagerly, clips.getOrNull(player.currentMediaItemIndex.value)) - val positionMs = player.positionMs - val durationMs = player.durationMs - val isPlaying = player.isPlaying + .stateIn(viewModelScope, SharingStarted.Eagerly, clips.getOrNull(currentPlayer.currentMediaItemIndex.value)) + val positionMs = playerFlow + .flatMapLatest { it.positionMs } + .stateIn(viewModelScope, SharingStarted.Eagerly, currentPlayer.positionMs.value) + val durationMs = playerFlow + .flatMapLatest { it.durationMs } + .stateIn(viewModelScope, SharingStarted.Eagerly, currentPlayer.durationMs.value) + val isPlaying = playerFlow + .flatMapLatest { it.isPlaying } + .stateIn(viewModelScope, SharingStarted.Eagerly, currentPlayer.isPlaying.value) + val error = playerFlow + .flatMapLatest { it.error } + .stateIn(viewModelScope, SharingStarted.Eagerly, currentPlayer.error.value) + val playback = mode.playback val speed = MutableStateFlow(1f) - val skipSilence = MutableStateFlow(false) - val playNext = MutableStateFlow(true) + + private val modeLoadJob = viewModelScope.launch { + val initialPlayback = mode.playback.value + val persistedPlayback = mode.load() + if (persistedPlayback != initialPlayback) { + replacePlayer(persistedPlayback) + } + } fun play(clip: AudioClip) { - val selection = playbackSelectionFor(clip, clips) - player.speed = speed.value - player.skipSilence = skipSilence.value - player.pauseAtEndOfMediaItems = pauseAtEndOfMediaItemsFor(playNext.value) - player.setMediaQueue(selection.queue.map(AudioClip::toLightAudioItem), selection.startIndex) - player.play() + val selectedPlayer = currentPlayer + viewModelScope.launch { + if (!selectedPlayer.awaitReady() || selectedPlayer !== currentPlayer) return@launch + val selection = playbackSelectionFor(clip, clips) + selectedPlayer.speed = speed.value + selectedPlayer.setMediaQueue( + selection.queue.map(AudioClip::toLightAudioItem), + selection.startIndex, + ) + selectedPlayer.play() + } } fun togglePlayPause() { - if (isPlaying.value) player.pause() else player.play() + if (isPlaying.value) currentPlayer.pause() else currentPlayer.play() } - fun skipBack() = player.skipBack() - fun skipForward() = player.skipForward() - fun skipToPrevious() = player.skipToPrevious() - fun skipToNext() = player.skipToNext() + fun skipBack() = currentPlayer.skipBack() + fun skipForward() = currentPlayer.skipForward() + fun skipToPrevious() = currentPlayer.skipToPrevious() + fun skipToNext() = currentPlayer.skipToNext() fun cycleSpeed() { val next = SPEEDS[(SPEEDS.indexOf(speed.value) + 1).mod(SPEEDS.size)] speed.value = next - player.speed = next + currentPlayer.speed = next } - fun toggleSkipSilence() { - val enabled = !skipSilence.value - skipSilence.value = enabled - player.skipSilence = enabled + fun toggleDetached() { + viewModelScope.launch { + modeLoadJob.join() + val nextPlayback = nextPlaybackMode(playback.value) + replacePlayer(nextPlayback) + mode.set(nextPlayback) + } } - fun togglePlayNext() { - val enabled = !playNext.value - playNext.value = enabled - player.pauseAtEndOfMediaItems = pauseAtEndOfMediaItemsFor(enabled) + private fun replacePlayer(nextPlayback: LightAudioPlayback) { + val oldPlayer = currentPlayer + oldPlayer.stop() + oldPlayer.setMediaQueue(emptyList()) + oldPlayer.release() + + val nextPlayer = audio.newPlayer(playback = nextPlayback).apply { + speed = this@PlayerViewModel.speed.value + } + mode.playback.value = nextPlayback + playerFlow.value = nextPlayer } override fun onCleared() { - player.release() + currentPlayer.release() super.onCleared() } @@ -123,9 +221,7 @@ internal data class PlaybackSelection( val startIndex: Int, ) -internal fun pauseAtEndOfMediaItemsFor(playNext: Boolean): Boolean = !playNext - -/** The whole library is one playlist; play-next only controls auto-advance. */ +/** The whole library is one playlist. */ internal fun playbackSelectionFor( selected: AudioClip, clips: List, @@ -138,7 +234,11 @@ internal fun playbackSelectionFor( class PlayerScreen(private val sealedActivity: SealedLightActivity) : LightScreen(sealedActivity) { override val viewModelClass = PlayerViewModel::class.java - override fun createViewModel() = PlayerViewModel(lightContext.filesDir, DefaultLightAudio(sealedActivity)) + override fun createViewModel() = PlayerViewModel( + filesDir = lightContext.filesDir, + audio = DefaultLightAudio(sealedActivity), + modeStore = DataStorePlayerModeStore(lightContext.dataStore), + ) @Composable override fun Content() { @@ -147,9 +247,9 @@ class PlayerScreen(private val sealedActivity: SealedLightActivity) : val position by viewModel.positionMs.collectAsState() val duration by viewModel.durationMs.collectAsState() val playing by viewModel.isPlaying.collectAsState() + val error by viewModel.error.collectAsState() val speed by viewModel.speed.collectAsState() - val skipSilence by viewModel.skipSilence.collectAsState() - val playNext by viewModel.playNext.collectAsState() + val playback by viewModel.playback.collectAsState() val durationDisplay = playerDurationDisplay(duration) LightTheme(colors = colors) { @@ -178,23 +278,35 @@ class PlayerScreen(private val sealedActivity: SealedLightActivity) : .fillMaxWidth() .padding(bottom = 0.5f.gridUnitsAsDp()), ) - Row(Modifier - .fillMaxWidth() - .padding(horizontal = 1f.gridUnitsAsDp())) { - PlayerOption("SPEED", "${speed}x", Modifier.weight(1f), viewModel::cycleSpeed) - PlayerOption( - "SKIP SILENCE", - if (skipSilence) "ON" else "OFF", - Modifier.weight(1f), - viewModel::toggleSkipSilence, + error?.let { + LightText( + text = playbackErrorMessage(it), + variant = LightTextVariant.Fine, + align = TextAlign.Center, + modifier = Modifier + .fillMaxWidth() + .padding(bottom = 0.5f.gridUnitsAsDp()), ) + } + Row( + Modifier + .fillMaxWidth() + .padding(horizontal = 1f.gridUnitsAsDp()), + ) { + PlayerOption("SPEED", "${speed}x", Modifier.weight(1f), viewModel::cycleSpeed) PlayerOption( - "PLAY NEXT", - if (playNext) "ON" else "OFF", + "DETACHED", + if (playback == LightAudioPlayback.Detached) "ON" else "OFF", Modifier.weight(1f), - viewModel::togglePlayNext, + viewModel::toggleDetached, ) } + LightText( + text = "MODE SWITCH STOPS PLAYBACK", + variant = LightTextVariant.Superfine, + align = TextAlign.Center, + modifier = Modifier.fillMaxWidth(), + ) LightBottomBar( items = listOf( LightBarButton.LightIcon( @@ -226,6 +338,9 @@ class PlayerScreen(private val sealedActivity: SealedLightActivity) : } } +internal fun playbackErrorMessage(error: LightAudioError): String = + "${error.kind}: ${error.diagnostic}. Select another item to continue." + @Composable private fun AudioClipRow(number: Int, clip: AudioClip, selected: Boolean, onClick: () -> Unit) { Column( diff --git a/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/AudioLibraryRepositoryTest.kt b/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/AudioLibraryRepositoryTest.kt index dcf64597..74478b9f 100644 --- a/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/AudioLibraryRepositoryTest.kt +++ b/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/AudioLibraryRepositoryTest.kt @@ -15,9 +15,9 @@ class AudioLibraryRepositoryTest { fun sampleCatalogIncludesBundledMusicSpeechAndRemoteStreams() { val clips = SampleAudioCatalog.clips - assertEquals(5, clips.size) + assertEquals(6, clips.size) assertEquals(3, clips.count { it.source is AudioClipSource.AssetSource }) - assertEquals(2, clips.count { it.source is AudioClipSource.UrlSource }) + assertEquals(3, clips.count { it.source is AudioClipSource.UrlSource }) assertEquals(2, clips.count { it.kind == AudioContentKind.Speech }) assertTrue(clips.all { it.usage == LightAudioUsage.Music }) assertTrue(clips.any { it.formatLabel == "OGG" }) @@ -44,12 +44,6 @@ class AudioLibraryRepositoryTest { } } - @Test - fun playNextControlsPauseAtEndWithoutChangingQueue() { - assertEquals(false, pauseAtEndOfMediaItemsFor(playNext = true)) - assertEquals(true, pauseAtEndOfMediaItemsFor(playNext = false)) - } - @Test fun listCombinesBundledClipsAndNewestRecordingsFirst() { val filesDir = createTempDirectory("audio-library").toFile() diff --git a/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/PlayerScreenTest.kt b/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/PlayerScreenTest.kt index ae36a68d..9559f137 100644 --- a/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/PlayerScreenTest.kt +++ b/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/PlayerScreenTest.kt @@ -1,9 +1,44 @@ package com.thelightphone.audiodemo +import com.thelightphone.sdk.audio.LightAudioError +import com.thelightphone.sdk.audio.LightAudioErrorKind +import com.thelightphone.sdk.audio.LightAudioPlayback import kotlin.test.Test import kotlin.test.assertEquals +import kotlinx.coroutines.runBlocking class PlayerScreenTest { + @Test + fun playbackModeAlternatesBetweenAttachedAndDetached() { + assertEquals(LightAudioPlayback.Detached, nextPlaybackMode(LightAudioPlayback.Attached)) + assertEquals(LightAudioPlayback.Attached, nextPlaybackMode(LightAudioPlayback.Detached)) + } + + @Test + fun detachedModePersistsAcrossViewModelInstances() = runBlocking { + val store = InMemoryPlayerModeStore() + PlayerDemoMode(store).set(LightAudioPlayback.Detached) + + assertEquals(LightAudioPlayback.Detached, PlayerDemoMode(store).load()) + + PlayerDemoMode(store).set(LightAudioPlayback.Attached) + assertEquals(LightAudioPlayback.Attached, PlayerDemoMode(store).load()) + } + + @Test + fun playbackErrorMessageExplainsHowToContinue() { + assertEquals( + "Source: ERROR_CODE_IO_FILE_NOT_FOUND. Select another item to continue.", + playbackErrorMessage( + LightAudioError( + LightAudioErrorKind.Source, + "ERROR_CODE_IO_FILE_NOT_FOUND", + itemIndex = 2, + ), + ), + ) + } + @Test fun unknownDurationShowsDashAndDisablesSeeking() { assertEquals( @@ -24,3 +59,13 @@ class PlayerScreenTest { ) } } + +private class InMemoryPlayerModeStore : PlayerModeStore { + private var playback = LightAudioPlayback.Attached + + override suspend fun getPlayback(): LightAudioPlayback = playback + + override suspend fun setPlayback(playback: LightAudioPlayback) { + this.playback = playback + } +} diff --git a/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/ToneScreenTest.kt b/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/ToneScreenTest.kt index 14c57bb0..a9721d28 100644 --- a/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/ToneScreenTest.kt +++ b/examples/audio-demo/src/test/kotlin/com/thelightphone/audiodemo/ToneScreenTest.kt @@ -5,6 +5,7 @@ import com.thelightphone.sdk.audio.CaptureConfig import com.thelightphone.sdk.audio.LightAudio import com.thelightphone.sdk.audio.LightAudioCapture import com.thelightphone.sdk.audio.LightAudioPlayer +import com.thelightphone.sdk.audio.LightAudioPlayback import com.thelightphone.sdk.audio.LightAudioRecorder import com.thelightphone.sdk.audio.LightAudioUsage import com.thelightphone.sdk.audio.LightAudioVoice @@ -26,7 +27,10 @@ class ToneScreenTest { val vm = ToneViewModel(object : LightAudio { override val capabilities: AudioCapabilities = AudioCapabilities(67) - override fun newPlayer(usage: LightAudioUsage): LightAudioPlayer { + override fun newPlayer( + usage: LightAudioUsage, + playback: LightAudioPlayback, + ): LightAudioPlayer { TODO("Should not be called") } diff --git a/plugin/src/main/kotlin/com/thelightphone/plugin/LightToolMetadata.kt b/plugin/src/main/kotlin/com/thelightphone/plugin/LightToolMetadata.kt index 26697e10..72693706 100644 --- a/plugin/src/main/kotlin/com/thelightphone/plugin/LightToolMetadata.kt +++ b/plugin/src/main/kotlin/com/thelightphone/plugin/LightToolMetadata.kt @@ -20,6 +20,7 @@ data class LightToolMetadata( val versionCode: Int, val versionName: String, val permissions: List, + val capabilities: List = emptyList(), val serverPackage: String, val orientation: String? = null, ) { @@ -58,6 +59,7 @@ data class LightToolMetadata( versionCode = validateVersionCode(tool.tomlLong("versionCode")), versionName = validateVersionName(tool.tomlString("versionName")), permissions = validatePermissions(tool.tomlStringList("permissions")), + capabilities = validateCapabilities(tool.tomlStringList("capabilities")), serverPackage = validateServerPackage(tool.tomlString("serverPackage")), orientation = validateOrientation(tool.tomlString("orientation")), ) @@ -120,6 +122,16 @@ data class LightToolMetadata( val seen = mutableSetOf() for (item in list) { require(seen.add(item)) { "duplicate permission: $item" } + // A capability-generated permission is never hand-written: the + // capability is the single source of truth for it, so point the + // dev at the capability instead of the bare allowlist. + capabilityGenerating(item)?.let { capability -> + throw LightToolMetadataException( + "permission not allowed: $item — it is generated by the " + + "'$capability' capability; declare " + + "capabilities = [\"$capability\"] instead" + ) + } require(item in LightToolPolicy.ALLOWED_PERMISSIONS) { "permission not allowed: $item\nallowed: ${LightToolPolicy.ALLOWED_PERMISSIONS.sorted().joinToString()}" } @@ -127,6 +139,23 @@ data class LightToolMetadata( return list } + private fun validateCapabilities(values: List?): List { + val list = values ?: emptyList() + val seen = mutableSetOf() + for (item in list) { + require(seen.add(item)) { "duplicate capability: $item" } + require(item in LightToolPolicy.ALLOWED_CAPABILITIES) { + "capability not allowed: $item\nallowed: ${LightToolPolicy.ALLOWED_CAPABILITIES.sorted().joinToString()}" + } + } + return list + } + + private fun capabilityGenerating(permission: String): String? = + LightToolPolicy.CAPABILITY_IMPLIED_PERMISSIONS.entries + .firstOrNull { permission in it.value } + ?.key + private fun require(condition: Boolean, lazyMessage: () -> String) { if (!condition) throw LightToolMetadataException(lazyMessage()) } @@ -158,6 +187,31 @@ object LightToolPolicy { "android.permission.NFC", ) + const val DETACHED_AUDIO: String = "detached-audio" + + val ALLOWED_CAPABILITIES: Set = setOf(DETACHED_AUDIO) + + /** + * Permissions a capability contributes to the generated manifest. These are + * deliberately absent from [ALLOWED_PERMISSIONS]: a bare platform permission + * says nothing about what the tool asked for (any transitive dependency can + * declare one), so the capability owns them instead. + */ + val CAPABILITY_IMPLIED_PERMISSIONS: Map> = mapOf( + DETACHED_AUDIO to listOf( + "android.permission.FOREGROUND_SERVICE", + "android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK", + ), + ) + + /** + * Application `` key marking [capability] as declared. Under our + * own namespace, so only our generator can produce it — the property a + * platform permission cannot have. + */ + fun capabilityMarker(capability: String): String = + "com.thelightphone.sdk.CAPABILITY_" + capability.uppercase().replace('-', '_') + /** * Permissions that Play Store / lint infer as also requiring a hardware * feature. Lacking a matching `` element triggers diff --git a/plugin/src/main/kotlin/com/thelightphone/plugin/ManifestGenerator.kt b/plugin/src/main/kotlin/com/thelightphone/plugin/ManifestGenerator.kt index e9af9d08..65a2c4a2 100644 --- a/plugin/src/main/kotlin/com/thelightphone/plugin/ManifestGenerator.kt +++ b/plugin/src/main/kotlin/com/thelightphone/plugin/ManifestGenerator.kt @@ -15,7 +15,13 @@ object ManifestGenerator { fun render(metadata: LightToolMetadata): String = buildString { appendLine("""""") appendLine("""""") - val permissions = metadata.permissions + // A capability declares what the tool does and the permissions it needs + // follow from that, so they are unioned in here rather than written by the tool. + val permissions = ( + metadata.permissions + metadata.capabilities.flatMap { + LightToolPolicy.CAPABILITY_IMPLIED_PERMISSIONS[it].orEmpty() + } + ).distinct() for (perm in permissions) { appendLine(""" """) } @@ -32,6 +38,29 @@ object ManifestGenerator { val screenOrientation = metadata.orientation?.let { "\n | android:screenOrientation=\"${xmlAttr(it)}\"" }.orEmpty() + val capabilityMarkers = marginBlock( + metadata.capabilities.flatMap { capability -> + listOf( + """ """, + ) + } + ) + // Only tools that opted in declare the audio service. Shipping it in the + // SDK library manifest would put a mediaPlayback claim in every tool. + val detachedAudioService = marginBlock( + if (LightToolPolicy.DETACHED_AUDIO !in metadata.capabilities) emptyList() else listOf( + """ """, + """ """, + """ """, + """ """, + """ """, + ) + ) appendLine( """ | | + | android:value="${xmlAttr(metadata.serverPackage)}" />$capabilityMarkers | - | + | $detachedAudioService | | | @@ -73,6 +102,13 @@ object ManifestGenerator { ) } + /** + * Splices [lines] into the trimMargin template below. Interpolation happens + * before trimMargin runs, so every inserted line carries its own margin. + */ + private fun marginBlock(lines: List): String = + lines.joinToString("") { "\n |$it" } + private fun xmlAttr(value: String): String = buildString(value.length) { for (ch in value) { when (ch) { diff --git a/plugin/src/test/kotlin/com/thelightphone/plugin/LightToolMetadataTest.kt b/plugin/src/test/kotlin/com/thelightphone/plugin/LightToolMetadataTest.kt index 32aa1412..dde2aa03 100644 --- a/plugin/src/test/kotlin/com/thelightphone/plugin/LightToolMetadataTest.kt +++ b/plugin/src/test/kotlin/com/thelightphone/plugin/LightToolMetadataTest.kt @@ -24,6 +24,7 @@ class LightToolMetadataTest { versionCode = 7 versionName = "1.2.0" permissions = ["android.permission.INTERNET"] + capabilities = ["detached-audio"] serverPackage = "com.lightos" orientation = "portrait" """.trimIndent()) @@ -35,6 +36,7 @@ class LightToolMetadataTest { assertEquals(7, meta.versionCode) assertEquals("1.2.0", meta.versionName) assertEquals(listOf("android.permission.INTERNET"), meta.permissions) + assertEquals(listOf("detached-audio"), meta.capabilities) assertEquals("com.lightos", meta.serverPackage) assertEquals("portrait", meta.orientation) } @@ -46,7 +48,7 @@ class LightToolMetadataTest { id = "com.example.mytool" label = "My Tool" versionCode = 1 - versionName = "1.0" + versionName = "1.0.0" serverPackage = "com.lightos" """.trimIndent()) @@ -60,7 +62,7 @@ class LightToolMetadataTest { id = "com.example.mytool" label = "My Tool" versionCode = 1 - versionName = "1.0" + versionName = "1.0.0" serverPackage = "com.lightos" orientation = "landscape" """.trimIndent()) @@ -70,19 +72,71 @@ class LightToolMetadataTest { } @Test - fun `foreground service permission is not allowed`(@TempDir dir: Path) { + fun `capabilities default to empty`(@TempDir dir: Path) { val file = writeToml(dir, """ [tool] id = "com.example.mytool" label = "X" versionCode = 1 - versionName = "1.0" + versionName = "1.0.0" serverPackage = "com.lightos" - permissions = ["android.permission.FOREGROUND_SERVICE"] + """.trimIndent()) + + assertEquals(emptyList(), LightToolMetadata.parse(file).capabilities) + } + + @Test + fun `unlisted capability fails`(@TempDir dir: Path) { + val file = writeToml(dir, """ + [tool] + id = "com.example.mytool" + label = "X" + versionCode = 1 + versionName = "1.0.0" + serverPackage = "com.lightos" + capabilities = ["detached-video"] """.trimIndent()) val ex = assertThrows { LightToolMetadata.parse(file) } - assert(ex.message!!.contains("not allowed")) + assert(ex.message!!.contains("capability not allowed")) + } + + @Test + fun `duplicate capability fails`(@TempDir dir: Path) { + val file = writeToml(dir, """ + [tool] + id = "com.example.mytool" + label = "X" + versionCode = 1 + versionName = "1.0.0" + serverPackage = "com.lightos" + capabilities = ["detached-audio", "detached-audio"] + """.trimIndent()) + + val ex = assertThrows { LightToolMetadata.parse(file) } + assert(ex.message!!.contains("duplicate capability")) + } + + @Test + fun `explicit capability-generated permission fails naming the capability`(@TempDir dir: Path) { + // The capability is the single source of truth for these permissions, + // so hand-writing one is rejected even alongside the capability itself. + val file = writeToml(dir, """ + [tool] + id = "com.example.mytool" + label = "X" + versionCode = 1 + versionName = "1.0.0" + serverPackage = "com.lightos" + permissions = ["android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK"] + capabilities = ["detached-audio"] + """.trimIndent()) + + val ex = assertThrows { LightToolMetadata.parse(file) } + assert(ex.message!!.contains("android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK")) { + ex.message ?: "" + } + assert(ex.message!!.contains("detached-audio")) { ex.message ?: "" } } @Test diff --git a/plugin/src/test/kotlin/com/thelightphone/plugin/ManifestGeneratorTest.kt b/plugin/src/test/kotlin/com/thelightphone/plugin/ManifestGeneratorTest.kt index 7bc998a8..f57fd0ce 100644 --- a/plugin/src/test/kotlin/com/thelightphone/plugin/ManifestGeneratorTest.kt +++ b/plugin/src/test/kotlin/com/thelightphone/plugin/ManifestGeneratorTest.kt @@ -9,6 +9,7 @@ class ManifestGeneratorTest { private fun render( label: String = "My App", permissions: List = emptyList(), + capabilities: List = emptyList(), serverPackage: String = "com.lightos", orientation: String? = null, ): String = ManifestGenerator.render( @@ -18,6 +19,7 @@ class ManifestGeneratorTest { versionCode = 1, versionName = "1.0.0", permissions = permissions, + capabilities = capabilities, serverPackage = serverPackage, orientation = orientation, ) @@ -97,6 +99,38 @@ class ManifestGeneratorTest { assertFalse(xml.contains("uses-feature")) } + @Test + fun `detached-audio capability generates its foreground service permissions`() { + val xml = render(capabilities = listOf("detached-audio")) + + assertTrue(xml.contains("""""")) + assertTrue(xml.contains("""""")) + // Neither permission implies hardware, so nothing narrows the install pool. + assertFalse(xml.contains("uses-feature")) + } + + @Test + fun `detached-audio capability emits its marker meta-data`() { + val xml = render(capabilities = listOf("detached-audio")) + + assertTrue( + xml.contains("""android:name="com.thelightphone.sdk.CAPABILITY_DETACHED_AUDIO""""), + "expected the capability marker; got:\n$xml" + ) + } + + @Test + fun `detached-audio capability declares the audio service`() { + val xml = render(capabilities = listOf("detached-audio")) + + assertTrue( + xml.contains("""android:name="com.thelightphone.sdk.audio.LightAudioService""""), + "expected LightAudioService; got:\n$xml" + ) + assertTrue(xml.contains("""android:foregroundServiceType="mediaPlayback"""")) + assertTrue(xml.contains("""""")) + } + @Test fun `server package is emitted as meta-data in application element`() { val xml = render(serverPackage = "com.lightos") @@ -111,13 +145,14 @@ class ManifestGeneratorTest { } @Test - fun `manifest emits no service or foreground-service permissions`() { - // Background audio was removed for the MVP; the manifest must never - // declare a media service or foreground-service permissions. + fun `without the capability nothing detached-audio is emitted`() { + // A tool that never asked for detached playback must carry no service, + // no mediaPlayback claim, no marker, and no foreground permissions. val xml = render(permissions = listOf("android.permission.RECORD_AUDIO")) assertFalse(xml.contains("() { @@ -105,7 +112,8 @@ class PlayerScreen(private val sealedActivity: SealedLightActivity) : LightScree #### Player -`LightAudioPlayer` plays files, bundled assets, and local or remote URLs. A player owns one queue and exposes position, duration, playback state, and queue index through `StateFlow` objects. +`LightAudioPlayer` plays files, bundled assets, and local or remote URLs. A player owns one queue and exposes its state through `StateFlow` objects: `positionMs`, `durationMs`, `isPlaying`, `currentMediaItemIndex`, `error`, and `availability`. +`currentMediaItemIndex` is `NO_MEDIA_ITEM` while the queue is empty. ```kotlin player.setMediaQueue( @@ -119,12 +127,81 @@ player.setMediaQueue( player.play() ``` -- Use `pause`, `stop`, `seekTo`, or the skip methods for transport controls. +- Use `pause`, `stop`, and `seekTo` for transport controls. +- `skipBack()` and `skipForward()` seek 15 seconds within the current item; `skipToPrevious()` and `skipToNext()` move through the queue. +- Set `speed` to change playback rate; values at or below zero clamp to the minimum supported rate. +- Use `setSource(File)` as a convenience for a one-item local-file queue. - Playback requests audio focus automatically. - If focus is unavailable, `play()` does nothing. - Observe `isPlaying` for the actual state. - Transient focus loss pauses and later resumes playback, while duckable loss lowers the volume. +##### Playback failures + +Player creation failures and playback failures use different mechanisms. + +`LightAudioPlayerException` is **thrown** when you create a player, for things the tool got wrong: a missing capability, a second detached handle, or a usage that conflicts with a live detached session. +Correct the cause, then create the player again. + +`LightAudioError` is **observed** while playing, for things the content or the device got wrong. Playback stopped, but the player still works; select another item to continue. + +```kotlin +player.error.collect { error -> + when (error?.kind) { + LightAudioErrorKind.Source -> // network or file I/O; worth retrying + LightAudioErrorKind.Unsupported -> // bad container or codec; skip it + LightAudioErrorKind.Output -> // the audio device could not be opened + LightAudioErrorKind.Unknown -> // unclassified + null -> // no current failure + } +} +``` + +`error.diagnostic` carries the underlying platform error name for logs, and `error.itemIndex` is the queue position that failed, or `-1` when unavailable. + +When an item fails, playback stops rather than skipping to the next item. +The SDK does not advance automatically, because unplayable content would advance in a loop. +Setting a queue again or selecting a healthy item clears `error`. + +##### Detached playback and reconnecting + +Create a detached player when playback must continue after its tool screen is released. +Enable detached audio in `lighttool.toml`: + +```toml +[tool] +capabilities = ["detached-audio"] +``` + +Wait for its controller before deciding whether the session is fresh: + +```kotlin +val player = audio.newPlayer(playback = LightAudioPlayback.Detached) +if (player.awaitReady() && player.currentMediaItemIndex.value == NO_MEDIA_ITEM) { + player.setMediaQueue(items) +} +``` + +The player `availability` field exposes the connection lifecycle: + +- `Initializing` — the detached controller is connecting. +- `Ready` — commands can be handled. Attached players start here. +- `Released` — the handle is closed and cannot accept commands. + +`awaitReady()` suspends through initialization and returns `true` only when the player reaches `Ready`. Returns `false` if the player is released first. + +A non-empty queue belongs to the surviving service. Reuse it instead of setting the queue again, which would replace live playback. Position, duration, playing state, queue index, and playback error are populated when the controller connects. +Once the service stops, the next player is fresh and restoring its queue and position is the tool's responsibility. + +`release()` disconnects a detached handle but does not stop its playback. +To end detached playback, call `stop()` before `release()`. Calling `release()` +alone only disconnects the handle; playback may continue. +Only one detached handle may exist per tool process. + +A live session must be reopened with the same `LightAudioUsage`; requesting a different usage throws `LightAudioPlayerException`. + +For more details on the detached playback design, refer to [Design Decisions - Detached audio](../../docs/design_decisions/detached_audio.md). + #### PCM voice `LightAudioVoice` plays short mono signed 16-bit PCM buffers. diff --git a/sdk/client/build.gradle.kts b/sdk/client/build.gradle.kts index df4774cf..034afa1e 100644 --- a/sdk/client/build.gradle.kts +++ b/sdk/client/build.gradle.kts @@ -64,6 +64,7 @@ dependencies { implementation(libs.androidx.work.runtime) implementation(libs.androidx.media3.common) implementation(libs.androidx.media3.exoplayer) + implementation(libs.androidx.media3.session) lintChecks(project(":lint-rules")) testImplementation(libs.kotlin.test) diff --git a/sdk/client/src/main/AndroidManifest.xml b/sdk/client/src/main/AndroidManifest.xml index d99d556e..f680fc85 100644 --- a/sdk/client/src/main/AndroidManifest.xml +++ b/sdk/client/src/main/AndroidManifest.xml @@ -14,5 +14,8 @@ + diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioExceptions.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioExceptions.kt index 8fe14a62..c8e6d6ea 100644 --- a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioExceptions.kt +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioExceptions.kt @@ -31,3 +31,7 @@ class LightAudioRecorderException(message: String, cause: Throwable? = null) : */ class LightAudioCaptureException(message: String, cause: Throwable? = null) : LightAudioException(message, cause) + +/** Thrown when detached playback cannot be created. */ +class LightAudioPlayerException(message: String, cause: Throwable? = null) : + LightAudioException(message, cause) diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioTypes.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioTypes.kt index f3e058a2..b059156a 100644 --- a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioTypes.kt +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/AudioTypes.kt @@ -3,6 +3,7 @@ package com.thelightphone.sdk.audio import android.media.AudioAttributes import android.media.MediaRecorder import androidx.media3.common.C +import androidx.media3.common.PlaybackException import androidx.media3.common.AudioAttributes as Media3AudioAttributes /** Describes how the platform should classify, process, and prioritize audio. */ @@ -32,6 +33,54 @@ enum class LightAudioUsage { VoiceCall } +/** Whether playback is tied to the tool screen or survives leaving it. */ +enum class LightAudioPlayback { + /** Playback stops when the player is released. */ + Attached, + /** Playback continues after the player handle is released. */ + Detached, +} + +/** What went wrong during playback, at the granularity a tool can act on. */ +enum class LightAudioErrorKind { + /** Network or file I/O failure. */ + Source, + /** Container, codec, or decoding support failure. */ + Unsupported, + /** Audio output initialization or write failure. */ + Output, + /** Failure without a more actionable classification. */ + Unknown, +} + +/** Current playback failure, using SDK-owned types shared by both playback modes. */ +data class LightAudioError( + /** Actionable failure category. */ + val kind: LightAudioErrorKind, + /** Stable media3 error-code name for logs and bug reports. */ + val diagnostic: String, + /** Queue index that failed, or `-1` when unavailable. */ + val itemIndex: Int, +) + +internal fun errorKindFor(errorCode: Int): LightAudioErrorKind = when (errorCode) { + in PlaybackException.ERROR_CODE_IO_UNSPECIFIED.. + PlaybackException.ERROR_CODE_IO_READ_POSITION_OUT_OF_RANGE -> LightAudioErrorKind.Source + in PlaybackException.ERROR_CODE_PARSING_CONTAINER_MALFORMED.. + PlaybackException.ERROR_CODE_PARSING_MANIFEST_UNSUPPORTED, + in PlaybackException.ERROR_CODE_DECODER_INIT_FAILED.. + PlaybackException.ERROR_CODE_DECODING_RESOURCES_RECLAIMED -> LightAudioErrorKind.Unsupported + in PlaybackException.ERROR_CODE_AUDIO_TRACK_INIT_FAILED.. + PlaybackException.ERROR_CODE_AUDIO_TRACK_OFFLOAD_INIT_FAILED -> LightAudioErrorKind.Output + else -> LightAudioErrorKind.Unknown +} + +internal fun PlaybackException.toLightAudioError(itemIndex: Int): LightAudioError = LightAudioError( + kind = errorKindFor(errorCode), + diagnostic = PlaybackException.getErrorCodeName(errorCode), + itemIndex = itemIndex, +) + internal data class AudioAttributeSpec( val usage: Int, val contentType: Int diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedAudioCapability.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedAudioCapability.kt new file mode 100644 index 00000000..004f5eca --- /dev/null +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedAudioCapability.kt @@ -0,0 +1,24 @@ +package com.thelightphone.sdk.audio + +import android.content.Context +import android.content.pm.PackageManager + +/** + * Written by the Light Gradle plugin for tools that declare the capability in + * `lighttool.toml`. Nothing else can contribute it — unlike the foreground + * service permissions, which any transitive dependency may also declare. + */ +private const val DETACHED_AUDIO_MARKER = "com.thelightphone.sdk.CAPABILITY_DETACHED_AUDIO" + +@Suppress("DEPRECATION") +internal fun Context.requireDetachedAudioCapability() { + val appInfo = packageManager.getApplicationInfo(packageName, PackageManager.GET_META_DATA) + if (appInfo.metaData?.getBoolean(DETACHED_AUDIO_MARKER) != true) { + throw LightAudioPlayerException( + """ + Detached audio requires this lighttool.toml capability: + capabilities = ["detached-audio"] + """.trimIndent() + ) + } +} diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedConnectionHints.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedConnectionHints.kt new file mode 100644 index 00000000..3d9af3d4 --- /dev/null +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedConnectionHints.kt @@ -0,0 +1,35 @@ +package com.thelightphone.sdk.audio + +import android.os.Bundle + +/** + * What a tool's `MediaController` tells [LightAudioService] as it connects. + * + * The service is constructed by the system, not by us, so connection hints are + * the only channel for per-connection context — and they arrive before the + * controller is allowed to do anything. + */ +internal fun detachedConnectionHints(usage: LightAudioUsage): Bundle = + Bundle().apply { + putBoolean(TOOL_CONTROLLER_HINT, true) + putString(USAGE_HINT, usage.name) + } + +internal fun Bundle.requestedUsage(): LightAudioUsage = + getString(USAGE_HINT) + ?.let { name -> LightAudioUsage.entries.firstOrNull { it.name == name } } + ?: LightAudioUsage.Music + +/** + * Whether these hints came from a `LightAudioPlayer` rather than from someone + * else holding a controller on the session. + * + * media3 connects a controller of its own to render the notification, the + * platform connects legacy ones for media buttons, and a LightOS now-playing + * surface will connect one to display playback. None of them should decide the + * session's usage. + */ +internal fun Bundle.isToolController(): Boolean = getBoolean(TOOL_CONTROLLER_HINT, false) + +private const val USAGE_HINT = "com.thelightphone.sdk.audio.usage" +private const val TOOL_CONTROLLER_HINT = "com.thelightphone.sdk.audio.toolController" diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedSessionState.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedSessionState.kt new file mode 100644 index 00000000..5480581c --- /dev/null +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/DetachedSessionState.kt @@ -0,0 +1,82 @@ +package com.thelightphone.sdk.audio + +/** + * The one detached session a tool process can have. + * + * Two facts with deliberately different lifetimes, behind one lock: + * + * - **The handle** spans `newPlayer` to `release`. It caps detached players at + * one per process, and it is what keeps [LightAudioService] alive — a handle + * exists before its `MediaController` connects and after it disconnects, so + * counting controllers instead would let the service stop underneath an + * arriving reconnect. + * - **The session usage** spans the service adopting it to the service stopping. + * It outlives the handle on purpose: releasing a handle does not stop detached + * playback, and a later handle must not silently change the usage of audio + * that is still playing. + * + * **This type is process-local and the service depends on that.** + * [LightAudioService] is declared without `android:process` so it shares the + * tool's process and therefore this instance. Splitting the process gives each + * side its own copy, and the service's copy would report no open handle — it + * would stop itself 60s into any pause with a tool still attached. The service + * asserts its process on startup rather than letting that happen quietly. + */ +internal class DetachedSessionState { + private var handleOpen = false + private var sessionUsage: LightAudioUsage? = null + private var handleChanged: (() -> Unit)? = null + + /** Takes the single detached handle, or fails when one is already out. */ + fun openHandle(): Boolean { + val listener = synchronized(this) { + if (handleOpen) return false + handleOpen = true + handleChanged + } + listener?.invoke() + return true + } + + /** Frees the handle. Does not stop playback — the service owns that. */ + fun closeHandle() { + val listener = synchronized(this) { + handleOpen = false + handleChanged + } + listener?.invoke() + } + + /** Service side: re-evaluate liveness whenever handle ownership changes. */ + @Synchronized + fun setHandleChangedListener(listener: (() -> Unit)?) { + handleChanged = listener + } + + /** Whether a tool holds a handle, and so whether the service must stay up. */ + @Synchronized + fun isHandleOpen(): Boolean = handleOpen + + /** Service side: the live session settled on a usage. */ + @Synchronized + fun adoptUsage(usage: LightAudioUsage) { + sessionUsage = usage + } + + /** Service side: the session is gone, so the next handle starts fresh. */ + @Synchronized + fun clearSession() { + sessionUsage = null + } + + /** The live session's usage, or `null` when no session is running. */ + @Synchronized + fun activeUsage(): LightAudioUsage? = sessionUsage +} + +internal val detachedSessionState = DetachedSessionState() + +internal fun isDetachedUsageCompatible( + activeUsage: LightAudioUsage?, + requestedUsage: LightAudioUsage, +): Boolean = activeUsage == null || activeUsage == requestedUsage diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudio.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudio.kt index ace86f85..fc70aba1 100644 --- a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudio.kt +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudio.kt @@ -6,7 +6,10 @@ import com.thelightphone.sdk.SealedLightActivity interface LightAudio { val capabilities: AudioCapabilities - fun newPlayer(usage: LightAudioUsage = LightAudioUsage.Music): LightAudioPlayer + fun newPlayer( + usage: LightAudioUsage = LightAudioUsage.Music, + playback: LightAudioPlayback = LightAudioPlayback.Attached, + ): LightAudioPlayer fun newRecorder(cfg: RecorderConfig = RecorderConfig()): LightAudioRecorder fun newCapture(cfg: CaptureConfig = CaptureConfig()): LightAudioCapture fun newVoice( @@ -24,9 +27,44 @@ value class DefaultLightAudio( override val capabilities: AudioCapabilities get() = sealedActivity.activity.readAudioCapabilities() - /** Create a player that requests audio focus appropriate for [usage]. */ - override fun newPlayer(usage: LightAudioUsage): LightAudioPlayer { - return LightAudioPlayer(sealedActivity.activity, usage) + /** + * Create a player that requests audio focus appropriate for [usage]. + * + * Only one [LightAudioPlayback.Detached] player handle may exist in the + * process at a time. + */ + override fun newPlayer( + usage: LightAudioUsage, + playback: LightAudioPlayback, + ): LightAudioPlayer { + if (playback == LightAudioPlayback.Attached) { + return LightAudioPlayer(sealedActivity.activity, usage, playback) + } + + sealedActivity.activity.requireDetachedAudioCapability() + val activeUsage = detachedSessionState.activeUsage() + if (!isDetachedUsageCompatible(activeUsage, usage)) { + throw LightAudioPlayerException( + "Detached audio is already using $activeUsage; requested $usage. " + + "Reconnect with the active usage or wait for the detached session to stop." + ) + } + if (!detachedSessionState.openHandle()) { + throw LightAudioPlayerException( + "Only one detached LightAudioPlayer may exist at a time; release the existing player first" + ) + } + return try { + LightAudioPlayer( + sealedActivity.activity, + usage, + playback, + detachedSessionState::closeHandle, + ) + } catch (error: Throwable) { + detachedSessionState.closeHandle() + throw error + } } /** Create a recorder using [cfg]. Call [LightAudioRecorder.release] when done. */ diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioPlayer.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioPlayer.kt index 30334a21..43957d1c 100644 --- a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioPlayer.kt +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioPlayer.kt @@ -1,15 +1,17 @@ package com.thelightphone.sdk.audio +import android.content.ComponentName import android.content.Context -import android.media.AudioManager import android.net.Uri import androidx.media3.common.C import androidx.media3.common.MediaItem import androidx.media3.common.MediaMetadata import androidx.media3.common.PlaybackParameters +import androidx.media3.common.PlaybackException import androidx.media3.common.Player -import androidx.media3.common.util.UnstableApi import androidx.media3.exoplayer.ExoPlayer +import androidx.media3.session.MediaController +import androidx.media3.session.SessionToken import java.io.File import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers @@ -21,6 +23,14 @@ import kotlinx.coroutines.isActive import kotlinx.coroutines.launch import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.first + +/** Whether a [LightAudioPlayer] is initializing, accepts commands, or is terminal. */ +enum class LightAudioPlayerAvailability { + Initializing, + Ready, + Released, +} /** * Plays a queue of local, bundled, or remote audio with observable playback @@ -31,7 +41,9 @@ import kotlinx.coroutines.flow.StateFlow */ class LightAudioPlayer internal constructor( context: Context, - usage: LightAudioUsage = LightAudioUsage.Music + usage: LightAudioUsage = LightAudioUsage.Music, + internal val playback: LightAudioPlayback = LightAudioPlayback.Attached, + private val onRelease: () -> Unit = {}, ) { private val scopeJob = SupervisorJob() private val scope = CoroutineScope(scopeJob + Dispatchers.Main.immediate) @@ -39,8 +51,11 @@ class LightAudioPlayer internal constructor( private val _durationMs = MutableStateFlow(0L) private val _isPlaying = MutableStateFlow(false) private val _currentMediaItemIndex = MutableStateFlow(NO_MEDIA_ITEM) + private val _error = MutableStateFlow(null) + private val commands = PendingPlayerCommands() private var positionJob: Job? = null - private var pausedForTransientLoss = false + private var player: Player? = null + private var cancelPendingConnection: (() -> Unit)? = null private var released = false /** Current position in milliseconds, updated while playing. */ @@ -51,16 +66,34 @@ class LightAudioPlayer internal constructor( val isPlaying: StateFlow = _isPlaying /** Current queue index, or `-1` when the queue is empty. */ val currentMediaItemIndex: StateFlow = _currentMediaItemIndex + /** Current playback failure, or `null` after successful re-preparation. */ + val error: StateFlow = _error + /** Connection and command-acceptance lifecycle of this player. */ + val availability: StateFlow = commands.availability + + init { + when (playback) { + LightAudioPlayback.Attached -> connectPlayer( + ExoPlayer.Builder(context).build().apply { + setAudioAttributes(usage.toMedia3AudioAttributes(), true) + }, + ) + LightAudioPlayback.Detached -> connectDetachedPlayer(context, usage) + } + } - private val player = ExoPlayer.Builder(context).build().apply player@{ - setAudioAttributes(usage.toMedia3AudioAttributes(), false) - addListener(object : Player.Listener { + private fun connectPlayer(connectedPlayer: Player) { + if (released) { + connectedPlayer.release() + return + } + player = connectedPlayer + connectedPlayer.addListener(object : Player.Listener { override fun onMediaItemTransition(mediaItem: MediaItem?, reason: Int) { - // `this@player` is the ExoPlayer (Int index), not the wrapper's StateFlow. _currentMediaItemIndex.value = if (mediaItem == null) { NO_MEDIA_ITEM } else { - this@player.currentMediaItemIndex + connectedPlayer.currentMediaItemIndex } } @@ -70,49 +103,60 @@ class LightAudioPlayer internal constructor( startPositionUpdates() } else { stopPositionUpdates() - updatePosition() + updatePosition(connectedPlayer) } } override fun onPlaybackStateChanged(playbackState: Int) { - updateDuration() - updatePosition() + updateDuration(connectedPlayer) + updatePosition(connectedPlayer) if (playbackState == Player.STATE_ENDED) { stopPositionUpdates() - abandonFocus() } } + + override fun onPlayerErrorChanged(error: PlaybackException?) { + _error.value = error?.toLightAudioError(connectedPlayer.currentMediaItemIndex) + } }) + val state = connectedPlayer.snapshotState() + _currentMediaItemIndex.value = state.currentMediaItemIndex + _positionMs.value = state.positionMs + _durationMs.value = state.durationMs + _isPlaying.value = state.isPlaying + _error.value = connectedPlayer.playerError + ?.toLightAudioError(connectedPlayer.currentMediaItemIndex) + if (state.isPlaying) { + startPositionUpdates() + } else { + stopPositionUpdates() + } + commands.ready(connectedPlayer) } - private val focus = AudioFocusHelper( - context = context, - usage = usage, - gainType = AudioManager.AUDIOFOCUS_GAIN, - onFocusChange = ::onAudioFocusChange - ) + private fun connectDetachedPlayer(context: Context, usage: LightAudioUsage) { + val token = SessionToken(context, ComponentName(context, LightAudioService::class.java)) + val future = MediaController.Builder(context, token) + .setConnectionHints(detachedConnectionHints(usage)) + .buildAsync() + cancelPendingConnection = { future.cancel(false) } + future.addListener( + { + cancelPendingConnection = null + runCatching(future::get) + .onSuccess(::connectPlayer) + .onFailure { release() } + }, + context.mainExecutor, + ) + } /** Playback rate, clamped to a minimum positive rate. */ var speed: Float = 1.0f set(value) { - field = value.coerceAtLeast(MIN_SPEED) - player.playbackParameters = PlaybackParameters(field) - } - - /** Enables the platform player's silence-skipping behavior. */ - var skipSilence: Boolean = false - @androidx.annotation.OptIn(markerClass = [UnstableApi::class]) - set(value) { - field = value - player.skipSilenceEnabled = value - } - - /** When `true`, playback pauses at the end of each queue item instead of advancing. */ - var pauseAtEndOfMediaItems: Boolean = false - @androidx.annotation.OptIn(markerClass = [UnstableApi::class]) - set(value) { - field = value - player.pauseAtEndOfMediaItems = value + val speed = value.coerceAtLeast(MIN_SPEED) + commands.dispatch { it.playbackParameters = PlaybackParameters(speed) } + field = speed } /** Replaces the queue with [file] and prepares it for playback. */ @@ -138,19 +182,23 @@ class LightAudioPlayer internal constructor( */ fun setMediaQueue(items: List, startIndex: Int = 0) { if (items.isEmpty()) { - player.clearMediaItems() - _currentMediaItemIndex.value = NO_MEDIA_ITEM - updateDuration() - updatePosition() + commands.dispatch { player -> + player.clearMediaItems() + _currentMediaItemIndex.value = NO_MEDIA_ITEM + updateDuration(player) + updatePosition(player) + } return } require(startIndex in items.indices) { "Start index must reference a queue item" } val mediaItems = items.mapIndexed { index, item -> item.toMediaItem(index) } - player.setMediaItems(mediaItems, startIndex, C.TIME_UNSET) - _currentMediaItemIndex.value = startIndex - player.prepare() - updateDuration() - updatePosition() + commands.dispatch { player -> + player.setMediaItems(mediaItems, startIndex, C.TIME_UNSET) + _currentMediaItemIndex.value = startIndex + player.prepare() + updateDuration(player) + updatePosition(player) + } } /** @@ -159,32 +207,29 @@ class LightAudioPlayer internal constructor( * Observe [isPlaying] for the actual playback state. */ fun play() { - if (released || !focus.request()) { - return - } - player.play() + commands.dispatch(Player::play) } - /** Pauses playback and abandons audio focus. */ + /** Pauses playback. */ fun pause() { - pausedForTransientLoss = false - player.pause() - abandonFocus() + commands.dispatch(Player::pause) } - /** Stops playback, returns to position zero, and abandons audio focus. */ + /** Stops playback and returns to position zero. */ fun stop() { - pausedForTransientLoss = false - player.stop() - player.seekTo(0L) - updatePosition() - abandonFocus() + commands.dispatch { player -> + player.stop() + player.seekTo(0L) + updatePosition(player) + } } /** Seeks to [ms], clamped to the resolved duration. Unknown duration clamps to zero. */ fun seekTo(ms: Long) { - player.seekTo(ms.coerceIn(0L, player.duration.validDuration())) - updatePosition() + commands.dispatch { player -> + player.seekTo(ms.coerceIn(0L, player.duration.validDuration())) + updatePosition(player) + } } /** Seeks backward 15 seconds, clamped to the item bounds. */ @@ -199,60 +244,45 @@ class LightAudioPlayer internal constructor( /** Selects the next queue item when one exists. */ fun skipToNext() { - player.seekToNextMediaItem() + commands.dispatch(Player::seekToNextMediaItem) } /** Selects the previous queue item when one exists. */ fun skipToPrevious() { - player.seekToPreviousMediaItem() + commands.dispatch(Player::seekToPreviousMediaItem) } - /** Permanently releases playback, focus, and state-update resources. Idempotent. */ + /** Waits for connection, returning `false` if this player is released first. */ + suspend fun awaitReady(): Boolean = awaitPlayerReady(availability) + + /** + * Releases this handle. Attached playback stops; detached playback continues + * until [stop] is called or the service's idle rule fires. Idempotent. + */ fun release() { if (released) return released = true stopPositionUpdates() - abandonFocus() - player.release() - scope.cancel() - } - - private fun onAudioFocusChange(change: Int) { - when (change) { - AudioManager.AUDIOFOCUS_LOSS -> { - pausedForTransientLoss = false - scope.launch { player.pause() } - } - - AudioManager.AUDIOFOCUS_LOSS_TRANSIENT -> { - pausedForTransientLoss = player.isPlaying - scope.launch { player.pause() } - } - - AudioManager.AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK -> { - player.volume = DUCKED_VOLUME - } - - AudioManager.AUDIOFOCUS_GAIN -> { - player.volume = FULL_VOLUME - if (pausedForTransientLoss) { - pausedForTransientLoss = false - scope.launch { play() } - } - } + commands.release() + try { + cancelPendingConnection?.invoke() + cancelPendingConnection = null + player?.release() + player = null + scope.cancel() + } finally { + onRelease() } } - private fun abandonFocus() { - focus.abandon() - } - private fun startPositionUpdates() { if (positionJob?.isActive == true) return positionJob = scope.launch { while (isActive) { - updatePosition() - updateDuration() + player?.let { + updatePosition(it) + updateDuration(it) + } delay(POSITION_POLL_MS) } } @@ -263,15 +293,20 @@ class LightAudioPlayer internal constructor( positionJob = null } - private fun updatePosition() { + private fun updatePosition(player: Player) { _positionMs.value = player.currentPosition.coerceAtLeast(0L) } - private fun updateDuration() { + private fun updateDuration(player: Player) { _durationMs.value = player.duration.validDuration() } } +internal suspend fun awaitPlayerReady( + availability: StateFlow, +): Boolean = availability.first { it != LightAudioPlayerAvailability.Initializing } == + LightAudioPlayerAvailability.Ready + internal fun LightAudioItem.toMediaItem(queueIndex: Int): MediaItem { val uri = Uri.parse(source.uriString()) return MediaItem.Builder() @@ -301,11 +336,24 @@ internal fun skipPosition(positionMs: Long, durationMs: Long, deltaMs: Long): Lo return (positionMs + deltaMs).coerceIn(0L, durationMs.validDuration()) } +internal data class ConnectedPlayerState( + val currentMediaItemIndex: Int, + val positionMs: Long, + val durationMs: Long, + val isPlaying: Boolean, +) + +internal fun Player.snapshotState(): ConnectedPlayerState = ConnectedPlayerState( + currentMediaItemIndex = if (mediaItemCount == 0) NO_MEDIA_ITEM else currentMediaItemIndex, + positionMs = currentPosition.coerceAtLeast(0L), + durationMs = duration.validDuration(), + isPlaying = isPlaying, +) + private fun Long.validDuration(): Long = takeIf { it > 0L && it != C.TIME_UNSET } ?: 0L private const val SKIP_INTERVAL_MS = 15_000L private const val POSITION_POLL_MS = 250L private const val MIN_SPEED = 0.1f -private const val DUCKED_VOLUME = 0.2f -private const val FULL_VOLUME = 1.0f -private const val NO_MEDIA_ITEM = -1 +/** Queue index reported when a player has no media item. */ +const val NO_MEDIA_ITEM = -1 diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioService.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioService.kt new file mode 100644 index 00000000..7fc40457 --- /dev/null +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/LightAudioService.kt @@ -0,0 +1,146 @@ +package com.thelightphone.sdk.audio + +import android.app.Application +import android.os.Handler +import android.os.Looper +import androidx.media3.common.Player +import androidx.media3.exoplayer.ExoPlayer +import androidx.media3.session.MediaSession +import androidx.media3.session.MediaSessionService + +/** + * Hosts detached playback: one [ExoPlayer] and one [MediaSession], shared by + * every controller that connects. + * + * Declared without `android:process`, so it runs in the tool's own process and + * shares [detachedSessionState] with the tool that created the handle. That is + * load-bearing, not incidental — see [DetachedSessionState]. + * + * Started by the system when a tool builds a `MediaController` against it, and + * stopped by itself once playback is paused and no tool holds a handle — see + * [shouldStartIdleStop]. + */ +internal class LightAudioService : MediaSessionService() { + + private lateinit var player: ExoPlayer + private lateinit var session: MediaSession + private val idleHandler = Handler(Looper.getMainLooper()) + private val idleStop = Runnable { stopSelf() } + + override fun onCreate() { + super.onCreate() + + assertSharedProcess() + + player = ExoPlayer.Builder(this).build().apply { + setAudioAttributes(LightAudioUsage.Music.toMedia3AudioAttributes(), true) + addListener(object : Player.Listener { + override fun onIsPlayingChanged(isPlaying: Boolean) { + refreshIdleStop() + } + }) + } + session = MediaSession.Builder(this, player) + .setId(packageName) + .setCallback(SessionCallback()) + .build() + detachedSessionState.setHandleChangedListener { + idleHandler.post(::refreshIdleStop) + } + refreshIdleStop() + } + + override fun onGetSession(controllerInfo: MediaSession.ControllerInfo): MediaSession = session + + override fun onDestroy() { + idleHandler.removeCallbacks(idleStop) + detachedSessionState.setHandleChangedListener(null) + detachedSessionState.clearSession() + session.release() + player.release() + super.onDestroy() + } + + private inner class SessionCallback : MediaSession.Callback { + override fun onConnect( + session: MediaSession, + controller: MediaSession.ControllerInfo, + ): MediaSession.ConnectionResult { + // Notification, Bluetooth, and system media controllers also connect + // here, but only our LightAudioPlayer controller requests a usage. + if (controller.connectionHints.isToolController()) { + val requestedUsage = controller.connectionHints.requestedUsage() + // newPlayer checks this synchronously. Repeat it here because the + // service state may change before the async controller connects. + if (!isDetachedUsageCompatible(detachedSessionState.activeUsage(), requestedUsage)) { + return MediaSession.ConnectionResult.reject() + } + // An empty player has no live playback to preserve, so this fresh + // session may take the connecting tool's requested usage. + if (player.mediaItemCount == 0) { + adoptUsage(requestedUsage) + } + } + return super.onConnect(session, controller) + } + + // Connections are only a prompt to re-evaluate; the answer comes from + // the handle, not from who is attached. + override fun onPostConnect(session: MediaSession, controller: MediaSession.ControllerInfo) { + refreshIdleStop() + } + + override fun onDisconnected(session: MediaSession, controller: MediaSession.ControllerInfo) { + refreshIdleStop() + } + } + + /** A fresh session takes the connecting handle's usage; a live one keeps its own. */ + private fun adoptUsage(usage: LightAudioUsage) { + player.setAudioAttributes(usage.toMedia3AudioAttributes(), true) + detachedSessionState.adoptUsage(usage) + } + + private fun refreshIdleStop() { + idleHandler.removeCallbacks(idleStop) + if (shouldStartIdleStop(player.isPlaying, detachedSessionState.isHandleOpen())) { + idleHandler.postDelayed(idleStop, IDLE_STOP_MS) + } + } + + /** + * Fails loudly if someone gives this service its own process, which would + * silently give it a second [detachedSessionState] that no tool ever opens + * a handle on. Nothing else can catch that: it is a manifest attribute, so + * there is no compile error, and the symptom is playback stopping a minute + * into a pause. + */ + private fun assertSharedProcess() { + val process = Application.getProcessName() + // Compared against the application's own process rather than the package + // name: a tool may legitimately rename its default process, and that is + // not a split. + val appProcess = applicationInfo.processName + check(process == appProcess) { + "LightAudioService must run in the tool's default process ('$appProcess') " + + "but is in '$process'. Remove android:process from its manifest entry — " + + "detached playback state is shared with the tool as process-local state." + } + } +} + +/** + * `MediaSessionService` manages *foreground* state, not *service* state: a + * paused session without a holder would keep an [ExoPlayer] and the tool's + * process alive indefinitely. + * + * A handle exists before its controller connects and after it disconnects, + * so counting controllers would let the service stop when a recconnect arrives, + * and it would also count media3's notification controller, the platform's + * media-button controllers, and the LightOS now-playing surface, + * none of which own the session. + */ +internal fun shouldStartIdleStop(isPlaying: Boolean, handleOpen: Boolean): Boolean = + !isPlaying && !handleOpen + +internal const val IDLE_STOP_MS = 60_000L diff --git a/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/PendingPlayerCommands.kt b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/PendingPlayerCommands.kt new file mode 100644 index 00000000..6de736e5 --- /dev/null +++ b/sdk/client/src/main/kotlin/com/thelightphone/sdk/audio/PendingPlayerCommands.kt @@ -0,0 +1,57 @@ +package com.thelightphone.sdk.audio + +import androidx.media3.common.Player +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow + +internal class PendingPlayerCommands { + private val pending = mutableListOf<(Player) -> Unit>() + private val _availability = MutableStateFlow(LightAudioPlayerAvailability.Initializing) + private var player: Player? = null + private var released = false + + val availability: StateFlow = _availability + + fun dispatch(command: (Player) -> Unit) { + val readyPlayer = synchronized(this) { + checkNotReleased() + player?.also { return@synchronized it } ?: run { + pending += command + null + } + } + readyPlayer?.let(command) + } + + fun ready(player: Player) { + while (true) { + val commands = synchronized(this) { + if (released) return + if (pending.isEmpty()) { + this.player = player + _availability.value = LightAudioPlayerAvailability.Ready + return + } + pending.toList().also { pending.clear() } + } + commands.forEach { it(player) } + } + } + + fun requireActive() { + synchronized(this) { checkNotReleased() } + } + + fun release() { + synchronized(this) { + released = true + player = null + pending.clear() + _availability.value = LightAudioPlayerAvailability.Released + } + } + + private fun checkNotReleased() { + check(!released) { "LightAudioPlayer has been released" } + } +} diff --git a/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioPlayerTest.kt b/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioPlayerTest.kt index 722261c9..345cbe50 100644 --- a/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioPlayerTest.kt +++ b/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioPlayerTest.kt @@ -1,9 +1,163 @@ package com.thelightphone.sdk.audio +import androidx.media3.common.C +import androidx.media3.common.Player +import androidx.media3.common.PlaybackException +import java.lang.reflect.Proxy +import kotlinx.coroutines.async +import kotlinx.coroutines.runBlocking +import kotlinx.coroutines.yield import kotlin.test.Test import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertTrue class LightAudioPlayerTest { + private val player = Proxy.newProxyInstance( + Player::class.java.classLoader, + arrayOf(Player::class.java), + ) { _, _, _ -> null } as Player + + @Test + fun playbackErrorCodesMapToActionableKinds() { + assertEquals( + LightAudioErrorKind.Source, + errorKindFor(PlaybackException.ERROR_CODE_IO_FILE_NOT_FOUND), + ) + assertEquals( + LightAudioErrorKind.Unsupported, + errorKindFor(PlaybackException.ERROR_CODE_DECODING_FORMAT_UNSUPPORTED), + ) + assertEquals( + LightAudioErrorKind.Output, + errorKindFor(PlaybackException.ERROR_CODE_AUDIO_TRACK_INIT_FAILED), + ) + assertEquals(LightAudioErrorKind.Unknown, errorKindFor(Int.MAX_VALUE)) + } + + @Test + fun pendingCommandsWaitUntilReadyAndFlushInOrder() { + val commands = PendingPlayerCommands() + val calls = mutableListOf() + + commands.dispatch { calls += 1 } + commands.dispatch { calls += 2 } + assertTrue(calls.isEmpty()) + + commands.ready(player) + + assertEquals(listOf(1, 2), calls) + } + + @Test + fun connectedPlayerSnapshotRestoresLivePlaybackState() { + val livePlayer = Proxy.newProxyInstance( + Player::class.java.classLoader, + arrayOf(Player::class.java), + ) { _, method, _ -> + when (method.name) { + "getMediaItemCount" -> 3 + "getCurrentMediaItemIndex" -> 1 + "getCurrentPosition" -> 12_345L + "getDuration" -> 60_000L + "isPlaying" -> true + else -> null + } + } as Player + + assertEquals( + ConnectedPlayerState( + currentMediaItemIndex = 1, + positionMs = 12_345L, + durationMs = 60_000L, + isPlaying = true, + ), + livePlayer.snapshotState(), + ) + } + + @Test + fun connectedPlayerSnapshotIdentifiesFreshSession() { + val freshPlayer = Proxy.newProxyInstance( + Player::class.java.classLoader, + arrayOf(Player::class.java), + ) { _, method, _ -> + when (method.name) { + "getMediaItemCount" -> 0 + "getCurrentMediaItemIndex" -> 0 + "getCurrentPosition" -> 0L + "getDuration" -> C.TIME_UNSET + "isPlaying" -> false + else -> null + } + } as Player + + val state = freshPlayer.snapshotState() + assertEquals(NO_MEDIA_ITEM, state.currentMediaItemIndex) + assertEquals(0L, state.durationMs) + } + + @Test + fun commandsRunImmediatelyAfterReady() { + val commands = PendingPlayerCommands() + val calls = mutableListOf() + commands.ready(player) + + commands.dispatch { calls += 1 } + + assertEquals(listOf(1), calls) + } + + @Test + fun availabilityMovesFromInitializingToReady() { + val commands = PendingPlayerCommands() + assertEquals(LightAudioPlayerAvailability.Initializing, commands.availability.value) + + commands.ready(player) + + assertEquals(LightAudioPlayerAvailability.Ready, commands.availability.value) + assertTrue(runBlocking { awaitPlayerReady(commands.availability) }) + } + + @Test + fun releaseDropsPendingCommandsAndRejectsNewOnes() { + val commands = PendingPlayerCommands() + var called = false + commands.dispatch { called = true } + + commands.release() + commands.ready(player) + + assertEquals(false, called) + assertEquals(LightAudioPlayerAvailability.Released, commands.availability.value) + assertEquals(false, runBlocking { awaitPlayerReady(commands.availability) }) + assertFailsWith { commands.dispatch { called = true } } + assertFailsWith { commands.requireActive() } + } + + @Test + fun releaseAfterReadyIsTerminal() { + val commands = PendingPlayerCommands() + commands.ready(player) + + commands.release() + commands.ready(player) + + assertEquals(LightAudioPlayerAvailability.Released, commands.availability.value) + } + + @Test + fun awaitingConnectionReturnsFalseWhenReleased() = runBlocking { + val commands = PendingPlayerCommands() + val result = async { awaitPlayerReady(commands.availability) } + yield() + assertEquals(false, result.isCompleted) + + commands.release() + + assertEquals(false, result.await()) + } + @Test fun skipPositionClampsToStartAndDuration() { assertEquals(0L, skipPosition(positionMs = 5_000L, durationMs = 60_000L, deltaMs = -15_000L)) diff --git a/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioServiceTest.kt b/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioServiceTest.kt new file mode 100644 index 00000000..69f38ce2 --- /dev/null +++ b/sdk/client/src/test/kotlin/com/thelightphone/sdk/audio/LightAudioServiceTest.kt @@ -0,0 +1,85 @@ +package com.thelightphone.sdk.audio + +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFalse +import kotlin.test.assertNull +import kotlin.test.assertTrue + +class LightAudioServiceTest { + @Test + fun idleStopsOnlyWhenPausedAndUnheld() { + assertTrue(shouldStartIdleStop(isPlaying = false, handleOpen = false)) + assertFalse(shouldStartIdleStop(isPlaying = false, handleOpen = true)) + assertFalse(shouldStartIdleStop(isPlaying = true, handleOpen = false)) + assertFalse(shouldStartIdleStop(isPlaying = true, handleOpen = true)) + } +} + +class DetachedSessionStateTest { + @Test + fun usageCompatibilityAcceptsFreshAndSameUsageOnly() { + assertTrue(isDetachedUsageCompatible(activeUsage = null, LightAudioUsage.Alarm)) + assertTrue(isDetachedUsageCompatible(LightAudioUsage.Music, LightAudioUsage.Music)) + assertFalse(isDetachedUsageCompatible(LightAudioUsage.Music, LightAudioUsage.Alarm)) + } + + @Test + fun allowsOneHandleUntilClosed() { + val state = DetachedSessionState() + + assertTrue(state.openHandle()) + assertFalse(state.openHandle()) + state.closeHandle() + assertTrue(state.openHandle()) + } + + @Test + fun reportsWhetherAHandleIsOpen() { + val state = DetachedSessionState() + + assertFalse(state.isHandleOpen()) + state.openHandle() + assertTrue(state.isHandleOpen()) + state.closeHandle() + assertFalse(state.isHandleOpen()) + } + + @Test + fun handleChangesNotifyTheServiceLivenessObserver() { + val state = DetachedSessionState() + var changes = 0 + state.setHandleChangedListener { changes++ } + + state.openHandle() + state.closeHandle() + + assertEquals(2, changes) + } + + /** + * Releasing a handle does not stop detached playback, + * so the live session's usage has to outlive it. + */ + @Test + fun sessionUsageOutlivesTheHandle() { + val state = DetachedSessionState() + + state.openHandle() + state.adoptUsage(LightAudioUsage.Alarm) + state.closeHandle() + + assertFalse(state.isHandleOpen()) + assertEquals(LightAudioUsage.Alarm, state.activeUsage()) + } + + @Test + fun clearingTheSessionLetsTheNextHandleStartFresh() { + val state = DetachedSessionState() + + state.adoptUsage(LightAudioUsage.Alarm) + state.clearSession() + + assertNull(state.activeUsage()) + } +} diff --git a/tool/lighttool.toml b/tool/lighttool.toml index 169c8696..a6a0d5ba 100644 --- a/tool/lighttool.toml +++ b/tool/lighttool.toml @@ -4,6 +4,7 @@ label = "Light SDK Tool" versionCode = 1 versionName = "1.0.0" permissions = ["android.permission.INTERNET"] +capabilities = [] # change if you run this on an LP3! # serverPackage = "com.lightos" serverPackage = "com.thelightphone.sdk.emulator"