diff --git a/theoplayer/how-to-guides/web/theolive/metadata.mdx b/theoplayer/how-to-guides/web/theolive/metadata.mdx index 2d68cbb2be89..3c6a4ed2d404 100644 --- a/theoplayer/how-to-guides/web/theolive/metadata.mdx +++ b/theoplayer/how-to-guides/web/theolive/metadata.mdx @@ -8,40 +8,103 @@ sidebar_position: 3 Metadata that travels through your OptiView Live stream — whether [pushed via the API or embedded as SEI](/theolive/channel/metadata-insertion/) — surfaces on the player as cues on a text track. The general approach is the same in every case: 1. Listen for `addtrack` on `player.textTracks` so you find out when a new metadata track shows up. -2. Inspect the track to make sure it carries the metadata you care about. -3. Subscribe to a cue event on that track and read the payload from `cue.content`. +2. Subscribe to a cue event on that track. +3. Inspect the cue to make sure it carries the metadata you care about. +4. Read the payload from `cue.content`. The properties you match on, the cue event you subscribe to, and the shape of `cue.content` all depend on which kind of metadata is being delivered. ## User-data-unregistered SEI and API push -UDU SEI and API-pushed payloads surface on a text track with `type === 'emsg'` and an `inBandMetadataTrackDispatchType` of `urn:uuid:`. Pick the cue event that matches when you want to react: `addcue` fires as soon as a payload becomes available on the track, before playback reaches it — useful when you want to prefetch data or update application state ahead of time. `entercue` fires when playback enters the cue — useful when you want to act in sync with the video, for example to render an overlay at the moment the cue was inserted. +UDU SEI and API-pushed payloads surface as ID3 `PRIV` frames inside an ID3 v2.4 tag embedded in an `emsg` box, carried by the CMAF segments of the stream. + +- The owner identifier of the `PRIV` frame will be set to: `optiview.live:meta:` +- The frame's private data will contain your metadata as raw binary data. + +(See [section 4.27 of the ID3 v2.4 Native Frames specification](https://id3.org/id3v2.4.0-frames) and [ID3 in CMAF](https://aomediacodec.github.io/id3-emsg/) for more details.) + +The OptiView Player SDK exposes these ID3 frames on a text track with [`TextTrack.type`](pathname:///theoplayer/v11/api-reference/web/interfaces/TextTrack.html#type) equal to `'id3'`. Listen for the cue event that matches when you want to react: + +- `addcue` fires as soon as a cue becomes available on the track, before playback reaches it — useful when you want to prefetch data or update application state ahead of time. +- `entercue` fires when playback enters the cue — useful when you want to act in sync with the video, for example to render an overlay at the moment the cue was inserted. + +The ID3 frame will be contained in `cue.content` as a [ID3PrivateFrame](pathname:///theoplayer/v11/api-reference/web/interfaces/ID3PrivateFrame.html) with `ownerIdentifier` equal to `optiview.live:meta:`. + +:::important +Make sure to always check the frame's `id` and `ownerIdentifier` before using it, to avoid processing unrelated ID3 frames. +::: Suppose your UUID is `11111111-1111-1111-1111-111111111111`: ```javascript -player.textTracks.addEventListener('addtrack', (event) => { +player.textTracks.addEventListener('addtrack', function trackListener(event) { const track = event.track; - if (track.type === 'emsg' && track.inBandMetadataTrackDispatchType === 'urn:uuid:11111111-1111-1111-1111-111111111111') { - track.addEventListener('addcue', (e) => { - console.log('cue', e.cue.content); + if (track.type === 'id3') { + // ID3 track was added. Listen for incoming ID3 cues. + track.addEventListener('addcue' /* or 'entercue' */, (e) => { + const frame = e.cue.content; + if (frame.id === 'PRIV' && frame.ownerIdentifier === 'optiview.live:meta:11111111-1111-1111-1111-111111111111') { + console.log('metadata', frame.data); + } }); + track.removeEventListener('typechange', trackListener); + } else if (track.type === '') { + // Track type is not yet known. Check again when it becomes known. + track.addEventListener('typechange', trackListener); + } else { + // Track type is known, but is not ID3. Stop listening. + track.removeEventListener('typechange', trackListener); + } +}); +``` + +Here the `frame.data` carries the binary payload exactly as it was sent — your application is responsible for parsing it (for example UTF-8 JSON, protobuf, or your own format). + +
+Using third-party players + +When using a third-party player SDK, please refer to their respective documentation on how to retrieve ID3 from the stream. + +For example, with [Shaka Player](https://github.com/shaka-project/shaka-player), +you can listen for the [`metadata` event](https://shaka-player-demo.appspot.com/docs/api/shaka.Player.html#.event:MetadataEvent): + +```javascript +player.addEventListener('metadata', (event) => { + if (event.metadataType === 'org.id3') { + const frame = event.payload; // see https://shaka-player-demo.appspot.com/docs/api/shaka.extern.html#.MetadataFrame + if (frame.key === 'PRIV' && frame.description === 'optiview.live:meta:11111111-1111-1111-1111-111111111111') { + console.log('metadata', frame.data); + } + } +}); +``` + +With [hls.js](https://github.com/video-dev/hls.js/), you can listen for the `FRAG_PARSING_METADATA` event. +However, hls.js doesn't come with a built-in ID3 parser, so you'll need to parse the ID3 tag manually: + +```javascript +hls.on(Hls.Events.FRAG_PARSING_METADATA, (event, data) => { + for (const sample of data.samples) { + if (sample.type === Hls.MetadataSchema.emsg) { + const tag = sample.data; + // TODO Parse ID3 tag and look for ID3 PRIV frame with correct owner identifier + } } }); ``` -Here the cue `content` carries the binary payload exactly as it was sent — your application is responsible for parsing it (for example UTF-8 JSON, protobuf, or your own format). +
## Picture timing SEI -Picture timing SEI cues surface on a text track with `id === 'timecode'`. Each cue is anchored to a frame, so `entercue` is usually the most useful event: it fires when playback reaches the cue, letting you react in sync with the video. +Picture timing SEI cues surface on a text track with [`TextTrack.id`](pathname:///theoplayer/v11/api-reference/web/interfaces/TextTrack.html#id) equal to `'timecode'`. Each cue is anchored to a frame, so `entercue` is usually the most useful event: it fires when playback reaches the cue, letting you react in sync with the video. ```javascript player.textTracks.addEventListener('addtrack', (event) => { const track = event.track; if (track.id === 'timecode') { track.mode = 'showing'; // Setting the mode to showing will enable the entercue events - track.addEventListener('entercue', this.onTimeCode); + track.addEventListener('entercue', onTimeCode); } }); ```