[Proposal] SE-00001: Implementation of RTP & RTCP Packet Integration and Parsing in libAES67 for Real-Time Audio Transport

[N3v1]

Proposal Title

Implementation of RTP Packet Integration and Parsing in libAES67 for Real-Time Audio Transport

Proposal ID

SE-00001

Author

Nevio Hirani

Motivation

The AES67 standard depends on the Real-time Transport Protocol (RTP) to facilitate low-latency, high-precision audio transmission over IP networks (AoIP). Currently, libAES67 lacks native implementations of RTP in its initial release, necessitating the development of functionalities for decoding, encoding, synchronization, and real-time audio stream processing.

Integrating RTP packet parsing capabilities into libAES67 will empower the library to directly ingest, interpret, and manage audio data streams transported via RTP. This enhancement will support deployment scenarios such as media gateways, broadcast infrastructure, and live IP audio distribution. Moreover, this integration will promote interoperability with other AES67-compliant systems by enabling efficient jitter buffering, payload handling, extraction, and timestamp synchronization within a unified, standards-compliant framework.

Specification

This specification defines the structure, interpretation, and required behavior for parsing RTP packets within the libAES67 library. According to AES67, the real-time transport of audio streams is performed using the Real-time Transport Protocol (RTP), as specified in RFC 3550. To ensure compliance and interoperability with AES67-compliant devices and networks, libAES67 must implement a parser that is capable of validating, interpreting, and extracting essential information from incoming RTP packets carrying audio payloads (namely PCM, L16, L24, Opus, etc.).

RTP Packet Header Specification

An RTP packet begins with a fixed 12-byte header, optionally followed by a list of Contributing Source (CSRC) identifiers. Depending on the packet, it may also include extension headers, audio payload data, and optional padding. The layout of the standard RTP header is illustrated below:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X|  CC   |M|     PT      |       sequence number         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           timestamp                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           synchronization source (SSRC) identifier            |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|            contributing source (CSRC) identifiers             |
|                             ....                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Header Fields

• Version (V, 2 bits): This field indicates the version of the RTP protocol. It must always be set to 2 in compliant packets. Any other version should cause the packet to be discarded by libAES67.
• Padding (P, 1 bit): If this bit is set, the packet contains extra padding bytes at the end. The last byte of the packet specifies how many padding bytes are present. Padding may be used for alignment or encryption schemes.
• Extension (X, 1 bit): When set, this bit signals the presence of an RTP header extension. The extension follows immediately after the CSRC list and includes a 16-bit defined identifier, a length field, and extension-specific data.
• CSRC Count (CC, 4 bits): Specifies the number (0–15) of 32-bit CSRC (Contributing Source) identifiers that follow the SSRC field. These identify the sources of contributing streams in a mixed session. In most AES67 use cases, this is typically zero.
• Marker (M, 1 bit): The interpretation of this bit is defined by the application profile. It may be used to mark significant events in the stream (e.g., the beginning of a talk spurt). AES67 does not explicitly require its use.
• Payload Type (PT, 7 bits): Indicates the format of the audio payload. For AES67, this is typically a dynamic value such as 96 (used for L16 stereo) or another negotiated format via SDP. It must correspond to a known encoding and sample format.
• Sequence Number (16 bits): This value increments by one for each RTP packet sent and is used to detect packet loss or reordering. Proper sequencing is essential for jitter buffering and stream continuity.
• Timestamp (32 bits): Represents the sampling instant of the first byte in the payload. This timestamp is derived from a clock that increments at the sampling rate (e.g. 48,000 Hz). It is used for playback timing and synchronization.
• SSRC (Synchronization Source Identifier, 32 bits): A randomly chosen identifier that uniquely distinguishes the source of the RTP stream. All packets from a given stream must have the same SSRC. This is essential for distinguishing multiple simultaneous streams.
• CSRC Identifiers (0–15 × 32 bits, optional): A list of contributing source identifiers used in cases where a mixer combines multiple streams into one. Each CSRC represents one original source. This field is rarely used in typical AES67 deployments.

RTP Package Validation

To support AES67-compliant streaming, the libAES67 RTP parser must be able to validate that the incoming packet conforms to version 2 of the protocol. It must correctly extract and interpret the sequence number, timestamp, payload type, and SSRC fields. The parser must also identify and skip any CSRC identifiers and RTP extensions to isolate the audio payload. The payload must then be passed to the decoding layer for the library, along with metadata such as the timestamp and sequence number for synchronization and jitter management.

The parser must detect and report anomalies such as out-of-order packets, duplicate sequence numbers, and malformed headers. If padding is indicated by the padding flag, the parser must inspect the last byte of the packet to determine the correct payload length. If RTP header extensions are used, these must either be ignored or parsed according to the specific profile in use.

For the initial implementation, support for jitter buffering, packet reordering, and loss concealment may be deferred or simplified, but must remain extensible for future enhancements. The parser must be modular and designed to integrate with existing time synchronization and audio decoding components of libAES67.

RTCP Considerations

Although RTP is responsible for transporting audio data, it is designed to operate in conjunction with the Real-time Transport Control Protocol (RTCP), which provides out-of-band control messages for quality monitoring, synchronization, and session reporting. According to AES67 and RFC 3550, every RTC session must implement RTCP in some form to support sender and receiver reports. These reports convey metrics such as packet loss, jitter, round-trip time, and stream clock correlation, which are essential for maintaining and diagnosing audio stream performance.

RTCP Packet Specification

The Real-time Transport Control Protocol (RTCP) operates alongside RTP to provide essential control and quality monitoring for real-time audio streams. In AES67 environments, RTCP packets serve to report transmission statistics, synchronize media streams, and convey identification and session information. Every RTP session must also include RTCP to ensure compliance and provide monitoring of Quality of Service (QoS) metrics. RTCP packets are transmitted periodically by both senders and receivers and are multiplexed over the same network channel as RTP, typically using an even-odd port pair.

RTCP packets are structured differently from RTP packets. Unlike RTP, which transports media payloads, RTCP transmits metadata and reports in a modular format. Each RTCP packet consists of a fixed header followed by a variable-length payload depending on the packet type. Multiple RTCP packets may be concatenated within a single compound RTCP message, which is the standard transmission unit.

The general structure of an RTCP packet header is as follows:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|   RC    |      PT       |             length            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         packet-specific fields                |
|                              ...                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

RTCP Header Fields

• Version (V, 2 bits): Like RTP, this field must be set to 2, indicating compliance with the second version of the protocol. This field ensures compatibility and version negotiation between participants.
• Padding (P, 1 bit): If this bit is set, one or more padding bytes are appended to the end of the packet. The last byte of the packet indicates the number of padding bytes, similar to RTP padding handling.
• Reception Report Count (RC, 5 bits): Also referred to as the count field, this specifies either:
  • The number of reception report blocks included in the RTCP packet (for Receiver and Sender Reports),
  • Or, depending on the packet type, the number of sources described or identifiers included.
• Packet Type (PT, 8 bits): Indicates the type of RTCP packet. Common types include:
  • 200 – Sender Report (SR)
  • 201 – Receiver Report (RR)
  • 202 – Source Description (SDES)
  • 203 – Goodbye (BYE)
  • 204 – Application-defined (APP)
• Length (16 bits): The length of the RTCP packet in 32-bit words minus one, not including the header. This allows the receiver to locate the start of the next RTCP packet in a compound message.

Common RTCP Packet Types and Functions

Sender Report (SR - PT=200)

Sent by active RTP senders, this packet provides information on transmitted packets, timestamps, and includes at least one reception report block. It enables receivers to measure round-trip times and assess sender clock drift for synchronization.

Receiver Report (RR - PT=201)

Sent by participants who are receiving but not actively sending RTP data. RR packets contain statistics such as fraction lost, cumulative packets lost, highest sequence number received, jitter, and delay since the last SR. These are critical for evaluating stream quality.

Source Description (SDES - PT=202)

Conveys canonical names (CNAME), user names, email addresses, and other descriptive metadata about session participants. The CNAME is essential for linking multiple RTP streams from the same source (e.g., audio and video) for synchronization.

Goodbye (BYE - PT=203)

Indicates that one or more sources are leaving the session. This allows receivers to release state associated with those SSRCs.

Application-Defined (APP - PT=204)

A flexible message type used for proprietary or experimental extensions. Its interpretation is left to the specific application and is not typically used in AES67 contexts.

Compound RTCP Packets

RTCP is often sent as compound packets, which include:

• 1× SR or RR (Sender/Receiver Report)
• 1× SDES (must include CNAME)
• Optional: APP, BYE

Compound packets ensure that necessary context (like the source’s CNAME) always accompanies statistical reports.

Required Types and Functions

To support this specification, libAES67 should define:

Core RTP Structures

• aes67_rtp_packet_t: Struct with parsed fields — version, PT, sequence, timestamp, SSRC, payload pointer, length, and flags for padding/extension.
• aes67_rtp_payload_format_t: Optional enum or struct for known payload formats (L16, L24, Opus, etc.).
• aes67_rtp_header_extension_t: Struct for handling RTP header extensions (if you plan to support them).

Core RTCP Structures

• aes67_rtcp_packet_t: Struct with version, PT, length, report blocks, and optional SDES fields.
• aes67_rtcp_type_t: Defines RTCP packet types as per RFC 3550, mapped from the PT field.
• aes67_rtcp_report_block_t: For fields in SR/RR packets — packet loss, jitter, etc.
• aes67_rtcp_sdes_item_t: For parsing SDES items (e.g., CNAME).
• aes67_rtcp_compound_packet_t: For holding multiple RTCP packets together (common in real streams).

Parsing & Validation

• parse_rtp_packet(const uint8_t* data, size_t len, aes67_rtp_packet_t* out): Parses raw RTP data into structured form, checks validity.

• parse_rtcp_packet(const uint8_t* data, size_t len, aes67_rtcp_packet_t* out): Parses RTCP compound packets and extracts metadata.

• parse_rtcp_compound(const uint8_t* data, size_t len, aes67_rtcp_packet_t* out[], size_t max): Splits compound RTCP packets into individual structs.

• is_valid_rtp(const uint8_t* data, size_t len): Quick sanity check before parsing.

• is_valid_rtcp(const uint8_t* data, size_t len): Same for RTCP.

RTCP Feedback & Handlers

• handle_rtcp_feedback(...): Interface for monitoring tools or stream managers to act on RTCP stats.
• handle_rtcp_sr(...) / handle_rtcp_rr(...): Application hooks to respond to sender/receiver reports (e.g., sync or metrics).
• handle_rtcp_bye(...): Handle stream termination logic.
• handle_rtp_payload(...): Hook for decoding audio or passing to a jitter buffer.

Debugging Tools

• dump_rtp_packet(...): Pretty-prints packet for logging/debugging.
• dump_rtcp_packet(...): Same for RTCP.
• rtcp_type_name(aes67_rtcp_type_t type): Optional utility for debugging/logging.

Timestamp Utilities

• rtp_timestamp_to_usec(uint32_t ts, uint32_t sample_rate)
• usec_to_rtp_timestamp(uint64_t usec, uint32_t sample_rate)

Jitter Buffer API

• aes67_jitter_buffer_t: Main jitter buffer structure holding buffered RTP packets and state for managing playback timing.

• aes67_jitter_packet_t: Represents a single RTP packet stored in the jitter buffer, including arrival time and payload data.

• aes67_jitter_buffer_init(aes67_jitter_buffer_t* jb, uint32_t sample_rate, uint16_t frame_size, size_t max_packets): Initializes the jitter buffer with audio sample rate, expected RTP frame size, and max packet capacity.

• aes67_jitter_buffer_free(aes67_jitter_buffer_t* jb): Frees resources and cleans up the jitter buffer.

• aes67_jitter_buffer_push(aes67_jitter_buffer_t* jb, const aes67_rtp_packet_t* pkt): Adds a new RTP packet into the jitter buffer queue, returning success or failure.

• aes67_jitter_packet_t* aes67_jitter_buffer_peek_next(aes67_jitter_buffer_t* jb, uint64_t now_usec): Peeks at the next RTP packet ready for playback based on the current timestamp.

• aes67_jitter_buffer_pop(aes67_jitter_buffer_t* jb): Removes the next RTP packet from the jitter buffer after playback.

Affected Layer

network

Alternatives Considered

None

Impact

This proposal will not break backward compatibility and introduces no breaking changes.

References

• https://www.rfc-editor.org/rfc/rfc3550
• https://www.rfc-editor.org/rfc/rfc3551
• https://www.rfc-editor.org/rfc/rfc4855
• https://www.rfc-editor.org/rfc/rfc4856
• https://www.rfc-editor.org/rfc/rfc7656
• https://www.rfc-editor.org/rfc/rfc3190
• https://www.rfc-editor.org/rfc/rfc3640
• https://www.rfc-editor.org/rfc/rfc6416
• https://www.rfc-editor.org/rfc/rfc2250
• https://www.rfc-editor.org/rfc/rfc6295
• https://www.rfc-editor.org/rfc/rfc4696
• https://www.rfc-editor.org/rfc/rfc7164
• https://www.rfc-editor.org/rfc/rfc7587
• https://www.rfc-editor.org/rfc/rfc7587
• https://www.rfc-editor.org/rfc/rfc7798
• https://en.wikipedia.org/wiki/Real-time_Transport_Protocol

Additional Notes

This proposal is part of a larger effort to enhance AES67 interoperability; feedback and contributions are welcome.

Is there an existing proposal for this?

• I have searched the existing proposals