diff --git a/docs/framing.md b/docs/framing.md index 6bc89d55..9cb5a587 100644 --- a/docs/framing.md +++ b/docs/framing.md @@ -2,6 +2,49 @@ Framing wraps messages with headers and checksums so receivers can identify message boundaries and verify integrity. +## Frame Format Definitions + +All supported frame formats are defined in [`examples/frame_formats.proto`](../examples/frame_formats.proto). This file provides: + +- Protocol Buffer definitions for each frame format +- Enumeration of all frame format types +- Configuration message for runtime format selection + +## Recommended Frame Formats + +| Format | Start Bytes | Length | CRC | Total Overhead | Use Case | +|--------|-------------|--------|-----|----------------|----------| +| BasicFrame | 2 (0x90, 0x91) | 0 | 2 | 5 | When all messages are known to both systems | +| TinyFrame | 1 (0x70) | 0 | 2 | 4 | Constrained environments with known messages | +| BasicFrameWithLen | 2 (0x90, 0x92) | 1 | 2 | 6 | When systems may not have matching message definitions | +| TinyFrameWithLen | 1 (0x71) | 1 | 2 | 5 | Constrained environments with variable messages | +| BasicFrameWithLen16 | 2 (0x90, 0x93) | 2 | 2 | 7 | Large payloads up to 64KB | +| BasicFrameWithSysComp | 2 (0x90, 0x94) | 0 | 2 | 7 | Multi-system networks with system and component IDs | + +## Extended Frame Formats + +| Format | Start Bytes | Length | CRC | Total Overhead | Use Case | +|--------|-------------|--------|-----|----------------|----------| +| NoFormat | 0 | 0 | 0 | 0 | Trusted links, nested protocols | +| MinimalFrame | 0 | 0 | 2 | 3 | Minimal framing with CRC | +| MinimalFrameNoCrc | 0 | 0 | 0 | 1 | Minimal framing, trusted link | +| MinimalFrameWithLen | 0 | 1 | 2 | 4 | Variable length, minimal | + +## Parametric Frame Formats + +| Format | Start Bytes | Length | CRC | Total Overhead | Use Case | +|--------|-------------|--------|-----|----------------|----------| +| BasicFrameNoCrc | 2 (0x90, 0x95) | 0 | 0 | 3 | Sync recovery, no CRC | +| BasicFrameWithLenNoCrc | 2 (0x90, 0x96) | 1 | 0 | 4 | Variable length, no CRC | +| BasicFrameWithLen16NoCrc | 2 (0x90, 0x97) | 2 | 0 | 5 | Large payloads, no CRC | +| TinyFrameNoCrc | 1 (0x72) | 0 | 0 | 2 | Minimal overhead | +| TinyFrameWithLenNoCrc | 1 (0x73) | 1 | 0 | 3 | Variable, minimal overhead | +| TinyFrameWithLen16 | 1 (0x74) | 2 | 2 | 6 | Large payloads, constrained | +| TinyFrameWithLen16NoCrc | 1 (0x75) | 2 | 0 | 4 | Large, minimal overhead | +| MinimalFrameWithLenNoCrc | 0 | 1 | 0 | 2 | Variable length, no CRC | +| MinimalFrameWithLen16 | 0 | 2 | 2 | 5 | Large payloads, minimal | +| MinimalFrameWithLen16NoCrc | 0 | 2 | 0 | 3 | Large payloads, no CRC | + ## No Frame Format For trusted point-to-point links where you control both ends, you can skip framing entirely. Messages are sent as raw bytes with no header or checksum. @@ -21,16 +64,16 @@ Limitations: The default frame format used by Struct Frame: ``` -[Start Byte] [Message ID] [Payload Data...] [Checksum 1] [Checksum 2] - 0x90 1 byte Variable Length 1 byte 1 byte +[Start Byte 1] [Start Byte 2] [Message ID] [Payload Data...] [Checksum 1] [Checksum 2] + 0x90 0x91 1 byte Variable Length 1 byte 1 byte ``` ### Components -**Start Byte (0x90)** -- Fixed marker to identify frame boundaries -- Parser scans for this byte to synchronize -- Allows recovery after corruption +**Start Bytes (0x90, 0x91)** +- Two-byte marker to identify frame boundaries +- Parser scans for this sequence to synchronize +- Second byte varies by frame type (0x91=Basic, 0x92=WithLen, etc.) **Message ID (1 byte)** - Maps to specific message type defined in proto @@ -52,15 +95,66 @@ The default frame format used by Struct Frame: Message: VehicleHeartbeat (ID=42) with payload [0x01, 0x02, 0x03, 0x04] ``` -Frame: [0x90] [0x2A] [0x01, 0x02, 0x03, 0x04] [0x7F] [0x8A] - Start ID Payload Checksum +Frame: [0x90] [0x91] [0x2A] [0x01, 0x02, 0x03, 0x04] [0x7F] [0x8A] + Start1 Start2 ID Payload Checksum ``` ### Frame Size -Total bytes = 2 (header) + payload size + 2 (checksum) +Total bytes = 3 (header) + payload size + 2 (checksum) + +For a message with 5 bytes of data: 3 + 5 + 2 = 10 bytes total + +## Minimal Frame Variants + +Minimal framing with just message ID and optional CRC (0 start bytes): + +**MinimalFrame**: `[MSG_ID (1)] [PAYLOAD] [CRC (2)]` +- 3 bytes overhead +- For synchronized links with error detection + +**MinimalFrameNoCrc**: `[MSG_ID (1)] [PAYLOAD]` +- 1 byte overhead +- For trusted, synchronized links + +## Tiny Frame Variants + +Low overhead framing for constrained environments (1 start byte): + +**TinyFrame**: `[START (0x70)] [MSG_ID (1)] [PAYLOAD] [CRC (2)]` +- 4 bytes overhead +- Battery-powered devices, high message rates -For a message with 5 bytes of data: 2 + 5 + 2 = 9 bytes total +**TinyFrameNoCrc**: `[START (0x72)] [MSG_ID (1)] [PAYLOAD]` +- 2 bytes overhead +- Minimal overhead with sync recovery + +## Length-Prefixed Variants + +All frame formats have variants with explicit length fields: + +**1-byte length (WithLen)**: Supports payloads up to 255 bytes +- Adds 1 byte to header +- Use for variable-length messages + +**2-byte length (WithLen16)**: Supports payloads up to 65,535 bytes +- Adds 2 bytes to header (little-endian) +- Use for large data transfers, firmware updates + +Example: `BasicFrameWithLen16` +``` +[START1 (0x90)] [START2 (0x93)] [MSG_ID (1)] [LEN_LO (1)] [LEN_HI (1)] [PAYLOAD] [CRC (2)] +``` + +## System/Component ID Variant + +**BasicFrameWithSysComp**: For multi-system networks +``` +[START1 (0x90)] [START2 (0x94)] [SYS_ID (1)] [COMP_ID (1)] [MSG_ID (1)] [PAYLOAD] [CRC (2)] +``` +- 7 bytes overhead +- System ID identifies the vehicle/ground station +- Component ID identifies the component (autopilot, camera, etc.) ## Parser State Machine @@ -109,29 +203,54 @@ for each byte in (message_id + payload): checksum = [sum1, sum2] ``` -## Custom Frame Formats +## Third Party Protocols + +| Format | Start Bytes | Length | CRC | Total Overhead | Use Case | +|--------|-------------|--------|-----|----------------|----------| +| UBX | 2 (0xB5, 0x62) | 2 | 2 | 8 | u-blox GPS compatibility | +| MavlinkV1 | 1 (0xFE) | 1 | 2 | 8 | Legacy drone communication | +| MavlinkV2 | 1 (0xFD) | 1 | 2-15 | 12-25 | Modern drone communication | -The architecture supports alternative frame formats. Planned implementations: +### UBX Format -**UBX Format** -- Used by u-blox GPS receivers -- 2-byte sync sequence (0xB5, 0x62) -- Class + ID message identification -- Little-endian 2-byte length field +u-blox proprietary binary protocol for GPS/GNSS receivers: + +``` +[SYNC1 (0xB5)] [SYNC2 (0x62)] [CLASS (1)] [ID (1)] [LEN (2)] [PAYLOAD] [CK_A] [CK_B] +``` + +- 8 bytes overhead +- Class + ID for message routing - Fletcher-8 checksum +- Use for u-blox GPS integration + +### MAVLink v1 + +Legacy drone communication protocol: -**Mavlink v1** -- Used by ArduPilot and PX4 -- 0xFE start byte -- Sequence counter -- System ID and Component ID -- CRC-16 checksum +``` +[STX (0xFE)] [LEN (1)] [SEQ (1)] [SYS (1)] [COMP (1)] [MSG (1)] [PAYLOAD] [CRC (2)] +``` + +- 8 bytes overhead +- Sequence counter for packet loss detection +- System ID and Component ID for multi-vehicle networks +- CRC-16/X.25 checksum +- Use for ArduPilot/PX4 compatibility + +### MAVLink v2 + +Modern drone communication with extended features: + +``` +[STX (0xFD)] [LEN (1)] [INCOMPAT (1)] [COMPAT (1)] [SEQ (1)] [SYS (1)] [COMP (1)] [MSG_ID (3)] [PAYLOAD] [CRC (2)] [SIGNATURE (13, optional)] +``` -**Mavlink v2** -- 0xFD start byte -- Incompatibility flags -- Compatibility flags -- Optional signature for authentication +- 12-25 bytes overhead +- 24-bit message ID (16.7M message types) +- Incompatibility/compatibility flags for version negotiation +- Optional 13-byte signature for authentication +- Use for secure drone communication ## Framing Compatibility @@ -139,8 +258,15 @@ The architecture supports alternative frame formats. Planned implementations: |--------------|---|-----|------------|--------| | No Header (No Framing) | Yes | Yes | Yes | Yes | | Basic Frame Format | Yes | Yes | Yes | Yes | -| UBX | Planned | Planned | Planned | Planned | -| Mavlink v1 | Planned | Planned | Planned | Planned | -| Mavlink v2 | Planned | Planned | Planned | Planned | +| MSG ID Frames | Defined | Defined | Defined | Defined | +| Tiny Frames | Defined | Defined | Defined | Defined | +| Length-Prefixed | Defined | Defined | Defined | Defined | +| UBX | Defined | Defined | Defined | Defined | +| Mavlink v1 | Defined | Defined | Defined | Defined | +| Mavlink v2 | Defined | Defined | Defined | Defined | + +**Legend:** +- **Yes**: Fully implemented runtime support +- **Defined**: Frame format defined in proto, implementation pending All frame formats are binary compatible across languages. A frame created in Python can be parsed in C and vice versa. diff --git a/examples/frame_formats.proto b/examples/frame_formats.proto new file mode 100644 index 00000000..89ffe90b --- /dev/null +++ b/examples/frame_formats.proto @@ -0,0 +1,326 @@ +// Frame Format Definitions for struct-frame +// Defines message framing formats for communication over serial, network, or byte streams. + +package frame_formats; + +// Frame format type enumeration +enum FrameFormatType { + NO_FORMAT = 0; + MINIMAL_FRAME = 1; + MINIMAL_FRAME_NO_CRC = 2; + BASIC_FRAME = 3; + BASIC_FRAME_NO_CRC = 4; + TINY_FRAME = 5; + TINY_FRAME_NO_CRC = 6; + MINIMAL_FRAME_WITH_LEN = 7; + MINIMAL_FRAME_WITH_LEN_NO_CRC = 8; + MINIMAL_FRAME_WITH_LEN16 = 9; + MINIMAL_FRAME_WITH_LEN16_NO_CRC = 10; + BASIC_FRAME_WITH_LEN = 11; + BASIC_FRAME_WITH_LEN_NO_CRC = 12; + BASIC_FRAME_WITH_LEN16 = 13; + BASIC_FRAME_WITH_LEN16_NO_CRC = 14; + TINY_FRAME_WITH_LEN = 15; + TINY_FRAME_WITH_LEN_NO_CRC = 16; + TINY_FRAME_WITH_LEN16 = 17; + TINY_FRAME_WITH_LEN16_NO_CRC = 18; + BASIC_FRAME_WITH_SYS_COMP = 19; + UBX = 20; + MAVLINK_V1 = 21; + MAVLINK_V2 = 22; +} + +// Basic message payload included in all frame formats +message BasicMessage { + uint8 msg_id = 1; +} + +// ============================================================================ +// NO FORMAT - Raw message, no framing +// Format: [MSG] +// Overhead: 0 bytes +message NoFormat { + BasicMessage payload = 1; +} + +// ============================================================================ +// MINIMAL FRAME - No start bytes, just MSG_ID with CRC +// Format: [MSG_ID] [MSG] [CRC1] [CRC2] +// Overhead: 3 bytes +message MinimalFrame { + BasicMessage payload = 1; + uint8 crc_byte1 = 2; + uint8 crc_byte2 = 3; +} + +// MINIMAL FRAME NO CRC +// Format: [MSG_ID] [MSG] +// Overhead: 1 byte +message MinimalFrameNoCrc { + BasicMessage payload = 1; +} + +// ============================================================================ +// BASIC FRAME - 2 start bytes with CRC (Recommended) +// Format: [START1=0x90] [START2=0x91] [MSG_ID] [MSG] [CRC1] [CRC2] +// Overhead: 5 bytes +message BasicFrame { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x91]; + BasicMessage payload = 3; + uint8 crc_byte1 = 4; + uint8 crc_byte2 = 5; +} + +// BASIC FRAME NO CRC +// Format: [START1=0x90] [START2=0x95] [MSG_ID] [MSG] +// Overhead: 3 bytes +message BasicFrameNoCrc { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x95]; + BasicMessage payload = 3; +} + +// ============================================================================ +// TINY FRAME - 1 start byte with CRC (Recommended) +// Format: [START=0x70] [MSG_ID] [MSG] [CRC1] [CRC2] +// Overhead: 4 bytes +message TinyFrame { + uint8 start_byte = 1 [(hex) = 0x70]; + BasicMessage payload = 2; + uint8 crc_byte1 = 3; + uint8 crc_byte2 = 4; +} + +// TINY FRAME NO CRC +// Format: [START=0x72] [MSG_ID] [MSG] +// Overhead: 2 bytes +message TinyFrameNoCrc { + uint8 start_byte = 1 [(hex) = 0x72]; + BasicMessage payload = 2; +} + +// ============================================================================ +// LENGTH-PREFIXED VARIANTS (1-byte length, up to 255 bytes) +// ============================================================================ + +// MINIMAL FRAME WITH LEN +// Format: [MSG_ID] [LEN] [MSG] [CRC1] [CRC2] +// Overhead: 4 bytes +message MinimalFrameWithLen { + BasicMessage payload = 1; + uint8 length = 2; + uint8 crc_byte1 = 3; + uint8 crc_byte2 = 4; +} + +// MINIMAL FRAME WITH LEN NO CRC +// Format: [MSG_ID] [LEN] [MSG] +// Overhead: 2 bytes +message MinimalFrameWithLenNoCrc { + BasicMessage payload = 1; + uint8 length = 2; +} + +// BASIC FRAME WITH LEN (Recommended for variable length) +// Format: [START1=0x90] [START2=0x92] [MSG_ID] [LEN] [MSG] [CRC1] [CRC2] +// Overhead: 6 bytes +message BasicFrameWithLen { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x92]; + BasicMessage payload = 3; + uint8 length = 4; + uint8 crc_byte1 = 5; + uint8 crc_byte2 = 6; +} + +// BASIC FRAME WITH LEN NO CRC +// Format: [START1=0x90] [START2=0x96] [MSG_ID] [LEN] [MSG] +// Overhead: 4 bytes +message BasicFrameWithLenNoCrc { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x96]; + BasicMessage payload = 3; + uint8 length = 4; +} + +// TINY FRAME WITH LEN (Recommended) +// Format: [START=0x71] [MSG_ID] [LEN] [MSG] [CRC1] [CRC2] +// Overhead: 5 bytes +message TinyFrameWithLen { + uint8 start_byte = 1 [(hex) = 0x71]; + BasicMessage payload = 2; + uint8 length = 3; + uint8 crc_byte1 = 4; + uint8 crc_byte2 = 5; +} + +// TINY FRAME WITH LEN NO CRC +// Format: [START=0x73] [MSG_ID] [LEN] [MSG] +// Overhead: 3 bytes +message TinyFrameWithLenNoCrc { + uint8 start_byte = 1 [(hex) = 0x73]; + BasicMessage payload = 2; + uint8 length = 3; +} + +// ============================================================================ +// LENGTH-PREFIXED VARIANTS (2-byte length, up to 65535 bytes) +// ============================================================================ + +// MINIMAL FRAME WITH LEN16 +// Format: [MSG_ID] [LEN_LO] [LEN_HI] [MSG] [CRC1] [CRC2] +// Overhead: 5 bytes +message MinimalFrameWithLen16 { + BasicMessage payload = 1; + uint16 length = 2; + uint8 crc_byte1 = 3; + uint8 crc_byte2 = 4; +} + +// MINIMAL FRAME WITH LEN16 NO CRC +// Format: [MSG_ID] [LEN_LO] [LEN_HI] [MSG] +// Overhead: 3 bytes +message MinimalFrameWithLen16NoCrc { + BasicMessage payload = 1; + uint16 length = 2; +} + +// BASIC FRAME WITH LEN16 (Recommended for large payloads) +// Format: [START1=0x90] [START2=0x93] [MSG_ID] [LEN_LO] [LEN_HI] [MSG] [CRC1] [CRC2] +// Overhead: 7 bytes +message BasicFrameWithLen16 { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x93]; + BasicMessage payload = 3; + uint16 length = 4; + uint8 crc_byte1 = 5; + uint8 crc_byte2 = 6; +} + +// BASIC FRAME WITH LEN16 NO CRC +// Format: [START1=0x90] [START2=0x97] [MSG_ID] [LEN_LO] [LEN_HI] [MSG] +// Overhead: 5 bytes +message BasicFrameWithLen16NoCrc { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x97]; + BasicMessage payload = 3; + uint16 length = 4; +} + +// TINY FRAME WITH LEN16 +// Format: [START=0x74] [MSG_ID] [LEN_LO] [LEN_HI] [MSG] [CRC1] [CRC2] +// Overhead: 6 bytes +message TinyFrameWithLen16 { + uint8 start_byte = 1 [(hex) = 0x74]; + BasicMessage payload = 2; + uint16 length = 3; + uint8 crc_byte1 = 4; + uint8 crc_byte2 = 5; +} + +// TINY FRAME WITH LEN16 NO CRC +// Format: [START=0x75] [MSG_ID] [LEN_LO] [LEN_HI] [MSG] +// Overhead: 4 bytes +message TinyFrameWithLen16NoCrc { + uint8 start_byte = 1 [(hex) = 0x75]; + BasicMessage payload = 2; + uint16 length = 3; +} + +// ============================================================================ +// SYSTEM/COMPONENT ID VARIANT (Recommended for multi-system networks) +// ============================================================================ + +// BASIC FRAME WITH SYS COMP +// Format: [START1=0x90] [START2=0x94] [SYS_ID] [COMP_ID] [MSG_ID] [MSG] [CRC1] [CRC2] +// Overhead: 7 bytes +message BasicFrameWithSysComp { + uint8 start_byte1 = 1 [(hex) = 0x90]; + uint8 start_byte2 = 2 [(hex) = 0x94]; + uint8 system_id = 3; + uint8 component_id = 4; + BasicMessage payload = 5; + uint8 crc_byte1 = 6; + uint8 crc_byte2 = 7; +} + +// ============================================================================ +// THIRD PARTY PROTOCOLS +// ============================================================================ + +// UBX FRAME - u-blox GPS/GNSS protocol +// Format: [SYNC1=0xB5] [SYNC2=0x62] [CLASS] [ID] [LEN_LO] [LEN_HI] [MSG] [CK_A] [CK_B] +// Overhead: 8 bytes +message UbxFrame { + uint8 sync1 = 1 [(hex) = 0xB5]; + uint8 sync2 = 2 [(hex) = 0x62]; + uint8 msg_class = 3; + uint8 msg_id = 4; + uint8 length_lo = 5; + uint8 length_hi = 6; + uint8 ck_a = 7; + uint8 ck_b = 8; +} + +// MAVLINK V1 FRAME - Legacy drone protocol +// Format: [STX=0xFE] [LEN] [SEQ] [SYS] [COMP] [MSG] [PAYLOAD] [CRC_LO] [CRC_HI] +// Overhead: 8 bytes +message MavlinkV1Frame { + uint8 stx = 1 [(hex) = 0xFE]; + uint8 length = 2; + uint8 sequence = 3; + uint8 system_id = 4; + uint8 component_id = 5; + uint8 msg_id = 6; + uint8 crc_lo = 7; + uint8 crc_hi = 8; +} + +// MAVLINK V2 FRAME - Modern drone protocol with extended message IDs +// Format: [STX=0xFD] [LEN] [INCOMPAT] [COMPAT] [SEQ] [SYS] [COMP] [MSG_ID x3] [PAYLOAD] [CRC] [SIG?] +// Overhead: 12-25 bytes +message MavlinkV2Frame { + uint8 stx = 1 [(hex) = 0xFD]; + uint8 length = 2; + uint8 incompat_flags = 3; + uint8 compat_flags = 4; + uint8 sequence = 5; + uint8 system_id = 6; + uint8 component_id = 7; + uint8 msg_id_0 = 8; + uint8 msg_id_1 = 9; + uint8 msg_id_2 = 10; + uint8 crc_lo = 11; + uint8 crc_hi = 12; +} + +// MAVLINK V2 SIGNATURE - Optional authentication (13 bytes) +message MavlinkV2Signature { + uint8 link_id = 1; + uint8 timestamp_0 = 2; + uint8 timestamp_1 = 3; + uint8 timestamp_2 = 4; + uint8 timestamp_3 = 5; + uint8 timestamp_4 = 6; + uint8 timestamp_5 = 7; + uint8 signature_0 = 8; + uint8 signature_1 = 9; + uint8 signature_2 = 10; + uint8 signature_3 = 11; + uint8 signature_4 = 12; + uint8 signature_5 = 13; +} + +// ============================================================================ +// CONFIGURATION +// ============================================================================ + +// Runtime frame format configuration +message FrameFormatConfig { + FrameFormatType format_type = 1; + uint8 start_byte1 = 2 [(hex) = 0x90]; + uint8 start_byte2 = 3 [(hex) = 0x91]; + bool crc_enabled = 4; + bool signing_enabled = 5; +}