sparkplug-host

A Sparkplug B Host Application in Python that gets the awkward parts right — stale death certificates, sequence gaps that are really reordering, aliases that were never defined, and rebirth requests that would otherwise be a denial-of-service amplifier.

git clone https://github.com/boheastill/sparkplug-host
cd sparkplug-host
python3 check_it.py        # no dependencies, no broker, ~1 second

SPARKPLUG B HOST — CONFORMANCE SCENARIOS
14 situations a real deployment produces and a first draft
gets wrong. Several times, the correct behaviour is to do nothing.

  [ 1/14] Stale NDEATH from a previous session             PASS
         node stayed online: the late NDEATH carried bdSeq 0
         while the live session is bdSeq 1
  [ 3/14] Death timestamped by the host, not the payload   PASS
  [ 4/14] Sequence gap: wait, then request rebirth         PASS
  [ 5/14] Gap that turns out to be reordering              PASS
         timer cancelled when the missing message arrived
  [ 7/14] One sequence counter per node, not per device    PASS
  [ 8/14] Alias that was never defined in a BIRTH          PASS
  [10/14] Rebirth requests are rate-limited                PASS
         one request, ten suppressed, allowed again after the cooldown
  [12/14] Signed integers in an unsigned field             PASS
         round-tripped [-23, -1234, -70000, -5000000000, 4000000000]
  [13/14] BooleanArray bit order                           PASS
  [14/14] Malformed topics are refused                     PASS

All 14 scenarios hold.

Why this exists

Eclipse Tahu is the reference implementation, and its own release notes are precise about scope: "Initial complete Java based Sparkplug v3.0.0 compatible implementation … Partial example implementations exist for C, C#, JavaScript, Node RED, and Python." Tahu's python/ directory has not been touched since February 2023.

On the Python side, pysparkplug is the cleanest modern library and handles edge-node bdSeq and seq correctly — but it implements the edge node, not the host: there is no Host Application, no primary-host STATE handling, and no Node Control/Rebirth responder. mqtt-spb-wrapper builds on Tahu's Python core and inherits its array quirks.

So: there is no maintained, open-source Python Host Application that does bdSeq correlation, STATE, sequence-gap handling and rebirth policy correctly. That's the gap this fills. It is small on purpose.

The four things it exists to get right

1. A death certificate is matched by bdSeq, never by its timestamp. NDEATH is delivered by the broker from a Will registered at CONNECT, so its payload timestamp describes when the node connected, not when it died. Worse, Will delivery races reconnection — a stale NDEATH can arrive after the new NBIRTH. Match on timestamp and you will mark healthy machines dead. Scenario 1 replays exactly that sequence.

2. seq is per Edge Node, across every topic it owns. A node's DBIRTH and DDATA for its devices draw from the same mod-256 counter as its own NBIRTH and NDATA. Implementations that keep a counter per device see phantom gaps forever.

3. A gap is usually reordering, not loss. Sparkplug data is QoS 0 by mandate, and QoS 0 across different topics can be reordered by a clustered broker. So a gap starts a timer; only if the hole is still there when it expires do we ask for a rebirth. Scenario 5 is the case that matters: a straggler arrives, the timer is cancelled, and nothing happens.

4. Rebirth is a denial-of-service amplifier. One NCMD makes a node retransmit its entire metric set. Anyone permitted to publish NCMD can turn a single message into a storm, so requests are rate-limited per node.

What it does not do

• No transport. The host is a pure state machine: you feed it topics and bytes, it hands back commands to publish. That is deliberate — everything hard about Sparkplug is state, and state you can hand-crank is state you can test without a broker, a network or a sleep(). Wiring it to paho-mqtt is a few lines and left to you.
• No edge node. Use pysparkplug for that; it is good.
• Partial payload decoding. DataSet, Template and property sets are parsed as far as their tags and then skipped. A host that pretended to understand Templates would be lying about its scope.
• Not TCK-certified. The Eclipse TCK is the only thing that can make a conformance claim, and I have not run it. These fourteen scenarios are my own reading of the specification, not a certificate.

Where the specification contradicts itself

Read against Sparkplug 3.0.0 (the normative AsciiDoc at tag v3.0.0, not the rendered PDF). Four places where I had to make a call — flagged here rather than resolved silently, because the next person deserves to know:

• The starting seq in an NBIRTH. Chapter 4 says it "MUST have a value of 0". Chapters 5 and 6 both say "between 0 and 255 (inclusive)". Two of three readings allow any value. This host emits 0 and accepts anything 0–255, which satisfies all three.
• The array examples are wrong. The prose says little-endian; the worked examples for FloatArray and DoubleArray are printed big-endian, the Int8Array example encodes −23 as 0xEF when it is 0xE9, and the DateTimeArray example shows 6 bytes for an 8-byte type. The prose is right.
• BooleanArray bit order is a live interoperability bug. Eclipse Tahu's Java encoder packs MSB-first, agreeing with the specification's own worked example. Tahu's Python helper packs LSB-first. They do not interoperate. This implementation follows Java and the spec example — scenario 13 pins it.
• sparkplug_host_id has no stated character restriction. Sparkplug 2.2 said +, / and # were forbidden in scada_host_id; 3.0 dropped the sentence, apparently by accident. It is still a topic token, so it is still validated here.

One more worth knowing: STATE is a breaking change between 2.2 and 3.0. In 2.2 the topic was STATE/{scada_host_id} with the literal payload "ONLINE" or "OFFLINE". In 3.0 it is spBv1.0/STATE/{host_id} with a JSON body. A 2.2 host and a 3.0 edge node cannot agree on primary-host detection. Announcements around 3.0 emphasised backward compatibility; for STATE that is not true.

What Sparkplug does not give you

Worth stating plainly, because it is routinely oversold:

• It is not a security model. The security chapter is explicitly non-normative. There is no authentication, no authorization, no payload signing, and no binding between an MQTT client ID and an edge node identity. Nothing in the protocol prevents an authorized publisher from forging another node's NDEATH. Broker ACLs are the whole defence.
• It is not store-and-forward. The phrase appears once in the spec, as a use case for the is_historical flag. There is no buffering, no backfill, no replay ordering, no dedupe. Vendor products implement it; the standard does not.
• Data is QoS 0 by mandate, with Clean Session = true, so the broker queues nothing for a disconnected client. Loss on a TCP disruption is expected behaviour. Sparkplug's answer to loss is not redelivery — it is detection (sequence gaps) plus resynchronisation (rebirth). That is a genuinely different reliability model, and worth understanding before you rely on it.
• It does not solve semantics. It standardises transport, lifecycle and typing. It says nothing about what Temperature means or what unit it is in. Two conformant systems can parse each other's bytes and agree on liveness while understanding nothing about each other.

On "Unified Namespace"

A UNS is an architectural pattern — one event-driven data layer, organised to mirror the business — popularised by Walker Reynolds. There is no UNS specification, no conformance test and no standards body, so "we built a UNS" is not a falsifiable claim.

Sparkplug and UNS are orthogonal and frequently conflated. Plain MQTT with JSON and retained messages is a perfectly ordinary UNS. Sparkplug adds discovery, typing and liveness — but it also forbids retained messages on data, which removes the behaviour a UNS usually assumes ("subscribe and immediately see the current value of everything"). And its topic structure puts the message type in the middle, so spBv1.0/Plant1/+/# spans verbs rather than hierarchy levels; the community workarounds encode the plant hierarchy inside group_id with a delimiter, at the cost of topic-level filtering.

If someone tells you Sparkplug gives you a UNS, they have not thought about retained messages.

Layout

File	What it is
sparkplug_host/host.py	The session state machine: bdSeq correlation, seq tracking, alias table, rebirth policy, STATE
sparkplug_host/payload.py	Dependency-free proto2 codec for the Sparkplug Payload message, plus array unpacking
check_it.py	The fourteen scenarios. Also the CI gate

Related

• You can't see a refactor. So try to break the machine. — the same acceptance idea, on a physical machine
• mqtt-machine-safety-gates — commanding a machine over MQTT, with consequence-class gates
• More runnable demos: boheastill.com/demos

License

MIT — see LICENSE.