Breaking Down MQTT Sparkplug Payload Structures in IIoT Messaging

In Part 5 of Sparkplug Essentials, Exploring MQTT Sparkplug Session State Management in IIoT Communication, we discussed the core concepts of MQTT Sparkplug Session State Management and how it helps in IIoT. In Part 6, we will cover the challenges presented by inconsistent & non-standardized payload formats in IIoT & how to overcome them using MQTT Sparkplug Payload Structures.

The Industrial Internet of Things (IIoT) encompasses a wide variety of devices, from simple temperature sensors to complex industrial machines. These devices often communicate with different systems, platforms, and other devices. To ensure efficient and meaningful communication between these entities, it’s essential to have a structured and standardized data format.

On the other hand, MQTT doesn’t dictate a specific structure for its payload. This means the payload can be anything - from plain text and binary data to formats like JSON or XML, depending on the need. Essentially, the payload is merely treated as a binary array of bytes. While this versatility allows for diverse data types, it may not be optimal for IIoT implementations that require consistent and standardized MQTT payload formats.

Let’s take a closer look at the challenges presented by inconsistent and non-standardized payload formats.

Challenges of Inconsistent or Non-standardized MQTT Payloads

Incompatibility Issues: If payloads are not standardized, one device or application may send data in a format that another device or application cannot interpret correctly. This leads to integration nightmares, especially in large-scale IIoT deployments involving devices from multiple manufacturers.

Increased Complexity: Without standardized payloads, developers need to create custom parsers for each different payload format. This not only complicates the codebase but also increases the potential for errors.

Reduced Efficiency: Parsing non-standardized or inconsistent payloads often requires more computational resources. In the context of IIoT devices, which are typically constrained in terms of processing power and memory, this can be particularly problematic.

Maintenance Challenges: Inconsistent payload structures make system maintenance and updates harder. If a device changes its payload format, all subscribing entities must be updated to understand this new format.

Ambiguity and Data Corruption: Without a standardized structure, the chances of misinterpreting data or even losing it due to corruption increase. This can have dire consequences, especially in critical applications like healthcare or industrial automation.

How Sparkplug Enables Standardized and Structured MQTT Payloads

The MQTT Sparkplug specification aims to standardize MQTT messages, ensuring a consistent data structure for developing interoperable IIoT solutions with MQTT as the base messaging technology. It achieves this by offering a payload encoding mechanism that not only preserves the foundational features of MQTT – lightweight, bandwidth efficient, and low latency – but also integrates modern encoding schemes tailored for the IIoT landscape.

In that regard, Sparkplug has seen the evolution of two distinct encoding schemes; Sparkplug A (spAv1.0), which is now deprecated, and Sparkplug B (spBv1.0), the current version.

spBv1.0 or Sparkplug B payload format is a data encoding scheme that uses Google Protocol Buffers or Google Protobufs. Google Protobufs is a programming language neutral mechanism for serializing structured data, which allows Sparkplug B to provide a way for encoding structured MQTT data efficiently yet extensible.

Sparkplug B supports a rich data model through the following:

Complex Data Types Using Templates
Datasets
Enhanced Metrics
Metric Alias Support
Historical Data Integration
File Data Management

As stated in Part 4 of this guide, Sparkplug uses the MQTT Topic Namespace element, highlighted below, to specify the encoding scheme of a given payload.

spBv1.0/[Group ID]/[Message Type]/[EON Node ID]/[Device ID]

Key Components of the MQTT Sparkplug Payload

Primarily, a Sparkplug payload contains some basic information, such as a timestamp and a sequence number, as well as an array of metrics which contain key/value pairs of data. Let us take a look at these and more components in detail.

Timestamp

Sparkplug mandates the inclusion of a timestamp with every metric. This ensures that every data point is contextualized with its recording time. The timestamp represents the time at which the message was published.

Metrics

The metrics components of a Sparkplug payload consist of an array of metrics. Metrics in the Sparkplug payload context refer to the specific data points or values being communicated between devices, e.g., a temperature reading, pressure value, or switch status. To describe the information it contains, a Sparkplug metric represents a key, value, timestamp, datatype, and any metadata that may be associated with it. Let’s look at the components of a metric.

name: Identifier for the metric
alias: A numerical alias for the metric, used to reduce the payload size in repetitive messages
timestamp: Indicates when the metric was sampled or created
datatype: The kind of data the metric holds (e.g., Boolean, Int32, String)
value: The actual data
is_historical: A Boolean flag which denotes whether this metric represents a historical value
is_transient: A Boolean flag which denotes if this metric should not be historized
is_null: A Boolean flag which denotes whether this metric has a null value
metadata: Metadata object associated with the metric
properties: Propertyset object associated with the metrics

Sequence

Each Sparkplug message contains a sequence number, incrementing with every new message. If a receiver detects a gap in sequence numbers, it knows a message was lost. This feature is essential for data consistency, allowing devices or systems to request missing data or take corrective action.

Below is an example of a simple Sparkplug B payload.

{
        "timestamp": 1486144502122,
        "metrics": [{
            "name": "My Metric",
            "alias": 1,
            "timestamp": 1479123452194,
            "dataType": "String",
            "value": "Test"
        }],
        "seq": 2
}

MQTT Sparkplug Payload Representation for Message Types

For illustrative purposes, let’s look at the payload representations of a few MQTT Sparkplug messages types.

NBIRTH Payload Representation

The NBIRTH is responsible for informing host applications of all of the information about the edge node. This includes every metric it will publish data about in the future.

Consider the following representation of a simple NBIRTH message on the topic:

spBv1.0/DairyPlant/NBIRTH/Refrigeration

A Sparkplug B payload published in the NBIRTH message may look as follows:

{
    "timestamp": 1486144502122,
    "metrics": [{
        "name": "bdSeq",
        "timestamp": 1486144502122,
        "dataType": "Int64",
        "value": 0
}, {
        "name": "Node Control/Scan Rate",
        "timestamp": 1486144502122,
        "dataType": "Int64",
        "value": 3000
}, {
        "name": "Properties/Hardware Make",
        "timestamp": 1486144502122,
        "dataType": "String",
        "value": "Opto22 Groov EPIC"
}, {
        "name": "Inputs/Temperature",
        "timestamp": 1486144502122,
        "dataType": "Float",
        "value": 25.6
}, {
        "name": "Inputs/Humidity",
        "timestamp": 1486144502122,
        "dataType": "Float",
        "value": 67.8
}, {
        "name": "Outputs/Pump",
        "timestamp": 1486144502122,
        "dataType": "Boolean",
        "value": true
}],
"seq": 0
}

As can be seen above, the NBIRTH payload includes metrics such as Node Control/Reboot, which allows the edge node to be remotely configured to adjust scan rate by updating its value of Int64 type. It also includes a metric that indicates the hardware make of the edge node and metrics that represent the temperature, humidity and pump status.

DDATA Payload Representation

NDATA messages are used to update the values of any edge node metrics that were originally published in the NBIRTH message. Whenever there’s a change in the edge node’s input, an NDATA message will be generated and published to the MQTT server. If multiple metrics on the edge node change, they can all be included in a single NDATA message.

Consider the following representation of a simple NDATA message on the topic:

spBv1.0/DairyPlant/NDATA/Refrigeration

A Sparkplug B payload published in the NDATA message may look as follows:

{
        "timestamp": 1486144502122,
        "metrics": [{
            "name": "Inputs/Temperature",
            "timestamp": 1486144502122,
            "dataType": "Float",
            "value": 29.2
}, {
            "name": "Inputs/Humidity",
            "timestamp": 1486144502122,
            "dataType": "Float",
            "value": 55.9
}],
    "seq": 0
}

The payload includes the updated values of temperature and humidity metrics of the edge node. Notice the value of the pump status may be excluded from the metrics if it has not changed.

Conclusion

In conclusion, while MQTT offers flexibility in its payload structure, this very flexibility can result in integration challenges and increased complexities, especially when using devices and applications from diverse vendors. The MQTT Sparkplug specification addresses this challenge by offering a consistent and standardized payload structure tailored for IIoT. Through its Sparkplug B data encoding mechanisms utilizing Google Protocol Buffers, it ensures that devices can efficiently exchange information while preserving the key attributes of MQTT. This uniformity not only simplifies development and integration efforts but also bolsters the reliability and integrity of data communication in IIoT deployments.

In the final part of this series, Decoding the Operational Behavior of MQTT Sparkplug in IIoT Systems we’ll look at the complex mechanisms essential for effective IIoT communication. If you’re looking to improve the reliability of your IIoT communications and would like to discuss your specific needs, please contact us today.

Additional Reading

Kudzai Manditereza

Kudzai is a tech influencer and electronic engineer based in Germany. As a Sr. Industry Solutions Advocate at HiveMQ, he helps developers and architects adopt MQTT and HiveMQ for their IIoT projects. Kudzai runs a popular YouTube channel focused on IIoT and Smart Manufacturing technologies and he has been recognized as one of the Top 100 global influencers talking about Industry 4.0 online.