Essential MQTT Architecture Considerations for IoT Use Cases
The Internet of Things is growing rapidly, embedded in everything from smart homes to industrial manufacturing processes. An enormous volume of data fuels IoT applications, which require efficient and reliable data transport. The industry standard for IoT messaging is MQTT — a communication protocol designed to share data efficiently and to work well with devices with limited bandwidth and spotty network connectivity. MQTT is typically used in smart applications for cities, factories, homes, and vehicles. When you’re working with MQTT for IoT or IIoT projects, there are important architectural decisions to consider to meet the requirements of your unique implementation and ensure smooth, efficient project functioning.
Key Components of MQTT Architecture
MQTT is based on a publish/subscribe model. Devices publish data to a broker and other devices or systems subscribe to topics to receive data. While working with MQTT for IoT or IIoT, first, you need to consider the MQTT architecture itself and understand the roles of publishers and subscribers. Publishers send messages to a broker to share data. Subscribers receive that data. Publishers and subscribers don’t necessarily need to be devices — they could also be applications. Messages are sent to specified topics that clients can subscribe to. Pub/Sub architecture decouples publishers and subscribers in both space and time. Devices involved in an IoT system don’t need to be near each other and don’t need to run at the same time or without interruptions. MQTT’s pub/sub architecture allows developers to efficiently work with large amounts of IoT data.
MQTT brokers are a major reason why MQTT is so efficient. Each device only needs to establish one connection with a broker, rather than create a new connection to communicate with every other device in a connected system. Establishing a connection for each message uses more data than sending several messages through the same connection.
Key Architecture Considerations for IoT Use Cases
Optimizing your IoT use case when working with MQTT involves key architecture considerations. These can be summarized as follows.
Broker-Centric Model
Understanding the Role of the MQTT Broker: The MQTT broker is the central point that manages message delivery. All clients connect to the broker and either publish (send) or subscribe (receive) messages. Understand the central role of the MQTT broker in managing message delivery for all connected clients.
Scalability: As your project succeeds, the amount of data and devices you need to handle might grow. To avoid a negative impact on system performance, consider using a broker that uses clustered nodes, like HiveMQ MQTT Broker. The broker's ability to handle numerous connections and a high volume of messages is crucial, especially for large-scale deployments. Clustered nodes can distribute data loads across multiple servers and balance traffic so it is distributed evenly. HiveMQ can support up to 200 million connections at once.
Redundancy and High Availability: Implementing failover mechanisms and load balancing can ensure consistent availability of the MQTT service.
Client and Session Management
Client Identification: Each MQTT client is identified by a unique ID, which is crucial for managing sessions and subscriptions.
Persistent vs. Non-Persistent Sessions: Understanding and configuring session persistence correctly impacts message delivery behavior and resource usage.
Trade-offs Between QoS Levels: Quality of Service (QoS) levels enable reliable message delivery and can be customized to your project’s needs. Different QoS levels (0, 1, 2) offer trade-offs between delivery assurance and network overhead. The choice depends on the criticality of message delivery vs. bandwidth constraints. Level 0 is to send a message at most once, level 1 is to send a message at least once, and level 2 is to send a message exactly once. You might choose 0 if you don’t need a guarantee of message delivery. You might choose 1 if it’s important to guarantee message delivery and you have a system set up to handle potential duplicates. And you might choose 2 if it’s important that a message is delivered reliably with no duplicates. Depending on your application, you can strike the right balance between the importance of your data and your ability to handle duplicates on one hand, vs. how much time and bandwidth is acceptable for transmitting messages.
Topic Structure and Naming Conventions
Hierarchical Topics: MQTT uses a hierarchical topic structure that allows clients to publish and subscribe to specific topics. This allows for efficient message filtering.
Naming Strategy: A well-designed topic naming strategy is essential for maintaining an organized and scalable system.
Security
Encryption: MQTT has options to keep your data secure. One option is encryption. You can implement TLS/SSL for secure communication between clients and the broker.
Authentication and Authorization: Another good option is to use authentication, such as with a username and password. Leading brokers, like HiveMQ, will let you set up role-based access control so that different members of your team can have access to different sets of data depending on individual permissions. Setting up robust authentication mechanisms (like client certificates or user credentials) and defining authorization policies for topic access are recommended practices.
Network Considerations
Handling Unreliable Networks: MQTT's design caters to unreliable networks, but considerations for message retry, keep-alive intervals, and connection timeouts are crucial.
Bandwidth Optimization: This is especially important for IoT devices with limited bandwidth.
Integration with Other Systems
Monitoring and Logging:
System Monitoring: Implementing monitoring tools for the broker and clients to track system health, performance metrics, and usage statistics.
Logging: Efficient logging mechanisms for troubleshooting and auditing.
Adherence to MQTT Specification: Ensuring compliance with the official MQTT specification for compatibility and interoperability.
Custom Extensions: While MQTT is lightweight, specific use cases might require custom extensions or additional features in the broker or client implementations.
Retained Messages: MQTT can retain a message on a topic, ensuring that new subscribers receive the last message sent on that topic immediately after subscribing. This is particularly useful for status updates in IoT applications.
Last Will and Testament (LWT): This feature allows clients to notify other clients about an unexpected disconnection, which is vital for monitoring the status and connectivity of IoT devices.
Message Format and Payload: Payload Format Indicator and Content Type are MQTT packets that let you describe the payload content of a message. They allow standards and consistency across the MQTT industry. They also ensure messages can be properly processed and parsed without recipients having to delve into the content of the messages.
Clean Session Flag: This feature determines whether the server should remember the client's state from a previous connection, which is important for managing the persistence of subscriptions and messages.
Real-World MQTT Architecture Implementation in IoT and IIoT
To help you apply architecture considerations to your own MQTT applications, here are some real-world implementation examples.
Bi-directional data flow - This is when there’s a continuous flow of data between a broker and a database. For example, data may be sent to a database where it is processed or transformed into a new dataset, then the new data is sent back to the broker where IoT devices can use it. Learn more by reading this article on setting up a bi-directional data flow between HiveMQ and Google Cloud using Mapping and Transformers.
Edge-to-cloud data exchange - Edge computing can make IoT applications more efficient. Most enterprise-generated data is created outside of the cloud, so processing it closer to where it is created makes sense. MQTT has client libraries that allow it to integrate into applications for edge computing written in Python, Java, and other programming languages. Learn how MQTT can empower edge computing.
Data ingestion from OT to IT - Information Technology (IT) includes computer and data storage infrastructure. Operational Technology (OT) includes systems to monitor events and processes to assist with operations, such as industrial control systems in a manufacturing environment. IT and OT systems need efficient data ingestion and exchange to work together. MQTT can serve as a bridge to reliably route data between them. For an example of how this works and how HiveMQ fits in, read Green IT/OT: How to Optimize the Footprint of a Digital System Based on MQTT.
Conclusion
If you’re developing IoT or IIoT applications, you’re probably working with MQTT. Understanding the key components of MQTT architecture as well as making the right architectural choices for your unique implementation can help you realize the full potential of your IoT or IIoT application.
Watch Jean-Romain Bardet, Co-Founder at Scorp-io, share their company's story of how they built a robust MQTT architecture and UNS for scaling across various industrial locations.
HiveMQ Team
The HiveMQ team loves writing about MQTT, Sparkplug, Industrial IoT, protocols, how to deploy our platform, and more. We focus on industries ranging from energy, to transportation and logistics, to automotive manufacturing. Our experts are here to help, contact us with any questions.
