A Beginner's Guide to MQTT Brokers

In the world of the Internet of Things (IoT), the MQTT protocol has become the cornerstone for IoT simplicity and efficiency in handling device communication. At the heart of MQTT's functionality is the MQTT broker, a central hub that facilitates message exchange between devices. If you're an IoT enthusiast, solution architect, CTO, or anyone with a thirst for understanding the backbone of modern communication, you're in the right place. 

In this quick overview guide, we'll unravel the intricate world of MQTT brokers, exploring how they empower seamless data exchange. We'll dissect the broker’s inner workings, discuss how to configure an MQTT broker and guide you through choosing the ideal MQTT broker for your specific needs or industry vertical use case. We'll touch on open-source versus commercial options and on-premises versus cloud-based solutions. Let’s dive in.

What is an MQTT Broker?

An MQTT broker serves as the central hub in the publish/subscribe messaging system, where it receives messages from publishers and forwards them to subscribers, playing a critical role in overseeing communication between MQTT clients and ensuring reliable message delivery. These clients are typically IoT devices like sensors or actuators. The broker receives, filters, and distributes messages to appropriate clients, ensuring efficient and reliable communication. 

Today, MQTT brokers are used in large-scale production that connects millions of geographically dispersed clients to low-scale projects, such as creating Proof-of-Concepts (PoCs) or DIY.  In later sections of this post, we will cover how to choose the right MQTT broker for your use case. 

How to Make an MQTT Broker Communicate with an MQTT Client?

Here’s an overview of how to make an MQTT broker communicate with an MQTT client, such as a sensor, an edge device, a PLC, etc.:

Step 1: Install an MQTT Broker of Your Choice

MQTT brokers are available in commercial, open-source, cloud-managed and general-purpose editions. To get an MQTT broker to communicate with an MQTT client, first you need to find an edition of the broker that is right for your use case. 

Here are a few examples of how you can install an MQTT broker in different environments:

• Use a fully-managed MQTT Broker, like HiveMQ Cloud. Here’s how to get started with HiveMQ Cloud.  

• Download and install an MQTT broker on a server or computer of your choice. Here’s how to get started with the HiveMQ platform.

• Run an MQTT broker on Docker. Here’s how to get started with HiveMQ on-premise MQTT Broker.

• Deploy an MQTT broker on Amazon Web Services (AWS). Here’s how to get started with HiveMQ on AWS.

• Deploy an MQTT Broker on Microsoft Azure. Here’s how to get started with HiveMQ on Azure.

Step 2: Connect MQTT Clients to Your MQTT Broker

Depending on the broker you are using, you can connect to an MQTT client based on the broker installation. Here are a few ways to do it:

Example 1: Using MQTT CLI for On-Premise MQTT Brokers

If you are unaware, the MQTT CLI is an open-source, Java-based MQTT client tool that enables you to interact quickly and easily with any MQTT broker in various ways. The MQTT CLI comes in various binary packages that can be downloaded from the documentation homepage on GitHub.

Here are the steps: 

1. Open a terminal window and go to the tools directory of your HiveMQ MQTT broker. In the tools directory, go to the mqtt-cli/bin folder and enter mqtt sh to start the MQTT CLI in shell mode. The MQTT CLI starts and lists useful options and commands.

2. To connect your first MQTT client to your HiveMQ broker on localhost and give it a custom identifier, enter con -h localhost -i testClient1. This command creates the first MQTT client with the custom identifier testClient1 and connects the client to your MQTT broker on localhost.

3. To connect another MQTT client that you can use to test your installation, open a second terminal window and enter mqtt sh (keep your original terminal window open).

4. In the second terminal window, enter con -h localhost -i testClient2. This command creates a second MQTT client with the customer identifier testClient2 and connects the client to your MQTT broker on localhost.

Example 2: Using Cloud-managed MQTT Broker

Using a cloud-managed MQTT broker, like HiveMQ Cloud, you can connect to MQTT Clients via the console. Here’s an overview of how to connect to MQTT Clients via HiveMQ Cloud.

1. Sign up for Serverless FREE HiveMQ Cloud plan.

2. Log in to HiveMQ Cloud Portal.  

3. Configure your HiveMQ Cloud plan.

4. Create a free HiveMQ Cloud cluster.

5. Define the credentials that your MQTT clients use to connect to the cluster.

6. Create Access Credentials.

7. Add Topic filter-based permissions.

8. Test with the HiveMQ WebSocket Client if a client has established a connection. 

Step 3: Start Publishing MQTT Messages/Subscribing to an MQTT Topic

Using MQTT CLI or via the console, you can start publishing MQTT messages and subscribing to MQTT topics. 

Here’s an example using MQTT CLI: 

1. In the terminal window of the MQTT client (testClient2), enter sub -t testTopic -s.This command subscribes testClient2 to all messages that are published with the topic testTopic.

2. In the terminal window of another MQTT client (testClient1), enter pub -t testTopic -m Hello. This command publishes the message Hello from testClient1 with the topic testTopic.The message Hello appears immediately in the terminal window of testClient2.

For detailed steps, check out our documentation. 

Mechanism of How an MQTT Client Establishes a Connection with an MQTT Broker

Here's a basic overview of the mechanism of how an MQTT client establishes a connection with an MQTT broker:

• Client Initialization: The MQTT client initializes its MQTT library and sets up the required parameters, such as the broker's address and port.

• Connection to the broker: The client establishes a TCP/IP connection with the MQTT broker. The default port for MQTT is 1883 (or 8883 for encrypted connections using TLS/SSL).

• Handshake: Once the TCP/IP connection is established, an MQTT handshake occurs. This involves exchanging control packets to establish the connection and deal with parameters.

• Connect packet: The client sends a "CONNECT" packet to the broker, indicating its intention to establish a connection. This packet includes information such as client ID, connection flags, and other settings.

• Acknowledgment (CONNACK): The broker responds with a "Connack" packet, indicating whether the connection request is accepted or rejected.

• Subscriptions (optional): If the client is a subscriber, it can then send "Subscribe" packets to the broker, indicating the topics it wants to subscribe to.

• Publishing messages: Once the connection is established, clients can publish messages to topics or subscribe to topics to receive messages from other clients.

• Keep-Alive: To maintain the connection, clients periodically exchange "Ping" packets to confirm that the connection is still active.

Choosing the Right MQTT Broker

Choosing the right MQTT broker depends on your architectural requirements (a.k.a. Non-Functional Requirements). An MQTT broker comparison based on these architectural requirements should give you insight into how to find the best MQTT broker for your needs.

Here are some of the functionalities you should look for in an MQTT broker if you want to use it in large-scale production:

• Handling large numbers of concurrent connections reliably: Depending on its implementation, a broker has the capability to manage millions of MQTT client connections reliably and securely, with a robust load balancing feature. This facilitates communication between diverse devices, networks, and software systems in near real-time.

• Filtering and routing messages: MQTT brokers can filter messages based on the subscription topic and determine which client(s) should receive the message.

• Session management: MQTT brokers can maintain session data for all connected clients, including subscriptions and missed messages, for clients with persistent sessions.

• Authentication and authorization: The broker is responsible for authenticating and authorizing clients based on credentials provided by the client. The broker is extensible, facilitating custom authentication, authorization, and integration into backend systems. In addition to authentication and authorization, brokers may provide other security features, such as encryption of messages in transit and access control lists.

• Scalability and monitoring: An MQTT broker must be scalable to handle large volumes of messages and clients, integrate into backend systems, be easy to monitor, and be failure-resistant. To meet these requirements, the MQTT broker must use state-of-the-art event-driven network processing, an open extension system, and standard monitoring providers. Brokers may also provide advanced features for managing and monitoring the MQTT system, such as message filtering, message persistence, and real-time analytics.

• Clustering: MQTT brokers can support clustering, allowing multiple instances of the broker to work together to handle large numbers of clients and messages.

• Provides control over how IoT data flows within your data pipeline: Prominent MQTT brokers can provide you with the ability to have control over your data with visualization and management tools, powerful security features, and an integrated policy engine that can validate, enforce, and transform data in motion.

Here are some of the functionalities you should look for in an MQTT Broker if you want to use it in small-scale projects: 

• Ease-of-use: For small-scale projects, especially DIY projects and PoCs, an MQTT broker should be easy to deploy. There are several open-source and fully-managed MQTT brokers that help you connect devices with minimal effort.

• Scalability: MQTT brokers should be able to help you scale as you go so your PoCs turn into production-level projects soon.

• Cost-efficiency: There is pay-as-you-go pricing available for MQTT brokers so you don’t have to constrain yourself while working on a PoC

• Security: It goes without saying how important it is to secure your device connectivity from end to end, irrespective of your scale of the project.

Types of MQTT Brokers

There are different types of MQTT brokers available:

• Open source: These are license-free brokers that are suitable for small-scale projects.

• Commercial: These are available via licenses and offer several features and customization, making them suitable for production-level deployment.

• Cloud-managed: These are fully-managed MQTT brokers that need minimum maintenance and are easy to deploy.

• General purpose: These are available both license-free as well as for licenses and are suitable for small-scale projects.

The below image summarizes features offered by different types of brokers.

Open Source Vs. Commercial MQTT Brokers

Open-source MQTT brokers have limited scalability, limited security options, cannot cluster for higher availability, are hard to manage when coded in difficult libraries, and have no overload protection from overactive publishers. 

On the other hand, commercial MQTT brokers are customizable according to your needs, provide increased scalability,  security, flexibility & reliability, and have overload protection. 

On-premise MQTT Brokers Vs. Fully-Managed Cloud MQTT Brokers

On-premises MQTT brokers offer more control but require substantial setup and maintenance. Fully managed cloud-based MQTT brokers are easier to deploy, cost-effective, and have zero maintenance requirements. 

MQTT Broker Architecture in Smart Manufacturing

MQTT brokers can be used for several smart manufacturing use cases, such as intra-factory connections, inter-factory connections, etc.

Here’s an example of inter-factory communication using the HiveMQ MQTT broker.

To learn more, read our whitepaper Modernizing the Smart Manufacturing Industry with MQTT.

MQTT Broker Architecture in Connected Cars

MQTT brokers can be used for several connected car use cases such as remote monitoring, remote diagnostics, Over-the-Air (OTA) updates, theft recovery, energy management, etc. 

To learn more, read our blog Building New Connected Car Features on the Back of a Robust Connectivity Platform. 

MQTT Broker Architecture in Transportation and Logistics

MQTT brokers are well-suited for use in the transportation and logistics industry, where real-time data communication is crucial for efficiency and safety. Some of the use cases include fleet management, supply chain visibility, warehouse management, driver and cargo safety, etc. 

Here’s an example of how an MQTT broker, like HiveMQ, can help real-time communication between several vehicles and backend systems.

MQTT Broker Architecture in the Energy Industry

MQTT brokers can play a crucial role in optimizing the operations and OT/IT interoperability in the energy industry. MQTT brokers can be used in energy use cases such as Asset Performance Management, Predictive Maintenance, Asset Tracking, etc.  

Here’s an example of how an MQTT broker, like HiveMQ, can be used for Remote Asset Management in Oil & Gas.

Next Steps

As you explore an MQTT broker’s inner workings, learn how to configure an MQTT broker, and know how to choose the ideal MQTT broker for your specific needs, check out the HiveMQ MQTT platform as a next step.

Additional Reading

• Top 10 Criteria for Selecting a MQTT Broker

• Getting Started with Raspberry Pi Pico W for IoT: Micropython and MQTT

• Using MQTT and Raspberry Pi to Visualize Sensor Data on a TilesFX Dashboard