Hands-on Guide to LoRaWAN and HiveMQ MQTT Broker Integration for IoT
In today's interconnected world, the need for efficient and reliable communication between devices is paramount. Two significant players in this arena are LoRaWAN and MQTT. Individually, these technologies have carved out substantial niches in the IoT ecosystem. LoRaWAN, known for its long-range and low-power connectivity, is a champion in the field of wide-area IoT networks. On the other hand, MQTT, a lightweight messaging protocol, has become a cornerstone for IoT messaging due to its efficiency and reliability in low-bandwidth environments.
In this blog post, we dive into the synergy of HiveMQ MQTT Platform and ChirpStack Open Source LoRaWAN Network Server. The integration of LoRaWAN and MQTT opens a myriad of possibilities for IoT applications, enhancing capabilities in areas such as scalability, energy efficiency, and data management. From smart agriculture to urban planning and industrial automation, the convergence of LoRaWAN and MQTT is set to revolutionize the way we interact with and manage connected devices.
Let's embark on this journey of technological fusion and discover the endless possibilities that arise when we combine ChirpStack and HiveMQ Platform.
What is LoRaWAN?
LoRaWAN, short for Long Range Wide Area Network, is a protocol for wireless, low-power wide area networks (LPWAN). It's designed to allow long-range communications at a low bit rate among connected objects such as sensors and actuators, making it ideal for Internet of Things (IoT) applications.
Key features of LoRaWAN include:
Long Range: LoRaWAN can cover a large geographical area, often in the range of several kilometers, which is much more than traditional Wi-Fi networks.
Low Power: Devices connected to a LoRaWAN network have low power consumption, allowing them to operate for years on a small battery.
Secure Communication: It incorporates several layers of encryption to ensure secure communication between devices and the network server.
Bidirectional Communication: LoRaWAN supports two-way communication, allowing for both uplink data from devices to the server and downlink messages from the server to the devices.
Adaptability: It can be used across various industries — including agriculture, smart cities, and industrial IoT — due to its versatility and ability to handle a large number of connections.
LoRaWAN operates on unlicensed frequencies, which means it can be used without the need to purchase spectrum licenses, making it a cost-effective solution for many IoT applications.
LoRaWAN and HiveMQ Integration Architecture
Here is an overview of the global architecture that you can follow when you want to deploy your private LoRaWAN network.
In this architecture, HiveMQ Broker is the backbone for communication between components and services. Two types of connectivity are shown, one with a gateway that embeds the ChirpStack components and another that does not contain any Chirpstack components and forwards LoRaWAN packets to the gateway bridge. The ChirpStack server then consumes the events and commands published on HiveMQ Broker to serve the data to applications or gateways. You can find more architectures in the ChirpStack official documentation, including a multi-region example mixing US and EU frequencies.
Our Lab
To run our lab environment, we have multiple resources:
LoRaWAN server
LoRaWAN gateway
LoRaWAN device
HiveMQ Enterprise broker
To optimize the resources, we deploy the LoRaWAN server with HiveMQ broker on a single Linux server. In a production environment, we recommend to run HiveMQ as a cluster and the LoRaWAN server on a dedicated server (that also could be a cluster).
Server specifications:
4 CPU
8 Gb RAM
64 Gb SSD
1 network interface
Ubuntu 22.04
Don’t forget to update your system before starting the installation:
sudo apt update
sudo apt upgrade -yHiveMQ MQTT Broker Installation
To install HiveMQ Broker on our Linux server, we follow the official documentation.
Requirements
To run, our broker needs Java installed on the server. We can install the latest version or an older one starting at version OpenJDK JRE 11. We also install unzip to extract files from the source archive.
sudo apt install openjdk-19-jre
sudo apt install unzipYou can check the installation by running:
java -versionInstall HiveMQ Enterprise Broker
We will install our broker in the default directory for HiveMQ: /opt/hivemq. The default username will be HiveMQ.
Run the following commands to download the source, install all the files, create the user, and update the permissions:
cd /opt
wget --content-disposition https://releases.hivemq.com/hivemq-4.23.0.zip
unzip hivemq-4.23.0.zip
ln -s /opt/hivemq-4.23.0 /opt/hivemq
useradd -d /opt/hivemq hivemq
chown -R hivemq:hivemq /opt/hivemq-4.23.0
chown -R hivemq:hivemq /opt/hivemq
cd /opt/hivemq
chmod +x ./bin/run.shTo check the installation, start the broker manually by running this command (assuming you are in /opt/hivemq):
./bin/run.shNote: For our demo, we run the broker without licenses. This allows us to have up to 25 connections and use all extensions during 5 hours.
Install HiveMQ Service
To automatically start the broker, we will deploy the service on our server.
sudo cp /opt/hivemq/bin/init-script/hivemq.service /etc/systemd/system/hivemq.service
sudo systemctl start hivemq
sudo systemctl enable hivemqYou can check that HiveMQ Broker is listening for incoming connections by running the following command:
netstat -an | grep 1883ChirpStack LoraWan Gateway Installation
Requirements
To run Chirpstack, you need PostgreSQL and Redis. To install them, run the following command:
sudo apt install \
redis-server \
redis-tools \
postgresqlPostgreSQL setup
To enter in the command line utility for PostgreSQL, type:
sudo -u postgres psqlOnce connected as postgres user, you can prepare the database and the user that will be used by ChirpStack:
-- create role for authentication
create role chirpstack with login password 'chirpstack';
-- create database
create database chirpstack with owner chirpstack;
-- change to chirpstack database
\c chirpstack
-- create pg_trgm extension
create extension pg_trgm;
-- exit psql
\qAdd Chirpstack Repository
As Chirpstack provides a repository with packages for Ubuntu and Debian, we will use them in our deployment.
First, make sure that we have prerequisites for adding the repository:
sudo apt install apt-transport-https dirmngrSet up the key for the repository:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1CE2AFD36DBCCA00Add the repository:
sudo echo "deb https://artifacts.chirpstack.io/packages/4.x/deb stable main" | sudo tee /etc/apt/sources.list.d/chirpstack.listUpdate the package list with the new repository:
sudo apt updateInstall ChirpStack Gateway Bridge
Since we will use our SenseCap M2 LoRaWAN gateway in packet forwarder mode, we need to install the bridge to allow communication.
Install it using the package repository:
sudo apt install chirpstack-gateway-bridgeConfigure ChirpStack Gateway Bridge
The configuration file is located at /etc/chirpstack-gateway-bridge/chirpstack-gateway-bridge.toml. You need to update the [integration.mqtt] section to match the region prefix for the region that applies to this ChirpStack Gateway Bridge instance.
As an example, here is the configuration for Europe (EU868):
[integration.mqtt]
event_topic_template="eu868/gateway/{{ .GatewayID }}/event/{{ .EventType }}"
command_topic_template="eu868/gateway/{{ .GatewayID }}/command/#"More regions are available in the official documentation:
Configure the connection to your HiveMQ broker in the file /etc/chirpstack-gateway-bridge/chirpstack-gateway-bridge.toml
You need to update the [integration.mqtt.auth] section to match your broker address and authentication.
[integration.mqtt.auth.generic]
# MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
server="tcp://127.0.0.1:1883"
# Connect with the given username (optional)
username="chirpstack"
# Connect with the given password (optional)
password="chirpstack"Notes: If you are using HiveMQ Enterprise broker with the Enterprise Security Extension, make sure to have the appropriate permissions to allow publish and subscribe on the event and command topics.
Start ChirpStack Gateway Bridge
Start ChirpStack Gateway Bridge service and add it to the list of services to be started on boot:
# start chirpstack-gateway-bridge
sudo systemctl start chirpstack-gateway-bridge
# check that the service is properly running
sudo systemctl status chirpstack-gateway-bridge
# add chirpstack-gateway-bridge on boot
sudo systemctl enable chirpstack-gateway-bridgeInstall ChirpStack
We now install the code component of our Lorawan server. Install it using the package repository:
sudo apt install chirpstackConfigure ChirpStack
The configuration files are located at /etc/chirpstack. You will find one global configuration file called chirpstack.toml and various region configuration files. You can start by adjusting the region allowed on the server by adjusting the section [network]:
[network]
# Network identifier (NetID, 3 bytes) encoded as HEX (e.g. 010203).
net_id="000000"
# Enabled regions.
#
# Multiple regions can be enabled simultaneously. Each region must match
# the 'name' parameter of the region configuration in '[[regions]]'.
enabled_regions=[
"eu868",
]You also need to update the default API port since on our lab port 8080 is already used by HiveMQ Control Center. Update the section [api] and change port from 8080 to 8181:
# API interface configuration.
[api]
# interface:port to bind the API interface to.
bind="0.0.0.0:8181"In order to use HiveMQ broker, you need to adjust the settings in section [integration] :
[integration]
enabled=["mqtt"]
[integration.mqtt]
server="tcp://localhost:1883/"
json=trueStart ChirpStack
Start ChirpStack service and add it to the list of services to be started on boot:
# start chirpstack
sudo systemctl start chirpstack
# check service status for chirpstack
sudo systemctl status chirpstack
# start chirpstack on boot
sudo systemctl enable chirpstackCheck the logs with:
sudo journalctl -f -n 100 -u chirpstackAt this point, if no error is raised, you should be able to open the admin portal of ChirpStack by browsing http://<server-ip>:8181 .
Default username: admin and default password: admin.
Gateway Setup
For our lab, we’ve chosen to work with an indoor gateway provided by Seeed Studio: the SenseCAP M2 Multi-Platform LoRaWAN Gateway. While the company provides a lot of gateways preconfigured on dedicated public LoRaWAN networks (like longFi or Helium), the SenseCAPM2 Multi-Platform allows you to connect it to your private LoRaWAN network and use it without any limitation. If you want to explore advanced scenarios like deploying your own ChirpStack components (with the Gateway OS) and building a DIY gateway, you can have a look at a Raspberry PI with a LoRaWAN hat (compatibility list here).
The following instructions assume that you have a brand new gateway or you did a factory reset on an existing one (Don’t forget to plug the antenna before switching on the gateway. Otherwise, you can damage the LoRaWAN chip).
Open the admin interface of the gateway. You should find it at: http://<ip-address>/cgi-bin/luci/
Default login is: “admin” and default password is located on the sticker under the gateway. Once logged in, you end up at the homepage of the gateway.
From there, you need to go to the following dropdown menu “LoRa > LoRa Network” to configure the gateway to work with our local ChirpStack Network Server.
By default, the gateway is connected to the SenseCAP Network. We will change it to “Packet Forwarder” and configure the IP Address of our ChirpStack Gateway. Once configured, click “Save & Apply”. Before leaving the page, copy the EUI code for later.
Note: The second configuration available here is to set the Local Network Server which enables, on the gateway, some ChirpStack components that directly connect to our HiveMQ platform.
To check that the connectivity with the network server is working fine, go back to the home screen to check that the schema looks like below.
To allow the gateway to communicate with our ChirpStack Network Server, we need to add the gateway in our configuration. Go to the admin portal of ChirpStack Network Server > Tenant > Gateways and click Add.
Fill out the fields in the form to add our gateway information and click “Submit”.
After adding the gateway, you should see it as Online in the gateway list.
That is all you need to do to integrate the gateway. We will now configure a device to send data via our LoRaWAN gateway.
Device Setup To Send Data Via LoRaWAN Gateway
To connect a device, we need to follow a few steps.
Create a Device profile.
Create an Application.
Declare the device to work with the Application.
In LoRaWAN ChirpStack connectivity, there are two ways to authenticate a device ABP (DevAddr with Session-keys) and OTAA (Device root-keys).
Create a Device Profile
To create a device profile, go to Tenant > Device profiles. Create a new profile by entering the name and the region configuration.
On the Join tab, check that “Device supports OTAA” is activated.
Then click “Submit” to create the device profile.
Create an Application
To create an application, go to Tenant > Application. Create a new application by entering the name and then clicking “Submit”.
Once the application is created, you can add a new device to it.
Declare a New Device
Once the application is created, go into your application to add a new device. For our initial test, we will use a LoRa-E5 Dev board stock with the factory AT firmware. This board can be used in multiple ways:
AT commands via serial connection
With external MCU using UART
With a custom firmware developed with STM32Cube.
As the topic here is not to build a how-to on developing on STM32, we will use the AT firmware to do our tests.
You can connect to the board via USB-C and open a serial tool with baudrate to 9600 and both NL & CR. To validate the connection, you can send the command:
ATThat should respond with:
+AT: OKIf it’s not the case, you need to check the connection to your board.
We will start by requesting the Device EUI to the board by sending the following command:
AT+ID=DevEuiThis will return something similar to:
+ID: DevEui, 2C:F7:F1:20:43:30:A9:E4This is the Device EUI that you need to put in the device creation form with a name and the Device EUI on the Application that you’ve previously created.
We submit the form to create the device. We need then to generate an OTAA Key to finalize the configuration of the board and test the connectivity.
In order to generate the OTAA Key, go into the ChirpStack Network Server portal to the device property page and the tab OTAA keys. Use the round arrow to generate an MSB Application Key and copy it for later. Click on “Submit” to save it.
Go back to your serial connection on the board and type the following commands:
AT+KEY=APPKEY,"replace with OTAA Keys previously saved"You should receive a response similar to:
+KEY: APPKEY 8395725ADEDBCB75F629B199A2CB0DBCTo test the connectivity to our LoRaWAN server, we will set the board on the EU frequency and start a JOIN process with the network by running the following commands:
AT+DR=EU868
AT+CH=NUM,0-2
AT+MODE=LWOTAA
AT+JOINThe result should be something similar to:
+JOIN: NORMAL
+JOIN: Network joined
+JOIN: NetID 000000 DevAddr 01:7D:E4:97
+JOIN: DoneYou can also check that the server has received the communication by checking if you get the LoRaWAN frames JoinRequest and JoinAccept.
This means that our board is able to communicate via our gateway to our ChirpStack Network Server.
This is just a test communication, for a production use case. Now we need to build our own firmware on the board to do our logic and communicate over our LoRaWAN private network.
Application Integration with MQTT
In a production environment, the device is sending data over LoRaWAN. The ChirpStack Network Server allows integration with multiple external systems like IFTTT, HTTP POST, and others. In our case, we are interested in configuring the integration with HiveMQ Broker to benefit from its data quality (Data Hub) features and its extensions that allow integrations with streaming platforms, databases, and data lake systems.
Configuration
To configure our Application on ChirpStack to send the data to HiveMQ Broker, we need to update the configuration file of our ChirpStack Network Server.
The configuration files are located at /etc/chirpstack. You need to edit the global configuration file called chirpstack.toml:
Go to the integration section and update it as follows:
[integration]
enabled=["mqtt"]
[integration.mqtt]
server="tcp://localhost:1883/"
# Event topic template.
event_topic="application/{{application_id}}/device/{{dev_eui}}/event/{{event}}"
# Command topic.
#
# This is the topic on which the MQTT subscribes for receiving (enqueue) commands.
command_topic="application/{{application_id}}/device/{{dev_eui}}/command/{{command}}"
# Use JSON encoding instead of Protobuf (binary).
json=true
# Quality of service level
#
# 0: at most once
# 1: at least once
# 2: exactly once
#
# Note: an increase of this value will decrease the performance.
# For more information: <https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels>
qos=1
# Clean session
#
# Set the "clean session" flag in the connect message when this client
# connects to an MQTT broker. By setting this flag you are indicating
# that no messages saved by the broker for this client should be delivered.
clean_session=false
# Client ID
#
# Set the client id to be used by this client when connecting to the MQTT
# broker. A client id must be no longer than 23 characters. If left blank,
# a random id will be generated by ChirpStack.
client_id=""
# Keep alive interval.
#
# This defines the maximum time that that should pass without communication
# between the client and server.
keep_alive_interval="30s"Save the file and restart the service with:
sudo systemctl restart chirpstackAnd voilà, now HiveMQ Broker will receive the data sent by the devices on application/{{application_id}}/device/{{dev_eui}}/event/{{event}}
Test the MQTT Configuration
To test our configuration, we connect an MQTT client (MQTT.fx or MQTT Explorer) to HiveMQ Broker and configure it to listen for all the topics with the root path “application/#”. This will allow us to see the data going through the broker.
We will then come back to our Lora-E5 dev board connected via serial. We will use AT commands to send messages over LoRaWAN to our broker.
Run the following command to check the connection, join the network, and send a message.
AT
AT+DR=EU868
AT+CH=NUM,0-2
AT+MODE=LWOTAA
AT+JOIN
#For unconfirmed message
AT+MSG="HELLO FROM HIVEMQ"
#For confirmed message
AT+CMSG="HELLO FROM HIVEMQ CONFIRMED"The result of the should be:
#Unconfirmed message
+MSG: Start
+MSG: FPENDING
+MSG: RXWIN1, RSSI -65, SNR 6.0
+MSG: Done
#Confirmed message
+CMSG: Start
+CMSG: Wait ACK
+CMSG: FPENDING
+CMSG: ACK Received
+CMSG: RXWIN1, RSSI -66, SNR 8.0
+CMSG: DoneYou can then go to your MQTT client to see the message that was received:
In the JSON payload, you can search for the data value that is sent encoded as base64. You can also see the structure of the topic that is dynamically built from our configuration.
The LoRaWAN frame is also visible in the ChirpStack UI on the device details in the LoRaWAN frames tab:
And the confirmed message:
Quick note: If you plan to develop a custom firmware for the Lora-E5 board, you can refer to the guide from SeeedStudio here. There are also plenty of possibilities out there to develop products or sensors with LoRaWAN.
Wrap Up
In this guide, we’ve successfully navigated through the intricate process of installing ChirpStack alongside HiveMQ MQTT Broker, paving the way for a robust and scalable IoT infrastructure. Let's quickly recap the pivotal steps and benefits of this integration:
Seamless Installation: We began with the straightforward installation of ChirpStack, appreciating its user-friendly interface and flexibility. Next, integrating HiveMQ MQTT Broker into our setup brought an unprecedented level of efficiency and reliability in handling MQTT messaging.
Enhanced Connectivity: By combining ChirpStack’s LoRaWAN capabilities with HiveMQ’s MQTT protocol proficiency, we've established a communication backbone capable of supporting a wide range of IoT devices and applications.
Scalability and Reliability: The synergy between ChirpStack and HiveMQ ensures our system is not just powerful today, but also ready for future expansions. This setup promises high scalability and reliability, critical for any growing IoT network.
Real-World Applications: From smart city infrastructure to industrial IoT solutions, the possibilities are endless. This integration can serve diverse sectors, offering real-time data transmission, enhanced security, and efficient remote monitoring.
HiveMQ Added Value: HiveMQ not only offers a reliable backbone for ChirpStack but also allows extended scenarios with Applications data thanks to its multiple extensions to integrate data directly into streaming platforms, databases, or data lakes.
Whether it’s for personal projects or enterprise-level solutions, the ChirpStack and HiveMQ combination is a testament to the power and potential of modern IoT technologies. As we conclude, remember that the world of IoT is ever-evolving. Stay curious, keep experimenting, and don't hesitate to push the boundaries of what your IoT network can achieve.
In the next article focusing on LoRaWAN and HiveMQ, we will see how to create a device based on embedded chips that sends temperature information via our LoRaWAN Private Network. The data will be normalized with the HiveMQ Data Hub transformation feature and saved into a database thanks to the HiveMQ Enterprise Extension for PostgreSQL for historical purposes.
Anthony Olazabal
Anthony is part of the Solutions Engineering team at HiveMQ. He is a technology enthusiast with many years of experience working in infrastructures and development around Azure cloud architectures. His expertise extends to development, cloud technologies, and a keen interest in IaaS, PaaS, and SaaS services with a keen interest in writing about MQTT and IoT.
