Setting up TLS for your cloud-based MQTT broker

As head of support at HiveMQ, I’ve had the privilege of helping more than 100 companies to improve their existing products and services (and deliver entirely new ones) with state of the art Internet of Things (IoT) technologies. Connected car platforms are a great example. Using MQTT (the machine-communication-optimized standard protocol for IoT), average delivery time for commands such as opening a car door with your phone reduces from more than 40 seconds to less than 3. MQTT has made services such as fleet-wide deer sighting warnings or premiere navigation that considers the rhythm of traffic lights a reality.

Over the past 5 years, I have observed a clear trend that more and more companies, even large corporations with immense data centers of their own, are choosing to run modern connectivity services on public cloud providers such as Amazon Web Services, Microsoft Azure, or the Google Cloud Platform. Use of a public cloud provider has many advantages. These providers can guarantee extraordinarily high availability through the sheer number of logical and physical computers, networks, and availability zones they possess. However, one of the drawbacks of moving to the public cloud is that you must let physical control over infrastructure and network fall into the hands of the cloud provider. In a previous blog post, we discussed an easy way to adapt and implement enterprise-ready device authentication and authorization for IoT projects that are hosted in the cloud. Today’s post focuses on another pillar of solid IoT security: data encryption.
HiveMQ enables you to implement and configure server, client, and mutual TLS certificates to provide encrypted device to server communication. Communication between multiple servers in a HiveMQ cluster can also be encrypted via the use of TLS certificates. In this post, I will show you how to quickly enable complete end-to-end encryption in the cloud using HiveMQ and self-signed TLS certificates. If you would rather skip the technical details for now, feel free to jump ahead to the executive summary and resource list.

TLS, Java Keystores, and Truststores

Transport Layer Security (TLS) is a cryptographic protocol that allows secure and encrypted communication at the transport layer between a client application and a server. On top of the provided transport layer encryption, TLS also ensures data confidentiality. Participating communication partners use certificates to verify each other’s identity (authenticity). This method prevents man-in-the-middle attacks. The sender can be certain that the receiver gets exactly the same data as the sender sent (and vice versa).

HiveMQ allows three TLS configurations:

1) Server-side TLS, where the MQTT broker presents a certificate to the connecting clients, 2) Client-side TLS, where the client presents a certificate to the broker and 3) Mutual TLS, where both the client and the broker present certificates. We recommend using mutual TLS whenever possible. TLS certificates are typically issued by a so-called Certification Authority (CA) that acts as a trusted third party. The inherent trustworthiness of the certificate is what provides the authenticity for whoever presents the certificate. Certificates for public websites, where the HTTP connection is secured through TLS, are commonly signed by so-called trusted CAs. Web browsers view the certificates as trustworthy. This trust is advantageous when you want everyone who browses your website to be able to trust your certificate and thereby your website. In most IoT deployments, companies issue their own certificates through their own CA for lower cost and higher accessibility. To securely store certificates and private keys, Java applications such as HiveMQ typically use Java KeyStores(JKS). Truststores are a special kind of JKS that contains certificates of external systems that you trust. When you use a certificate issued by a custom CA or a so-called self-signed certificate, a truststore is required. In the following example we will use a self-signed certificate and truststore. Usually, certificates issued by trusted CAs are part of the default Java Truststore.

Creating the necessary certificates

For the purpose of this blog post, we use self-signed certificates. This allows you to replicate the process for your personal testing needs without purchasing a certificate or going through an internal certificate-issuing process. To create the necessary server and client certificates and the corresponding keystores, we will use the keytool and OpenSSL command line tools.

• Create Server Keystore

This creates a Java keystore containing a certificate and private key using the RSA algorithm, a key size of 2048 bits, which is valid for 360 days.
Make sure to use an appropriately secure password.

• Enter all the necessary data for your certificate

NOTE: The first question about the first and last name is the so-called common name. This common name should match the server address (URL, Load Balancer DNS name, localhost for testing purposes etc.)

• Export certificate from the keystore to a PEM file

We will use this PEM file with the MQTT Command Line Interface later.

• Generate a PEM based client certificate

This creates a certificate and private key in PEM format using the RSA algorithm and a key size of 2048 bits, which are valid for 360 days.

• When prompted, enter all prompted information

This creates two PEM files: mqtt-client-key.pem and mqtt-client-cert.pem

• Export the client certificate from the PEM file into an CRT file.

• Import the certificate into a Java KeyStore

Make sure that you use an appropriately secure password.

We have now successfully created all necessary files.

• hivemq.jks: The Java KeyStore that contains the HiveMQ server certificate that HiveMQ broker nodes present to connecting MQTT clients and to each other for internal communication

• mqtt-client-key.pem and mqtt-client-cert.pem: The certificate and private key pair in PEM format for our MQTT client

hivemq-trust-store.jks: The truststore that HiveMQ needs to trust the certificate that the client presents

NOTE: In production, we recommend using client and server certificates that are signed by a trusted CA or the internal CA of your company. To create individual client certificates for IoT devices, create an intermediary from your ROOT CA and sign individual certificates with this intermediary. Make sure that clients present the entire certificate chain on connection, this way, you only need to include your ROOT certificate in the HiveMQ truststore. Feel free to contact us, for additional guidance on certificate management.

Configuring HiveMQ appropriately

What’s left to do is using and configuring the created files appropriately. We will use a high-availability HiveMQ cluster that is hosted on AWS with an AWS network load balancer in front. You can follow this guide to set up your own HiveMQ cluster in the cloud.

• Copy both Java KeyStores (hivemq.jks and hivemq-trust-store.jks) to the /opt/hivemq/conf/ folder of all of your HiveMQ nodes

• Configure your TCP lister as a secured TCP listener over TLS

Notice that we set the client-authentication-mode to REQUIRED. This setting ensures that only MQTT clients that use a certificate that is part of the configured truststore are allowed to connect. More details on configuring a TLS-secured TCP listener can be found in the HiveMQ User Guide.

• Configure your cluster transport to use TLS as well.

Because we use the same certificate for all of our HiveMQ broker nodes, we use the same hivemq.jks as our keystore and truststore. When using individual certificates for each node, make sure to configure appropriate truststores. More details on configuring cluster transport over TLS secured TCP can be found in the HiveMQ User Guide.

• After you make the changes to the configuration, restart the HiveMQ process on both broker nodes

• Check that the encryption of the internal server communication is working

To confirm this, we look for the following statement in the hivemq.log file.

INFO - Cluster size = 2, members : [8Jojp, WlF1S].

Testing the mutual TLS listener configuration

Now that we have verified that our internal server communication is using TLS, we want to test the TLS listener configuration that ensures that the communication between the client and server uses mutual encryption (use of both a client and server certificate).

We will use the MQTT Command Line Interface to see if everything is working as we expected.

• Let’s try connecting to the broker cluster without the use of any certificates

Since our self-signed certificate is not trusted, we see that without adding the CA file to connection, the client won’t even start the TLS handshake.

Let’s add the the server.pem CA file that we created earlier to our connection attempt

This time a TLS handshake is attempted and fails because we configured our HiveMQ broker to require a client certificate

• Let’s add the client certificate mqtt-client-cert.pem and the client key mqtt-client-key.pem that we created earlier to the connection attempt

• We need to enter our private key password to access the key.

Now the connection is established successfully and we are allowed to subscribe to our desired topic ’topic’.

This demonstrates the correct usage of certificates with the MQTT CLI. Please feel free to contact us is you have any questions about configuring mutual TLS encryption with the MQTT client of your personal choice.

Executive summary

Here are the key take-aways from this post in a nutshell:

• Moving connectivity services to the cloud yields significant benefits in terms of scaling and availability

• Encryption is a key pillar of enterprise-grade security compliance

• HiveMQ delivers powerful TLS-based encryption for your MQTT connections and internal server communication

• Use of TLS-based encryption means that you do not have to trust the cloud provider’s network security to protect your data on network level

• Setting up a state of the art, fully end-to-end encrypted MQTT broker cluster in the cloud with the HiveMQ Enterprise Edition takes less than an hour

Necessary resources

Here’s a list of all the resources you need to set up our fully end-to-end encrypted MQTT broker cluster with Amazon Webservices:

• Blog post: HiveMQ Cluster on AWS

• We have also prepared a download package, with all the necessary certificates and a correctly set up config.xml file. Just enter the correct IP address and you are ready to go.
  Hint: The server certificate in this package uses the common name ’localhost’. When using it on a hosted server and not a local machine, depending on the MQTT client you are using, you may need to set the “insecure” flag, as the common name and IP address will not match.

Contact Us

We know that topics such as cloud security can be extremely complicated. Let us know about your personal and legal requirements. We are happy to find away to achieve security compliance with you!

Check out the video below that provides the summary of this blog