Implementing Authentication in HiveMQ Without Active Directory Schema Changes

Introduction

In a high-demand enterprise, IoT implementation requires scale, scalability, ease of implementation, and security (authentication, authorization, secure data in-air and at rest), and secure networks. Most enterprise systems manage access controls for end users using some sort of end-user directory service. 

Yet in an asset-intensive scenario/sector, managing the entire device/sensor lifecycle and access control within a corporate directory service is a daunting task. Typically, a corporate directory service (like Active Directory) is used for end user management then for device, sensor, site, tower, and/or operations access control.

In a few cases, it is not practical to import schema classes and start implementing dynamic linking of users and groups for managing access control. 

Imagine a Sparkplug-B-based (or MQTT-based) Report by Exception, edge-driven, UDT-based SCADA logic on top-end to auto render visualisation or ablity to control remote assets requiring very granular role-based access control (RBAC). For example:,

• A sensor can only publish to a specific topic with a certain QoS level.

• In “site to site” or “machine to machine” (M2M) use cases where there is local command-control between these M2M with specific topics local broker (or namespace) as well as different access to centralized broker with different federated access control strategy. It will get more complicated when there is a need for two way command-control for remote asset operational control, which needs clear approval and audit.

• Certain OT applications can only subscribe to specific topics with or without shared subscriptions.

• In a federated broker set-up, where one needs to implement “Do not allow” a specific site/device/sensor/asset/system to write to other site/asset/device/sensor topics to ensure there is clear isolation.

• Allow a group of applications doing the same thing to have shared subscription – meaning certain broker users subscribe to a topic to deliver arriving messages only to one in the group (load distribution). Please note: Shared Subscriptions allow multiple MQTT clients to share a single subscription on the broker. When shared subscriptions are used, all clients that share the same subscription with a subscription group receive messages in an alternating fashion. This mechanism improves load balancing and fault tolerance in an MQTT system.

Please note: The term “users” in above cases refers to the user-id or identity used in establishing an MQTT connection to connect to a broker by sites/edges/devices (as in a system account).

In the context of IoT lifecycle and access control management, and the challenges these present, I was wrangling with different possibilities for authentication and authorization around HiveMQ Broker for an enterprise implementation. This blog post shares my experience of implementing that without using Active Directory schema. 

HiveMQ Broker is a robust and flexible broker that supports scale with built-in security and can work with integrations, such as MQTT, Kafka, Sparkplug B, and more. However, finding the optimal combination of authentication and authorization methods can be challenging, as there are many factors to consider, such as the corporate approach to device management, how end devices participate, whether they use X.509 certificates, whether there is a need for isolation, whether they are remotely controlled, and many other considerations. It will become particularly tricky when one wants to use Active Directory to manage end site/device/edges and other enterprise system’s authentication and authorization in HiveMQ.  

After evaluating different options, such as using OAuth, JWT, X.509 certificates, or custom plugins, I discovered that the best solution for me was to use Active Directory (AD) for authentication and Database for authorization. This way, I could leverage the existing corporate Active Directory schema for user (remote/site/device – system accounts) authentication through AD/LDAP, and have the flexibility to manage access control (RBAC) in a secure database by mapping AD groups to roles, and in turn, to permissions in the database. 

This solution also allowed me to use the HiveMQ Control Center and the HiveMQ Enterprise Security Extension (ESE) for monitoring and auditing broker activities and permissions.

Below I walk you through how I solved a few of the challenges, plus the solutions I encountered, while working on the project involving Active Directory (AD) tool sets and configuration. I hope this information will be useful for anyone who is dealing with similar issues or wants to learn more about this topic.

I’ve documented all the steps I followed and the tools I used. Hopefully, this walkthrough will help those struggling with AD (like I was) or those seeking to set up a prototype with a given combination of AD and Database.

Following is the step-by-step configuration of HiveMQ Broker with Authentication through Microsoft Active Directory and Authorization (role-based access control) using Database using (PostgreSQL) 

Before we deep-dive into the configuration, here’s an overview of ESE — the extension we’re going to use in HiveMQ for authentication and authorization.  

HiveMQ Enterprise Security Extension (ESE) & Realms

HiveMQ Enterprise Security Extension (ESE) add-on is central for security controls around the broker. If you are familiar with ESE, fundamental blocks to manage Authentication and Authorization within ESE are “Realms”. There are various realms for various controls. Please refer to HiveMQ documentation for in-depth understanding of realms.

Access control can be applied to the Broker (MQTT Client), Control Center (portal for monitoring and few administration tasks) and REST API (API to interact programmatically with the broker), and HiveMQ Broker components. The following table illustrates high-level applicability of Realm to broker components.

LDAP Realm can currently be leveraged for MQTT broker and Control Center but not yet for REST API. LDAP-based REST API should be underway.

The below setup and configuration tutorial is divided into three sections:

1. Set up Active Directory and tools needed.

2. Set up Database with permissions, roles and role-permission mapping.

3. Update HiveMQ configuration.

Section 1: Setting up Authentication using Active Directory

Step 1: Get ready to configure AD

Following are the steps to prepare AD groups and users, beginning with AD schema enablement.

Open command prompt with admin rights.

Enable Schema editing.

Open Microsoft Management Console by typing mmc on command prompt. 

Select “Add/Remove Snap-in...”

Add “Active Directory Users and Computers” and “Active Directory Schema” from Available snap-ins to Selected snap-ins as shown above.

This will open both “Active Directory Users and Computers” and “Active Directory Schema”, where we can administer users, groups, and attributes for setting authentication.

Step 2: Create Groups

At this particular point there should be a fair idea of what topics access are needed for given user groups at the organization level. You can refer to the HiveMQ Realm and SQL database structure documentation.

At a high level, there are two ways access control can be provisioned:

1. Create permissions and assign them directly to users.

2. Create permissions, create roles, assign permissions to roles, and then add these roles to users.

The second method is very flexible and easy to manage. This tutorial will set up this permission to role and assign role to users.

Permission — the heart of authorization — is covered in detail in “Section 2: Database Setup”. As users and groups are provisioned in AD, let us set them up.

Groups/Roles: Typically, Security groups in AD are logical ways to group users into specific roles. Following are the use cases for setting up permissions.

1. Devices from edge, which should access only specific topics and publish only. Say SiteAUser – only publish to topics (say PUB - OrgA/Sites/A/#), can send messages at any QoS. However, should not be allowed to subscribe to any topic.

2. Reporting user group, who can only read specific topics. Say – Ignition/Grafana user will be allowed to subscribe all sites (say SUB – OrgA/Sites/#) and not allowed to publish.

3. Application users, who need to subscribe to specific topics and can publish back a command. Say PUB + SUB to SiteA topics only (PUB+SUB – OrgA/Sites/A)

Thus, we need to design groups as security groups in AD.

To create a group, right-click, select New and Group, as shown below:

And provide a group name, as follows:

Note: Please consider the right naming conventions. What is given in the example is a sample.

Create the groups needed, reflecting the role segregation required in this setup.

Step 3: Create Users and Assign Groups

The next step is to create the users and assign the groups as needed.

Create a user and assign the right group as “Member Of”.

Click Next and finish.

Right-click on the created user and click Properties.

Select “Member Of” tab.

Type in the group/role you want to assign and click Check Names. If given group matches, it will get highlighted with underscore. Click OK.

Please note: All the groups, users, and assignment can also be automated through scripts. 

So, in the above steps “Roles” in the form of groups are created and assigned to users as “Member Of”. In Section 2, we will assign permissions to given roles.

 Section 2: Database Setup

To set up the database for authentication, please refer to SQL Realm.

Though I used Postgres, there is a long list of supported databases. Please refer to the documentation for the same.

When you set up SQL Realm with given scripts, you get the following tables in DB:

To set up authorizations in the DB, we need to configure data/details in “permission”, “role” and “role_permission” tables.

Permission table: As previously mentioned, “permissions” here mean topic-level permissions. Following is the permissions table structure:

In the table, provide the topic patterns in the 'topic' column, set the Boolean value to allow publishing and subscribing, and use the other fields for finer control.

Following is an example for a set of users as per “Step: 2 Create Groups” in “Section 1” of this tutorial.

ID – 1 – represent permission to allow all.

ID – 2 – allow Publish only on OrgA/Sites/A topics.

ID – 3 – allow Subscription on All sites (i.e OrgA/Sites/#).

ID – 4 – allow both Subscription and Publish on Site A topic (i.e. OrgA/Sites/A).

Role Table: Create a role reflecting DN of the group created in AD in “Step 3” in “Section 1”. Permission table structure is as follows:

One can look up the DN for any user or group using the “ADAM” tool in windows or ADSI Edit.

For connect-all group, DN is “CN=connect-all,DC=hivemqad,DC=eastus,DC=cloudapp,DC=azure,DC=com”

We need to insert this DN as a role in “role” table:

As the documentation specifies, we need to map permission to role in the “role_permissions” table.

The table structure is: 

The table in the DB will appear as follows for “connect-all” to Permission-ID-1:

Hopefully, you can fetch other roles and map permissions with this given example.

Section 3: Broker Configuration

Now that we configured AD and the DB, let us update the needed configurations in “hivemq-enterprise-security-extension”, which is present in the “extensions” folder in HiveMQ installation software.

Step 1: Add Database and LDAP realm in configurations

Please provide the right host, port, user_id, and password for the DB.

Provide DN of the AD service and RDN for users. Also provide the password for HiveMQ LDAP Realm to connect to AD.

Step 2: Define pipelines

Provide “ldap-authentication-manager” pointing to “ldap-realm” for authentication and “sql-authorization-manager” for “sql-realm”.

Bonus tip: Here is how to configure the HiveMQ ID mapping to AD user attribute. 

The “uid-attribute” in HiveMQ configuration points to the attribute of the user object in Active Directory. If you wish to use another attribute (say email-id or sAMAccount) one can update this field to point to another attribute. Following is the way to look at attributes in AD.

To view all the attributes of a user, select “Advanced Features” in Microsoft Management Console (MMC), right-click on user, and select Properties:

In the “Attribute Editor” one can look at all the attributes. I have used “uid” in this instance. If you want, you can use email-id or sAMAccount etc.

Summary

To avoid customization of AD schema to set up LDAP-based authentication and authorization, one can set up authentication through LDAP and authorization through a supported database such as a SQL database. This is easier to set up and easy to manage when dealing with millions of devices.

One of the challenges I encountered was configuring LDAP and SQL without making any changes to the AD schema. This specific combination required additional effort. I utilized network sniffing and database logs to gather the necessary details. Although it took some time, I was able to connect the dots and understand the logic behind the LDAP authentication and database authorization processes.

Another challenge I overcame was learning how to use various AD tools to set up my prototype. Some of the tools I used were LDIFDE, MMC, ADSM, and Apache LDAP studio. These tools helped me create, modify, query, and delete objects in the AD. I am sure there are smarter and more efficient ways to do this with scripts, but I was not familiar with them at the time.

I hope this tutorial helps those who are starting out on AD and want to use AD for authentication and SQL for authorization.

Disclaimer: The views expressed are personal and are not a reflection of Cognizant or any other organization.