Debunking Common MQTT QoS Misconceptions

In the modern world of IoT, understanding the intricacies of MQTT's Quality of Service (QoS) is crucial for designing robust and efficient systems. As a solution architect, industrial automation engineer, OT engineer, or IT Systems Architect, you may have encountered common misconceptions about MQTT QoS, particularly regarding message retransmission and QoS downgrading. To avoid these pitfalls, it is essential to familiarize yourself with the MQTT specifications and implement best practices, such as enabling persistent sessions and carefully selecting QoS levels based on your application's requirements. This blog post aims to debunk two common misconceptions and provide clarity on how MQTT QoS truly operates, empowering you to make informed decisions in your IoT architectures.

Common Misconception 1: Message Retransmission

First of all it is very important to understand: There is no timeout for message retransmission in MQTT QoS. Message retransmission only happens on client reconnection!

“After the timeout expires, the sender retransmits the original message and resets the timer, waiting again for the PUBACK (QoS 1+2) and PUBREC, PUBREL, PUBCOMP (QoS 2). This process continues until the awaited acknowledgment is received, ensuring that the message is delivered, even though duplicates might occur.”

This is a very widespread misunderstanding of how QoS with MQTT works, and this is not right at all!

Following the MQTT specification there is no timeout on missing QoS 1/2 acknowledgment control packets (PUBACK, PUBREC, PUBREL, PUBCOMP) and no retransmission in an ongoing client session. 

Again, retransmission of unacknowledged PUBLISH packets is only required by client or broker on client reconnects (MQTT v3.1.1+5,.0). MQTT v5.0 is even much stricter by saying: Clients and Servers MUST NOT resend messages at any other time than on reconnects.

As Stated in MQTT v3.1.1 Specification

The MQTT v3.1.1 about retransmission of messages:

4.4 Message delivery retry

When a Client reconnects with CleanSession set to 0, both the Client and Server MUST re-send any unacknowledged PUBLISH Packets (where QoS > 0) and PUBREL Packets using their original Packet Identifiers [MQTT-4.4.0-1]. This is the only circumstance where a Client or Server is REQUIRED to redeliver messages.

Non normative comment

Historically retransmission of Control Packets was required to overcome data loss on some older TCP networks. This might remain a concern where MQTT 3.1.1 implementations are to be deployed in such environments.

As Stated in MQTT v5.0 Specification

The MQTT v5.0 about retransmission of messages:

4.4 Message delivery retry

When a Client reconnects with Clean Start set to 0 and a session is present, both the Client and Server MUST resend any unacknowledged PUBLISH packets (where QoS > 0) and PUBREL packets using their original Packet Identifiers. This is the only circumstance where a Client or Server is REQUIRED to resend messages. Clients and Servers MUST NOT resend messages at any other time [MQTT-4.4.0-1].

If PUBACK or PUBREC is received containing a Reason Code of 0x80 or greater the corresponding PUBLISH packet is treated as acknowledged and MUST NOT be retransmitted [MQTT-4.4.0-2].

Please note: A MQTT broker will always initiate a session for every client, and this is how it manages the state in MQTT. MQTT is stateful because it requires the broker to maintain the client's session information, which includes the state of subscriptions, message queues, and other session-related data. However a persistent session is crucial in order to queue undelivered Qo1 + 2 messages, to be retransmitted on reconnect:

The client explicitly requests a persistent session (MQTT v3.1.1: cleanSession=false, MQTT v5.0: cleanStart=false + sessionExpiry>0): The broker retains both unacknowledged messages and the client’s subscription state. When the client reconnects, all undelivered messages (QoS 1 and 2) are re-transmitted, and the session picks up where it left off.

The client explicitly requests a non-persistent session:

MQTT v3.1.1: cleanSession=true, the broker does not retain the session after disconnection, meaning previous subscriptions, queued messages (QoS 1 and 2), and other session state are discarded.

MQTT v5.0: cleanStart=true + sessionExpiry=0, with cleanStart=true, a new session is initiated with no previous state, and sessionExpiry=0 ensures that the session is deleted immediately when the client disconnects. In this case, undelivered QoS 1 and 2 messages are discarded.

In summary – unlike TCP, where retransmission is driven by timeouts at the transport layer, MQTT handles retransmissions upon the re-establishment of a connection. The session persistence of MQTT plays an important role in this: session persistence must be enabled, in order to store unacknowledged messages (QoS 1 and QoS 2) by both the client and broker, and retransmissions occur automatically when the client reconnects. QoS 0 doesn’t support retransmission or queuing of undelivered messages at all.

Common Misconception 2: Downgrade of QoS

There are a few common misconceptions around downgrading MQTT QoS levels, such as:

1. QoS is end-to-end between publisher and subscriber.

2. Higher QoS always means better reliability.

3. QoS levels can be upgraded by the broker.

4. QoS 0 messages are unreliable.

5. Subscribers always receive messages at their requested QoS level.

Let’s dive in to debunk the downgrade of QoS. MQTT is an asynchronous one-to-many protocol. Publishers, as well as subscribers, are free to choose their QoS level. It is entirely possible to have a number of subscribers with different Quality of Service levels. The difference in QoS levels between the publisher and the subscriber provides flexibility in MQTT communication. It allows the sender to define the reliability of message delivery to the broker independently of how the message is ultimately delivered to the receiver. This is particularly useful in scenarios where different devices or applications have varying reliability and bandwidth requirements.

The MQTT broker plays a crucial role in managing these different QoS levels. Even if the message is published with a higher QoS, the broker will downgrade it as necessary to match the QoS level granted to each subscriber. Conversely, if the message is published with a lower QoS, the broker cannot upgrade it beyond that level, even if a subscriber requests a higher QoS.

No Per-Subscriber QoS for Publishers

Given each subscriber to a particular topic can specify its desired QoS level independently when it subscribes to the topic:

• Subscriber A subscribes with QoS 0

• Subscriber B subscribes with QoS 1

• Subscriber C subscribes with QoS 2

PUBLISH QoS 2 Messages

Given a Publisher sends a message with a QoS of 2:

• Subscriber A will receive the message with QoS 0

• Subscriber B will receive the message with QoS 1

• Subscriber C will receive the message with QoS 2

PUBLISH QoS 1 Messages

Given a Publisher sends a message with a QoS of 1:

• Subscriber A will receive the message with QoS 0

• Subscriber B will receive the message with QoS 1

• Subscriber C will receive the message with QoS 1

PUBLISH QoS 0 Messages

Given a Publisher sends a message with a QoS of 0:

• Subscriber A will receive the message with QoS 0

• Subscriber B will receive the message with QoS 0

• Subscriber C will receive the message with QoS 0

Summary

Conclusion

Understanding the nuances of MQTT QoS is key to building reliable, high-performance systems. By clearing up common misconceptions around message retransmission and QoS downgrading, we’ve shed light on how MQTT’s QoS mechanisms truly work. With this knowledge, you’re now better equipped to design IoT solutions that fully harness MQTT’s flexibility and reliability, ensuring seamless communication between devices and applications across your IoT ecosystems.