The Origin of MQTT

Welcome to the first blog in a series on the history of MQTT written by Ian Craggs, a technology thought leader currently working on the standardization of MQTT-SN at OASIS, and project lead of Eclipse Paho and the maintainer of several components there, as well as contributing to Eclipse Sparkplug. Ian’s four part series gives a behind-the-scenes look at the creation of MQTT and its evolution to the beloved protocol we have today.

MQTT is the protocol for industrial device communication that has existed since 1999. Its name has changed several times during that time, usually as an abbreviation for a longer phrase. During standardization at OASIS, it was agreed that the name MQTT would not represent anything else, but itself. Unfortunately, the standardization committee was then informed that according to OASIS convention, the name of the committee could not be changed once it had been created, so that historical name lives on.

Many people have worked on MQTT over the years, contributing to its success. This is an attempt to condense the story of MQTT and acknowledge those people and contributions.

Pre-standardization (1999 - 2012)

The polling approach to collecting data from industrial devices was designed to avoid packet collisions when more than one device on the same network tried to send data at the same time. Each device manufacturer wanted to beat the competition to market, tolerating no delay to participate in or wait for standards to appear, so each developed its own method.

Polling relies on the server to regularly contact all connected devices whether or not the data they held has changed. A server can be overloaded with communications to a large number of devices, even if their data changes infrequently.

Arlen Nipper (then of Arcom Control Systems) and Andy Stanford-Clark of IBM drew up the first version of MQTT at the start of 1999. It was then known as the “Argo Lightweight On The Wire Protocol”, Argo being an IBM product codename. Its primary aim was to move away from the “legacy of literally hundreds of proprietary poll/response protocols” that existed at the time. It was the bare bones of what was to become MQTT, without DISCONNECT, UNSUBSCRIBE or PING messages. 

Version 1 didn’t last long. Arlen and Andy defined version 2 later that year, changing the name to “MQ Integrator Pervasive Device Protocol'' (MQIpdp). The changes incorporated into version 2 made it look very like the MQTT 3.1.1 that we know today, with will messages, keepalive processing and the ability to unsubscribe being added for the first time. 

Version 3 followed the next year, 2000, with minor changes and clarifications that were to remain stable for the next ten years until version 3.1 came out in 2010. Those ten years saw the adoption of MQTT in Arcom and IBM in various projects including:

• 2000: Arcom Control Systems “Director” - MQTT enabled industrial network gateway

• 2002: SCADA Mousetrap (Andy SC)

• 2004: IBM Microbroker - a Java MQTT broker implementation 

• 2005: UIB - a smart phone app to interact with MQTT data (Bharat Bedi)

• 2006: MQTT.org Launched

  • IA92 Java Client

  • IA93 C reference implementation

  • Websphere MQ

  • MQ Message and Event Broker

  • Websphere Business Integration Message Broker

  • CPAN Websphere::MQTT::Client

  • Combichem (Dr. Jeremy Frey)

• 2007: RSMB - a minimalist MQTT broker in C (Ian Craggs)

• 2007: Home power and water use monitoring (Andy SC)

• 2009: Isle of Wight Redjet Ferry status on Twitter (Andy SC)

• 2009: the house that Tweets (Andy SC)

• 2010: the open source Mosquitto project (Roger Light)

• 2010: IBM and Eurotech published MQTT 3.1, which made the following changes:

  • Username and password could now be sent with a CONNECT packet

  • New return codes on CONNACK packets, for security problems

  • Clarification that clients are not informed of un-authorized PUBLISH or SUBSCRIBE commands, and that the normal MQTT flow should complete even though the command has not been performed

  • Strings now supported full UTF-8, instead of just the US-ASCII subset

• 2011: use in Facebook messenger

• 2011: the open source Eclipse Paho project

• 2012: the Chale community project

• 2012: IBM MessageSight

• 2012: Mosquitto joins the Eclipse Foundation

By this time, enough use was being made of MQTT that standardization became a top priority for the community to ensure its future success.