MQTT Experts Answer Your Questions - May 2022

Watch the Webinar

Webinar Overview

Drawing upon the success of our previous ‘Ask Me Anything About MQTT' webinar and on popular demand, we started a webinar series under the same name. In this May 2022 edition, our MQTT experts answered questions around MQTT 3, MQTT 5, MQTT Sparkplug, OPC-UA, etc.

Jens Deters, Head of Professional Services at HiveMQ and Florian Raschbichler, Head of Support at HiveMQ, personally answered questions live during the webinar.

Feel free to ask questions on the HiveMQ Community Forum.

Key Takeaways

• Sparkplug is an open-source software specification for MQTT that defines how to use MQTT for industrial IoT.

• OPC UA is not directly compatible with MQTT due to different architectural patterns (request-response vs. pub/sub). OPC UA is not open source and is considered heavyweight, making it less suitable for IIoT.

• Recommended approaches for timestamps in MQTT include adding timestamp within payload (for MQTT 3.1.1) and using user properties (for MQTT 5).

• MQTT 5 adoption is increasing, especially for new projects where users have control over the implementation.

• Hyperscalers like AWS may not fully support MQTT standards due to their specific purposes and implementation choices.

• HiveMQ is 100% compliant with the MQTT 3.1, MQTT 3.1.1 and MQTT 5.0 specifications. For all MQTT-related settings in the specifications that are open to the broker implementation, HiveMQ uses sensible default values.

• MQTT is ideal for ingesting large amounts of sensor data, and its bi-directionality enables sending control messages to machines.

Transcript

Welcome and poll

Jayashree Hegde: 00:00:08.324 All right. Let's kick off. Good morning, good afternoon, good evening. Very warm welcome to our Ask Me Anything session. I am Jayshree Hegde. I'm super excited to have you all today. As always, we have MQTT experts from our team. Today we have on our panel, Florian Raschbichler, Head of Support. And we have Jens Deters, who is Head of Professional Services. Joined by Florian and Jens is Gaurav, the Director of Product Marketing, who will be moderating the Q&A. And before we kick off, a couple of housekeeping pointers. We'll be recording this session, and we will share the recording in a follow-up email. And then there are two polls running for this session, one right at the beginning, just so that we have an idea of your MQTT understanding. This will help our panelists craft their answers according to your MQTT understanding. And then there's the second poll right at the end of the session. Then I request you all to participate in all the polls. And without further ado, I will hand it over to Gaurav, and I will start the polls. So welcome, everyone. It's great to have you all. Gaurav, over to you.

Gaurav Suman: 00:01:32.345 Thank you. [crosstalk]. Yeah. So if you could put the poll up.

Jayashree Hegde: 00:01:38.261 Yeah.

Gaurav Suman: 00:01:46.983 So we're looking to get a sense of what your level of knowledge about MQTT is and your plans on using MQTT. We'll give you a couple of minutes here to pick your response.

[silence]

Jayashree Hegde: 00:02:44.647 I'll end the polls now. Thank you so much for participating.

Gaurav Suman: 00:02:49.365 Great. Thank you, Jayashree. Okay. Let's dive right in. I want to begin with this question around Sparkplug. And we might have an audience from different backgrounds here for everybody's benefit. Sparkplug has more to do with industrial IoT, so industrial IoT use cases essentially. So the question we have here is: They're looking for more information around the Sparkplug specification. And if those specifications are included in MQTT 5, or are they separated like with MQTT 3.1.1? This question is coming from David Wantigam, and that's all they're asking. Jens or Florian?

Jens Deters: 00:03:31.600 Well, I got this question in the way that Sparkplug is part of the certain specification. And yeah, well, it's not. The Sparkplug specification is about how to use MQTT in a way to achieve certain goals. So the focus of Sparkplug is the workflow, how to implement an asset or a state management for assets or devices. It defines some key principles like Report by Exception. It has a certain topic structure — describes a certain topic structure, and it's a specification on how to implement and format the payload. But the Sparkplug specification is on top of MQTT. So MQTT is a communication protocol, and Sparkplug is how to use it to achieve certain goals. And it's important. The current Sparkplug specification is making use of MQTT 3.1.1, not the magic stuff coming with MQTT 5 by now.

Florian Raschbichler: 00:05:05.746 Yeah. And maybe if you allow me to add something here based on what I read from the question, right? So you can use Sparkplug with both MQTT 3.1.1 as well as MQTT 5. That's what that means. If it's based on 3.1.1, that means MQTT 5 will cover everything 3.1.1 has, plus more. And that means, with both MQTT 5 as well as 3.1.1 clients, you can implement the Spark block specification for payloads, topics, message flows, everything that Ian sent.

Jens Deters: 00:05:41.119 Yes, and here's why. So the MQTT feature Sparkplug is using or depending on is quality of service zero and one (QoS 0 and 1). It makes heavy use of retained messages and Last Will and Testament. And you need a broker and also a client which implements this. And yeah, basically this is defined by MQTT 3.1.1. And yeah, that's why you have to use at least 3.1.1.

Is MQTT able to interact with an OPC UA server?

Gaurav Suman: 00:06:14.988 Makes sense. Thank you. And a resource I would love to point out here is a blog post series and a set of videos we have done late in 2020 around Sparkplug. We have the MQTT Sparkplug Essentials eBook. And thank you, Jayashree, for directing people to that resource. All right. Thank you for that answer, Jens and Florian. Next, we have, in fact, several questions on OPC UA. So there's great interest in OPC UA. And I'm sure there's some myth busting also we'll have to do here because we can see people referring to OPC UA as a protocol in some of these questions. So the first one I'll pick here is from Frey Patrick from Andreshauser. And Frey is asking: Is MQTT able to interact with an OPC UA server? And the example they have in mind is, say, an MQTT broker or a client publishing into an OPC UA node, which is part of an OPC UA deployment, and also the inverse. Can an OPCUA client publish and connect to an MQTT subscriber? I think some of the terminology may not be 100% accurate here, but largely they're talking about MQTT client and broker interacting with an OPC UA infrastructure. So let me start with you, Jens. What are your thoughts on this?

Jens Deters: 00:07:29.356 So basically, yes, it can interact. You can bridge between the two worlds. So OPC UA architecture and Pub/Sub architecture using, for instance, MQTT. But this needs some implementation. So you have to implement or purchase a gateway which does the bridging for you. So OPC UA, basically, depends on the — or the architecture is based on a request-response pattern. So there's a client server connection and relation. So all the data in the OPC UA environment is provided by a server, and clients are requesting data from the server. In the Pub/Sub environment, you will get notified if there is something for you. So it's a push mechanism. So it's a really different pattern and a really different architecture. So to use the best of both worlds, you have to have a gateway, right, or implement a gateway to handle the data flow. How to do this is specified by the OPC UA specification part 14. I think this is the Pub/Sub specification. But this is not about — there is a Pub/Sub extension for OPCUA. It's about how to deal with the OPC UA data format in terms of using a Pub/Sub protocol. The part 14 specification recommends using AMQP or MQTT as the Pub/Sub protocol here. So yeah, again, the OPC UA Pub/Sub is all about how to deal with the data format OPC UA describes.

Gaurav Suman: 00:09:43.982 Makes sense. Anything you want to add, Florian?

Florian Raschbichler: 00:09:49.340 Not really. I think I covered it all too well. My biased follow-up question to the question would be, "Why?" So certainly, from a messaging point of view, a Pub/Sub push communication is superior to a request-response. So I would want to go there. But of course, I understand in reality there's legacy systems, there are existing Brownfield deployments where you just don't have the choice of replacing everything. And for these cases, like Jens said, you can basically — maybe not get the best of both worlds, but bring both worlds together by either implementing this by yourself, or also as mentioned, there are some vendors out there that offer pre-built solutions to basically translate between request-response and Pub/Sub.

Jens Deters: 00:10:58.838 If you want to play with this — and you are familiar with Node-RED — there are really interesting nodes you can use to deal with OPC UA infrastructure and connections to bind it or transfer it into the MQTT world. So if you would like to get an impression how — a really easy impression because it's low code using Node-RED, this might be a good starting point for that to get a feeling how to deal with it and how to use it.

Can OPC UA be implemented in industrial IoT infrastructure?

Gaurav Suman: 00:11:40.545 Right. And any other sort of element to consider here, Jens, is around the fact that OPC UA — it appears like a closed club, right? There are only a specific number of vendors who continue to focus on this. Yeah.

Jens Deters: 00:11:58.242 If this question goes in the direction to implement an industrial IoT infrastructure, OPC UA is not the answer to industrial IoT. So to implement industrial IoT, you have to really go the hard way and to implement a very new architecture system in parallel. Industrial IoT needs to be — first of all, it needs to be open. It needs to be an open standard. It needs to make use of open standards, really open. So OPC UA is not open. It's a closed shop. It's a pay-to-play thing. So you have to be a member of the OPC Foundation. And then you are free to implement the OPC UA. I think it's way more than 1,000 pages, all the standards and specifications in OPC UA. But you only have access to this documentation when, yeah, being part of the OPC Foundation. So it's not open, which MQTT is, for instance. MQTT is a really open source protocol and open standard and specification.

Jens Deters: 00:13:25.742 In an IoT environment, one really fundamental target is to be lightweight, which OPC UA is not at all. It's HTTP [inaudible]. So heavyweight communication structure with client-server relation and huge data format specification. So in comparison with the payload format of MQTT and all the lightweight communication protocol, OPC UA is not lightweight at all. And you need a kind of a Report by Exception principle. So as a data consumer, you just want to be notified if there is new information for you. If a sensor value has not changed during the last hour — you don't want to be notified every 30 seconds that there is no change or with the same value. You just only want to be notified of a data change. And this can be achieved also with the approach with a Pub/Sub pattern. So just to notify all subscribers that there's new data by push and not forcing all clients permanently to pull the data. And OPC UA is not supporting this in a good way.

Gaurav Suman: 00:15:09.388 Makes sense.

Jens Deters: 00:15:10.261 That's why OPC UA cannot be used to achieve industrial IoT goals.

In case of requirement of timestamps, is there a recommended approach? 

Gaurav Suman: 00:15:20.203 Thank you. All right. So the next one here we have is around timestamps. And this question is by Svetozar Yolov. And Svetozar, I think he's also on the call. So let's read this question, get our initial take on it. And if Svetozar would prefer to do a follow-up, we will make sure that we unmute Svetozar. So what we are saying is that in case of the requirement of timestamps, is there a recommended approach? And the use cases in applications where DNP3 is applicable, can we use MQTT as a replacement, which is where timestamp becomes critical, and especially when the client is in a persistent session? So what are your thoughts on timestamp and keeping that integrity of timestamp?

Florian Raschbichler: 00:16:12.596 Yeah. Certainly. So first of all, what did you say? Did you say DNP3?

Gaurav Suman: 00:16:21.957 DNP3, yeah.

Florian Raschbichler: 00:16:23.699 So I'm not familiar with what that is.

Jens Deters: 00:16:29.192 It's a distributed network protocol, which is used for [inaudible] also.

Florian Raschbichler: 00:16:38.518 What's the requirement here? Because if you want to get the original timestamp from a message as it was sent by a sender, I would say immediately there's two ways to go about it, right? If you're using the more known but older version of MQTT, which is version 3.1.1 — but it's commonly referred to as MQTT version 3 — then you can add this timestamp within the payload. We've seen from some of our customers here, for example — what's very, very popular here is basically that you have something which is called a data envelope, right? So you have some type of key value-based data structure, such as JSON or more popular these days is something like Protobuf, which is a — yeah. You can save data by using a compressed structure like Protobuf. And then you have basically one key timestamp, and you have the timestamp value. And then you have a second key that's like payload, and then you put the actual payload there. And this way you can correlate with MQTT version three, the timestamp with the payload. With MQTT 5, it's even much, much, I would say, easier or more elegant, elegant ways of solving this by the introduction of the so-called user properties. Just a quick reminder for those here that might not be extremely familiar with the MQTT 5 specification, what the user properties are as a key value string pair that can be added to almost all MQTT packets. So very similar to what headers are in HTTP. And this is specifically designed for use cases like this, right? So you do not have to go into the payload of the published packet and put information there, but you can actually put an envelope around the whole published packet and then put the information there with the timestamp. And then of course, typically the Unix time of the time the message was sent.

Florian Raschbichler: 00:19:02.910 The publisher can set this, and then the broker will automatically forward this timestamp to the receiver. And the receiver can even read the timestamp without having to go into the published packet and actually unpacking the whole payload. This is the added benefit here between using a payload versus user properties. If you want to go super technical, the payload is technically a byte stream, whereas the user properties, they're already ready to read string pairs. It's already UTF-8.

Gaurav Suman: 00:19:39.030 Sounds good. Anything to add, Jens?

Jens Deters: 00:19:42.531 No. I mean, you should just avoid skipping the payload as the data layer and put any information into the user properties, because technically, that is possible, but yeah, should not be the right way.

Florian Raschbichler: 00:20:06.767 Yeah. That's correct. Does this answer your question or do you have a follow-up?

Svetozar Yolov: 00:20:14.350 Oh, yes. Thank you, sir. I'm asking this question because nowadays there are a lot of systems, especially in the water industry, which rely on this protocol, DNP3, which has some features. For example, something similar for persistent sessions which MQTT got. So my idea is to replace, in some of our projects, such a solution with MQTT. But in this solution where you have to send data information with timestamp of the sender, that is why I'm persistent — not persistent, but looking for a solution using MQTT and making the timestamp of the publisher to be available for the application, which is actually subscriber. Thank you.

Jens Deters: 00:21:18.675 What you can do, as a proposal for a specified payload format, also, depending on the timestamp, you might have a look at the Sparkplug specification because, yeah, then you have a specified way of how to format the payload. So basically it's a JSON object, and then you have a defined structure for your payload.

Svetozar Yolov: 00:21:51.905 I see. Well, as far as I know, I'm still not dealing with Sparkplug B. But as far as I know, it's absolutely transparent in point of view of MQTT 3.1, for example, because the whole functionality is in the payload. So any broker doesn't care if there is Sparkplug B or not, right?

Jens Deters: 00:22:20.325 Yeah.

Florian Raschbichler: 00:22:24.595 He's correct.

Jens Deters: 00:22:26.873 Sorry?

Florian Raschbichler: 00:22:27.301 No, he's correct, right? The broker doesn't care about the page.

Jens Deters: 00:22:30.870 Yeah, he is correct. Yeah. It's basically just MQTT traffic in a certain way in terms of the topic structure and the payload format and so on. But it is normal MQTT traffic. You don't have to implement all the stuff the Sparkplug specification defines. In your case, you can just have a look at the payload format and implement that despite what else is defined in the Sparkplug specification.

Svetozar Yolov: 00:23:18.256 Okay. Thank you very much for your explanation. I'll get myself familiar with Spark with more details with the documentation. We'll see. Thank you very much, sir.

What’s your view on the adoption around MQTT 5?

Gaurav Suman: 00:23:31.846 Thank you so much. Yeah. Keep in touch. Keep sharing with us how it's going. And if there's anything we could help with, please keep in touch with us. So I want to pick up this question from Steve Urbansky. And Steve is looking for our comments on the adoption around MQTT 5, and he's also pointing out to the fact that, say, AWS is — not even AWS is supporting MQTT 5. Florian, how about I come to you first on adoption and then why the hyperscalers are not going there yet?

Florian Raschbichler: 00:24:05.244 Yeah, absolutely. Happy to take that question, and happy to also report from the field from out there, right, because in my position as Head of Support at HiveMQ, I'm fortunate enough to talk directly to the people that are implementing IoT MQTT use cases in production and have been doing so for many years now. And I have to say, actually, that now in 2022, which is pretty much probably — MQTT 5 has been out now for 26 months, I believe. So a little bit more than two years. So very, very young in terms of if you speak of a communications protocol years, that's very young, very fresh. And the adoption that we are seeing is actually increasing a lot. So we see many companies, whenever they get into a state where they build an application themselves that consumes the messages, they tend to go with MQTT 5 because if you look into the ecosystem of client libraries out there, there are many available that use MQTT 5. And also, if you look at the ecosystem of MQTT brokers — and I'm talking about MQTT brokers — then also most of them support MQTT 5 at this point. I have to agree with Steve that if you look at purely industry and if you look at machines, PLCs, pre-built devices, pre-built software, such as historians, for example, they typically are not yet there at the use of MQTT 5. But from the trends that we are seeing, I have an absolutely — I have no doubt in my mind that in a few years MQTT 5 will be absolutely dominant for the user properties feature alone. And there's other features that really, really make it, especially in the IoT world, very useful, such as the Topic Alias, just to name one example. A shameless plug for myself. So similar to the Sparkplug Essentials that Gaurav mentioned, we also have MQTT 5 Essentials up on the HiveMQ blog with an excellent speaker explaining to you all the MQTT 5 features in a seven-part series.

Jens Deters: 00:26:32.459 What's his name?

Florian Raschbichler: 00:26:34.001 I forgot. I think he's a support guy. And coming back to the mention of AWS not supporting MQTT 5, so I'm assuming you're referencing IoT Core. I think it's called AWS IoT Core, which is their somewhat MQTT-compliant messaging solution. But if you want to look at that, it's also not MQTT 3.1.1 compliant. So AWS, assuming built this for a very particular purpose, and it's a different purpose to what a pure MQTT broker delivers. And if you're looking at the pure MQTT brokers — they all support MQTT 5 these days. And again, many of the hyperscaler solutions are actually not even MQTT 3.1.1 compliant because there's some hard-to-implement stuff there that they'd rather skip and not implement. And that's, yeah, my take on that. I hope that helped you in some way, Steve. Or if you want to jump on the call, we can also have a discussion about it if you have follow-up questions.

Gaurav Suman: 00:27:56.522 So Jayashree, if you could give Steve the right to unmute himself if he wants, and if there's any thoughts they still have. Steve Urabanski — that's the person. Are you able to, Jayashree? Are you able to give them the rights?

Jayashree Hegde: 00:28:21.282 Yes. Sorry.

Florian Raschbichler: 00:28:26.762 He seems to have some connectivity issues.

Gaurav Suman: 00:28:28.313 Yeah. It looks like Jayashree is having some connectivity challenges here. Okay. We'll keep going, and then we'll come back. We'll come back when Jayashree is back. Okay. I think, Steve, you have the option to unmute yourself and ask a follow-up if you like.

Steve Urabanski: 00:28:45.268 Oh, hi. Can you hear me?

Florian Raschbichler: 00:28:46.493 Yes, we can.

Steve Urabanski: 00:28:48.259 Okay. Great. No. Thanks for taking the question. And you might have received that question before. So this is my first time on one of these chats, so I appreciate you addressing that question. MQTT 5 is something we've been looking into and just monitoring it, trying to decide if that is something that we should pursue. AWS does provide frameworks to add MQTT 5 capabilities, even though they really only support 3.1.1. So we're just trying to monitor that and figure out what the benefits are for going to MQTT 5 and if that's something that we should be pursuing. So just kind of figuring that out on our side and just kind of wanted to gauge where the industry was going and how new it was, and if people are still trying to digest the new standard, or if they're just early adopters right now picking it up.

Florian Raschbichler: 00:30:07.056 Okay. Yeah, happy to go a little bit more deeply. And thank you for providing the context, right? And it makes sense. But my honest advice to you is, if you have the option, if you're building whatever you're using yourself and you're not reliant on a PLC or a piece of gateway in your system that you don't have control over, there is no downside for going to MQTT 5 or MQTT 3.1.1. It's actually improved even the use for constrained devices as well as very powerful devices. And it's definitely, definitely the future. And then also, once you implement something with MQTT 5, you definitely always have the option to go back to 3 with the implementation that you built around 5, right? But if you choose a solution that's not at 5 yet, you can't easily go up, but you can always very easily go back. And professional enterprise MQTT brokers, such as HiveMQ, of course, also allow you a way of implementing a heterogeneous ecosystem where you have some MQTT 3 clients and some MQTT 5 clients. And you can utilize the cool features based on whether your publisher or subscriber is MQTT 5 or not. So yeah, I would definitely recommend going with MQTT 5 if you have that option. And I'm also very convinced that the industry will adopt it in due time. Again, two and a half years is still pretty young in the space we're talking about.

Steve Urabanski: 00:31:54.917 Yeah. Okay. That's encouraging. Thanks for your comments on that, and thanks for taking the question.

Florian Raschbichler: 00:32:01.766 Thank you for the question.

Jens Deters: 00:32:03.389 And do you rely on AWS? So you basically intend to use AWS services, and?

Steve Urabanski: 00:32:13.490 Yeah. We've used a number before, you know, but we tend to stick with mainstream web services for our products. And so we've used IBM, AWS. And so it looks like AWS. Like I said before, it looks like they do provide some guidance on how to use or how to incorporate some MQTT 5 capabilities. But we really haven't dug into it in any detail.

Jens Deters: 00:32:53.789 As an alternative, you can think about hosting your own MQTT broker supporting all that stuff, and then connect it to the cloud or the hyperscaler you intend to use.

Steve Urabanski: 00:33:08.245 So that would be hosting our own broker on like an AWS service?

Jens Deters: 00:33:15.701 Yeah. Yeah. For instance, when using HiveMQ Broker, you can make use of extensions. We have a default available extension, which is called Kafka Extension. And with the help of this extension, you are able to connect, for instance, to the Azure Event Hub services or the event services of AWS. And that's the entry point into the cloud world there. And you have a fully compatible and implemented broker available.

Steve Urabanski: 00:34:07.364 Yeah. That's definitely something to consider. We do have a number of legacy products out there that strictly use 3.1.1. But I'm assuming it would be backwards compatible so that those devices would still work with a MQTT 5 broker.

Jens Deters: 00:34:28.324 Yeah. When using HiveMQ, you have the support of both in parallel. You can use MQTT 3 and 5 clients in parallel, because internally, in the broker, everything is mapped to MQTT 5, and the broker will handle that.

Florian Raschbichler: 00:34:48.188 Yeah. And I tried to mention this, right? You are even capable of using some of the features that MQTT 5 have with your MQTT 3 clients when you use HiveMQ as your broker. So just one example, MQTT 3 by default does not support something like a session expiry. But if you're using HiveMQ where internally we treat everything as MQTT 5, you can add a global session expiry that would even apply for MQTT 3 clients. So this is just one example showing you, if it's possible without violating the protocol towards the MQTT 3 client, we are capable of using MQTT 5 features even with MQTT 3 clients when using HiveMQ as a broker.

Jens Deters: 00:35:37.120 Yeah. The same goes for shared subscriber groups, which is part of MQTT 5, but it was also supported by broker for MQTT 3.1.1.

Gaurav Suman: 00:35:55.002 So Steve, we also did a short webinar on ingesting IoT data into the cloud. And with the cloud, we really are trying to decouple from any one hyperscaler platform. So if you're okay, I [inaudible], or I could send you a follow-up email with a link to that and you can see all of this visualized and some walkthroughs on the three or four factors you want to keep in mind or other can begin to evaluate as you're thinking through what comes next for your evolution here.

Steve Urabanski: 00:36:23.928 Yeah. No. That would be good if you could send that to me. That would be great.

How can IoT systems be used in the predictive maintenance use case? 

Gaurav Suman: 00:36:28.372 We will do. Thank you, Steve. Appreciate the question. And thank you, Jens and Florian, for your deep dive here on MQTT 5. And all that comes with that. Thank you for that. So I'm going back to the list of questions we have here, and this one is from Sravana Kumar S, who is on the call. So I'm going to read the question from Sravana Kumar. And then if they want to ask a follow-up, we'll give them the opportunity. So they are looking at predictive maintenance. And this question is with regard to the railway industry. I think that's the domain they are in, signaling and transportation, judging by the name of the business they registered through. And they're asking how can IoT systems be used in the predictive maintenance use case and if it can be related to logistics or the railway industry. Jens? Florian?

Florian Raschbichler: 00:37:23.828 I mean, so I can certainly talk about the — I mean, predictive maintenance is one of the more typical use cases that we always [inaudible] and also that some of our key customers are using. And IoT technologies or with MQTT in particular comes into play here because it's such a great communication protocol, ingestion protocol really, right? So MQTT with all its characteristics, with the way it allows the broker as a single entrypoint for the connectivity, with being able to handle high latencies or bad connectivity really well, and still ensuring guaranteed message delivery — it's really well-built to basically connect anything even with a bad internet connection to the broker. And then we get into the conversation that Jens also just alluded to when talking about cloud providers. From the MQTT broker, which is the ingestion of the data, from there, you can put it to wherever you have more resources to do your predictive maintenance, right, because to do predictive maintenance, you need a machine learning model, you need compute resources, ideally a lot of them, right? So this is why predictive maintenance on the edge is sometimes not feasible.

Florian Raschbichler: 00:39:00.338 But if you use MQTT to get the data into the cloud, then you are into the cloud — using technologies like Kafka works really well, already mentioned also by Jens as the intermediate step to get the data from MQTT to Kafka. And then from Kafka, you can put it into whatever machine learning technology you want to be using. And then going one step further, the cool thing about MQTT is it's really bidirectional. So it's an ingestion technology, or it works well as an ingestion technology, yes. But in case of you finding something in your predictive maintenance, and you're being like, "Hey, I don't know. This machine here is five degrees celsius warmer than all the other machines that have produced similar amounts of products. Let's please take it offline to be sure and check." And you can even send this message then back to the machine via MQTT, since it's bi-directional and basically controls the machines as well. And yeah, so its bi-directional characteristics as well as its functioning so well as an ingestion layer for all kinds of data into the cloud — that's the reason why I think MQTT is really well-suited for any predictive maintenance in general.

Jens Deters: 00:40:38.662 And in addition to that, if I'm thinking of the railway industry, so I assume it's railways, and it's a huge distributed system with all the signaling stuff and sensoring and maybe also there are sensors on trains. In this kind of environment, MQTT is the solution for that because — as Florian already mentioned — it's built to be very lightweight, and you have to deal with uncertain connection. And so this kind of environment and ecosystem, it is one of the best protocols you can use here. So to collect all the data you need. And predictive maintenance basically is based on a huge amount of data to detect anomalies or trends or something. So it's not based on 10 or 1,000 or 100,000 values. We're talking about millions and petabytes of storage and the kind of time series database for a long period of time, so for one, two, three years. So you need reliable data ingestion. And you also need to make use of all the hyperscaling stuff the cloud providers do provide. So huge amount of storage and certain algorithms to some machine learning or AI, yeah, to get the results you would like to see or also to get the results you won't expect because using the machine learning will also lead to detecting patterns that engineers won't have had in mind. So it's really interesting which kind of results you get to let machine learning algorithms analyze your data.

Since MQTT is a lightweight protocol, and we are transmitting payload, is there any restriction on the total number of bytes we transmit?

Gaurav Suman: 00:43:18.604 Thank you. Thank you, Jens. So Sravana Kumar has one more question. Let me ask that, and then we can give them the opportunity to ask a follow-up. And they are talking about OPC UA and MQTT, which I think is a question we answered a little while back. And then the other question they have is around nominal data-carrying capacity or transmission capacity. I assume they're talking about how fast the throughput can be around data analytics. But how would we do this? We'll give Sravan Kumar an opportunity to share with us more, just any context that they can give us. So Sravan Kumar, you have the floor now.

Sravan Kumar: 00:43:59.915 Hi. Hi. This is Sravan.

Gaurav Suman: 00:44:04.936 Hi there. So can you just have any more context you can provide around throughput? Yes?

Sravan Kumar: 00:44:11.167 Yeah. Since it is saying as a lightweight protocol, when we are transmitting as a payload, is there any restriction in the total number of bytes we are transmitting?

Jens Deters: 00:44:24.269 You mean the size of the payload?

Sravan Kumar: 00:44:26.367 Yeah.

Jens Deters: 00:44:28.365 So yeah, according to the specification, you are free to add 265 MB of payload per message.

Florian Raschbichler: 00:44:41.826 Sorry to be [inaudible] here, Jens, but 255 megabytes is the total package size. So that leaves almost that for the payload. But yeah, so very large payloads, very large payloads that are, from what we've seen, always larger than what is used.

Jens Deters: 00:45:02.370 You can subtract some bytes for the header and so on. But yeah, question is if this is really a good way to create such huge messages. Why are you asking? So which message size do you expect in your system?

Sravan Kumar: 00:45:36.490 No. Our Indian Railways customer — they are asking for what will be the payload message size. So when we are using, for this predictive maintenance system, for the maximum number of payload size.

Florian Raschbichler: 00:45:58.108 Yeah. So like we said, it's close to 255 MB for a single payload. Again, the question is, does it really make sense to send such large payloads? Again, when referencing to [inaudible] — sorry, MQTT as a lightweight protocol, it references that the overhead on top of the payload is comparatively low compared to other protocols. Also, we are having the fact of a standing TCP connection. So there's not a handshake necessary every time we send a message. It only happens once and then we can continuously send messages in a stream without these additional overheads, right?

Florian Raschbichler: 00:46:50.694 So this typically means that the customers decide to go with smaller payload sizes to take advantage of the lightweight nature of the protocol. Also, MQTT not being designed for large transfers of large payloads, while it's possible, you have to consider that there is no resume capabilities for canceled transmissions. So if you send a 200 MB message and after 180 MB, you lose the connection, you'll have to start fresh. So if you do look to transfer large files via MQTT, what the typical approach is to send a link to an FTP or HTTP file that then can be downloaded with resume capabilities by the receiver of the message. Thank you.

Jens Deters: 00:47:51.197 And Florian, isn't there an element of such a large payload if the connection is disrupted, and then you have a degree of authentication also getting impacted, right? The client would have to then register. How does that element work around bringing those devices online? And I'm assuming Sravan Kumar is talking about — he mentioned Indian Railways, which would have, I can imagine, tens of thousands of endpoints perhaps. So this disconnection of a large payload, and then you are — obviously, session resumption is a fact authentication that has to happen again. Any thoughts?

Florian Raschbichler: 00:48:30.126 So you were a little bit choppy there at the end, but I think I got the question. So yes, a client would need to re-authenticate, reauthorize if they lose the connectivity, pretty much regardless of the size of the payload or if they even were sending up a payload. So looking at the time, I'm not sure if we can go into all of that because that could fill a whole hour-long session probably on its own. So what I can tell you from extensive experience with other customers that have hundreds of thousands and millions, even tens of millions of connected devices that utilize authentication, authorization with the client-broker approach, so you can really authorize and authenticate clients really quickly. And a lot of them simultaneously compare to what you know from a peer-to-peer type of communication approach. I'm not sure if I actually covered your question, but.

Gaurav Suman: 00:49:53.453 No. You did. And Florian, again, I think time is a factor here. So in fact, perhaps a good opportunity for us to mention that May 25th is when we are doing a webinar on IoT security basics, which is where we'll cover this in much, much better detail. And Florian, hopefully you'll be able to join me also so we can answer some real-life scenarios. So Sravan Kumar and anybody else who's on here, please do join us for that. And we'll be able to talk about not just the issues we covered today, but also at scale, how the security element comes into play and at IoT scale, how other such complications need to be factored in. Jens, do you have any thoughts?

Jens Deters: 00:50:32.137 Yeah. I just would like to mention in terms of the large messages. For sure, you have a kind of delivery service. So in MQTT, you have the quality of service one and two (Qos 1 and 2), which grants a kind of delivery. But in terms of large messages and large payloads, for instance, the client loses the broker connection and all these messages are stored in a session — as long as the client does not reconnect and does receive the messages, the messages are stored on the broker. And this might lead very quickly into memory problems or storage issues.

Closing words

Gaurav Suman: 00:51:29.552 Thank you. Thank you, Jens. So that's the end of the questions which were submitted to us. And I don't think there's any other pending questions here in the Q&A part. Jayshree, can you just confirm there's nothing else that we haven't been able to get to?

Jayashree Hegde: 00:51:42.340 No, I think we have covered everything. I'll quickly run the second polls if that's fine.

Gaurav Suman: 00:51:47.920 Please go ahead.

Jayashree Hegde: 00:51:49.689 Cool. Request all attendees to participate. If you have any further questions, please reach out to us. We are here to help. If you need any links to resources, just email me. I can help you out. So before we close the session, any thoughts? Gaurav, Jens, Florian?

Gaurav Suman: 00:52:37.225 I'm good. Good. I'm glad there were some really interesting questions, very, very broad conversations. Florian or Jens, any final thoughts from you?

Jens Deters: 00:52:42.841 Yes. During the last sessions and also during the last conferences I attended, I really see a strong demand or interest in Sparkplug, but also in how to deal with the current OPC UA infrastructures and architectures and how to do the next step toward more digitalization in terms of industrial IoT. So that's, yeah, really interesting for me that the demand on this is really increasing, especially in America or outside of Germany, because I still got the impression that the German industry is so proud about its grade of automation that they should not miss to do the necessary next steps.

Gaurav Suman: 00:53:43.179 Great. Thank you for sharing.

Florian Raschbichler: 00:53:44.425 Yeah. And I'm the MQTT 5 guy. I always advocate for MQTT 5. So having someone like Steve come in here and ask about it fills my heart with joy because I can only tell everybody you should really move to MQTT 5 if you have the opportunity. The feature set might seem not as severe, but if you really look into it, it can save you money. So go for it.

Gaurav Suman: 00:54:14.046 Thank you so much. Back to you, Jayshree.

Jayashree Hegde: 00:54:18.168 I think we have covered everything. Thank you so much for participating in the polls. We will see you next time and also to register for our next webinar, which is on May 25th. It's on IoT security. So yep. See you all next time. Thank you so much.

Gaurav Suman: 00:54:38.100 Thanks everybody.

Jayashree Hegde: 00:54:39.074 Thanks, Gaurav. [crosstalk].

Florian Raschbichler: 00:54:39.869 All right. Thank you, [inaudible].

Jayashree Hegde: 00:54:41.653 See you. Bye-bye.

Florian Raschbichler: 00:54:43.015 Bye-bye.