MQTT

When a program wants to send and receive data to and from another program, it can do it in two ways: one, send the data directly to the receiving program, and two, send the data to an intermediary program, which will then forward it to the receiving program. In computing solutions, the first approach is referred to as a request-response architecture, while the second approach is referred to as a messaging architecture.

The messaging architecture uses an asynchronous communication mechanism. The sending program hands off some data (referred to as a message) to the intermediary program (referred to as the message broker) and continue its processing without awaiting a response from the message broker. The receiving program receives the message from the message broker whenever it is ready, which could be immediately or anytime later. The receiving program may send a response or acknowledgment to the sending program in the same asynchronous manner through the message broker.

This architectural pattern minimizes the mutual awareness that applications need to have of each other (in terms of technology, design, data formats, etc.) to be able to exchange messages. This approach leads to a decoupled architecture. It is helpful and necessary when diverse applications need to communicate with each other.

This decoupling can have two dimensions: Space, where the sending and receiving programs do not need to know deployment or configuration details (i.e., server IP or port). They only need to know details about the broker and the time of sending and receiving programs, where it is not required to run parallelly. As you can imagine, asynchronous communication is considerably helpful in IoT solutions since sensor events may occur at random time intervals. The processor does not need to listen to an event frequently and may check with the message broker every once in a while using the application's criticality frequency. Also, the sensor can relay the data to the message broker and continue to read more data without waiting for an acknowledgment where data will be processed as long as it knows the message broker will be continued to receive and will deliver it to the processor eventually.

MQTT (short for Message Queue Telemetry Transport) is the preferred standard protocol for messaging-based communication in an IoT solution. It is an extremely lightweight protocol compared to other messaging protocols used in traditional enterprise applications, and therefore it is ideal for connecting remote devices with low-powered processors, simple programming, and low network bandwidth. MQTT, a lightweight communication protocol specifically designed to tolerate intermittent connections, minimize the code footprint on devices and reduce network bandwidth requirements.

MQTT has the following characteristics that make it useful for IoT solutions:

• Lightweight and Efficient MQTT client applications are tiny and require minimal resources so they can be used on small microcontrollers. MQTT message headers are small to optimize network bandwidth.
• Reliable Message Delivery Reliability of message delivery is important for many IoT use cases. That is why MQTT has 3 defined quality of service levels: 0 - at most once, 1- at least once, 2 - exactly once
• Support for Unreliable Networks Many IoT devices connect over unreliable cellular networks. MQTT’s support for persistent sessions reduces the time to reconnect the client with the broker.
• Security Enabled MQTT makes it easy to encrypt messages using TLS and authenticate clients using modern authentication protocols, such as OAuth.

Several message broker solutions implement this protocol. You can install them on your local machines, or in some cases, use them directly on the cloud without requiring any local installation.

There are two design patterns that messaging systems follow: Message Queues and Publish-Subscribe.

• Message Queues

This message queue pattern is designed for scenarios like a task list or work queue where each message stored in it must be read and processed only once by any one consumer. It is sometimes referred to as point-to-point messaging. Generally, in this pattern, the consumers are multiple instances of the same application that do the same processing. Let's take the example of an online image processing service. Multiple users will upload images to be processed (the messages in this case), at the same time using their web browsers (the producers in this case). The image processing service needs to process these uploaded images (resize, change colors, convert formats, etc.), which could take some time.

Instead of making a client (the browser and the user) wait for the processing to complete, images are received and held in a queue, making them waiting to be processed. The users receive an acknowledgment that their uploaded image has been received and will be processed shortly, so they can move on to doing something else on their browser and check back later to view their uploaded image. The consumer - the image processing application, will read the first message in the queue (the image data in this case), process it, and post the processed image to a location from which the browser can access it. After that, it will read the next message in the queue, process it and post it, and so on, until all messages (images) present in a queue are processed. In order to increase the processing capacity, we can also add multiple consumers, where each of them will read the next message, which is waiting in the queue, and process it. Since an image should not be processed more than once, only one consumer should receive each message. In order to ensure such, each message present in a queue will be deleted once read by a consumer. This pattern works well as a load balancing system since messages (or work items) are picked up by the next available consumer, and all consumers are equally loaded.

• Publish-Subscribe In the publish-subscribe (pub-sub) pattern, one message can be read by multiple consumers. It is a one-to-many pattern, where, unlike in the message queue model, it does not matter if the message is processed multiple times. It is even possible that a message may be processed in different ways by each consumer. Here a category is referred to as a Topic for a particular type of message and is created in the message broker. Depending on the message, the producer published it to a topic. Consumers can subscribe to a topic, and each one of them will receive all the messages published on that topic. One topic can have multiple consumers, and one consumer can subscribe to various topics, where both patterns can be used in an IoT solution. Where vast amounts of data are read by sensors and sent for processing, a message queue pattern with multiple consumers can be used to process the data.

Alternately, where processing speed may not be important, the Pub-Sub pattern can be used. Each type of sensor will publish its data on a topic. The consumers, whether a smartphone app, a Node-RED flow, or any other process, will read the sensor data that they are interested in by subscribing to the relevant topics. Each consumer may subscribe to one or more topics, and it will receive all the messages published to that topic. While both models can be used in any use case, there are some advantages and constraints to each model; thus, they must be used judiciously depending on the requirements of the solution.