What is the Message Queue Telemetry Transport Protocol?

MQTT  is an ISO standard publish-subscribe-based “lightweight” messaging transport protocol for use on top of the TCP/IP protocol. It is designed for connections with remote locations where a “small code footprint” is required or high latency/ low-bandwidth networks. Andy Stanford-Clark and Arlen Nipper of Cirrus Link authored the first version of the protocol in 1999.

Publish / Subscribe

The publish-subscribe messaging pattern requires  message client and broker. The client can be any device, from a micro controller to a server, which runs the MQTT library and is connected to MQTT broker over any network. The broker is responsible for receiving all messages, filtering, making a decision and distributing messages to subscribed clients based on the topic of a message. Since MQTT is based on TCP/IP, both client and broker are expected to have TCP/IP stack.

Verbs

MQTT defines 14 different methods (sometimes referred to as verbs) to indicate the desired action to be performed on the identified resource. What this resource represents, whether pre-existing data or data that is generated dynamically, depends on the implementation of the server. Often, the resource corresponds to a file or the output of an executable found on the server.

Typically, end users only need to employ the CONNECT, DISCONNECT, PUBLISH, SUBSCRIBE, and UNSUBSCRIBE message types. The other message types are used for internal mechanisms and message flows.

A topic is a UTF-8 string, which is used by the broker to filter messages for each connected client. A topic consists of one or more topic levels. Each topic level is separated by a forward slash (topic level separator).

MQTT Topic Wildcards can be used for topic filters when subscribing to MQTT messages. They are useful if a client wants to receive messages for different topics with similar structure at once.

“+” – a wildcard that matches one complete topic level. It must occupy an entire topic level. This wildcard can be used more than once in a topic subscription.

“#” – a wildcard that matches any number of levels within a topic. It must be the last character of a topic subscription.

QoS

Each MQTT publish is sent with one of three Quality of Service (QoS) levels. These levels are associated with different guarantees with regards to the reliability of the message delivery. Both client and broker provide additional persistence and redelivery mechanisms to increase reliability in case of network failures, restarts of the application, and other unforeseen circumstances.

MQTT relies on TCP, which has reliability guarantees on its own. Historically QoS levels were needed to overcome data loss on older and unreliable TCP networks. This can still be a valid concern for mobile networks today.

There are 3 QoS levels in MQTT:

0 – at most once delivery: The sender tries with best effort to send the message and relies on the reliability of TCP. No retransmission takes place.

1 – at least once delivery: The receiver will get the message at least once. If the receiver does not acknowledge the message or the acknowledge gets lost on the way, it will be resent until the sender gets an acknowledgement. Duplicate messages can occur.

2 – exactly once delivery: The protocol makes sure that the message will arrive exactly once at the receiver. This increases communication overhead but is the best option when neither loss nor duplication of messages are acceptable.

QoS flows between a publishing and subscribing client are two different things. That means the QoS level can be different from client A, who publishes a message, and client B, who receives the published message. Between the sender and the broker the QoS is defined by the sender. When the broker sends out the message to all subscribers, the QoS of the subscription from client B is used. If client B has subscribed to the broker with QoS 1 and client A sends a QoS 2 message, it will be received by client B with QoS 1. And of course, it could be delivered more than once to client B, because QoS 1 only guarantees to deliver the message at least once. So QoS guarantees can get downgraded for a particular receiving client if subscribed with a lower QoS.

Security

Security is a very important part of any communication. MQTT itself keeps everything as simple as possible and relies on other proven technologies for safeguards

Username / Password Authentication

An MQTT CONNECT message can contain a username and password. The broker can authenticate and authorize with this information if such a mechanism is implemented. Many open-source brokers rely on Access Control Lists while other enterprise brokers allow coupling with user databases and/or LDAP systems.

Transport Security: TLS

A best practice when using MQTT is to add transport layer security if possible. With TLS, the complete communication between client and broker is encrypted, and no attacker can read any message exchanged. If feasible, X509 client certificate authentication adds an additional layer of security to the clients: trust.

Let’s take a look how to install MQTT broker on RPI and connect Node-RED client to it.

Preparation

To get rolling, we need a Raspberry PI3 and an SD card with Raspbian preinstalled. If you don’t have a preinstalled card, you can install Raspbian yourself by following this instruction.

Next steps after installing Raspbian:
1. Open a terminal and run these commands to update to latest Raspbian and packages:

(This will take a short while)
2. Next open a terminal and run command:
sudo raspi-config
3. Now you will have to make some smaller configs to Raspbian
4. First open Advanced Options and choose Expand Filesystem
5. Secondly open Interfacing Option and Enable SSH and VNC
6. Exit the raspi-config and it will ask you to reboot

Steps to install working Java for ActiveMQ:
1. Open a terminal using CTRL+T and then run

ActiveMQ Installation

ActiveMQ is an open-source multi-protocol message broker with a core written around JMS. It supports MQTT and maps MQTT semantics over JMS.

Here are few steps to get it installed on RPI:

1. Create folder named in a place of your choosing, preferably in home folder

2. Download compressed file from Apache website, either manually or commandline:

3. Uncompress the downloaded file

4. To run ActiveMQ

(To run as service) or

(To run in terminal)

5. To enable start from boot:
a.

b. And then this command to update rc.d

c. Now ActiveMQ should be started during boot

Node RED Installation

1. Run this command to update nodejs and Node-RED since it’s already installed in Raspbian

2. To run Node-RED, open a terminal and run

3. To enable Node-RED from boot:

4. Open browser and browse to localhost:1880 to configure flows and deploy them

Simple Flow

Here is a simple Node-RED flow illustrating subscribe/publish scenario.

Flow above is represented by the following json:

[{“id”:”8788d23c.6f5b2″,”type”:”mqtt in”,”z”:”b9bd2972.30ee88″,”name”:”Subscribe to /test/topic”,”topic”:”/test/topic”,”qos”:”1″,”broker”:”212b12a2.a8c08e”,”x”:350,”y”:276,”wires”:[[“d2b166e9.2fda58”]]},{“id”:”d2b166e9.2fda58″,”type”:”debug”,”z”:”b9bd2972.30ee88″,”name”:”Show message”,”active”:true,”console”:”false”,”complete”:”true”,”x”:595,”y”:276,”wires”:[]},{“id”:”d4762c5e.b8f34″,”type”:”mqtt out”,”z”:”b9bd2972.30ee88″,”name”:”Publish to /test/topic”,”topic”:”/test/topic”,”qos”:”0″,”retain”:”true”,”broker”:”212b12a2.a8c08e”,”x”:576,”y”:182,”wires”:[]},{“id”:”d8358446.c5e948″,”type”:”inject”,”z”:”b9bd2972.30ee88″,”name”:”Message”,”topic”:””,”payload”:””,”payloadType”:”date”,”repeat”:””,”crontab”:””,”once”:false,”x”:332,”y”:182,”wires”:[[“d4762c5e.b8f34”]]},{“id”:”212b12a2.a8c08e”,”type”:”mqtt-broker”,”z”:””,”broker”:”localhost”,”port”:”1883″,”clientid”:””,”usetls”:false,”compatmode”:true,”keepalive”:”60″,”cleansession”:true,”willTopic”:””,”willQos”:”0″,”willPayload”:””,”birthTopic”:””,”birthQos”:”0″,”birthPayload”:””}]

Conclusion

You now have a small IoT gadget ready to start your own IoT adventure. Later we will look into how we can add more hardware and even interconnect multiple devices, such as sensors and relays.

Many thanx to Taner Yavuz and Maksym Gerasymchuk for the help with this post 🙂