MQTT topic management conundrums

Well, this sounds like a gripping title for a post eh?

But actually for me it’s a thing. I’m getting my head around MQTT as a way of sending data from our air quality sensors to the systems that want to work on the data. MQTT is a “publish and subscribe” transport. An MQTT node can send messages to a particular address managed by sn MQTT broker process running on a machine somewhere. Other processes can subscribe to that address on the broker and, when the mode sends a packet of data (string of bytes) to the that address the broker will then send that message to everyone who has subscribed to that address.

Make sense? OK, think of it as a mailing list for a pop star, perhaps David Essex. Anyone interested in what David is up to can subscribe to the mailing list, which has a particular name. David’s publicist, who produces the content describing the amazing life that I’m sure David has, (other pop stars are available for this exercise) connects to the mailing list as a publisher, and sends messages which are then sent out to all the subscribers. So MQTT Is basically a fan mail management system.

The address expressing the a particular endpoint is called a topic. Topics are hierarchical. So, an air quality sensor could send a messages to the following topics with the respective data items from that sensor:

Sensor01/temp
Sensor01/pressure
Sensor01/humidity

Alternatively, an air quality sensor could send a single message containing all three values, encoded as a lump of JSON (JavaScript Object Notation) to a single endpoint:

Sensor01/data

Which is better? I’m not sure. Object Oriented Rob says that having all the related items in a single cohesive object which can also contain things like timestamp and location information will make it much easier to process. Free Wheeling Rob says that having individual data on separate topics makes it possible to handle data produced at different rates by different sensors, and also makes it easier for something only interested in temperature (a fan perhaps) to just subscribe to that data item. And both Robs need to remember that the broker is just that, a broker. It doesn’t have any bearing on what will happen to the values that are passed through it from a device to a process that is doing something with it.

As usual, when faced with a problem like this, I’ve gone a bit meta. To me the important thing is probably not which particular way the values are configured for a given application, but that it should be easy to change this configuration during the lifetime of a node. To this end I’m working on a little protocol which will let a node advertise the readings that it has available and then be told which readings to be sent where. Once a node has been told where to send its data it will do that until told otherwise. This means that we can change how we use the sensors so that we could use either of the above approaches.

The good news is that this will be wonderful. The bad news is that I’ve got to do it. Oh well. Today I got a Python program talking to MQTT. The next step is to get JSON working on the Arduino so our systems can start exchanging configuration information.