Roger Light
> When a client subscribes to a particular topic ”forever”, the server will
> keep saving messages for it. Potentially, the server is now heading for the
> problem I guess any publish-and-subscribe system can suffer from: Some
> messages just pile up on the server and never get removed because one or
> more hanging subscriptions are never re-activated or stopped.
This is correct. Mosquitto offers a beyond-the-spec feature which is
persistent client expiration. This allows persistent clients to be
removed if they do not reconnect within a certain time frame. When
persistent clients are removed, any subscriptions and queued messages
for them are also removed. The broker running on test.mosquitto.org
will remove persistent clients when they have not reconnected for a
week.
Adding something similar for retained messages would be pretty
straightforward. I'd think an option along the lines of "remove
retained messages with a payload over X bytes that are at least X
weeks old" would go a long way.
> I have one suggestion for making such a ”space watcher”: The Mosquitto
> package includes the source code for a program, db_dump.c, capable of
> writing the contents of the Mosquitto database in a format, readable for
> humans.
Just to be clear about this, the mosquitto.db file is only
periodically written to disk if enabled, as well as when mosquitto
exits. It is only ever read on startup. It's really just a
serialisation of the appropriate internal data structures - calling it
a database is really a throwback to when everything was stored using
an in-memory sqlite database back in version 0.8 and earlier.
> One challenge I am having here is to fully understand the structure of the
> output file from the db_dump program (or to understand the database file
> itself). If somebody could help me here I could give a documentation back to
> this community in return.
Neither of those is documented - I do regard it as an internal file
format that may change at any point. In practice there are unlikely to
be many/big changes though.
There is a 15-byte file "magic" file identification header, followed
by a 32-bit CRC (currently unused), and a 32-bit int format version
number. After that data is stored as "chunks" with a 16-bit int chunk
type, 32-bit int length and then data of the appropriate length. It's
all covered in db_dump.c or src/persist.c and src/persist.h. Some data
structures do depend on others, so order can be important. I doubt a
full description would be that interesting to everyone here.
I have talked about making a plugin interface for mosquitto (but
haven't made much progress so far). I was mostly thinking of
privileged in-broker clients for e.g. separating out bridging support
but it's possible this would be a good match as well.
Cheers,
Roger
Christian Hauris Sorensen
Thank you, Roger, for your through reply; I am honored to get a reply from, of all people, you!
Well well, I guess it's not worth the trouble making a monitor based on the Mosquitto "database". One question to you, though: You were saying: "I have talked about making a plugin interface for mosquitto..." - Did you (amongst other things) think of the possibililty of hooking a monitor kind of thing on to the Mosquitto server?
And a couple of questions to thís community: Would a kind of monitor be useful? How could/should it be made?
Thanks to everyone spending time on this.
Regards, Christian.
Den mandag den 22. oktober 2012 15.57.47 UTC+2 skrev Christian Hauris Sorensen:
Hi!
I am an old programmer with experience in WebSpere MQ. I have now been looking at the MQTT world and I am finding it interesting. As an old WMQ guy I am missing one thing in MQTT: Monitoring. I am aware of the $SYS topics concept but I am thinking of another area:
When a client subscribes to a particular topic ”forever”, the server will keep saving messages for it. Potentially, the server is now heading for the problem I guess any publish-and-subscribe system can suffer from: Some messages just pile up on the server and never get removed because one or more hanging subscriptions are never re-activated or stopped.
What one could want then is a system for watching the space consumption for the server ”data base” and a way of drilling down to the root causes of excessive space usage. Also, the age of stored messages would be interesting but, as far as I got it, this is not normally available in MQTT systems.
I have one suggestion for making such a ”space watcher”: The Mosquitto package includes the source code for a program, db_dump.c, capable of writing the contents of the Mosquitto database in a format, readable for humans. I have tried this program on my own Mosquitto database file, giving a text file, which tells about topics, their retained messages, etc. These entries are ”keyed” with ”Store ID” values or similar and can therefore be put together and structured into something interesting. I could imagine a monitoring program that could work on the output file from this db_dump program (or, of course, on the database directly):
The program could show all topics of the database and for each topic, the messages retained for it, with size, contents etc. Also subscribers subscribing to this topic could be shown. Such a program could show aggregated numbers, e.g. the total space consumption for each topic. There could be options for sorting the topics on name (to find something particular in there) or size (to spot space hogs).
One challenge I am having here is to fully understand the structure of the output file from the db_dump program (or to understand the database file itself). If somebody could help me here I could give a documentation back to this community in return.
Another challenge: My suggested ”monitor” would work for Mosquitto only. A general standard for this would maybe be a better idea, although that cannot deliver any monitor in the short run.
Besides: Time stamps in the stored messages for being published and for being read would be very useful for monitoring purposes – when can the administrator drop a space hog? Well, maybe when nobody has touched it for six months. Would time stamps in the database be considered an extension to the standard or could it just be done on a particular server system, such as Mosquitto? I guess it can be found already in the larger systems, such as the MQTT part of Websphere MQ.
I have made a ”prototype of a prototype” for the monitor suggested above. The first level ”prototype” means that the logic is only rudimentary; the second level that I imagine a GUI with a tree kind of display and options for searching and for looking at details, and there is nothing like a GUI now. Anyway; based on data like this (output from the program db_dump):
DB_CHUNK_MSG_STORE:
Length: 86
Source ID: PC8
Source MID: 1
MID: 0
Topic: samples/topic02
QoS: 2
Retain: 1
Payload Length: 46
Payload: Message from TryPublish at 2012-10-09 15:28:03
DB_CHUNK_MSG_STORE:
Length: 86
Store ID: 8324
Source ID: PC8
Source MID: 1
MID: 0
Topic: samples/topic02
QoS: 2
Retain: 1
Payload Length: 46
Payload: Message from TryPublish at 2012-10-09 15:29:08
DB_CHUNK_CLIENT_MSG:
Length: 21
Client ID: PC8R
Store ID: 8295
MID: 1
QoS: 2
Retain: 1
Direction: 1
State: 4
Dup: 1
DB_CHUNK_CLIENT_MSG:
Length: 21
Client ID: PC8R
Store ID: 8324
MID: 2
QoS: 2
Retain: 0
Direction: 1
State: 11
Dup: 0
DB_CHUNK_SUB:
Length: 24
Client ID: PC8R
Topic: samples/topic02
QoS: 2
my prototype can produce this text on the console:
Topic: samples/topic02
- MsgStore: 46 - Message from TryPublish at 2012-10-09 15:28:03
- ClientMsg: PC8R
- MsgStore: 46 - Message from TryPublish at 2012-10-09 15:29:08
- ClientMsg: PC8R
- Sub: PC8R
just to use the pedagogical principle called ”a picture is worth a thousand words” (well, it seems like you are getting both :-)).
Or: Did I just overlook something here - does it already exist? Is there a better way of obtaining this kind of data; is there a better way of showing it?
If my idea seems right to this community, I will offer to make such a monitor with some open source license. I guess I should then take a discussion on functional and technical requirements with whoever of you who would take on the trouble to participate.
Kind regards, Christian Hauris Sorensen.
Roger Light
> Well well, I guess it's not worth the trouble making a monitor based on the
> Mosquitto "database". One question to you, though: You were saying: "I have
> talked about making a plugin interface for mosquitto..." - Did you (amongst
> other things) think of the possibililty of hooking a monitor kind of thing
> on to the Mosquitto server?
bridge case was just an example. Likewise adding mqtt-s support could
be a plugin as well.
Cheers,
Roger