Get queue age/timestamp of the first message from a specific queue

[Aneci Adrian]

Hi team,

I was wondering if there is method to get the queue age or the timestamp of the oldest message from a specific queue.

I've made some researches but didn't find anything related to this metric. I've also read that the producer can set a timestamp in the header of the message and in this way the head_message_timestamp property will be set on the queue side. Am I right?

I want to be able to get this metric without involving the development team, to be aware of the producer and the consumer.

I've enabled the RabbitMQ Message Timestamp Plugin which is supposed to automatically timestamp all messages that arrive in the queue but the head_message_timestamp property is still null.

In the official documentation for the head_message_timestamp property it is stated that "The timestamp property of the first message in the queue, if present. Timestamps of messages only appear when they are in the paged-in state".

Is there a way to accomplish this this task?

Regards,

Adrian.

[Michael Klishin]

`head_message_timestamp` is orthogonal to http://github.com/rabbitmq/rabbitmq-message-timestamp.

`head_message_timestamp` is only set for messages that go through the message store.

Are you sure they are not going straight to the consumer in your case or get embedded into the

queue index (this is done for messages < 4096 bytes in size by default, see [1]).

In general I'd recommend not making your system depend on `head_message_timestamp` and consider

an alternative design, whatever it may be.

1. https://github.com/rabbitmq/rabbitmq-server/blob/stable/docs/rabbitmq.config.example#L315

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Aneci Adrian]

Thanks for your quick reply.

It could be possible that almost all the messages to have a size < 4096 bytes in my test environment.

What would be your recommendation as an alternative to head_message_timestamp property from the monitoring perspective? I'm looking for a solution to monitor the entire pipeline or flow, from the time the message is sent from the producer until the time it arrive to the consumer. In this way I can be sure that the communication, at least at network/middle-ware layer, is working properly.

Can be a better approach to implement some checks at the application/end-to-end layer?

Regards,

Adrian.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/cKnw_AmKAAk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-users+unsubscribe@googlegroups.com.

[Mat Jaggard]

The silence is deafening. Did you ever get any answer to this?

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/cKnw_AmKAAk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.

[Michael Klishin]

> The silence is deafening

You sound quite privileged to me, as if someone owes you a response.

This is a community forum. There are no guarantees of a response here. Across this mailing list, GitHub Discussions, Slack and Discord, no one has the capacity to follow up on every discussion.

This was even worse in 2017 when the team was not just small but tiny.

I'm not sure what "monitoring a message path from producer to consumer" means specifically but in 2023 there are several options:

 * Message expiration (possibly combined with dead-lettering)

 * Timestamping the message on the publisher side or using the plugin mentioned above, and using that data as an application metric on the consumer side

 * Tracing tools (as in OpenTracing, for example), which is a very broad area

> In general I'd recommend not making your system depend on `head_message_timestamp` and consider

> an alternative design, whatever it may be.

is something I still stand by today, six years later. Monitoring rate and rate fluctuations is a better option that more tools support one way or another.