[rabbitmq-discuss] High-number of unack-ed messages

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi there,

doing some "mass" tests with RabbitMQ 1.7.2.

I inserted some 500k messages with one produce into the queue.
Then I attached one consumer to the queue feeding the messages into
Solr. The consumer is implemented as callback using the Carrot bindings
for Python. Every message is directly acknowledged. However "Alice"
always shows me between 300 and 500 unacknowledged messages. Shouldn't
that number be 0 or 1?

Andreas




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvrzhwACgkQCJIWIbr9KYzYzACgwmkBIyvXaDIMsHCg58oM5Gty
tIIAnjpSEk9EH66tW/Uhw9upyIeM/J/Y
=9t+D
-----END PGP SIGNATURE-----

[Matthew Sackman]

Hi Andreas,

On Thu, May 13, 2010 at 12:02:04PM +0200, Andreas Jung wrote:
> I inserted some 500k messages with one produce into the queue.
> Then I attached one consumer to the queue feeding the messages into
> Solr. The consumer is implemented as callback using the Carrot bindings
> for Python. Every message is directly acknowledged. However "Alice"
> always shows me between 300 and 500 unacknowledged messages. Shouldn't
> that number be 0 or 1?

Well. It should be low, but if there's no QoS being set by Alice then it
can get quite high. I would only trust it to be 0 or 1 if Alice was also
setting a QoS prefetch of 1, and even then, there are cases where the
unack count can be higher than that.

After Alice had drained all 500k messages, was the unack count still
high, or had it dropped to 0 by then?

Matthew

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

--
You received this message because you are subscribed to the Google Groups "rabbitmq-discuss" group.
To post to this group, send email to rabbitmq...@googlegroups.com.
To unsubscribe from this group, send email to rabbitmq-discu...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rabbitmq-discuss?hl=en.

[Brendan Doyle]

On 1.7.2 I often see unack count reported by rabbitmqctl much higher than my QoS x number of consumers

I always wrote it off to high throughput and different priorities on handling new incoming messages vs. acks or another internal implementation detail

Brendan

[Matthew Sackman]

On Thu, May 13, 2010 at 10:20:06AM -0500, Brendan Doyle wrote:
> On 1.7.2 I often see unack count reported by rabbitmqctl much higher than my QoS x number of consumers
>
> I always wrote it off to high throughput and different priorities on handling new incoming messages vs. acks or another internal implementation detail

Yeah, that's generally wise - in general the numbers reported by
rabbitmqctl can't be trusted too much as they're accurate only for
things that have *really* made it to the queue - there are plenty of
places within rabbitmq where buffering can occur.

Matthew

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Matthias Radestock]

Brendan,

Matthew Sackman wrote:
> On Thu, May 13, 2010 at 10:20:06AM -0500, Brendan Doyle wrote:
>> On 1.7.2 I often see unack count reported by rabbitmqctl much
>> higher than my QoS x number of consumers
>>
>> I always wrote it off to high throughput and different priorities
>> on handling new incoming messages vs. acks or another internal
>> implementation detail
>
> Yeah, that's generally wise - in general the numbers reported by
> rabbitmqctl can't be trusted too much as they're accurate only for
> things that have *really* made it to the queue - there are plenty of
> places within rabbitmq where buffering can occur.

What Matthew says is true, but the discrepancy you are seeing still
strikes me as somewhat unusual. For the reported unacked message count
to be much higher than the prefetch limit x number of consumers one of
the following likely has happened:

a) the prefetch limit was only set after the consumer have been created

b) messages were consumed with basic.get rather than basic.consume

Is that the case? If not, could you post the code to reproduce the
results you are seeing? There may be a completely innocent explanation,
but there could also be a bug lurking somewhere, so I'd like to
understand what is happening here.

Regards,

Matthias.

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Matthew Sackman wrote:
> Hi Andreas,
>
> On Thu, May 13, 2010 at 12:02:04PM +0200, Andreas Jung wrote:
>> I inserted some 500k messages with one produce into the queue.
>> Then I attached one consumer to the queue feeding the messages into
>> Solr. The consumer is implemented as callback using the Carrot bindings
>> for Python. Every message is directly acknowledged. However "Alice"
>> always shows me between 300 and 500 unacknowledged messages. Shouldn't
>> that number be 0 or 1?
>
> Well. It should be low, but if there's no QoS being set by Alice then it
> can get quite high. I would only trust it to be 0 or 1 if Alice was also
> setting a QoS prefetch of 1, and even then, there are cases where the
> unack count can be higher than that.
>
> After Alice had drained all 500k messages, was the unack count still
> high, or had it dropped to 0 by then?
>

Since I am not very happy with Alice+Wonderland I would like bring
the discussion back the point: what is the easiest way for getting hold
of the queue names, the number of messages in each queue and the number
of unacknowledged messages? rabbitctl seems to expose only queue names
and the number of messages but not the number of unack-ed messages.

Andreas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvxjLkACgkQCJIWIbr9KYxioQCeMLYYSrsTTmXsG5i1sa/wi/mp
T7YAoL25EuCObjZ5rbj/oIb5JkEP6K1m
=+mNG
-----END PGP SIGNATURE-----

[Matthew Sackman]

On Mon, May 17, 2010 at 08:36:41PM +0200, Andreas Jung wrote:
> Since I am not very happy with Alice+Wonderland I would like bring
> the discussion back the point: what is the easiest way for getting hold
> of the queue names, the number of messages in each queue and the number
> of unacknowledged messages? rabbitctl seems to expose only queue names
> and the number of messages but not the number of unack-ed messages.

Please read the manual. rabbitmqctl:
list_queues [-p <vhostpath>] [<queueinfoitem> ...]

<queueinfoitem> must be a member of the list [name, durable,
auto_delete, arguments, pid, owner_pid, exclusive_consumer_pid,
exclusive_consumer_tag, messages_ready, messages_unacknowledged,
messages, consumers, memory].

messages_unacknowledged is what you're after. You probably want in
total:

rabbitmqctl list_queues name messages_ready messages_unacknowledged

as "messages" on its own is messages_ready + messages_unacknowledged.