'x-delayed-message' exchange bindings stops working after broker restart

[Cristian Moreno]

Hello everyone,

I am working on an application where we use RabbiMQ with rabbitmq_delayed_message_exchange plugin installed, and we use an 'x-delayed-message' exchange (with 'x-delayed-type: topic') where we send all messages, which are later routed to one of our queues, based on the routingKey. The setup works fine when we create it, but after we restart the broker, new messages are not routed any more.

Platforms

I've been able to reproduce this issue on those setups:

1. Running Docker on mac, from rabbitmq:3-management:

RabbitMQ 3.8.5

Erlang 23.0.2

Plugin (rabbitmq_delayed_message_exchange) 3.8.0

2. Running the server locally in my macOS Catalina:

RabbitMQ 3.8.6

Erlang 23.0.3

Plugin (rabbitmq_delayed_message_exchange) 3.8.0

Client Libraries

I am able to reproduce the issue both using the management UI/CLI (with the management plugin enabled), and with the NodeJS amqplib package.

Steps to reproduce the issue

Using RabbitMQ server, with rabbitmq_delayed_message_exchange:

1. Add a new exchange with

    - type: x-delayed-message

    - durability: durable

    - arguments: x-delayed-type: topic

2. Create a Classic Queue, with Durability: Durable.

3. Create a binding in the exchange created at (1) to route messages to the queue created at (2), specifying a routingKey.

4. Publish some message to the exchange, providing the same routingKey used in (3)

    - Message should be routed successfully to the queue.

5. Restart the server

6. Repeat step (4)

    - Messages are not routed any more

Commands to reproduce (above steps)

$ rabbitmqadmin declare exchange name=exchange durable=true type=x-delayed-message 'arguments={"x-delayed-type":"topic"}'

$ rabbitmqadmin declare queue name=queue durable=true queue_type=classic

$ rabbitmqadmin declare binding source=exchange destination=queue routing_key=queue

$ rabbitmqadmin publish routing_key=queue exchange=exchange payload=test

> Message published

## Now we restart the server

$ rabbitmqadmin publish routing_key=queue exchange=exchange payload=test

> Message published but NOT routed

How to Fix it

When this happens, you can delete the binding created in step (3), and then repeat step (3). Now, new messages created like in (4), will be routed properly again.

It is actually not possible to fix it from the CLI, because trying to delete results in  "Not found" (maybe related with the issue), and redeclaring the binding doesn't fix it neither:

$ rabbitmqadmin delete binding source=exchange destination_type=queue destination=queue

> *** Not found: /api/bindings/%2F/e/exchange/q/queue/~

Additional Notes:

Some of my colleagues have an older version of RabbitMQ installed and can't reproduce the issue:

1. Running Docker on mac, from rabbitmq:3-management:

RabbitMQ 3.8.2

Erlang 22.2.2

Plugin (rabbitmq_delayed_message_exchange) 3.8.0

2. Running the server locally in macOS Catalina:

RabbitMQ 3.8.2

Erlang 23.0.3

Plugin (rabbitmq_delayed_message_exchange) 3.8.0

Finally, I could also not reproduce the mentioned issue when using an x-delay-message exchange with 'x-delayed-type: direct', this will only happen with topic.

I appreciate any help on this, an I would be glad to clarify any of the steps to reproduce the issue.

Best regards.

[TNSRE Engineering]

We use RabbitMQ 3.8.8-1 on Ubuntu 20 and face similar issue; with every rabbitmq restart, we are forced to re-create the delayed exchange. if we do not do it, the DELAY exchange behaviour is not consistent i.e. only works intermittently. We also cannot find any errors in the rabbitmq logs - validated with "rabbitmq-diagnostics log_tail". So not sure what is the issue here. Any help will be appreciated, thanks.

RabbitMQ 3.8.8-1

Erlang 23.0.3-1

Ubuntu 20.04.1

Thanks.

[Karl Nilsson]

Hi - I just tested this. Would you mind raising an issue with as much detail as possible at: https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/issues

Cheers

Karl 

---

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of TNSRE Engineering <tnn...@gmail.com>
Sent: 15 October 2020 7:42 AM
To: rabbitmq-users <rabbitm...@googlegroups.com>
Subject: [rabbitmq-users] Re: 'x-delayed-message' exchange bindings stops working after broker restart

 

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/7a20eb47-eea3-4cd6-9a02-df211ffd732cn%40googlegroups.com.

[Cristian Moreno]

Hi, thanks for your response, I just created an issue as you suggested: https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/issues/149

Regards,

Cristian