RabbitMQ integration issues with IBM Sterling Integrator

[Vinod Shanker]

We are having RabbitMQ integrated with IBM Sterling B2B Integrator using the JMS interface. We have used the .bindings file along with the Async Receive Adapter in the Sterling SI for integrating the rabbitmq. Following are the jars that are used for integration

commons-io-2.6.jar

fscontext.jar

geronimo-jms_1.1_spec-1.1.1.jar

jndi-1.2.1.jar

amqp-client-4.1.1

rabbitmq-jms-1.7.0.jar

slf4j-api-1.7.5.jar

The integration works well and SI is able to receive the messages posted in the RabbitMQ. 

Now our problem is that when there are huge number of messages in the RabbitMQ, the Sterling SI will consume around 1500+ messages and kept in its memory in one shot and they will be shown as un acknowledged messages in the rabbitmq console until all of those are processed by the Sterling. So when the Sterling goes down abruptly, following issues happen

1. Some messages are getting lost (around 7-9 of them)

2. Once the sterling comes up, it gets back lots of messages even before the rabbitmq connection is reestablished. All these messages are having the redelivered flag as true. So in such cases, suppose there is one original message and one redelivered one; both are having the redelivered flag as true. 

  Due to the above mentioned facts, the Enterprise Intergation team is not confident to go with RabbitMQ and Sterling integrator combination. 

  

 When we use the Tibco (which was existing queue), the messages received by the Sterling was one at a time (instead of 1500+ in rabbit case). So even if the SI goes down, the possiblity of messages missing and redelivering is minimal.

 

 Setting of the QoS will not be practical here, I guess, as the adapter is not programmed one. There seems to be no provision to add the QoS in this setup.

 

 So the question is that if there is any rabbitmq client jars that use the receive method instead of message listener? Can they work with rabbit-jms-1.7.0 jar? 

 Any sugestions or pointers in this regard would be really helpful.

 

 Thanks,

 Vinod 

[Arnaud Cogoluègnes]

You can use MessageConsumer#receive(), which does not use an
asynchronous consumer. This is not the most effective solution, I
strongly advise to use asynchronous consumers and set QoS with
RMQConnectionFactory#setChannelsQos(int).

It's also possible to set a default prefetch/qos for all channels, but
this is a global setting that will affect all applications connecting
to the cluster, see [1].

[1] https://groups.google.com/d/msg/rabbitmq-users/9fjfnaMbMNk/vVuihVikAgAJ

> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/d9e85d6f-74a4-41aa-b487-50a246403f37%40googlegroups.com.

[Vinod Shanker]

Thanks for the suggestion, Arnaud Cogoluègnes.  I  tried with QoS as 1 and now the maximum unacknowledged message is 1. 

So this almost solves the issue of having multiple instances of same message with redelivered flag as true.  

But since we use the message listener interface, we lose around 9 messages from a lot of 2000 messages when the Sterling Integrator abruptly went down. 

Maybe if we have to reduce the number of messages in flight when the Sterling Integrator goes down, may have to try the receive message approach. 

But does the RMQConnectionFactory use the MessageConsumer receive method?

Thanks and Regards,

Vinod Sivashankaran

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.

[Luke Bakken]

Hi Vinod,

Please realize that when you say "lose messages" it is not correct. The un-acknowledged messages are re-enqueued, and re-delivered the next time a consumer comes up.

Thanks,

Luke