RabbitMQ publish message TLS/SSL through AWS Network Load Balancer (NLB)

[James Ward]

Hi all!

I am having an issue with TLS through an AWS network load balancer (NLB) to my rabbitmq cluster (hosted on EC2 redhat).

I have managed to publish to a queue through the load balancer without TLS

My spring boot code can publish fine to a queue directly to any of the nodes with TLS.

 

As soon as I go through this load balancer it almost immediately throws an EOF exception (please see below).

I have uploaded the same client-server certificates to the load balancer as is on all nodes in the rabbitmq cluster.

What I am really after here is steps to troubleshoot this, or if any one has seen this before and has a suggestion on how to fix it.

Humbly appreciate any help with this one.

2021-09-16 18:15:07.192 ERROR 23117 --- [Thread-11] [CID:] c.n.m.t.s.mq.send.RabbitMqProducer       : Error publishing to RabbitMq

org.springframework.amqp.AmqpIOException: java.io.IOException

      at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:70)

      at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:602)

      at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:724)

      at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:216)

      at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2167)

      at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2140)

      at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:1063)

      at org.springframework.amqp.rabbit.core.RabbitTemplate.convertAndSend(RabbitTemplate.java:1128)

      at org.springframework.amqp.rabbit.core.RabbitTemplate.convertAndSend(RabbitTemplate.java:1121)

      at com.nab.ms.tfdp.screening.mq.send.RabbitMqProducer.send(RabbitMqProducer.java:32)

      at com.nab.ms.tfdp.screening.processor.EventCreateProcessor.lambda$sendForScreening$0(EventCreateProcessor.java:50)

      at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)

      at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: java.io.IOException

      at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:129)

      at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:125)

      at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:396)

      at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1139)

      at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1087)

      at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.connectAddresses(AbstractConnectionFactory.java:640)

      at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.connect(AbstractConnectionFactory.java:615)

      at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:565)

      ... 11 more

Caused by: com.rabbitmq.client.ShutdownSignalException: connection error

      at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)

      at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)

      at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:502)

      at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:326)

      ... 16 more

Caused by: java.io.EOFException

      at java.base/java.io.DataInputStream.readUnsignedByte(DataInputStream.java:295)

      at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)

      at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:184)

      at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:665)

[James Ward]

I have the answer

It seems mTLS isn't supported with NLBs

I set:

ssl_options.verify = verify_none 

ssl_options.fail_if_no_peer_cert = false

And it started working. 

Bit of a shame though. I'm going to raise a ticket with AWS