rabbitmq to prefer IPv6?

[rolsh...@gmail.com]

Our rabbitmq configured to listen on both IPv4 and IPv6 interfaces. But the requirement is for the IPv6 to be preferred.

I.e., if there are both AAAA and A DNS records, AAAA should be preferred.

We tried all kind of config changes, but nothing seems to do the trick.

Is there any option for the rabbit /erlang to set this preference?

Thank you!

Rose

[Michael Klishin]

What specifically have you tried? How is DNS involved? (RabbitMQ does not use DNS

when determining what interfaces to bind to).

What's the end goal? Please be more specific than "prefer IPv6" as that doesn't mean much.

Are you looking to use IPv6 for *everything*? Client connections? Inter-node communication?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

Instead of starting duplicate threads [1] please describe what you are trying to achieve

in as much detail as possible.

In [1] I see heartbeats, AAAA records and "preferr IPv6" (like I said, it doesn't mean much).

Heartbeats are a feature of messaging protocols and sent by both RabbiitMQ and clients. A client connection

cannot use IPv6 for some frames and IPv4 for others, at least I have never heard of such feature of TCP or its adjacent protocols.

So if it is client connections that we are talking about, RabbitMQ does not control what interface clients connect on.

If you only want to accept client connections over IPv6, configure the node to listen only on IPv6 [2].

On some platforms that actually means "listen on both IPv6 and IPv4", as a usability feature. Again, RabbitMQ does not

control that behavior.

For CLI tools and inter-node communication using IPv6 is possible but epmd [3] can only bind to IPv4-enabled interfaces at the moment.

This is an Erlang limitation [4].

1. https://groups.google.com/forum/#!topic/rabbitmq-users/NlLCXE8WNI0

2. https://www.rabbitmq.com/networking.html#single-stack-ipv6

3. https://www.rabbitmq.com/networking.html#epmd-inet-dist-port-range

4. https://groups.google.com/d/msg/rabbitmq-users/mbsT8PSJTHQ/WkSGpVYJBgAJ

[Michael Klishin]

To add to the AAAA claim in a parallel thread 👀: client libraries and their networking setup

control which stack is used. It is a pretty nuanced process that's programming language/platform-dependent.

On the JVM there's a VM property that controls that, for example. The OS also has a say (obviously), for example,

my local development on MacOS strongly suggests that even though RabbitMQ listens on all interfaces, clients actually

connect over IPv6, very likely because it's a LAN-local default (but I'm not knowledgeable enough of MacOS to be sure).

[rolsh...@gmail.com]

To clarify, we have a rabbitmq client and a server. Both are configured with dual stack, i.e., both have IPv4 and IPv6 addresses. Both have 2 records in the DNS: A and AAAA. And both rabbits listening on both interfaces.

(We cannot force them to listen on IPv6 only. The idea of the dual stack is that you communicate to anybody who has IPv6 through the IPv6. But you also have the IPv4 stack for those who don't have the IPv6).

The Federal IPv6 certification requirement for this kind of config is that all communication between two such hosts uses IPv6 only -as they are IPv6 capable. However rabbit using the IPv4 (we are seeing it in the Wireshark).

Here is what we tried:

1) creating advanced config for rabbit with

%% -- ERLANG INET CONFIGURATION FILE -

{inet6, true}.

and RABBITMQ_ADVANCED_CONFIG_FILE=... in  rabbitmq-env.conf

2)  

SERVER_ADDITIONAL_ERL_ARGS="-proto_dist inet6_tcp"

CTL_ERL_ARGS="-proto_dist inet6_tcp"

3)

RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-proto_dist inet6_tcp"

RABBITMQ_CTL_ERL_ARGS="-proto_dist inet6_tcp"

They don't seem to make any difference. 

You are right, some languages have options for the IP stack/addresses to be preferred.

E.g., Java https://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.html

That's kind of a flag i was asking about.

Regarding the link for the empd you sent. I don't know much about the rabbitmq or epmd, and have no idea what epmd does/who connects to whom.

Does this link mean that rabbitmq will not work on a node that has only IPv6 as an external address (it may have the IPv4 loopback) ?  The link mentions using  RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS, but it doesn't work

[rolsh...@gmail.com]

[Michael Klishin]

proto_dist controls what transport (IP version in this casee) inter-node and CLI tools use. They are irrelevant

if your goal is to make clients connect over IPv6.

The preference has to be configured on the client end. I have no recommendations here because I have no details on the client

library/-ies used and target platform[s].

[rolsh...@gmail.com]

if you mean the OS level, it's already preferring  the IPv6 (it's Suse 11SP4). 

We have multiple app-s on these 2 hosts communicating to peers, and they all using the IPv6 with no problem.

The issue is only with the rabbitmq. 

What we are seeing is it gets both DNS records for the host, but then it's using the A record.

[Michael Klishin]

What recommendation do you expect if I cannot get any details on what client libraries you use

after asking several times? I am really terrible at mind reading.

RabbitMQ does not control how clients resolve hostnames. If you need to change something, it has to change

on the client side.

I give up.

[rolsh...@gmail.com]

Michael,

i have troubles understanding you too. If you don't mean the OS, which libraries you talking about?

As i said i don't know much about the rabbitmq, and have no idea what it's using.

Do you mean the erlang? 

I am not sure the statement about Rabbitmq not resolving hostnames is correct.

I am seeing it getting IPv4 and IPv6  DNS records in rabbit_peer_discovery_dns.erl...

[Michael Klishin]

I tried out of curiosity and with a RabbitMQ node that is configured to listen on [::]1:

listeners.tcp.1 = ::1:5672

I can connect using Bunny with

c = Bunny.new(hostname: 'localhost'); c.start

and Pika with

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost')) Bunny specifies AF_NET for protocol family when it instantiates its socket. According to RabbitMQ log and Wireshark, both peers have ::1 for their respective hostnames. So my hypothesis that socket family used when instantiating a client socket matters does not hold on MacOS. SUSE and particularly SUSE 11 might work differently. I couldn't find any way to disable IPv4 via sysctl the same way IPv6 can kind of be disabled. So whatever the solution is, chances are it is programming language/runtime/client-specific and may require manual hostname resolution and AAAA vs. A result arbitration in client library code.

[Michael Klishin]

Against my better judgement, I'm responding one last time to what seems to be an act of elaborate trolling.

rabbit_peer_discovery_dns is a cluster discovery mechanism [1]. That is completely orthogonal

to client library connections and must be enabled separately.

RabbitMQ is messaging middleware. It supports multiple protocols, accepts connections from client libraries [2] (which is what I am

asking about: what client libraries do your apps use? Surely you don't run RabbitMQ for the kicks of it?),

CLI tools and peer nodes. [3] lists them all. If you are not sure what a client library is, it is that thing applications

use to connect to RabbitMQ and define topology, publish and consume. In the MySQL community the same type of library

are called database drivers. They dictate what

All those things can use IPv4 or IPv6. The fact that DNS peer discovery (which you probably don't use but we don't know,

you are really keen on not sharing relevant information) may perform AAAA queries does not mean that your apps

will, or CLI tools, for that matter.

I already stated earlier than for CLI-to-node and inter-node communication, including peer discovery, IPv6

exclusively can be used with the exception of epmd. It is an Erlang limitation.

There is no way RabbitMQ can control how *client libraries* (those Ruby, Java, Python, PHP, Go, and other things listed in [2]),

resolve hostnames and whether they prefer IPv4 or IPv6.

On the JVM there is a preference that at least allows for IPv4 to be preferred.

I could not find a way to do something similar for Ruby, Python. Go has a way that requires client library developer to make

the call.

In Python plenty of people recommend performing hostname resolution manually if IPv6 preference is desired, then connecting

using the discovered IPv6 address. I don't know of a RabbitMQ client library that does something like that.

Several happily resolve addressees using AAAA records and connect using IPv6 if target RabbitMQ only binds to an IPv6 interface.

So, using IPv6 to the extent that is possible today:

 * Bind the node to an IPv6 only interface for every protocol enabled

 * Add `-proto_dist inet6_tcp` to Erlang VM arguments, including CLI tools [4]

 * If there is a way, configure all clients to use a hostname that resolves only via AAAA, or otherwise prefer IPv6. How that's done depends on the client library.

epmd and requests to it will still use IPv4 as mentioned above.

I'm out.

1. httpstra://www.rabbitmq.com/cluster-formation.html

2. https://www.rabbitmq.com/getstarted.html

3. https://www.rabbitmq.com/networking.html

4. https://www.rabbitmq.com/configure.html#supported-environment-variables, RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS and RABBITMQ_CTL_ERL_ARGS

[Michael Klishin]

[3] should have been https://www.rabbitmq.com/networking.html#ports specifically, not the entire Networking guide.

[rolsh...@gmail.com]

This is much better! :)

I am going to research all these, and will get back to you on Monday.

Have a good weekend!

[Luke Bakken]

Hello,

You might also need to use the inet_dist_use_interface argument wtih an IPv6 address - http://erlang.org/doc/man/kernel_app.html

[rolsh...@gmail.com]

An update on this issue. We tried all kind of IPv6 options, and we couldn't make rabbitmq management bind on IPv6 interface. Possible issue is that one of the systems is SUSE 11 SP4, and the latest rabbitmq it was verified with was from 2 years ago. So we ended with manually resolving the hostname into the IPv6 address before passing it to rabbitmq.

Which worked for the rabbitmq, but created other issues. Because we can't use preferIPv6Addresses as a Java option for this process (as the rabbitmq management doesn't bind on IPv6), the communication with other systems becomes non-IPv6 compliant /or needs to be resolved manually.

So the question is, assuming we upgrade to the latest SUSE + the latest rabbitmq, would the rabbitmq management bind to the IPv6? 

Also my understanding is epmd still doesn't bind to the IPv6, even in latest releases. Who exactly communicates with it? Would rabbitmq  IPv6 options affect the communication with the epmd/ would rabbitmq know to communicate to the epmd on the IPv4 loopback if it's configured with IPv6 options?

Thanks! 

[Michael Klishin]

What are the "all kinds of IPv6 options" you have tried for management? What exactly is the "version from 2 years ago"?

See, we are still not any better at mind

reading on this list and we can't suggest much without information to work with.

`management.tcp.ip` is the option that controls what interface Cowboy, the embedded HTTP[S] and WebSocket server RabbitMQ uses.

3.6.16 used `rabbitmq_management.listener.ip`.

The following config file with 3.7.13

management.tcp.ip = ::1
management.tcp.port = 15673

makes the node listen on `localhost` (according to lsof) and Wireshark suggests that communication with it using curl happens over [::1].

Nodes and CLI tools that need to communicate with a node (or multiple nodes) on host A

contact epmd on host A to find out what port(s) to use.