RabbitMQ opens additional non-ssl port when SSL is enabled for clustering.

[Diptesh Chatterjee]

Hi everyone,

I have a two-node RabbitMQ cluster with SSL set up. I am using TLSv1.2 as the encryption standard. After starting RabbitMQ (version 3.6.0 with OTP 18.1) , I wanted to see the ports it opened, and I found these:

tcp        0      0 127.0.0.1:5672              0.0.0.0:*                   LISTEN      29827/beam.smp

tcp        0      0 0.0.0.0:51696               0.0.0.0:*                   LISTEN      29827/beam.smp

tcp        0      0 0.0.0.0:46324               0.0.0.0:*                   LISTEN      29827/beam.smp

tcp        0      0 127.0.0.1:15672             0.0.0.0:*                   LISTEN      29827/beam.smp

tcp        0      0 127.0.0.1:48227             127.0.0.1:4369              ESTABLISHED 29827/beam.smp

tcp        0      0 :::5671                     :::*                        LISTEN      29827/beam.smp

tcp        0      0 ::ffff::5671  ::ffff::59458 ESTABLISHED 29827/beam.smp

Out of these, I know the port 46324 is used for clustering (since I have SSL enabled). I can see it has opened another port 51696 which is not SSL-encrypted. Does anyone know what this port does and why this is not SSL-encrypted?

[Michael Klishin]

It can be anything from non-standard configuration to plugins. What does `rabbitmqctl environment` output?

What about `rabbitmqctl status`?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Diptesh Chatterjee]

Hi Michael,

Here's the information you requested:

Output of "rabbitmqctl environment":

Application environment of node 'rabbit@rabbitnode' ...

[{amqp_client,[{prefer_ipv6,false},{ssl_options,[]}]},

 {asn1,[]},

 {compiler,[]},

 {crypto,[]},

 {inets,[]},

 {kernel,

     [{error_logger,tty},

      {inet_default_connect_options,[{nodelay,true}]},

      {inet_dist_listen_max,25672},

      {inet_dist_listen_min,25672}]},

 {mnesia,[{dir,"/data/rabbitmq/rabbit@rabbitnode"}]},

 {mochiweb,[]},

 {os_mon,

     [{start_cpu_sup,false},

      {start_disksup,false},

      {start_memsup,false},

      {start_os_sup,false}]},

 {public_key,[]},

 {rabbit,

     [{auth_backends,[rabbit_auth_backend_internal]},

      {auth_mechanisms,['PLAIN','AMQPLAIN']},

      {backing_queue_module,rabbit_priority_queue},

      {channel_max,0},

      {cluster_keepalive_interval,10000},

      {cluster_nodes,{[],disc}},

      {cluster_partition_handling,autoheal},

      {collect_statistics,fine},

      {collect_statistics_interval,1000},

      {credit_flow_default_credit,{200,50}},

      {default_permissions,[<<".*">>,<<".*">>,<<".*">>]},

      {default_user,<<"rabbit">>},

      {default_user_tags,[administrator]},

      {default_vhost,<<"/">>},

      {delegate_count,16},

      {disk_free_limit,50000000},

      {enabled_plugins_file,"/data/rabbitmq/enabled_plugins"},

      {error_logger,

          {file,"/var/log/rabbitmq/rab...@rabbitnode.log"}},

      {fhc_read_buffering,false},

      {fhc_write_buffering,true},

      {frame_max,131072},

      {halt_on_upgrade_failure,true},

      {handshake_timeout,10000},

      {heartbeat,60},

      {hipe_compile,false},

      {hipe_modules,

          [rabbit_reader,rabbit_channel,gen_server2,rabbit_exchange,

           rabbit_command_assembler,rabbit_framing_amqp_0_9_1,rabbit_basic,

           rabbit_event,lists,queue,priority_queue,rabbit_router,rabbit_trace,

           rabbit_misc,rabbit_binary_parser,rabbit_exchange_type_direct,

           rabbit_guid,rabbit_net,rabbit_amqqueue_process,

           rabbit_variable_queue,rabbit_binary_generator,rabbit_writer,

           delegate,gb_sets,lqueue,sets,orddict,rabbit_amqqueue,

           rabbit_limiter,gb_trees,rabbit_queue_index,

           rabbit_exchange_decorator,gen,dict,ordsets,file_handle_cache,

           rabbit_msg_store,array,rabbit_msg_store_ets_index,rabbit_msg_file,

           rabbit_exchange_type_fanout,rabbit_exchange_type_topic,mnesia,

           mnesia_lib,rpc,mnesia_tm,qlc,sofs,proplists,credit_flow,pmon,

           ssl_connection,tls_connection,ssl_record,tls_record,gen_fsm,ssl]},

      {log_levels,[{connection,info}]},

      {loopback_users,[<<"guest">>]},

      {memory_monitor_interval,2500},

      {mirroring_flow_control,true},

      {mirroring_sync_batch_size,4096},

      {mnesia_table_loading_timeout,30000},

      {msg_store_credit_disc_bound,{2000,500}},

      {msg_store_file_size_limit,16777216},

      {msg_store_index_module,rabbit_msg_store_ets_index},

      {msg_store_io_batch_size,2048},

      {password_hashing_module,rabbit_password_hashing_sha256},

      {plugins_dir,"/usr/lib/rabbitmq/lib/rabbitmq_server-3.6.0/plugins"},

      {plugins_expand_dir,

          "/data/rabbitmq/rabbit@rabbitnode-plugins-expand"},

      {queue_index_embed_msgs_below,4096},

      {queue_index_max_journal_entries,32768},

      {reverse_dns_lookups,false},

      {sasl_error_logger,

          {file,"/var/log/rabbitmq/rab...@rabbitnode-sasl.log"}},

      {server_properties,[]},

      {ssl_allow_poodle_attack,false},

      {ssl_apps,[asn1,crypto,public_key,ssl]},

      {ssl_cert_login_from,distinguished_name},

      {ssl_handshake_timeout,5000},

      {ssl_listeners,[5671]},

      {ssl_options,

          [{cacertfile,"/data/rabbitmq/certs/ca/cacert.pem"},

           {certfile,"/data/rabbitmq/certs/server/cert.pem"},

           {keyfile,"/data/rabbitmq/certs/server/key.pem"},

           {verify,verify_peer},

           {versions,['tlsv1.2']},

           {ciphers,[{ecdhe_ecdsa,aes_256_cbc,sha384}]},

           {fail_if_no_peer_cert,true}]},

      {tcp_listen_options,

          [{backlog,128},

           {nodelay,true},

           {linger,{true,0}},

           {exit_on_close,false}]},

      {tcp_listeners,[{"127.0.0.1",5672}]},

      {trace_vhosts,[]},

      {vm_memory_high_watermark,0.4},

      {vm_memory_high_watermark_paging_ratio,0.5}]},

 {rabbit_common,[]},

 {rabbitmq_management,

     [{http_log_dir,none},

      {listener,[{port,15672},{ip,"127.0.0.1"}]},

      {load_definitions,none},

      {rates_mode,basic},

      {sample_retention_policies,

          [{global,[{605,5},{3660,60},{29400,600},{86400,1800}]},

           {basic,[{605,5},{3600,60}]},

           {detailed,[{10,5}]}]},

      {stats_mode,detailed}]},

 {rabbitmq_management_agent,[]},

 {rabbitmq_web_dispatch,[]},

 {ranch,[]},

 {sasl,[{errlog_type,error},{sasl_error_logger,false}]},

 {ssl,[{protocol_version,['tlsv1.2','tlsv1.1',tlsv1]}]},

 {stdlib,[]},

 {syntax_tools,[]},

 {webmachine,[{error_handler,rabbit_webmachine_error_handler}]},

 {xmerl,[]}]

Output of "rabbitmqctl status":

Status of node 'rabbit@rabbitnode' ...

[{pid,21610},

 {running_applications,

     [{rabbitmq_management,"RabbitMQ Management Console","3.6.0"},

      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.6.0"},

      {rabbit,"RabbitMQ","3.6.0"},

      {mnesia,"MNESIA  CXC 138 12","4.13.1"},

      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.6.0"},

      {webmachine,"webmachine","git"},

      {mochiweb,"MochiMedia Web Server","2.13.0"},

      {ssl,"Erlang/OTP SSL application","7.1"},

      {public_key,"Public key infrastructure","1.0.1"},

      {asn1,"The Erlang ASN1 compiler version 4.0","4.0"},

      {compiler,"ERTS  CXC 138 10","6.0.1"},

      {os_mon,"CPO  CXC 138 46","2.4"},

      {crypto,"CRYPTO","3.6.1"},

      {amqp_client,"RabbitMQ AMQP Client","3.6.0"},

      {rabbit_common,[],"3.6.0"},

      {inets,"INETS  CXC 138 49","6.0.1"},

      {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"},

      {syntax_tools,"Syntax tools","1.7"},

      {xmerl,"XML parser","1.3.8"},

      {sasl,"SASL  CXC 138 11","2.6"},

      {stdlib,"ERTS  CXC 138 10","2.6"},

      {kernel,"ERTS  CXC 138 10","4.1"}]},

 {os,{unix,linux}},

 {erlang_version,

     "Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:32:32] [async-threads:64] [hipe] [kernel-poll:true]\n"},

 {memory,

     [{total,76262192},

      {connection_readers,0},

      {connection_writers,0},

      {connection_channels,0},

      {connection_other,2808},

      {queue_procs,2808},

      {queue_slave_procs,0},

      {plugins,384336},

      {other_proc,20900184},

      {mnesia,72360},

      {mgmt_db,65184},

      {msg_index,47200},

      {other_ets,1417128},

      {binary,123008},

      {code,27229047},

      {atom,992409},

      {other_system,25025720}]},

 {alarms,[]},

 {listeners,

     [{clustering,48229,"::"},{amqp,5672,"127.0.0.1"},{'amqp/ssl',5671,"::"}]},

 {vm_memory_high_watermark,0.4},

 {vm_memory_limit,54182217318},

 {disk_free_limit,50000000},

 {disk_free,337776873472},

 {file_descriptors,

     [{total_limit,3996},

      {total_used,2},

      {sockets_limit,3594},

      {sockets_used,0}]},

 {processes,[{limit,1048576},{used,232}]},

 {run_queue,0},

 {uptime,273},

 {kernel,{net_ticktime,60}}]

Does any of this help narrow down the issue?

Regards,

Diptesh

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/rneVaXIgfhA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

[Michael Klishin]

Your node uses ports 5672, 5671 (AMQP over TLS), 48229 and 25672 (for clustering).

46324 is not used for clustering and I'm not sure what makes you believe it's
a standard port (clustering links are bi-directional so there are ephemeral ports used among
other things).

Nothing points at 51696 either. How did you set up your cluster "with TLS"? Are you
referring to TLS support for client connections or inter-node links?

What does

`rabbitmqctl eval 'erl_epmd:names().'`

output?

On 4 August 2016 at 16:52:30, Diptesh Chatterjee (diptesh...@gmail.com) wrote:
> Hi Michael,
>
> Here's the information you requested:
>

> *Output of "rabbitmqctl environment":*

> *Output of "rabbitmqctl status":*

> On Thu, Aug 4, 2016 at 4:04 PM, Michael Klishin wrote:
>
> > It can be anything from non-standard configuration to plugins. What does
> > `rabbitmqctl environment` output?
> > What about `rabbitmqctl status`?
> >
> > On 4 ago 2016, at 15:56, Diptesh Chatterjee

> > email to rabbitmq-user...@googlegroups.com.
> > To post to this group, send email to rabbitm...@googlegroups.com.

> > For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "rabbitmq-users" group.
> > To unsubscribe from this topic, visit https://groups.google.com/d/
> > topic/rabbitmq-users/rneVaXIgfhA/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to

> > rabbitmq-user...@googlegroups.com.
> > To post to this group, send email to rabbitm...@googlegroups.com.

> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --

> You received this message because you are subscribed to the Google Groups "rabbitmq-users"
> group.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
> To post to this group, send an email to rabbitm...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

+rabbitmq-users

RabbitMQ (actually, the Erlang runtime) opens client sockets in order to connect
to cluster peers (cluster nodes form a fully connected graph). The links are bi-directional.
TLS and non-TLS connections use separate sockets (this is also true for client connections).

On 4 August 2016 at 17:40:29, Diptesh Chatterjee (diptesh...@gmail.com) wrote:
> Yes, I am referring to TLS support for client connections and for
> inter-node links. I want to setup RabbitMQ with TLS over inter-node
> clustering links and for client connections. It looks like RabbitMq opens
> two dynamic ports when TLS is enabled. That's what my question is about.
> What are these two dynamic ports used for? And is there a way to make sure
> there is only one port opened for listening to inter-node connections when
> TLS is enabled?
>
> Here's a snippet of my rabbitmq.config file:
>
> [
> {rabbit, [
> {cluster_partition_handling, autoheal},
> {default_user, <<"rabbit">>},
> {default_pass, <<"rabbit">>},
> {default_user_tags, [administrator]},

> {default_permissions, [<<".*">>, <<".*">>, <<".*">>]},

> {collect_statistics_interval, 1000},
> {tcp_listeners, [{"127.0.0.1", 5672}]},

> {ssl_listeners, [5671]},
> {ssl_options, [{cacertfile,
> "/data/rabbitmq/certs/ca/cacert.pem"},
> {certfile,
> "/data/rabbitmq/certs/server/cert.pem"},
> {keyfile,
> "/data/rabbitmq/certs/server/key.pem"},

> {verify, verify_peer},

> {versions, ['tlsv1.2']},
> {ciphers,
> [{ecdhe_ecdsa,aes_256_cbc,sha384}]},
> {fail_if_no_peer_cert, true}]}

> ]
> },
> {rabbitmq_management, [
> {listener, [{port, 15672},
> {ip, "127.0.0.1"}
> ]
> },
> {stats_mode, detailed}
> ]}
> ].
>
> Here's the output you requested:
> {ok,[{"rabbit",45477},{"rabbitmq-cli-20189",35486}]}
>
> And the output from netstat -pan | grep beam
>
> tcp 0 0 0.0.0.0:45477 0.0.0.0:*
> LISTEN 4730/beam.smp

> tcp 0 0 127.0.0.1:5672 0.0.0.0:*

> LISTEN 4730/beam.smp
> tcp 0 0 0.0.0.0:34701 0.0.0.0:*
> LISTEN 4730/beam.smp

> tcp 0 0 127.0.0.1:15672 0.0.0.0:*

> LISTEN 4730/beam.smp
> tcp 0 0 127.0.0.1:39064 127.0.0.1:4369
> ESTABLISHED 4730/beam.smp

> tcp 0 0 :::5671 :::*

> LISTEN 4730/beam.smp
>
> As you can see, there is an additional port 34701 (dynamically selected
> apparently) which I am unable to account for.
> Can you help me figure out what this port does?
>
> I am using RabbitMQ 3.6.0 and Erlang OTP 18.1.
>
> Regards,
> Diptesh

[Diptesh Chatterjee]

The problem we have here is the ports opened are dynamic in nature. Despite setting the inet_dist_listen_min and inet_dist_listen_max (not present in the config I provided) parameters, the ports opened are still dynamic and do not respect these parameters. Is there at least a way to setup Erlang such that these parameters are obeyed?

Regards,

Diptesh

> > > > email to rabbitmq-users+unsubscribe@googlegroups.com.
> > > > To post to this group, send email to rabbitmq-users@googlegroups.com.

> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "rabbitmq-users" group.
> > > > To unsubscribe from this topic, visit https://groups.google.com/d/
> > > > topic/rabbitmq-users/rneVaXIgfhA/unsubscribe.
> > > > To unsubscribe from this group and all its topics, send an email to

> > > > rabbitmq-users+unsubscribe@googlegroups.com.
> > > > To post to this group, send email to rabbitmq-users@googlegroups.com.

> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "rabbitmq-users"
> > > group.
> > > To unsubscribe from this group and stop receiving emails from it, send

> > an email to rabbitmq-users+unsubscribe@googlegroups.com.
> > > To post to this group, send an email to rabbitmq-users@googlegroups.com.

[Michael Klishin]

I am not aware of any issues reported around kernel.inet_dist_listen_min. I found a similar

question on erlang-questions from 2006 that didn't arrive at any specific conclusion but I also see in the code

that the TLS distribution module in Erlang/OTP does retrieve those values from the config.

What does

`rabbitmqctl eval 'erl_epmd:names().'`

output?

Regards,

Diptesh

> > > {file,"/var/log/rabbitmq/rabbit...@rabbitnode.log"}},

> > > {file,"/var/log/rabbitmq/rabbit...@rabbitnode-sasl.log"}},

> > > > To post to this group, send email to rabbitm...@googlegroups.com.

> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "rabbitmq-users" group.
> > > > To unsubscribe from this topic, visit https://groups.google.com/d/
> > > > topic/rabbitmq-users/rneVaXIgfhA/unsubscribe.
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > rabbitmq-users+unsubscribe@googlegroups.com.

> > > > To post to this group, send email to rabbitm...@googlegroups.com.

> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "rabbitmq-users"
> > > group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > an email to rabbitmq-users+unsubscribe@googlegroups.com.

> > > To post to this group, send an email to rabbitm...@googlegroups.com.

> > > For more options, visit https://groups.google.com/d/optout.
> > >
> >
> > --
> > MK
> >
> > Staff Software Engineer, Pivotal/RabbitMQ
> >
> >
> >
>

--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

Also,

`rabbitmqctl eval 'inet:i().'`

[Diptesh Chatterjee]

This is the output of rabbitmq eval 'inet:i().'

Port  Module    Recv Sent Owner       Local Address        Foreign Address      State     Type   

6616  inet_tcp  0    0    <0.25.0>    *:45489              *:*                  ACCEPTING STREAM 

6624  inet_tcp  0    0    <0.25.0>    *:45672              *:*                  ACCEPTING STREAM 

6648  inet_tcp  4    21   <0.26.0>    localhost:51238      localhost:epmd       CONNECTED STREAM 

39914 inet_tcp  0    0    <0.375.0>   localhost:amqp       *:*                  ACCEPTING STREAM 

39946 inet6_tcp 0    0    <0.381.0>   *:amqps              *:*                  ACCEPTING STREAM 

39978 inet_tcp  0    0    <0.395.0>   localhost:15672      *:*                  ACCEPTING STREAM 

57898 inet_tcp  1168 915  <0.21755.0> 172.16.177.200:45672 172.16.177.200:48261 CONNECTED STREAM 

57914 inet_tcp  150  551  <0.21757.0> localhost:51778      localhost:45489      CONNECTED STREAM 

57922 inet_tcp  551  150  <0.21758.0> localhost:45489      localhost:51778      CONNECTED STREAM 

Regards,

Diptesh

[Diptesh Chatterjee]

Could it be that rabbitmqctl talks to the local RabbitMQ server on 45672? If so, is there a way to make this port listen to localhost only instead of exposing it for any incoming connection?

Regards,

Diptesh

[Michael Klishin]

You can see one connection to localhost:epmd (port 4369 by default IIRC). It is a *client* socket and it is not exposed to "incoming connections."

[Diptesh Chatterjee]

I'm more worried about 45489 and 45672. They are both server sockets. One of them is SSL and used for clustering (45672). However, the other one 45489 is non-SSL and a local beam process connects to it from port 51778. Data being exchanged on this connection is not encrypted. This is what I am more worried about. If you could tell me what this particular port is opened for it would be great. Is there any way to NOT open this port at all or make it listen to incoming connections from localhost ONLY?

Regards,

Diptesh

[Michael Klishin]

OK, so in the inet:i/0 output you see that both server sockets that we are not sure about

are used by the same Erlang process ("thread"), <0.25.0>. We can query that process for its

stack trace and other state bits:

`rabbitmqctl eval 'erlang:process_info(list_to_pid("<0.25.0>")).'`

(note that the <0.25.0> name will change between node restarts but the idea is hopefully clear)

On Fri, Aug 5, 2016 at 11:09 AM, Diptesh Chatterjee <diptesh...@gmail.com> wrote:

[Diptesh Chatterjee]

Looks like we are in gen_server:loop as evident from the following output:

[{registered_name,ssl_tls_dist_proxy},

 {current_function,{gen_server,loop,6}},

 {initial_call,{proc_lib,init_p,5}},

 {status,waiting},

 {message_queue_len,0},

 {messages,[]},

 {links,[<6652.29.0>,<6652.30.0>,#Port<6652.828>,<6652.21.0>,#Port<6652.827>]},

 {dictionary,[{'$ancestors',[ssl_dist_sup,net_sup,kernel_sup,<6652.10.0>]},

              {'$initial_call',{ssl_tls_dist_proxy,init,1}}]},

 {trap_exit,false},

 {error_handler,error_handler},

 {priority,max},

 {group_leader,<6652.9.0>},

 {total_heap_size,233},

 {heap_size,233},

 {stack_size,9},

 {reductions,4980},

 {garbage_collection,[{min_bin_vheap_size,46422},

                      {min_heap_size,233},

                      {fullsweep_after,65535},

                      {minor_gcs,0}]},

 {suspending,[]}]

Can you help me figure out how to make it listen on localhost? This is the first time I am playing with Erlang. Any help would be greatly appreciated.

Regards,

Diptesh

[Michael Klishin]

Its parent is a ssl_dist_sup process, so this is indeed a TLS inter-node link socket.

I cannot say how to make it listen on localhost for two reasons:

 * You haven't posted any details on how exactly you're configuring Erlang distribution to use TLS
 * It will take digging in the code to see how it may be different from the standard (non-TLS) distribution

On 5 August 2016 at 15:22:50, Diptesh Chatterjee (diptesh...@gmail.com) wrote:
> > ssl_dist_sup

[Diptesh Chatterjee]

We use Erlang OTP 18.1 off the shelf. Whatever configuration changes we make happens through rabbitmq.config and rabbitmq-env.conf files. Here's what these files look like:

rabbitmq.config:

[

    {rabbit, [

            {cluster_partition_handling, autoheal},

            {default_user, <<"rabbit">>},

            {default_pass, <<"rabbit">>},

            {default_user_tags, [administrator]},

            {default_permissions, [<<".*">>, <<".*">>, <<".*">>]},

            {collect_statistics_interval, 1000},

            {tcp_listeners, [{"127.0.0.1", 5672}]},

            {ssl_listeners, [5671]},

            {ssl_options, [{cacertfile, "/data/rabbitmq/certs/ca/cacert.pem"},

              {certfile, "/data/rabbitmq/certs/server/cert.pem"},

              {keyfile, "/data/rabbitmq/certs/server/key.pem"},

              {verify, verify_peer},

                                   {versions, ['tlsv1.2']},

                                   {ciphers, [{ecdhe_ecdsa,aes_256_cbc,sha384}]},

              {fail_if_no_peer_cert, true}]}

             ]

    },

    {rabbitmq_management, [

            {listener, [{port, 15672},

                        {ip, "127.0.0.1"}

                       ]

            },

            {stats_mode, detailed}

    ]}

].

rabbitmq-env.conf:

SSL_PATH=`erl -eval 'io:format("~p", [code:lib_dir(ssl, ebin)]),halt().' -noshell`

export ERL_SSL_PATH=`echo $SSL_PATH | cut -d "\"" -f 2`

RABBITMQ_PID_FILE=/data/rabbitmq/rabbitmq.pid

RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-pa ${ERL_SSL_PATH} -proto_dist inet_tls -ssl_dist_opt server_certfile /data/rabbitmq/certs/server/rabbit.pem -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

RABBITMQ_CTL_ERL_ARGS="-pa ${ERL_SSL_PATH} -proto_dist inet_tls -ssl_dist_opt server_certfile /data/rabbitmq/certs/server/rabbit.pem -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

RABBITMQ_PLUGINS_ERL_ARGS="-pa ${ERL_SSL_PATH} -proto_dist inet_tls -ssl_dist_opt server_certfile /data/rabbitmq/certs/server/rabbit.pem -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

We have not made any change in any other code or config, either on the Erlang OTP side or on RabbitMQ side.

Do you think we need to change any additional parameter or deploy do something on the Erlang side in order to achieve the expected behaviour? Just to make it clear, we are fine with either one of the following three outcomes:

1. We open only ONE port from Erlang (other than epmd) instead of two and enable TLS on connections to that port.

2, We open two ports and encrypt both with TLS. Given the tcpdump we captured from the ports, one of them is not TLS encrypted. Data is passed as plaintext, and that port is also used to exchange messages stored in queues, which is a worrying factor for us.

3. We open two ports and make the non-TLS port listen on localhost only.

Is there a way to make this happen?

Regards,

Diptesh

[Diptesh Chatterjee]

Michael,

Just wanted to update you about my findings. I upgraded Erlang OTP to 18.3 and that solved the problem. It seems the issue was with OTP 18.1 as it did not obey inet_dist_listen_min and inet_dist_listen_max settings. With OTP 18.3, it just opened two ports for outside connections - 25672 and 5671 as I had specified in the config file. Looks like the issue has been resolved. :-)

Regards,

Diptesh

[Michael Klishin]

Hi Diptesh,

Thank you very much for reporting back, good to know it's an OTP issue in that specific version!