MQTT TLS Authentication and Proxy Protocol

[Sebastiano Suraci]

Hi,

I've noticed that in this setup (TLS is terminated by HA proxy, after proper client validation):

MQTT CLIENT => (TLS) => HA-PROXY => (PROXY-PROTOCOL, NO TLS) => RABBITMQ

TLS Authentication won't work as user credentials do not get passed to rabbitmq:

MQTT login failed: no credentials provided

Is there any setting like VerneMQ's 
listener.tcp.proxy_protocol_use_cn_as_username  
that keeps TLS auth working also with proxy protocol ? Any workaround ?

Tnx
Sebastiano

[Luke Bakken]

Hello,

Did you see this in the documentation? https://www.rabbitmq.com/mqtt.html#proxy-protocol

That setting may not have an effect on authentication, however.

In the case where HA proxy is used to terminate TLS, what actually gets passed to RabbitMQ?

Finally, to expedite investigation you should share a complete, working example, including client code that uses client cert authentication. It would take me a long time to guess how your environment is set up and most likely I would get it wrong.

Thanks -

Luke

[Sebastiano Suraci]

Hi Luke,
you're right, may details were missing ... I'll try to add them:
I actually have this configuration that is working without problems:

MQTT CLIENT (based on PAHO C library, with client certificate) => TLS => RABBITMQ (SSL Based Auth) with the following configuration:

[{rabbit,        

           [{proxy_protocol,    false},
            {tcp_listeners,    [5672]},
            {loopback_users, ["guest", "proxyuser"]},
            {ssl_options, [
            {cacertfile, "/etc/rabbitmq/ssl/rabbitmq-ca.mqtt.test.crt"},
            {certfile,   "/etc/rabbitmq/ssl/mqtt.test.crt"},
            {keyfile,    "/etc/rabbitmq/ssl/mqtt.test.key"},
            {verify,     verify_peer},
            {fail_if_no_peer_cert, true}
                ]},
        {auth_backends, [rabbit_auth_backend_internal  ]},
        {reverse_dns_lookup, false}
]},
{rabbitmq_mqtt, [
                  {proxy_protocol,  false},
                  {allow_anonymous,  false},
                  {exchange,         <<"amq.topic">>},
                  {vhost,            <<"mqtt-test">>},
                  {subscription_ttl, 1800000},
                  {prefetch,         10},
                  {ssl_listeners,    [8883]},
                  {tcp_listeners,    [1883]},
                  {tcp_listen_options, [{backlog,   128},
                                        {nodelay,   true}]},
                  {ssl_cert_login, true},
                  {ssl_cert_login_from, common_name}

    ]
}].

Now I would like to use HAProxy as a load balancer to:

• verify client certificate
• terminate TLS connection
• load balance between different rabbitmq instances 

In this case the setup is as follows:

RABBITMQ CLIENT (based on PAHO C library, with client certificate) => TLS => HA-PROXY => (PROXY-PROTOCOL, NO TLS) => RABBITMQ

In this setup:

• I enabled proxy_protocol on previous rabbitmq config
• I used this configuration for ha-proxy mqtt backend:

listen mqtt
  bind *:8883 ssl crt /usr/local/etc/haproxy/ssl/mqtt-test.pem ca-file /usr/local/etc/haproxy/ssl/rabbitmq-ca.mqtt-test.crt verify required
  mode tcp
  #Use this to avoid the connection loss when client subscribed for a topic and its idle for sometime
  option clitcpka # For TCP keep-alive
  timeout client 3h #By default TCP keep-alive interval is 2hours in OS kernal, 'cat /proc/sys/net/ipv4/tcp_keepalive_time'
  timeout server 3h #By default TCP keep-alive interval is 2hours in OS kernal
  option tcplog
  balance leastconn
  server rabbitmq_1 rabbitmq_01:1883 send-proxy-v2-ssl-cn
  server rabbitmq_2 rabbitmq_02:1883 send-proxy-v2-ssl-cn

Proxy protocol should correctly forward TLS details to RabbitMQ (thanks to this PR: https://github.com/rabbitmq/rabbitmq-server/pull/3175 ), but it seems that RabbitMQ does not get original client certificate CN (which is sent thanks to send-proxy-v2-ssl-cn configuration, http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#5.2-send-proxy-v2-ssl-cn ) to be used as SSL login name , infact RabbitMQ complains that:

MQTT login failed: no credentials provided

VerneMQ has an option to use client certificate CN (forwarded by proxy-protocol) as ssl login name: https://vernemq.com/blog/2020/01/13/vernemq-and-the-proxyprotocol.html#what-about-tls 

I was wondering if there's (or is planned) something similar on rabbitmq, and if there is some workaround for the above use case.

Many thanks
Sebastiano 

[Luke Bakken]

Hi  Sebastiano,

Thank you for the detailed information.

Proxy protocol should correctly forward TLS details to RabbitMQ (thanks to this PR: https://github.com/rabbitmq/rabbitmq-server/pull/3175 ), but it seems that RabbitMQ does not get original client certificate CN (which is sent thanks to send-proxy-v2-ssl-cn configuration, http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#5.2-send-proxy-v2-ssl-cn ) to be used as SSL login name , infact RabbitMQ complains that:

That pull request only makes certain parts of the TLS connection available to display in the management UI. It does not add anything with regard to authentication.

I just spent some time reviewing the code that deals with the proxy protocol and I do not see support for passing on the CN information for authentication purposes. If you would like to take your previous email with all of the details, and turn it into a feature request here, we could evaluate it -

https://github.com/rabbitmq/rabbitmq-server/issues

Thanks,

Luke 

[Sebastiano Suraci]

OK thanks,
I opened this issue https://github.com/rabbitmq/rabbitmq-server/issues/4607 and also added some considerations about performance and security.

Best Regards
Sebastiano