.NET Core 3.1 running in Windows container gives TLS Unknown CA

[mjc]

Hi team!

Thank you for developing and maintaining RabbitMQ! I have a .NET Core 3.1 web API application that uses RabbitMQ.Client NuGet (6.2.1) to connect and use RabbitMQ (3.8.11).

I am running an application in a Windows Container; RabbitMQ is installed as a standalone locally on Windows 2019.

Docker version:

Client: Mirantis Container Runtime

 Version:           19.03.14

 API version:       1.40

 Go version:        go1.13.15

 Git commit:        e820475

 Built:             12/17/2020 19:30:16

 OS/Arch:           windows/amd64

 Experimental:      false

Server: Mirantis Container Runtime

 Engine:

  Version:          19.03.14

  API version:      1.40 (minimum version 1.24)

  Go version:       go1.13.15

  Git commit:       57e3a05525

  Built:            12/17/2020 19:29:00

  OS/Arch:          windows/amd64

  Experimental:     false

A part of the C# code, where we connect to the RabbitMQ:

_factory = new ConnectionFactory()

{

HostName = _cfg.RabbitMqHost,

VirtualHost =  _cfg.RabbitMqVHost,

Port =  _cfg.RabbitMqPort,

UserName = "",

Password = "",

Ssl = new SslOption()

{

Enabled = true,

Certs = new X509CertificateCollection {  _cfg.GetRabbitMqCertificate() },

Version =  _cfg.GetSslProtocol(),

ServerName =  _cfg.RabbitMqHost

}

};

_factory.AuthMechanisms.Clear();

_factory.AuthMechanisms.Add(new ExternalMechanismFactory());

_connection = _factory.CreateConnection();

_session = _connection.CreateModel();

Here is a RabbitMQ config:

log.console = true

log.console.level = debug

loopback_users.guest = false

listeners.tcp = none

listeners.ssl.default = 5671

ssl_options.cacertfile = C:\Certs\ca_certificate.pem

ssl_options.certfile = C:\Certs\server_certificate.pem

ssl_options.keyfile = C:\Certs\server_key.pem

ssl_options.fail_if_no_peer_cert = true

ssl_options.verify = verify_peer

auth_mechanisms.1 = EXTERNAL

ssl_cert_login_from = common_name

ssl_options.versions.1 = tlsv1.2

ssl_options.password = mypass

Here is a docker-compose.yml

version: '3.8'

services:

     myapp:

        container_name: myapp

        ports:

          - '5018:80'

          - '9059:9059'

        image: myimage

I've generated certificates with the tls-gen (basic), and I think I've set up things correctly (I've used Server alt name as 192.168.3.111 when generating the certs). I am using PFX as a client, which is also installed in the user and machine cert store on the Win 2019. I've also checked, and the generated CA is also in the cert store.

If I run my Web API locally without docker containers, the authentication, and app work.

I do not use usernames and passwords, only TLS cert for auth and encryption. 

But when I run it in Windows Docker Container (mcr.microsoft.com/dotnet/aspnet:3.1-nanoserver-1809 AS base) I get 2021-02-08 16:14:30.200 [error] <0.3113.0> closing AMQP connection <0.3113.0> (192.168.3.111:54215 -> 192.168.3.111:5671):

{inet_error,{tls_alert,{unknown_ca,"TLS server: In state connection received CLIENT ALERT: Fatal - Unknown CA\n"}}}

I do not want to use AcceptablePolicyErrors = SslPolicyErrors.RemoteCertificateChainErrors in my client code.

Also, another thing. If I am using MassTransit, there is also no error.

Any hint would be great. Thank you!

[Eray Arslan]

Hi.

I don't know the answer actually but i can give you some suggestions. Couple days ago, i tossed exact error the "Unknown CA". I was just running dotnet core 3.1 application on my localhost, not on docker container.
Anyway, i fixed the problem by adding/copying the CA certificate to the Trusted Certificates. But actually i dont know how you can do it inside docker container.

Its only a guess. Here is my setup code. Good luck.

X509Store store = new X509Store(StoreName.My, StoreLocation.LocalMachine); // also you may want to change this for docker

store.Open(OpenFlags.ReadOnly);

var factory = new ConnectionFactory();

factory.Ssl.Enabled = true;

factory.Ssl.ServerName = "server_name";

            

factory.Ssl.CertPath = @"C:\path\to\client_certificate.p12";

factory.Ssl.Certs = store.Certificates;

factory.HostName = "somedomain.com";

factory.Port = 5671;

factory.Ssl.Version = SslProtocols.Tls12;

factory.UserName = "username";

factory.Password = "password";

factory.Ssl.CertPassphrase = "passphrase";

factory.VirtualHost = "/";
var connection = factory.CreateConnection()

8 Şubat 2021 Pazartesi tarihinde saat 18:39:42 UTC+3 itibarıyla miha.j...@gmail.com şunları yazdı:

[mjc]

Hi!

Thank you for your reply. It does not help.

It looks like I can not access the Windows Trust store from the Windows Containers.

I'll need to investigate further.

Regards, Miha

[mjc]

I will try to add cert to the docker image tomorrow.

[mjc]

No luck so far.

I've tried to inject the cert to Docker (which is not a good idea). I've also installed the pfx on the host.

FROM microsoft/nanoserver:sac2016 as tool

COPY ["/client_demo.pfx", "client_demo.pfx"]

COPY --from=tool /Windows/System32/certoc.exe .

USER ContainerAdministrator

RUN certoc.exe -ImportPFX -p password My client_demo.pfx

The image successfully builds, also the certs look like are imported.

Then in my client code I tried:

//assuming it goes in the isolated cert store

X509Store store = new X509Store(StoreName.My, StoreLocation.CurrentUser); //also tried StoreLocation.LocalMachine

store.Open(OpenFlags.ReadOnly);

var coll = new X509CertificateCollection();

coll.AddRange(store.Certificates);

Still I get back

RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable

|  ---> System.AggregateException: One or more errors occurred. (Authentication failed, see inner exception.)

    |  ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.

|  ---> System.ComponentModel.Win32Exception (0x80090326): The message received was unexpected or badly formatted.

|    --- End of inner exception stack trace ---

|    at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)

    |    at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)

|    at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)

    at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)

|    at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)

|    at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)

| --- End of stack trace from previous location where exception was thrown ---

|    at System.Net.Security.SslStream.ThrowIfExceptional()

|    at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)

|    at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)

|    at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)

    |    at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__64_2(IAsyncResult iar)

|    at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)

| --- End of stack trace from previous location where exception was thrown ---

If I add AcceptablePolicyErrors = SslPolicyErrors.RemoteCertificateChainErrors to the SSLOptions then it works.

:)

[mjc]

Thank you!

I've made it. It was my stupid mistake :). Instead of injecting the cert into the base image, I've injected the CA cert only into the build image. I provide a client certificate from an environment variable. Also, I've had to use sdk:3.1-nanoserver-1809 as the base image.

Anyway, here is a dockerfile:

--------------------------------------------------------------------------------------------------------  

FROM mcr.microsoft.com/dotnet/sdk:3.1-nanoserver-1809 AS base

WORKDIR /App

FROM microsoft/nanoserver:sac2016 as tool

ENV ASPNETCORE_ENVIRONMENT="Docker"

EXPOSE 80

FROM mcr.microsoft.com/dotnet/sdk:3.1-nanoserver-1809 AS build

WORKDIR /src

COPY ["Service/Service.csproj", "Service/"]

COPY . .

WORKDIR "/src/Service"

RUN dotnet build "Service.csproj" -c Release -o /App/build -p:EnvironmentName=Docker

FROM build AS publish

RUN dotnet publish "Service.csproj" -c Release -o /App/publish --framework netcoreapp3.1 /p:EnvironmentName=Docker

FROM base AS final

WORKDIR /App

COPY --from=build /src/Docker/ca_certificate.crt ca_certificate.crt

COPY --from=tool /Windows/System32/certoc.exe .

USER ContainerAdministrator

RUN certoc.exe -addstore root ca_certificate.crt

COPY --from=publish /App/publish .

ENTRYPOINT ["dotnet", "Service.dll", "--console"]

--------------------------------------------------------------------------------------------------------

And a client code did not change:

--------------------------------------------------------------------------------------------------------  

_factory = new ConnectionFactory()

{

HostName = RabbitMqHost,

VirtualHost = RabbitMqVHost,

Port = RabbitMqPort,

UserName = "",

Password = "",

Ssl = new SslOption()

{

Enabled = true,

Certs = new X509CertificateCollection { GetRabbitMqCertificate() },

Version = GetSslProtocol(),

ServerName = RabbitMqHost

}

};

_factory.AuthMechanisms.Clear();

_factory.AuthMechanisms.Add(new ExternalMechanismFactory());

-------------------------------------------------------------------------------------------------------- 

 

Good stuff, I hope it helps someone.

Miha