TLS with the authorization plugin

[Maxime Gaillard]

Hi,

Does the authorization plugin support TLS ?

We’ve got those errors while querying our distant authorization server over https:

- While setting HttpsVerifyPeers to true and leaving HttpsCACertificates empty :

orthanc_1      | E0313 10:30:00.106030 HttpClient.cpp:223] libCURL error: Problem with the SSL CA cert (path? access rights?)

orthanc_1      | E0313 10:30:00.124633 PluginsManager.cpp:197] Exception while invoking plugin service 27: Error in the network protocol

nginx-proxy_1  | nginx.1    | nginx-proxy 172.20.0.4 - - [13/Mar/2018:10:30:00 +0000] “GET /changes?limit=10 HTTP/1.1” 403 0 “-” “python-requests/2.9.1”

orthanc_1      | E0313 10:30:00.129175 PluginsManager.cpp:164] Error in the network protocol

- While setting HttpsVerifyPeers to false OR leaving it to true AND setting HttpsCACertificates to both included /etc/ssl/certs/ca-certificates.crt or downloaded http://curl.haxx.se/ca/cacert.pem :

orthanc_1      | E0313 10:32:00.151727 HttpClient.cpp:223] libCURL error: SSL connect error

orthanc_1      | E0313 10:32:00.152659 PluginsManager.cpp:197] Exception while invoking plugin service 27: Error in the network protocol

orthanc_1      | E0313 10:32:00.153421 PluginsManager.cpp:164] Error in the network protocol

nginx-proxy_1  | nginx.1    | nginx-proxy 172.20.0.4 - - [13/Mar/2018:10:32:00 +0000] “GET /changes?limit=10 HTTP/1.1" 403 0 “-” “python-requests/2.9.1"

Our certificate is issued by Cloudflare (Comodo).

Step to reproduce: setting any https url (ie. https://requestb.in/1hmtlhy1) as WebService on the demo: https://bitbucket.org/osimis/orthanc-setup-samples/src/45bbe8fc92a860c585ed1bdec16155277f02335c/docker/authorization-plugin/orthanc/orthanc.json?at=master&fileviewer=file-view-default#orthanc.json-5

Have anyone run into that issue? Any thougth?

[Maxime Gaillard]

Well! No one runs the webservice over HTTPS? ;-)

In the meantime, we looked deeper in the plugin codebase, tried a lot of workaround, but we’re still stuck.

Regards,

[Sébastien Jodogne]

Maxime,

Please be patient.

You reported this issue less than one week ago, and the Orthanc community needs time to investigate things. We work on many topics in parallel to make all users happy.

If you need professional support, please get in touch with Osimis:

http://www.osimis.io/en/services.html

Regards,

Sébastien-

[Maxime Gaillard]

Sébastien,

Thank you for your reply.

Sorry if my previous message lead to misinterpretation, I was not asking for any support nor investigation for now. I was just wondering if TLS is supported and if anyone use it in this case. That said, we should determine if the problem is on our side or not, and then make further investigations and eventually refer to Osimis for support.

Regards,

--

Maxime Gaillard
Co-founder at Naeka

        
        

[Bryan Dearlove]

Have you confirmed it Is the Auth plugin causing the issue? If you use Orthanc without it and enable SSL do you get the same issue?

I am running Orthanc with HTTPS through Apache without issue. On the connecting Orthanc instance I have ssl verify enabled and the certificate to verify the peer has the chain in PEM format, I took the cert and ca certificate and have them in there for verification.

[Sébastien Jodogne]

Dear Maxime,

Indeed, we cannot make such an investigation on our side without spending several hours to reproduce it.

HTTPS queries are expected to be supported, as Orthanc relies on libcurl and openssl. Also, carry on your testing on Ubuntu 16.04, as this is our development platform (this could be a Windows-related issue).

Please let us know the results of your investigations.

Regards,

Sébastien-

[Maxime Gaillard]

Thank you both.

Bryan, I don’t know if the issue is strictly related to the auth plugin or may be caused by the packaged libcurl/openssl usage. Our issue is not related to Orthanc being over HTTPS (it’s proxied) but is related to querying an HTTPS webservice with the auth plugin.

As you can see in the first email, you can reproduce in seconds using the official Demo here: https://bitbucket.org/osimis/orthanc-setup-samples/src/45bbe8fc92a860c585ed1bdec16155277f02335c/docker/authorization-plugin/orthanc/orthanc.json?at=master&fileviewer=file-view-default#orthanc.json-5

To reproduce:

1 - Checkout the demo

2 - Just add an https url in the WebService setting, like `https://requestb.in/1hmtlhy1`

3 - docker-compose up --build

And yes Sébastien, we used the official docker images from Osimis, so it’s tested on Ubuntu 16.04 and on different docker hosts (Linux, macOS).

[Deon Kuhn]

Hi Maxime

We were also having trouble with the auth plugin pointing to an https plugin but managed to solve it. It was a number of issues:

1. Make sure you are mounting the CA certificate into the orthanc plugin docker container:

     volumes:

        - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt

2. Make sure your ca-certificates are up to date.

apt-get upgrade ca-certificates

3. Ensure that that auth end-point domain matches that of the SSL certificate. In your case, it should be fine as your auth service seems to be external. In that case, see the below point.

4. Ensure your docker container can reach the external web, as by default they are not on a network that can reach external URIs.

services:

   orthanc-viewer:

...

     networks:

        - host

networks:

    host:

      external:

        name: host

Let me know if you have already fixed the issue, or if any of these suggestions hel

[Deon Kuhn]

As an additional note for point 4, you will need to remove the ports config from the docker service, as joining an external host network exposes any ports the container listens to automatically, and will produce an error if present.

Another point, self signed certificates may not work, as cURL Http verify peers don't like them.

[Maxime Gaillard]

Hi Deon,

Thank you for your feedback. If I understand your case, your app and orthanc are on the same docker-compose. It’s not our case, our app and orthanc can reach the web: I can `wget` orthanc from my app, and vice versa.

I tried to mount the host CA on orthanc, it’s a good idea to have orthanc started with latest certificates. But no luck, it still doesn’t work.

To me, the most interesting case here is that you can have the auth plugin working through SSL/TLS. It shows me it’s possible, but even with the demo app of orthanc it doesn’t work 0_O

The most annoying thing, is the totally blackbox we’re into: if Osimis' dockerfile was at least public, it would be possible (and fast!) to debug, fix and give our own feedback...

[Deon Kuhn]

Hi Maxime

Just some extra info, we were also able to get it to work by changing HttpsVerifyPeers=false, but you are not. Looking at your error message, 'SSL connect error', and the fact that you get that same error message in both cases of having the HttpsVerifyPeers=false and true (with the CA mounted), it tells me that the problem is not with CA, but with the actual SSL connection and cURL.

This might be a good place to start: https://stackoverflow.com/questions/35055715/ssl-connect-error-in-httr-curl

In my work yesterday I saw the cURL error message be quite accurate in pointing out where the problem might lie, i.e. if it says 'Problem with the SSL CA cert (path? access rights?)', that does actually mean there is no CA cert. And in cases where the CA is not being validated correctly, it will say 'Unable to validate peer with known CAs'.

Good luck!

[Maxime Gaillard]

Thanks again!

You right, I’m also pretty sure the issue is how the plugin is compiled. But Osimis doesn't release publicly the Dockerfile, so we’re completely blinded and we can’t help the community or Osimis debugging this :(

Even after multiple days of hard work, the docker image is still a blackbox to us. It’s a too bad, Orthanc is a amazing software, but we can’t help make it better.

I came to the point where I’m thinking to make our own docker images. It’s unimaginable to have a great (critical?) software in a bad blackbox without documentation (ie. the osimis/orthanc repo info is really outdated) or a least a dev looking into a real issue (explained & reproduced) for a month!

[Sébastien Jodogne]

Maxime,

I came to the point where I’m thinking to make our own docker images. It’s unimaginable to have a great (critical?) software in a bad blackbox without documentation (ie. the osimis/orthanc repo info is really outdated) or a least a dev looking into a real issue (explained & reproduced) for a month!

Why aren't you using the official Orthanc Docker images?

http://book.orthanc-server.com/users/docker.html

https://github.com/jodogne/OrthancDocker => definitely not a blackbox!

Regarding your sentence "a dev looking into a real issue (explained & reproduced) for a month", please be respectful and constructive, and keep in mind that:

• You are not the only Orthanc user. Your particular issue is not considered as more or less important than the issues reported by other users of Orthanc.
• You benefited from several answers from Orthanc contributors (Deon and Bryan) that (freely) investigated your issue.
• The Orthanc project is currently in the process of releasing Orthanc 1.3.2. The source code of the Orthanc ecosystem is extremely large, and we can't simultaneously work on each of its individual aspects. Please respect our long-term roadmap, even if the latter is not aligned with your own, particular use cases.
• I personally replied to your message on March 19th, asking you to be patient. In the meantime, I have been on vacations with my family since the end of March, which is something I also deserve.
• By forcing me to personally answer your complaints during my spare time in order to preserve the motivation of the Orthanc community, you are preventing me from spending time on solving your actual issue.
  
• As Orthanc is entirely free and open-source software, you can investigate the issue by yourself, even if the Orthanc core team is busy on other tasks. If you don't have the required skills, please find other ways on contributing back, which will free some of our volunteer time, so that we can study your issue sooner: http://book.orthanc-server.com/contributing.html
  

Sébastien-

[Alain Mazy]

Concerning the osimis/orthanc images being a black box, the reason is that we have a single repo that builds the Windows Installer, the osimis/orthanc images and the osimis/orthanc-pro images.  Unfortunately, the orthanc-pro images contains proprietary components and we can not therefore open-source the repo (well, actually, we could do it but nobody would be able to build some of the components).  Open-sourcing a repo actually requires a lot of work in documentation, making sure everybody can reproduce the builds on their side, answering questions, ...  Open-source is free for users but very expensive for maintainers and that's the reason we limit open-source to the essential components.

--

Alain Mazy / CTO 
a...@osimis.io / +32 494 31 67 27

	OSIMIS S.A.  Rue bois Saint-Jean 15/1BE-4102 Seraing  www.osimis.io

[Sébastien Jodogne]

Note that Alain's answer is only about plugins that are bound to a proprietary framework to which license restrictions apply (notably Microsoft SQL Server), and about the Windows installers edited by Osimis.

As far as the Orthanc ecosystem is concerned, all of its components are free software, and their source code is readily accessible from:

https://bitbucket.org/sjodogne/

On behalf of the Orthanc project,

Sébastien-

[Maxime Gaillard]

First, I personnaly apologize to Sebastien if my messages were too pushy. It wasn’t my intentions, your work is amazing, your commitment to OSS is incontestable, my messages were strictly work relative and the personal side must always come first. Again, my apologizes.

Thanks Sebastien and Alain for your feedback.

> Why aren't you using the official Orthanc Docker images?

Our deployment of Orthanc is planned step-by-step. This step is to use the free Osimis web viewer. A next step is to use the licenced Osimis web viewer. So, we build our app on the Osimis images.

> https://github.com/jodogne/OrthancDocker => definitely not a blackbox!

You’re right, and we checked and read a lot of Orthanc source code (and theirs plugins). We even reported private issues by speaking to Osimis (Frederic & Thibault).

> Concerning the osimis/orthanc images being a black box

Ok, got it :)

My frustration is due to the fact that I think beeing able to debug and help you/the community on this issue without disturbing Sebastien and all the Osimis team.

But yes, you’re right, I’ll wait and be at your disposal :)

Have a great week!

--

Maxime Gaillard
Co-founder at Naeka

        
        

[Sébastien Jodogne]

Maxime,

I really appreciate your clarification. I was not informed of your private discussions with Frédéric and Thibault.

Please be sure that, if nobody is able to help you further, I will personally have a look at your issue asap, even if I am unable to tell you when because of the 1.3.2 release process.

Regarding the Osimis Web viewer and the authentication plugin, if you are unable to readily use the Docker images by Osimis because of a lack of documentation, you could rebuild these two plugins from source on the top of the "jodogne/orthanc-plugins" image:

https://bitbucket.org/osimis/osimis-webviewer-plugin

https://bitbucket.org/osimis/orthanc-authorization

Regards,

Sébastien-

[Alain Mazy]

Hi Maxime,

We've finally opened the repo orthanc-builder so you may have a look at the osimis/orthanc images internals: https://bitbucket.org/osimis/orthanc-builder

I have also updated the osimis/orthanc:18.4.0 images such that you may use env-vars to configure the auth plugin (note that they are not exactly the same as the one you had defined in your authz.sh file).

Could you send us a modified version of the setup sample https://bitbucket.org/osimis/orthanc-setup-samples/src/18292ddd7f0f94d324de3f3a1732c3366cf465f3/docker/authorization-plugin/ that reproduces your TLS issue ?

Best regards,

Alain.

[Maxime Gaillard]

Hi Alain,

Oh, great! I took time to check it. It’s rather complete, I’ll take more time to check it later!

Thanks for the env-vars update, it was fast :)

About the TLS issue. I forked the repo and commited here: https://bitbucket.org/naeka/orthanc-setup-samples/commits/96cfc443aaa1cae108b69474102ba03cf1ad1be6

The webservices aren’t answering something orthanc is able to understand, as desired, the issue is before that.

Just:

$ cd docker/authorization-plugin/

$ docker-compose up --build 

Go to http://localhost/orthanc-restricted/app/explorer.html

You’ll see the libCURL error in the logs.

As you can see, it’s very strange, the issue only appears (for now?) when the cert issuer is Cloudflare (I realized this yesterday during my debug) and it seems to work when it’s Let’s Encrypt (ie. in my commit). The bad thing is that Cloudflare is a major SSL issuer :(

I don’t know if it’s about the ciphers Cloudflare uses, or something else… 0_O

Regards,

[Sébastien Jodogne]

Hello,

Back to the original problem. 

You can reproduce this issue by installing a minimalist Lua script in Orthanc (http://book.orthanc-server.com/users/lua.html), instead of launching a whole Docker infrastructure as discussed before. Here is this Lua script, that simply calls your URLs on Orthanc startup:

function Initialize()

   print("Contacting homepage:")

   HttpGet("https://www.orthanc-server.com/")

   print("Contacting Let's Encrypt:")

   HttpGet("https://httpbin.org/post")

   print("Contacting Cloudflare:")

   HttpGet("https://mockbin.org")

end

The corresponding minimal Orthanc configuration is:

{

  "LuaScripts" : [ "Sample.lua" ],

  "HttpsCACertificates" : "/etc/ssl/certs/ca-certificates.crt",

  "HttpVerbose" : true

}

Note how we use the certificates that are generated by the "ca-certificate" core package on Debian/Ubuntu systems. Note also that we have enabled a newly-introduced configuration called "HttpVerbose" that is pending in the mainline, in order to debug HTTP connections:

https://bitbucket.org/sjodogne/orthanc/commits/959bd8857eb5a98a0976f95fdee1d7712f7aa694

As mentioned in your tests, Orthanc can connect to its homepage and to the second site that uses Let's Encrypt, but fails while contacting Cloudflare:

$ ./Orthanc Sample.json

[...]

W0726 12:47:47.013740 main.cpp:667] Orthanc has started

W0726 12:47:47.013803 LuaContext.cpp:103] Lua says: Contacting homepage:

W0726 12:47:47.312361 LuaContext.cpp:103] Lua says: Contacting Let's Encrypt:

W0726 12:47:47.956914 LuaContext.cpp:103] Lua says: Contacting Cloudflare:

E0726 12:47:48.027587 HttpClient.cpp:230] libCURL error: SSL connect error

E0726 12:47:48.027734 LuaContext.cpp:258] Lua: Error in HttpGet() for URL https://mockbin.org

Here is the detailed log produced by the "HttpVerbose" option:

$ ./Orthanc Sample.json --verbose

[...]

W0726 12:52:28.018071 LuaContext.cpp:103] Lua says: Contacting Cloudflare:

* Rebuilt URL to: https://mockbin.org/

*   Trying 104.28.22.173...

* TCP_NODELAY set

* Connected to mockbin.org (104.28.22.173) port 443 (#2)

* ALPN, offering http/1.1

* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH

* successfully set certificate verify locations:

*   CAfile: /etc/ssl/certs/ca-certificates.crt

  CApath: none

* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

* Closing connection 2

I0726 12:52:28.074234 HttpClient.cpp:674] HTTP status code 0 after GET request on: https://mockbin.org

I0726 12:52:28.074354 HttpClient.cpp:679] cURL status code: 35

E0726 12:52:28.074423 HttpClient.cpp:230] libCURL error: SSL connect error

E0726 12:52:28.074660 LuaContext.cpp:258] Lua: Error in HttpGet() for URL https://mockbin.org

The "routines:SSL23_GET_SERVER_HELLO:sslv3" error message is clearly the culprit. It seems to indicate a problem with the Cloudfare certificate:

https://stackoverflow.com/q/48003263/881731

I am stuck at this point for time being. Any help from the Orthanc community is welcome.

Regards,

Sébastien-

On Tuesday, March 13, 2018 at 12:35:39 PM UTC+1, Maxime Gaillard wrote:

[Sébastien Jodogne]

OK, I have finally found the culprit. The problem was that ECDSA support was not activated in the OpenSSL library, that is called by the "libcurl" library, that is in turn used by Orthanc.

For reference, I was able to detect this issue thanks to the "sslscan" command-line tool (https://github.com/rbsec/sslscan). Here is an excerpt of this command:

$ ./sslscan mockbin.org

[...]

  Supported Server Cipher(s):

Preferred TLSv1.2  256 bits  ECDHE-ECDSA-CHACHA20-POLY1305 Curve P-256 DHE 256

Accepted  TLSv1.2  128 bits  ECDHE-ECDSA-AES128-GCM-SHA256 Curve P-256 DHE 256

Accepted  TLSv1.2  128 bits  ECDHE-ECDSA-AES128-SHA        Curve P-256 DHE 256

Accepted  TLSv1.2  128 bits  ECDHE-ECDSA-AES128-SHA256     Curve P-256 DHE 256

Accepted  TLSv1.2  256 bits  ECDHE-ECDSA-AES256-GCM-SHA384 Curve P-256 DHE 256

Accepted  TLSv1.2  256 bits  ECDHE-ECDSA-AES256-SHA        Curve P-256 DHE 256

Accepted  TLSv1.2  256 bits  ECDHE-ECDSA-AES256-SHA384     Curve P-256 DHE 256

Preferred TLSv1.1  128 bits  ECDHE-ECDSA-AES128-SHA        Curve P-256 DHE 256

Accepted  TLSv1.1  256 bits  ECDHE-ECDSA-AES256-SHA        Curve P-256 DHE 256

Preferred TLSv1.0  128 bits  ECDHE-ECDSA-AES128-SHA        Curve P-256 DHE 256

Accepted  TLSv1.0  256 bits  ECDHE-ECDSA-AES256-SHA        Curve P-256 DHE 256

  SSL Certificate:

Signature Algorithm: ecdsa-with-SHA256

The fix is implemented by the following changeset:

https://bitbucket.org/sjodogne/orthanc/commits/2a4ac74da1ed2f9d5682300894554cd70087bf1f

Pay attention to the fact that this fix is only available if OpenSSL is statically linked.

The fix is pending in the mainline and will be part of forthcoming Orthanc 1.4.2. Updated mainline Docker images and LSB binaries (Linux Standard Base) will be available today, as soon as our continuous integration server finishes its built.

Regards,

Sébastien-