XDR Test 3 (Receive) failure

[Akshita Gupta]

Hi Team,

I am facing below error when trying to connect using below endpoint:
https://muexternal.athenahealth.com/xds/ws/v1/DIRECT::XDR


The endpoint is accessible from outside and I am successfully able to make a post request using below curl command:
curl -k -vvv https://muexternal.athenahealth.com/xds/ws/v1/DIRECT::XDR --data-binary XYZFileName

Can some one please help in resolving this.

Thanks
Akshita Gupta

[Sandeep Savarala]

You need to install the certs provided by the tool into your system to connect on HTTPS.

--
You received this message because you are subscribed to the Google Groups "Edge Test Tool (ETT)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-too...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/16fb5a0e-93ca-47cc-9d93-bdb0ce74543bn%40googlegroups.com.

[Akshita Gupta]

We have the required certificates installed. Still facing the issue. Also for test 8 and 5 we are facing error.

Can you please validate logs from your end once.

Thanks

[Sandeep Savarala]

Caused by: { code:"500" extendedCode:"0" reason:"XdsInternal error Service not available on this host:port (https://muexternal.athenahealth.com/xds/ws/v1/DIRECT::XDR)" reasonPhrase:"Internal Server Error"
Stack Trace:
  Service not available on this host:port (https://muexternal.athenahealth.com/xds/ws/v1/DIRECT::XDR)

This is what I see on the backend logs.

Make sure the endpoint is available to the outside world.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/f685cea2-82aa-4d58-9952-f1181697c122n%40googlegroups.com.

[Ross Helfand]

Hi Sandeep,

I am the sysadmin responsible for setting this up on our load balancer (Citrix Netscaler).  Note that we've setup mTLS for other services, so this is something I've done before.  Here is what I did:

1. Downloaded the XDR TLS certificates from https://github.com/onc-healthit/ett/raw/resources/certificates/common/xdr-tls/keyAndCert.zip and unzipped them on my laptop
2. Downloaded the Root CA cert from https://ett.healthit.gov/ett/#/direct and installed on the Netscaler (our load balancer)
3. Set the Netscaler to require client certificate authentication / mTLS

When I configure the load balancer to require client certificate authentication and try to test using 'curl' (with --cert, --key, and --cacert), I get an error "alert unknown ca."  If I disable client cert auth, I am able to connect.  So either the CA isn't correct, or the certificate that I downloaded to test does not match the CA.  Can you provide any further assistance?  I have had a hard time finding documentation for this process.

[Sandeep Savarala]

These are self signed certs-only used for test purposes. So the CA is not a real-world CA. Accpet the self signed cert from your side and continue to send/receive the XDR message.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/fe3fe859-b08c-4a20-851c-32b5ba720f8fn%40googlegroups.com.

[Ross Helfand]

Thank you.  I understand they are self-signed.  But I am telling the Netscaler to trust the client cert, and we're still getting that error.  This tells me that the CA cert I installed on the Netscaler does not trust the client certs XDR is using for the connection.

Are the links I provided in my implementation above correct?  Am I using the correct certificates to test with?

[Sandeep Savarala]

These are the certs.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/be65f9c2-1ca2-4dcf-87f3-010ac0486c56n%40googlegroups.com.

[Ross Helfand]

Yes, I have those.  So those are the *client certificates* used to connect to us.  But I also need the CA cert that signed these certs so that the load balancer will trust them.

[Sandeep Savarala]

The tool only provided these certs. You have to install these certs in your load balancer or tomcat or whatever server you are using.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/bb43f6d1-0ee7-4557-b428-4afa78f8a22dn%40googlegroups.com.

[Ross Helfand]

This is very odd.  The Netscaler needs a CA cert to compare with the client certificate used to connect.  That is how mutual TLS works.

For reference:  https://docs.citrix.com/en-us/citrix-adc/current-release/ssl/config-client-auth.html

Again, we have this configured with other services.  I don't really understand how I'm supposed to proceed here.

[Akshita Gupta]

Hi Sandeep,

Thanks for your support on this. But we are not able to configure the certs as required without using CA cert. 
Can we have a quick call to discuss this and complete the setup. 

We have certification pending, setting this endpoint is at at most priority for us. 

Thanks 
Akshita 

[Sandeep Savarala]

I am not sure how your network/load balancer is set up. If you are using a container like tomcat you need to edit the server configuration as here - https://tomcat.apache.org/tomcat-7.0-doc/ssl-howto.html#Edit_the_Tomcat_Configuration_File

The key and cert files are already provided. Essentially your server and the tool will have the same certs so that mutual trust is established.

<!-- Define an SSL Coyote HTTP/1.1 Connector on port 8443 -->
<Connector
           protocol="org.apache.coyote.http11.Http11AprProtocol"
           port="8443" maxThreads="200"
           scheme="https" secure="true" SSLEnabled="true"
           SSLCertificateFile="/usr/local/ssl/server.crt"
           SSLCertificateKeyFile="/usr/local/ssl/server.pem"
           SSLVerifyClient="optional" SSLProtocol="TLSv1+TLSv1.1+TLSv1.2"/>

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/410a497f-1932-4a1d-b210-fd5eed7662ffn%40googlegroups.com.

[Ross Helfand]

Hi Sandeep,

Maybe I'm misunderstanding how this test is supposed to work.  My understanding:

ETT tries to connect to our end-point, providing a client certificate that we are supposed to validate (for mutual TLS).

Our setup is:

Client -> Citrix Netscaler (terminates SSL) -> Apache (on HTTP)

The Netscaler terminates SSL.  So if the Client is trying to provide an SSL client certificate and requires Mutual TLS, the Netscaler has to be able to verify the authority of the client certificate.  The only way I'm able to do that is to bind the CA certificate to the VIP the application is listening on.  The VIP does not allow me to bind the certificates (cert.pem and key.pem) to the VIP.  If I try to bind it, I get an error:

> bind ssl vserver muexternal_cs_vs -certkeyName ett-selfsigned -CA -ocspCheck Optional
ERROR: Not a CA certificate

[Sandeep Savarala]

XDR Test 3 does not require TLS. It's optional. You can use an HTTP endpoint to test. 

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/c172fdb8-399e-4c62-a7e4-c73c0ca5fb97n%40googlegroups.com.

[Ross Helfand]

Hi Sandeep.  Ok, so re-reading your Tomcat config above, you say:

'SSLVerifyClient=optional'

This maybe should have been a clue for me.  I configured the Netscaler such that mutual TLS is *required*.  I can adjust that and make it optional, though I am still confused on how the ETT site will test that scenario.

Thanks,
Ross

[thrilok nath]

hi Dragon/Team ,

Could you please help us here with the set up for athena , we are unable to resolve connection issues with ETT tool to receive messages to athena.

We have shared details on the steps we followed for this set up in earlier conversations. 

Thanks in Advance.

Regards

Thrilok

[Sandeep Savarala]

Have you tried using the http endpoint?

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/19cd2d46-0698-461e-ba93-aa99269ac05bn%40googlegroups.com.