XnatRestClient vs https

[Miklos Espak]

Hi,

I have an XNAT site that has a valid, *not* self-signed certificate.

However, if I want to XnatRestClient to download anything through https, I get this error:

Tue May 21 17:08:53 BST 2013 mespak@https://hipposeg.cs.ucl.ac.uk/:java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:168)
    at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:422)
    at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:460)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1188)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1215)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1199)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:166)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1195)
    at java.net.URLConnection.getContent(URLConnection.java:688)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getContent(HttpsURLConnectionImpl.java:406)
    at java.net.URL.getContent(URL.java:1024)
    at WSTool.manageLogin(WSTool.java:360)
    at WSTool.perform(WSTool.java:124)
    at XNATRestClient.main(XNATRestClient.java:289)


The command was this:

XNATRestClient -u *** -p *** -host https://hipposeg.cs.ucl.ac.uk -remote "/data/archive/projects" -m GET


The same command works if I replace https by http.

The server is accessible only through https from outside of the university, and the pipelines fail because they use XnatRestClient to download the images. The host parameter is currently propagated with a value from the form that launches the pipeline. I can hard-code the http://hipposeg.cs.ucl.ac.uk value.

I am wondering if there is a more flexible solution, passing a parameter to the XnatRestClient to make it accept the certificate?

Thank you,
Miklos

[Herrick, Rick]

Try doing the same call using curl. This should be something like:

 

curl -X GET -u xxx:yyy https://hipposeg.cs.ucl.ac.uk/data/archive/projects

 

If that works, try to run the calls through a proxy to see what the difference in headers is.

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Miklos Espak]

Hi Rick,

Yes, this works.

What proxy? Should I install one? I am not familiar with this kind of debugging.

Actually, the XnatRestClient commands are invoked by the pipeline, and it works perfectly with http but not https.

I tried to add the site certificate to a Java keystore:

keytool -import -trustcacerts -alias root -file hipposeg.cs.ucl.ac.uk.crt -keystore keystore.jks

and specify the keystore as a parameter of the XnatRestClient command

XNATRestClient -u *** -p *** -ts /opt/tomcat/conf/keystore.jks -tsPass *** -host https://hipposeg.cs.ucl.ac.uk -remote "/data/archive/projects" -m GET

but no success.

Is this how it should work?

Thanks,
Miklos

[Herrick, Rick]

You’d need something that can decode the https traffic. The main tool that I use for this sort of thing is Fiddler2, with the drawback being that that is only usable on Windows (you can however run Fiddler2 on a Windows machine but that will work fine as long as you can reach the proxy machine and server destination). For cross-platform compatibility, there are Charles and BurpSuite, but I’m not sure how to make those work to capture the https traffic.

 

You can also have a look at the logs on your server. I think you guys have an Apache HTTPD front-end running to manage the SSL transaction. That’s where things are breaking, so the HTTPD logs may give you some idea as to what’s going wrong on that end.

 

To clarify one point, this is really a pro-active thing on the server side. The fault is not technically speaking with the client, since whatever’s servicing the call is closing the connection early. That said, there’s clearly something in the call coming from the client that’s causing the server to do that.

--

You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Miklos Espak]

On Tue, May 21, 2013 at 7:15 PM, Herrick, Rick <herr...@mir.wustl.edu> wrote:

You’d need something that can decode the https traffic. The main tool that I use for this sort of thing is Fiddler2, with the drawback being that that is only usable on Windows (you can however run Fiddler2 on a Windows machine but that will work fine as long as you can reach the proxy machine and server destination). For cross-platform compatibility, there are Charles and BurpSuite, but I’m not sure how to make those work to capture the https traffic.

 

You can also have a look at the logs on your server. I think you guys have an Apache HTTPD front-end running to manage the SSL transaction. That’s where things are breaking, so the HTTPD logs may give you some idea as to what’s going wrong on that end.

Not on this server, here tomcat is listening on 80 and 443. There is nothing in the tomcat log, nor in the log of the webapp.

The problem is on the server itself (linux), so Fiddler2 is not an option.

The first line of the error is suspicious for me:

Tue May 21 19:20:31 BST 2013 mespak@https://hipposeg.cs.ucl.ac.uk/:java.net.SocketException: Connection reset

    at java.net.SocketInputStream.read(SocketInputStream.java:168)
    at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:422)
    at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:460)

...


Should not it be https://mes...@hipposeg.cs.ucl.ac.uk ?

 

 

To clarify one point, this is really a pro-active thing on the server side. The fault is not technically speaking with the client, since whatever’s servicing the call is closing the connection early. That said, there’s clearly something in the call coming from the client that’s causing the server to do that.

 

Not sure, I would expect something to appear in the tomcat log. Maybe with some verbose option?

Miklos

[Herrick, Rick]

The problem is on the server, but what Fiddler2 (or another proxy) can do is to make clear precisely what is being sent on each request. That can help you determine why the request from XNATRestClient is causing the reset while the request from curl is being completed properly. Something in the XNATRestClient request is different from curl’s request and that’s the root of your problem.

 

Should you want to try Charles, there’s a free evaluation period that you can use. All you have to do with XNATRestClient is add the –proxy and –proxyPort options to the command line and point those at the proxy. For curl, it’s something like –proxy address:port. Any proxy that’s handling SSL will require certificate substitution, which basically means you’ll get the proxy’s SSL certificate instead of the destination server certificate and THAT usually requires you to import the proxy SSL certificate into your trusted certs repo, e.g.:

 

http://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

 

Regarding the logs, I’m surprised nothing’s getting output though. You can try running Tomcat with these options on the JVM:

 

-Djavax.net.debug=all -Djavax.net.ssl=ssl.handshake

 

That may spit a BUNCH more info that you’d really like to see, but you should at least get info about your SSL transactions.

 

From: xnat_di...@googlegroups.com [mailto:xnat_di...@googlegroups.com] On Behalf Of Miklos Espak
Sent: Tuesday, May 21, 2013 1:33 PM
To: xnat_di...@googlegroups.com
Subject: Re: [XNAT Discussion] XnatRestClient vs https

 

On Tue, May 21, 2013 at 7:15 PM, Herrick, Rick <herr...@mir.wustl.edu> wrote:

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Miklos Espak]

It was the Java version on the client side. With Java6 it does not work. Java7 is fine.

It is weird, because the error occurred only if I tried to access the site through https. Probably some libraries have been built with Java7 with no compatibility option.

Anyway, thank for your help, as always. :)

Miklos