Problems getting docker plugin dcm2bids-session to work in XNAT 1.7

[Stefan Pszczolkowski]

Hi there!

I've got an instance of XNAT 1.7 running on a server and I have configured a docker image plugin for dcm2bids (https://github.com/NrgXnat/docker-images/tree/master/dcm2bids-session) and installed the docker image on the server (by doing "docker pull xnat/dcm2bids-session"). I've also automatised the launching of the command for when a session is archived (SessionArchived event)

The problem is every time I upload a new subject, the image gets launched but it fails.

The command-line command looks like this:

python dcm2bids_wholeSession.py --session XNAT_E00011 --overwrite False --host $XNAT_HOST --user $XNAT_USER --pass $XNAT_PASS --upload-by-ref False --dicomdir /dicom --niftidir /nifti

The contents of StdOut.log are:

Making DICOM directory /dicom
Making NIFTI image directory /nifti/IMG
Making NIFTI BIDS directory /nifti/BIDS
Get project and subject ID for session ID XNAT_E00011.
Request Failed
HTTPSConnectionPool(host='$xnat_host', port=443): Max retries exceeded with url: /data/experiments XNAT_E00011?handler=values&columns=project%2Csubject_ID&format=json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',))

And the enviromental variables look as follows:

{"XNAT_HOST":"http://mslaprad02.nottingham.ac.uk:8080","XNAT_USER":"7a8bfb70-2c79-44f6-a573-25f654806251","XNAT_PASS":"******"}

I have the impression that the problem is that the host is not being prepended in the URL the images uses to query the project and subject ID. I also noted that the host variable in the HTTPSConnectionPool request looks like $xnat_host (in lowercase) and that the port is 443 instead of 8080. What might I be doing wrong? Do I have to somehow configure the host and port so they get correctly filled out in the python script?

Thanks a lot!

[John Flavin]

Stefan,

Sorry this is happening. I have seen things like this before, so I have ideas what might be going wrong. But I need to know a few more things that might help me see where the problem is.

1. What version of the container service plugin do you have? If you don't have the latest version (1.5.1) you can download it here: https://github.com/NrgXnat/container-service/releases/tag/1.5.1. Just replace the older plugin jar you have with the new one, and try launching again.

2. What version of the dcm2bids-session command do you have? The latest version of that command is 1.5. 

The reason I ask about the versions is that there is a behavior that I've been tinkering with in recent container service versions and versions of this command; it looks to me from the error message you’re seeing that something is going wrong in exactly this area. Hopefully with all the latest versions of everything, the problem will go away.

Flavin

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at https://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

[Stefan Pszczolkowski]

Hi John

Thanks for your reply. Unfortunately I do have the exact versions you mention for both the container service plugin and dcm2bids-session.

Best wishes!

[John Flavin]

Can you check in your command definition (Administer > Plugin settings > Images and Commands > find your command > Edit Command) whether the property "override-entrypoint" is set to "true" (with no quotes)? 

That property is supposed to tell docker to execute your command through a shell interpreter, which would handle the environment variable expansion.

[Stefan Pszczolkowski]

Thanks! That did the trick. It had a null value

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussion+unsub...@googlegroups.com.

[Stefan Pszczolkowski]

Hi John

Related to this question, I would like to report a small bug in XNAT. When I was setting up this command, I copied the corresponding json file from the NRG XNAT repository, which effectively contains the property "override-entrypoint" set to true. However, when copy-pasting the json string into XNAT for the first time and then saving, the property ends up with a null value. Therefore it is necessary to edit the json again and set the property to true.

Best wishes,

Stefan

On Friday, 13 July 2018 18:42:45 UTC+1, John Flavin wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussion+unsub...@googlegroups.com.

[John Flavin]

Thanks for reporting that. This bug has been fixed in the upcoming container service version.

Flavin

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/12a7ef2a-509f-43ff-84d0-54f943c26e06%40googlegroups.com.

[gABoCas g]

Dear John, 

I got the same error than Stefan, I have Xnat v1.7.4.1 (docker-composer version). I have tried the container service v1.5.1 and v1.6.0 RC1 with the parameter "override-entrypoint" set to true and I got the same error. In addition, the command dcm2niix works without error (although I don't know where the generated nifti files are saved). 

Thanks in advance,

Gabriel

my stdout.log:

Making DICOM directory /dicom

Making NIFTI image directory /nifti/IMG

Making NIFTI BIDS directory /nifti/BIDS

Get project and subject ID for session ID val_lab05_E00001.

Request Failed

    HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /xnat/data/experiments/val_lab05_E00001?handler=values&columns=project%2Csubject_ID&format=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f52d4d15110>: Failed to establish a new connection: [Errno 111] Connection refused',))

[John Flavin]

You’re in luck! You are encountering an problem that I have also just encountered, and I figured out how to fix it. So the whole thing is still in my short-term memory. Good thing I've been dogfooding the xnat-docker-compose project recently.

Short answer: Set the Processing URL (Administer > Site Administration > Pipeline Settings > Processing URL) to http://host.docker.internal. And if you intend to mount any data, which dcm2bids doesn't do but other containers will, you'll need to set up the Path Translation by editing your docker server settings in Administer > Plugin Settings > Container Server Setup.

Longer answer:

When using xnat-docker-compose, your XNAT is accessible to your host machine at 'localhost'. So 'localhost' is what gets set inside your XNAT as the site URL. Then when you go to run a container, the container service gets that 'localhost' value out of the site URL and sticks it into the XNAT_HOST environment variable. Then containers are then free to use that environment variable as they wish.

The problem here is that 'localhost' inside a container doesn't route to your host machine anymore. From a container, 'localhost' routes back to the container itself! And, obviously, we don't have any XNAT running inside the container. So your container tries to connect to an XNAT at 'localhost' but finds nothing there.

The solution is to set some value for your host's network address into the Processing URL (Administer > Site Administration > Pipeline Settings > Processing URL). [Side note: the "pipeline settings" part of that is inaccurately named, since this setting isn't just for pipelines. It should probably be changed to something more generic like "Processing Settings".] When you set a value there, the container service will use it preferentially over the site URL to set the XNAT_HOST value.

What should you set there? Well, if your host machine has a stable IP address, that will work. But what I use is the special fake URL that docker internally routes back to the host machine: http://host.docker.internal. I know it works for docker for mac, I think it also works on docker for windows. This thread (https://github.com/docker/for-linux/issues/264) tells me it does not work on docker for linux.

And on another topic, the xnat-docker-compose setup requires you to set up path translation. The reason is that the data is in a different location inside the XNAT container than it is outside the container on your host machine. When container service asks XNAT "hey, where is this data?", XNAT will give it a path like /data/xnat/whatever, which is what it sees. But that data is mounted in from your host machine, where it lives at something like /Users/you/xnat-docker-compose/xnat-data/whatever. When docker needs to launch a new container it needs to know the host path to be able to properly mount the data. 

I wrote this long answer for you, then decided it would be good to preserve this info on some wiki pages. So you can also consult https://wiki.xnat.org/display/CS/Path+Translation and https://wiki.xnat.org/display/CS/Processing+URL which say more or less the same thing as this.

Flavin

-- 
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at https://groups.google.com/group/xnat_discussion.

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/b561afe9-7994-4d92-916c-c3832eed34ae%40googlegroups.com.

[gABoCas g]

Thanks for your help and support, I also feel lucky, it is great that you found and fixed that bug. I am going to try and I'll let you know how it goes.

Best,

Gabriel

[gABoCas g]

Thanks again for all your help. The DNS http://host.docker.internal works for authenticating to the xnat server (now my dcm2niix container works); however, when I tried to run the dcm2bids-session container, I got this error:

Making DICOM directory /dicom

Making NIFTI image directory /nifti/IMG

Making NIFTI BIDS directory /nifti/BIDS

Get project and subject ID for session ID val_lab06_E00001.

Request Failed

    404 Client Error: Not Found for url: http://host.docker.internal/data/experiments/val_lab06_E00001?handler=values&columns=project%2Csubject_ID&format=json

Do you have any idea of what could I be doing wrong?

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussion+unsub...@googlegroups.com.

[gABoCas g]

I just realized that the Processing URL should be set to http://host.docker.internal/xnat

[John Flavin]

Ah, yes, good catch. I was assuming your XNAT was available at localhost. But if yours is at localhost/xnat, you would have to use host.docker.internal/xnat. 

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.

To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at https://groups.google.com/group/xnat_discussion.

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/beda8dca-fef8-4a7e-802f-dc18802cda86%40googlegroups.com.

[gABoCas g]

now I am trying to run the bids-mriqc; however, it seems like the xnat2bids-setup command ignores the Path Translation parameter because when I checked the command history, the "xnat-host- and container-host- paths are the same. Have you noticed that before? 

This is the StdOut.log of the xnat2bids-setup command:

Input dir: /input

Output dir: /output

Checking for session structure in /input

Found SCANS directory. Checking scans for BIDS data.

Checking scan 17.

SKIPPING. Scan 17 does not have a BIDS directory.

Checking scan 28.

SKIPPING. Scan 28 does not have a BIDS directory.

Done checking all scans.

Checking subdirectories of /input.

Checking for session structure in /input/SCANS

STOPPING. Could not find SCANS directory.

No BIDS data found in session SCANS.

No BIDS data found anywhere in inputDir /input.

Best,

Gabriel

[John Flavin]

That could definitely be a bug. I'll take a look.

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.

To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at https://groups.google.com/group/xnat_discussion.

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/2f12e118-c4f1-4378-ab30-b153e79fb798%40googlegroups.com.

[gABoCas g]

has this possible bug has been checked or fixed? 

[John Flavin]

This bug has been fixed in the latest container service development code. Let me check if that has made it into a release...

Nope, not in a release yet. This was the first thing that got committed to the development branch right after the 1.6.0 release.

If you are able to build your own jar from the dev branch of the container service repo (https://github.com/NrgXnat/container-service) that would be the fastest way for you to get this fixed. If not, maybe Matt could put out a 1.6.1 bug fix release for you (probably with just this one fix and not the other stuff about adding user controls).

Flavin

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/36e3c3e1-33f0-4636-b7cf-04b47ad423e6%40googlegroups.com.

[gABoCas g]

Thanks a lot for your help. I compiled the jar by myself (I guess is the one at build/libs/containers-1.6.1-SNAPSHOT-fat.jar). I will try to run the bids-mriqc again and I 'll let you know.

[gABoCas g]

Dear Flavin,

The containers v1.6.1 fixed the bug. Thanks for all your help.

Best,

Gabriel

[Hassen Ben Tkhayat]

Hi Flavin,

so i encountred the same problem, i've done what you advised. COnfigured the PATH_Translation and Processing URL.  This time the XNAT finds the images but not the containers. I'am using xnat-docker-compose so for the processing URL i had to write the ip of the hostmachine. But still same Problem.

I get this as ERROR: {"message":"No such container: cc7f00b18f3abf5d41942f43586544a4052a726776efea2ba811c0b9ea376416"} 

Hassen

[kel...@wustl.edu]

Hi Hassen,

  Are you deploying xnat-docker-compose on a machine with Docker Desktop for Mac? I am seeing this same issue on my machine running Desktop Docker and have yet to find a solution.  One note: on Docker for Windows or Mac, the processing URL can be set to `http://host.docker.internal`. This should allow the XNAT container to launch other containers on the Docker host. 

-Matt

[Hassen Ben Tkhayat]

Hi Matt,

i'am using Docker for linux, i can't use `http://host.docker.internal as processing URL. I tried using the ip adress as processing URL but it didn't work.

Hassen

[Richard Cole]

***From root home ...  I now .. this is just for testing...

[root@ch2dt3 ~]# git clone https://github.com/NrgXnat/xnat-docker-compose

[root@ch2dt3 ~]# cd xnat-docker-compose

[root@ch2dt3 xnat-docker-compose]# docker-compose up -d

Optional - Waiting for Tomcat to get going and showing Xnat Startup Process..

[root@ch2dt3 xnat-docker-compose]# docker-compose logs -f --tail=20 xnat-web

Monitor logs to see when server start completed

[root@ch2dt3 xnat-docker-compose]# cd xnat/plugins/

[root@ch2dt3 plugins]# wget https://github.com/NrgXnat/container-service/releases/download/2.1.0/containers-2.1.0-fat.jar

[root@ch2dt3 plugins]# cd ../../

***have done this multiple times, so this step may not be necessary

If necessary, Restart tomcat after adding .jar to plugins

[root@ch2dt3 xnat-docker-compose]# docker-compose exec xnat-web sh /usr/local/tomcat/bin/shutdown.sh

[root@ch2dt3 xnat-docker-compose]# docker-compose exec xnat-web sh /usr/local/tomcat/bin/startup.sh

**Pull the images before attempting to pull from XNAT Container Service to resolve the "timeout" issue with some larger images..

[root@ch2dt3 xnat-docker-compose]# docker pull xnat/freesurfer711-xnat:latest

[root@ch2dt3 xnat-docker-compose]# docker pull xnat/dcm2niix:latest

Login to XNAT and setup Container Service Information

1) Administer > Site Administer > Pipeline Settings > Processing URL 

http://host.docker.internal 

2) Administer > Site Administer > Plugin Settings > Local socket

XNAT Path Prefix "/data/xnat"

Docker Server Path Prefix "/root/xnat-docker-compose/xnat-data"

3) Administer > Site Administer > Plugin Settings > Images $ Commands > Add New Image

(Run Imaage Add per prevously pulled docker images to put the correct ties in the XNAT DB.)

a. Image Name  > xnat/freesurfer711-xnat

Version Tag > :latest

Pull Image

b. Image Name > xnat/dcm2niix

Version Tag > :latest

Pull Image