problems running SHORE on DSI data

[Isotta Rigoni]

Hi Sebastien,

I am trying to run the cmp on DSI data but, for some reason, the pipeline did not work.

As I have read that it is not recommended to do eddy current correction on DSI data for it high b value, I have removed that step from the preprocessing pipeline, which works.

My problem is that the cmp stopped working, without any clear error message, at later stages; here are the last lines of the log file:

01/diffusion_pipeline/registration_stage/mr_convert_b0/b0.nii.gz --transform /output_dir/nipype/sub-01/diffusion_pipeline/registration_stage/SyN_registration/final1Warp.nii.gz --transform /output_dir/nipype/sub-01/diffusion_pipeline/registration_stage/SyN_registration/final0GenericAffine.mat
210528-07:15:24,589 nipype.workflow INFO:
       [Node] Finished "diffusion_pipeline.registration_stage.apply_warp_brainmask".
210528-07:15:24,589 nipype.workflow INFO:
       [Node] Setting-up "diffusion_pipeline.diffusion_stage.reconstruction.dipy_SHORE" in "/output_dir/nipype/sub-01/diffusion_pipeline/diffusion_stage/reconstruction/dipy_SHORE".
210528-07:15:24,594 nipype.workflow INFO:
       [Node] Running "dipy_SHORE" ("cmtklib.interfaces.dipy.SHORE")

Killed

Do you have any idea why the SHORE reconstruction algorithm did not work? Do I need to install some additional package? Might it be a problem of storage space? Otherwise, could I use other algorithms on DSI data?

Thanks in advance,

Isotta 

[Sebastien Tourbier]

Hello Isotta!

Based on the error, I am suspecting you ran out of memory. How much ram memory are you using?

Best,

Seb

--
You received this message because you are subscribed to the Google Groups "cmtk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/9570ebd9-b388-466f-bb75-09d64a030f79n%40googlegroups.com.

[Isotta Rigoni]

Hi Seb,

I have just checked and my computer has 32 G of ram, so I guess that's not the problem....

Is any other package needed to run SHORE?

Best,

Isotta

To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/CAJJe7azMGjeYiFu-hWajhFS3Jq7nOcHpjLc1E3ve-K8GTCQUoA%40mail.gmail.com.

[Sebastien Tourbier]

SHORE is provided by Dipy which is installed inside the docker container. When a process is killed, this is usually due to problems in resources. What OS are you using? Which CMP3 version?

SHORE DSI reconstruction is indeed successfully ran during continuous integration testing on the sample dataset on circleci on a medium linux VM with 7.5 GB RAM (results from the last RC4 release: https://app.circleci.com/pipelines/github/connectomicslab/connectomemapper3/738/workflows/dbeb1330-49ed-4300-82d0-96770ac9aa8b/jobs/2534)

Despite your computer having 32GB of RAM memory, docker might not be configured to use all. For instance, on macosx, the default setting is using 2GB and should be adjusted (see https://docs.docker.com/docker-for-mac/)

Hope this helps,

Seb

To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/CAPNJry1wv9t7qCaY0CvcUfjMXPLAee9SYLW%3DQz2O%3D8JNXqp-wA%40mail.gmail.com.

[Isotta Rigoni]

Hi Seb,

I am working with the 3.0.0-RC4 connectome mapper, in Ubuntu 20.04. I have checked the ram used by the docker while running the diffusion pipeline on my data and there is no limitation: the 32 G are fully allocated to be used by the docker image. However, I have checked the docker stats to monitor the amount of memory used by the container and it quickly jumps to the limit of 32 G, and then the cmp is killed. So, it looks like there is something wrong with the container.

Do you have any idea about what could be going wrong here?

Thanks in advance,

Best,

Isotta

You received this message because you are subscribed to a topic in the Google Groups "cmtk-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cmtk-users/boO1bvlECK0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cmtk-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/CAJJe7awWvsYSrcceqpt0PBYvR_aFG-3K6OzS1-mqgkx-vDTX%3DA%40mail.gmail.com.

[Sebastien Tourbier]

Hi Isotta,

I had to leave for family reasons and I am now back to work.

So it seems it is really resource related and this means that the reconstruction need even more memory.

I see two solutions:

- using a machine with more memory, or

- resampling your data to a lower resolution (in the diffusion preprocessing steps), but this would come at the cost of the connectivity matrix estimation

Best,

Seb