multiscalebrainparcellator Error during parcellation

26 views
Skip to first unread message

Ariana Familiar

unread,
Jan 15, 2020, 9:32:04 AM1/15/20
to cmtk-users
Hello! I'm receiving the following error during a single subject analysis, I tried restarting Docker and re-running but still receive the error:

Processing /output_dir/freesurfer/sub-01/mri/ROIv_scale4.nii.gz:

200115-14:10:49,530 nipype.workflow WARNING:

[Node] Error on "anatomical_pipeline.parcellation_stage.Lausanne2018_parcellation" (/output_dir/cmp/sub-01/tmp/anatomical_pipeline/parcellation_stage/Lausanne2018_parcellation)

200115-14:10:49,559 nipype.workflow ERROR:

Node Lausanne2018_parcellation failed to run on host e4589b50fe0c.

200115-14:10:49,563 nipype.workflow ERROR:

Saving crash info to /app/crash-20200115-141049-root-Lausanne2018_parcellation-0f4a5c54-3bdc-402a-ab3b-50f120efe26d.txt

Traceback (most recent call last):

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/plugins/linear.py", line 44, in run

    node.run(updatehash=updatehash)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command

    result = self._interface.run(cwd=outdir)

  File "/opt/conda/lib/python2.7/site-packages/nipype/interfaces/base/core.py", line 511, in run

    runtime = self._run_interface(runtime)

  File "/opt/conda/lib/python2.7/site-packages/cmtklib/parcellation.py", line 1506, in _run_interface

    crop_and_move_datasets(self.inputs.parcellation_scheme,self.inputs.subject_id, self.inputs.subjects_dir)

  File "/opt/conda/lib/python2.7/site-packages/cmtklib/parcellation.py", line 2704, in crop_and_move_datasets

    raise Exception('File %s does not exist.' % d[0])

Exception: File /output_dir/freesurfer/sub-01/mri/ROIv_scale4.nii.gz does not exist.


Traceback (most recent call last):

  File "/opt/conda/bin/multiscalebrainparcellator", line 102, in <module>

    anat_pipeline.process()

  File "/opt/conda/lib/python2.7/site-packages/cmp/multiscalebrainparcellator/pipelines/anatomical/anatomical.py", line 464, in process

    anat_flow.run()

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/workflows.py", line 597, in run

    runner.run(execgraph, updatehash=updatehash, config=self.config)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/plugins/linear.py", line 44, in run

    node.run(updatehash=updatehash)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command

    result = self._interface.run(cwd=outdir)

  File "/opt/conda/lib/python2.7/site-packages/nipype/interfaces/base/core.py", line 511, in run

    runtime = self._run_interface(runtime)

  File "/opt/conda/lib/python2.7/site-packages/cmtklib/parcellation.py", line 1506, in _run_interface

    crop_and_move_datasets(self.inputs.parcellation_scheme,self.inputs.subject_id, self.inputs.subjects_dir)

  File "/opt/conda/lib/python2.7/site-packages/cmtklib/parcellation.py", line 2704, in crop_and_move_datasets

    raise Exception('File %s does not exist.' % d[0])

Exception: File /output_dir/freesurfer/sub-01/mri/ROIv_scale4.nii.gz does not exist.

Sebastien Tourbier

unread,
Jan 15, 2020, 11:28:06 AM1/15/20
to cmtk-users
Hello!

It reminds me an old bug. Please could you indicate me which version (docker image tag) of the multiscalebrainparcellator are you using?

I am currently working on deploying the docker image for the last version (v1.1.0-hotfix) .
Continuous integration testing is ongoing. The new image should available on dockerhub overnight.

I would strongly suggest you to use this new version which includes a number of other bug fixes and improvements.
Also, I would also suggest to remove all the folders (cmp/, nipype/, tmp/) except the freesurfer/ folder in your output_dir/

Ariana Familiar

unread,
Jan 15, 2020, 11:39:44 AM1/15/20
to cmtk-...@googlegroups.com
Ok, I can wait for the new version! Here is my current version (“latest”):

REPOSITORY                                     TAG                 IMAGE ID            CREATED             SIZE
sebastientourbier/multiscalebrainparcellator   latest              ab979e2c4a6b        6 months ago        18.5GB

Thank you!

--
You received this message because you are subscribed to the Google Groups "cmtk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/26de41ac-49ff-4c17-8679-aeeb42a4fed4%40googlegroups.com.

Sebastien Tourbier

unread,
Jan 16, 2020, 5:06:08 AM1/16/20
to cmtk-...@googlegroups.com

Hi Ariana,

Just to let you know the latest version (v1.1.0-hotfix) has been successfully gone through continuous integration testing on CircleCI and it is now deployed on DockerHub.
In the future, I would advice you to always take one of the release e.g. tag vX.Y.Z which is always archived on Zenodo, so that in the long term you can reprocess your dataset with the exact same version.

To pull this new version v1.1.0-hotfix

$ docker pull sebastientourbier/multiscalebrainparcellator:v1.1.0-hotfix

To remove the version with tag latest

1. Get the Docker image ID with the following command
$ docker images

2. Remove the docker image ID corresponding to the tag latest.
$ docker rmi ad91fad1e8d9
where ad91fad1e8d9 is for instance the image ID corresponding to the tag latest and  identified by running the first command ($ docker images)

NOTE: It seems the docker image with the tag latest is very old (6 months) and stopped to be automatically updated when some changes were pushed to the master branch. It is also why the input flag --fs_license could not be found as it was implemented later. I will take care of fixing it.

Example of successful call during continuous integration testing

docker run -it --rm \
   
-v /tmp/data/ds005:/bids_dir \
    -v /
tmp/ds005/derivatives:/output_dir \
    -v $FS_LICENSE:/
bids_dir/code/license.txt \
    sebastientourbier
/multiscalebrainparcellator:v1.1.0-hotfix \
   
/bids_dir /output_dir participant --participant_label 01 \
   
--isotropic_resolution 1.0 \
   
--thalamic_nuclei \
   
--hippocampal_subfields \
   
--brainstem_structures \
   
--fs_license /bids_dir/code/license.txt \
   
--skip_bids_validator

Here is link to the full execution report on CircleCI:  https://circleci.com/gh/sebastientourbier/multiscalebrainparcellator/430

I guess in your case a proper call might be:
docker run -it --rm \
    -v $parent_dir:/bids_dir \
-v $out_dir:/output_dir \
    -v $FREESURFER_HOME/license.txt:/bids_dir/code/license.txt \
    sebastientourbier
/multiscalebrainparcellator:v1.1.0-hotfix \
   
/bids_dir /output_dir participant --participant_label 01 \
   
--isotropic_resolution 1.0 \
   
--fs_license /bids_dir/code/license.txt \
   
--skip_bids_validator

Let me know if it works!

Best,
Sebastien

Ariana Familiar

unread,
Jan 16, 2020, 3:24:41 PM1/16/20
to cmtk-users
Thank you so much for your help! :)  I tried running with the new version (v1.1.0-hotfix) on existing freesurfer data and it gets much further in the pipeline, but now getting an error at the "CombineParcellations" stage (I've also attached the full log file):

200116-20:10:57,667 nipype.workflow INFO:

[Node] Running "parcCombiner" ("cmtklib.parcellation.CombineParcellations")

200116-20:10:57,670 nipype.interface INFO:

------------------------------------------------------------

200116-20:10:57,671 nipype.interface INFO:

Combination of Lausanne2008 parcellation with new structures

200116-20:10:57,673 nipype.interface INFO:

------------------------------------------------------------

200116-20:10:57,674 nipype.interface INFO:

  > Freesurfer subject directory: /output_dir/freesurfer/sub-01

200116-20:10:57,695 nipype.workflow WARNING:

[Node] Error on "anatomical_pipeline.parcellation_stage.parcCombiner" (/output_dir/nipype/sub-01/anatomical_pipeline/parcellation_stage/parcCombiner)

200116-20:10:57,725 nipype.workflow ERROR:

Node parcCombiner failed to run on host 738e9cb131ba.

200116-20:10:57,756 nipype.workflow ERROR:

Saving crash info to /app/crash-20200116-201057-root-parcCombiner-60621ba1-ba0c-46e5-9736-ff69c4164381.txt

Traceback (most recent call last):

  File "/opt/conda/envs/py27msbparc/lib/python2.7/site-packages/nipype/pipeline/plugins/linear.py", line 48, in run

    node.run(updatehash=updatehash)

  File "/opt/conda/envs/py27msbparc/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/envs/py27msbparc/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/envs/py27msbparc/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command

    result = self._interface.run(cwd=outdir)

  File "/opt/conda/envs/py27msbparc/lib/python2.7/site-packages/nipype/interfaces/base/core.py", line 369, in run

    runtime = self._run_interface(runtime)

  File "/opt/conda/envs/py27msbparc/lib/python2.7/site-packages/cmtklib/parcellation.py", line 439, in _run_interface

    iflogger.info(bcolors.WARNING + '  Subfields image (Left hemisphere) not provided' + bcolors.ENDC)

NameError: global name 'bcolors' is not defined



To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "cmtk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-...@googlegroups.com.
pypeline.log

Sebastien Tourbier

unread,
Jan 17, 2020, 5:58:32 AM1/17/20
to cmtk-...@googlegroups.com
Great to know you got much further in the pipeline.
It seems however there might be an import missing somewhere.
I just opened an issue on github (https://github.com/sebastientourbier/multiscalebrainparcellator/issues/11) and will fix it today.
Let you know when the new version with the fix will be available.

Best,
Sebastien

To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/0a69aad3-3847-47bb-a3b6-370b3435c391%40googlegroups.com.

Ariana Familiar

unread,
Jan 17, 2020, 3:59:13 PM1/17/20
to cmtk-users
I pulled v1.1.0-hotfix2 and it works perfectly! Thank you again :) This tool is awesome.

Ariana

Sebastien Tourbier

unread,
Jan 20, 2020, 7:19:31 AM1/20/20
to cmtk-...@googlegroups.com
Great!!
Thanks also for your feedback :)

Best,
Sebastien

To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cmtk-users/097fa435-8b36-44a7-b53e-38d30718f618%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages