RuntimeError with synthstrip

[Radha]

Helo,

I am trying out MRIQC on my Mac in a docker container.

I get the following RuntimeError and I can't figure out what has gone wrong:

[Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-p03..anat..sub-p03_T1w.nii.gz/synthstrip)
220621-15:08:14,893 nipype.workflow ERROR:
     Node synthstrip.a0 failed to run on host 628494dbde06.
220621-15:08:14,985 nipype.workflow ERROR:
     Saving crash info to /out/logs/crash-20220621-150814-root-synthstrip.a0-5ef6cd9b-762e-4be6-b2fe-ce4dd695271c.txt
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/mriqc/engine/plugin.py", line 60, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 419, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 814, in _run_interface
    self.raise_exception(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 741, in raise_exception
    raise RuntimeError(
RuntimeError: Command:
synthstrip -b 1 -i /tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-p03..anat..sub-p03_T1w.nii.gz/pre_n4/clipped_corrected.nii.gz --model /opt/freesurfer/models/synthstrip.1.pt -o clipped_corrected_desc-brain.nii.gz -m clipped_corrected_desc-brain_mask.nii.gz
Standard output:

Standard error:
Killed
Return code: 137

Traceback (most recent call last):
  File "/opt/conda/bin/mriqc", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.8/site-packages/mriqc/cli/run.py", line 158, in main
    mriqc_wf.run(**_plugin)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/workflows.py", line 632, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/opt/conda/lib/python3.8/site-packages/mriqc/engine/plugin.py", line 184, in run
    self._clean_queue(jobid, graph, result=result)
  File "/opt/conda/lib/python3.8/site-packages/mriqc/engine/plugin.py", line 256, in _clean_queue
    raise RuntimeError("".join(result["traceback"]))
RuntimeError: Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/mriqc/engine/plugin.py", line 60, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 419, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 814, in _run_interface
    self.raise_exception(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 741, in raise_exception
    raise RuntimeError(
RuntimeError: Command:
synthstrip -b 1 -i /tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-p03..anat..sub-p03_T1w.nii.gz/pre_n4/clipped_corrected.nii.gz --model /opt/freesurfer/models/synthstrip.1.pt -o clipped_corrected_desc-brain.nii.gz -m clipped_corrected_desc-brain_mask.nii.gz
Standard output:

Standard error:
Killed
Return code: 137

I would appreciate any help.

Thanks!

[Stef Meliss]

I have experienced the same problem on two different macs and would highly appreciate any advice in this regard.

Thanks!

[Oscar Esteban]

That's very likely memory issues

--
You received this message because you are subscribed to the Google Groups "mriqc-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mriqc-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mriqc-users/835bc8a8-4fba-4f76-8a63-a6d75cd7d9f8n%40googlegroups.com.

[Stef Meliss]

Thank you so much for the prompt reply.

I had set the memory within Docker Desktop to 7.9GB, what would you recommend increasing it to?

Cheers

[Oscar Esteban]

sorry for the slow response, still traveling.

I believe 16GB should be enough. With 8, I can see it could need more.

Cheers,

oe

To view this discussion on the web visit https://groups.google.com/d/msgid/mriqc-users/258d48d3-7b1e-48cb-b599-a7bf16135401n%40googlegroups.com.

[Victoria Shevchenko]

Hello,

I'm having the same issue and no memory manipulations helped so far. Also, 8GB of RAM is the max available on my Docker desktop. The runtime error is always thrown at the stage of SynthStrip. Is there a way maybe to bypass this step? I'm trying to quality check COBRE data and it seems that it's trying to do SynthStrip for each slice. I've attached the BIDS folder structure of one subject.

The last command I used:

docker run -it --memory="6g" --rm -v /Users/VictoriaShevchenko/Documents/PhD/SCZ/COBRE:/data:ro -v /Users/VictoriaShevchenko/Documents/PhD/SCZ:/out nipreps/mriqc:latest /data /out participant --participant_label A00000368 --no-sub

Here, I tried to reduce the RAM to 6, but without this flag the error is the same, so it makes no difference.

The output I got:

221102-15:12:00,348 cli IMPORTANT:

    Running MRIQC version 22.0.6:

      * BIDS dataset path: /data.

      * Output folder: /out.

      * Analysis levels: ['participant'].

221102-15:51:11,990 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-01_echo-02_T1w.nii.gz/synthstrip)

221102-15:51:15,379 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-01_echo-01_T1w.nii.gz/synthstrip)

221102-15:51:17,226 nipype.workflow ERROR:

    Node synthstrip.a01 failed to run on host 1f55701eb564.

221102-15:51:21,362 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-01_echo-03_T1w.nii.gz/synthstrip)

221102-15:51:21,214 nipype.workflow ERROR:

    Saving crash info to /out/logs/crash-20221102-155117-root-synthstrip.a01-23abbc1e-39dc-4d10-a4ad-86a8f56abc90.txt

Traceback (most recent call last):

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node

    result["result"] = node.run(updatehash=updatehash)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command

    raise NodeExecutionError(

nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node synthstrip.

RuntimeError: subprocess exited with code 137.

Traceback (most recent call last):

  File "/opt/conda/bin/mriqc", line 8, in <module>

    sys.exit(main())

  File "/opt/conda/lib/python3.9/site-packages/mriqc/cli/run.py", line 159, in main

    mriqc_wf.run(**_plugin)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run

    runner.run(execgraph, updatehash=updatehash, config=self.config)

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 184, in run

    self._clean_queue(jobid, graph, result=result)

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 256, in _clean_queue

    raise RuntimeError("".join(result["traceback"]))

RuntimeError: Traceback (most recent call last):

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node

    result["result"] = node.run(updatehash=updatehash)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command

    raise NodeExecutionError(

nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node synthstrip.

RuntimeError: subprocess exited with code 137.

221102-15:51:44,618 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-04_echo-01_T1w.nii.gz/synthstrip)

221102-15:54:47,773 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-02_T1w.nii.gz/synthstrip)

221102-16:14:23,912 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-03_T1w.nii.gz/synthstrip)

221102-16:21:25,646 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-01_echo-05_T1w.nii.gz/synthstrip)

221102-16:30:15,258 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000456..ses-20090101..anat..sub-A00000456_ses-20090101_acq-mprage_run-01_echo-04_T1w.nii.gz/synthstrip)

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$  service docker stop

-bash: service: command not found

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$ service docker stop

-bash: service: command not found

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$ sudo service docker stop

Password:

sudo: service: command not found

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$ dockerd --storage-opt dm.basesize=30G

-bash: dockerd: command not found

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$ docker run -it nipreps/mriqc:latest --version

MRIQC v22.0.6

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$ dockerd --storage-opt dm.basesize=30G

-bash: dockerd: command not found

(base) Victorias-MacBook-Pro-2:~ VictoriaShevchenko$ docker run -it --memory="6g" --rm -v /Users/VictoriaShevchenko/Documents/PhD/SCZ/COBRE:/data:ro -v /Users/VictoriaShevchenko/Documents/PhD/SCZ:/out nipreps/mriqc:latest /data /out participant --participant_label A00000368 --no-sub

221103-11:23:44,579 cli IMPORTANT:

    

    Running MRIQC version 22.0.6:

      * BIDS dataset path: /data.

      * Output folder: /out.

      * Analysis levels: ['participant'].

221103-11:33:40,689 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000368..ses-20110101..anat..sub-A00000368_ses-20110101_acq-mprage_run-02_echo-03_T1w.nii.gz/synthstrip)

221103-11:34:02,323 nipype.workflow ERROR:

    Node synthstrip.a3 failed to run on host 9e861bb8be5c.

221103-11:34:02,947 nipype.workflow ERROR:

    Saving crash info to /out/logs/crash-20221103-113402-root-synthstrip.a3-66d2b98c-9945-41e8-bd2e-7b1f998a1c2a.txt

Traceback (most recent call last):

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node

    result["result"] = node.run(updatehash=updatehash)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command

    raise NodeExecutionError(

nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node synthstrip.

RuntimeError: subprocess exited with code 137.

Traceback (most recent call last):

  File "/opt/conda/bin/mriqc", line 8, in <module>

    sys.exit(main())

  File "/opt/conda/lib/python3.9/site-packages/mriqc/cli/run.py", line 159, in main

    mriqc_wf.run(**_plugin)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run

    runner.run(execgraph, updatehash=updatehash, config=self.config)

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 184, in run

    self._clean_queue(jobid, graph, result=result)

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 256, in _clean_queue

    raise RuntimeError("".join(result["traceback"]))

RuntimeError: Traceback (most recent call last):

  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node

    result["result"] = node.run(updatehash=updatehash)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run

    result = self._run_interface(execute=True)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface

    return self._run_command(execute)

  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command

    raise NodeExecutionError(

nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node synthstrip.

RuntimeError: subprocess exited with code 137.

221103-11:34:29,925 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000368..ses-20110101..anat..sub-A00000368_ses-20110101_acq-mprage_run-02_echo-02_T1w.nii.gz/synthstrip)

221103-11:34:50,199 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000368..ses-20110101..anat..sub-A00000368_ses-20110101_acq-mprage_run-02_echo-01_T1w.nii.gz/synthstrip)

221103-11:38:08,374 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000368..ses-20110101..anat..sub-A00000368_ses-20110101_acq-mprage_run-02_echo-04_T1w.nii.gz/synthstrip)

221103-11:44:55,317 nipype.workflow WARNING:

    [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000368..ses-20110101..anat..sub-A00000368_ses-20110101_acq-mprage_run-01_T1w.nii.gz/synthstrip)

221103-11:50:52,672 nipype.workflow WARNING:

     [Node] Error on "mriqc_wf.anatMRIQC.synthstrip_wf.synthstrip" (/tmp/work/mriqc_wf/anatMRIQC/synthstrip_wf/_in_file_..data..sub-A00000368..ses-20110101..anat..sub-A00000368_ses-20110101_acq-mprage_run-02_echo-05_T1w.nii.gz/synthstrip)

[Xiaoyang Chen]

Same problem here. Has anyone figured it out? Thanks!

Best,

Xiaoyang

[Nicholas D'Cruz]

For me, increasing my swap memory did the trick.

Nick