Remove_node_directories option on config file

[Augustine Koh]

Dear All Nipype users and Developers,

I have a question regarding the suppression or perhaps deletion of the unique "sandbox" - like directories that Nipype creates when it is executing the nodes of workflows. After running several workflows, we have a lot of accumulation of these directories and they take up a lot of space even after DataGrabber has captured all the outputs from them.

Is there any way Nipype could automatically delete these directories? Is it through the remove_node_diretories option on the webpage: http://www.mit.edu/~satra/nipype-nightly/users/config_file.html#execution?

Thank you very much!

Best regards,

Augusine

[Augustine Koh]

I attempted to try to set this option via two ways as suggested on the documentation - 1. setting the global config options directly in my workflow script itself and 2. using the object.attribute notation of myworkflow.config['execution'] however they both did not work.

My workflow DAG is attached here for reference purposes because I'll be referring to it as I explain my problem.

For 1. I was met with error messages while executing my terminal - basically when the workflow was executed to the point of the register_std node (the second processing node in the pipeline), I got a series of errors (I was running the workflow for 16 subjects) repeatedly reporting:

140607-21:36:29,372 workflow INFO:

[Job finished] jobname: register_std.aI.a1.b10 jobid: 68

140607-21:36:29,392 workflow ERROR:

['Node register_std.aI.a1.b10 failed to run on host Kohs-MacBook-Pro.local.']

140607-21:36:29,392 workflow INFO:

Saving crash info to /Users/Kohaugustine/Desktop/Workflow_Demo/crash-20140607-213629-Kohaugustine-register_std.aI.a1.b10.pklz

140607-21:36:29,392 workflow INFO:

Traceback (most recent call last):

  File "/Users/Kohaugustine/continuum_miniconda/miniconda/lib/python2.7/site-packages/nipype/pipeline/plugins/base.py", line 243, in run

    self._remove_node_dirs()

  File "/Users/Kohaugustine/continuum_miniconda/miniconda/lib/python2.7/site-packages/nipype/pipeline/plugins/base.py", line 450, in _remove_node_dirs

    outdir = self.procs[idx]._output_directory()

AttributeError: 'Node' object has no attribute '_output_directory'

It seemed like the node got removed before it could finish executing? But then when I went to check for the sandbox directories, they were all still present. What could be the problem?

For 2., the problem was simply that nothing happened - it was as if that line of code to instruct nipype to remove the node directories was never present in the first place.

Any help/suggestions/feedback would be greatly appreciated! Thank you very much!

Best regards,

Augustine