Michael
Nov 13, 2013, 8:48:44 PM11/13/13
to nipy...@googlegroups.com
I have since recent days an important issue when using NiPype: when processing too many subjects the pipeline first initializes:
131113-20:27:33,55 workflow INFO:
['check', 'execution', 'logging']
131113-20:28:11,695 workflow INFO:
Running in parallel.
And then hangs, forcing me to split the number of subjects and defeating somewhat the great parallelization feature of NiPype. Here is the output when i send an interrupt signal:
Process NonDaemonPoolWorker-9:
Process NonDaemonPoolWorker-8:
Process NonDaemonPoolWorker-10:
Process NonDaemonPoolWorker-7:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Process NonDaemonPoolWorker-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process NonDaemonPoolWorker-5:
Process NonDaemonPoolWorker-1:
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process NonDaemonPoolWorker-3:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process NonDaemonPoolWorker-4:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process NonDaemonPoolWorker-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
self.run()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
self.run()
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self.run()
self.run()
self.run()
self.run()
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
self._target(*self._args, **self._kwargs)
self._target(*self._args, **self._kwargs)
self._target(*self._args, **self._kwargs)
self._target(*self._args, **self._kwargs)
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 85, in worker
task = get()
task = get()
task = get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
task = get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
task = get()
task = get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
task = get()
task = get()
task = get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 376, in get
task = get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
racquire()
racquire()
racquire()
KeyboardInterrupt
KeyboardInterrupt
racquire()
racquire()
return recv()
KeyboardInterrupt
KeyboardInterrupt
racquire()
racquire()
KeyboardInterrupt
KeyboardInterrupt
KeyboardInterrupt
KeyboardInterrupt
racquire()
KeyboardInterrupt
racquire()
KeyboardInterrupt
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/IPython/utils/py3compat.pyc in execfile(fname, *where)
202 else:
203 filename = fname
--> 204 __builtin__.execfile(filename, *where)
/my/dir/pipeline.py in <module>()
403 ])
404
--> 405 multimodality.run(plugin='MultiProc', plugin_args={'n_procs' : 10})
406 #multimodality.write_graph(graph2use='hierarchical')
407 #graph_path = os.path.join(multimodality.base_dir,'graph_color_t2.dot')
/usr/local/lib/python2.7/dist-packages/nipype/pipeline/engine.pyc in run(self, plugin, plugin_args, updatehash)
687 if str2bool(self.config['execution']['create_report']):
688 self._write_report_info(self.base_dir, self.name, execgraph)
--> 689 runner.run(execgraph, updatehash=updatehash, config=self.config)
690 datestr = datetime.utcnow().strftime('%Y%m%dT%H%M%S')
691 if str2bool(self.config['execution']['write_provenance']):
/usr/local/lib/python2.7/dist-packages/nipype/pipeline/plugins/base.pyc in run(self, graph, config, updatehash)
218 self._config = config
219 # Generate appropriate structures for worker-manager model
--> 220 self._generate_dependency_list(graph)
221 self.pending_tasks = []
222 self.readytorun = []
/usr/local/lib/python2.7/dist-packages/nipype/pipeline/plugins/base.pyc in _generate_dependency_list(self, graph)
417 except:
418 self.depidx = nx.to_scipy_sparse_matrix(graph)
--> 419 self.depidx = self.depidx[:, indices][indices, :]
420 self.refidx = deepcopy(self.depidx)
421 self.refidx.astype = np.int
/usr/local/lib/python2.7/dist-packages/scipy/sparse/lil.pyc in __getitem__(self, index)
243 return self.__class__([[self._get1(iii, jjj) for iii, jjj in
244 zip(ii, jj)] for ii, jj in
--> 245 zip(i.tolist(), j.tolist())])
246
247 def _insertat2(self, row, data, j, x):
/usr/local/lib/python2.7/dist-packages/scipy/sparse/lil.pyc in _get1(self, i, j)
209 if i < 0:
210 i += self.shape[0]
--> 211 if i < 0 or i >= self.shape[0]:
212 raise IndexError('row index out of bounds')
213
/usr/local/lib/python2.7/dist-packages/scipy/sparse/base.pyc in get_shape(self)
91 self._shape = shape
92
---> 93 def get_shape(self):
94 return self._shape
95
KeyboardInterrupt:
Satrajit Ghosh
Nov 14, 2013, 10:44:36 AM11/14/13
to nipy-user
hi michael,
i know exactly what the problem is. i'll work on a solution. in the meantime if you want to have it work, simply comment that line in plugins/base.py file:
--> self.depidx = self.depidx[:, indices][indices, :]
cheers,
satra
satra
--
---
You received this message because you are subscribed to the Google Groups "NiPy Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nipy-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Michael
Dec 2, 2013, 3:54:46 PM12/2/13
to nipy...@googlegroups.com
Hi Satra,
I'm running into the same issue with NiPype install on a server (which uses the last nightly build of NiPype). Is there a way to fix this without a user custom modification on the source code? i.e. by setting some parameters in the user pipeline itself or with a PR?
Satrajit Ghosh
Dec 7, 2013, 10:24:23 PM12/7/13
to nipy-user
hi mike,
i just sent a pull request. can you check it out. this should handle it.
cheers,
satra
satra
Reply all
Reply to author
Forward
0 new messages