Execution stops at gaussian scale space, but with fastr execute it works
Mattias Hansson
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF227_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF311_MRI_L_T30 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF311_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF390_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF390_MRI_L_T0 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF390_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF244_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF380_MRI_L_T0 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF380_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF380_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF197_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF197_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF036_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF036_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF173_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF173_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF173_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF227_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF380_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF128_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF103_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF197_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF357_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF345_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF244_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF345_MRI_L_T30 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF345_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF390_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF244_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF227_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF128_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF269_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF357_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF357_MRI_L_T0 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF357_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF269_MRI_L_T0 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF269_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] drmaaexecution:0205 >> Job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF403_MRI_L_T0 not found in DRMAA lookup
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF403_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF103_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF103_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF345_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF311_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF150_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___morph___PROOF150_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/morph/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF128_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF311_MRI_L_T30 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF403_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___threshold___PROOF269_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/threshold/outputs/image in {}!
[INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___scalespacefilter___PROOF403_MRI_L_T0 with status JobState.cancelled
[ERROR] node:1192 >> Could not find required data for fastr:///networks/MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier/nodelist/scalespacefilter/outputs/image in {}!
Traceback (most recent call last):
File "MultiAtlasApp_PROOF_FemkeSegmTEST_LEFT_TrainClassifier.py", line 124, in <module>
main()
File "MultiAtlasApp_PROOF_FemkeSegmTEST_LEFT_TrainClassifier.py", line 120, in main
network.execute(sourcedata, sinkdata)
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/network.py", line 995, in execute
joblist = node.execute()
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/node.py", line 1165, in execute
for sample_index, sample_id, job_data, job_depends, failed_annotations in self.input_group_combiner:
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/node.py", line 469, in iter_input_groups
for sample_payloads in itertools.product(*[ig.iterinputvalues for ig in self.input_groups.values()]):
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/node.py", line 300, in iterinputvalues
yield self[index]
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/node.py", line 164, in __getitem__
target_index=index)
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/node.py", line 260, in find_source_index
target_index=target_index)
File "/home/mhansson/virtual_env_fastr_develop/fastr/fastr/core/node.py", line 288, in solve_broadcast
raise exceptions.FastrSizeMismatchError('The estimated size after broadcast matching is incorrect! (estimated {}, actual {})'.format(estimated_source_size, source_size))
Mattias Hansson
Hakim Achterberg
The reason fastr crashes and throws and exception is because you appearantly are using 13 normal images and 15 label images in your sources:
16 [INFO] node:1380 >> Preparing Node images with size [13] and dimnames ('atlas',)
42 [INFO] node:1380 >> Preparing Node label_images with size [15] and dimnames ('atlas',)
that don't match and give trouble in the sampler node where they should come together.
I am not sure why you think the GaussianScaleSpace fails, but it seems to be doing okay. There was an issue with a parameter file that couldn't be found:
104 [INFO] network:1109 >> Finished job MultiAtlasApp_PROOF_FEMKE_LEFT_TrainClassifier___param_file___id_0 with status JobState.failed
which caused all the classification jobs to cancel at some point.
Cheers,
Hakim