Problems with the greedy command in ASHS

[helena.g...@googlemail.com]

Dear ASHS users,

lately I have been having problems running ASHS. The processing would stop either at stage 1 or during various steps of stage 2. After looking through log files I found that the problem is caused by `greedy` command. In most of the cases `greedy` co-registration would crash, but the processing pipeline would continue. In some cases it would simply freeze.

After extracting a single `greedy` command line from a log file and running exactly the same command multiple times, I have found out that:

it crashes or freezes at a different iteration step or different level (mostly at level 2 of 3, sometimes 1 or 3) each time;

most of the times it crashes with 'segmentation fault (core dumped)' error like below:

./ry_run_greedy_simple: line 1: 24785 Segmentation fault      (core dumped) /share/apps/ashs/ext/Linux/bin/greedy -d 3 -it greedy_atlas_to_subj_affine.mat -gm tse_to_chunktemp_left_regmask.nii.gz -i tse_to_chunktemp_left.nii.gz tse_to_chunktemp_left.nii.gz -o greedy_atlas_to_subj_warp.nii.gz -m NCC 2x2x2 -n 60x60x20 -e 0.5 -float

occasionally (maybe 1 out of 10 times) it freezes without any error message;

very rarely it would crash with this error message:

ABORTING PROGRAM DUE TO RUNTIME EXCEPTION -- /mnt/build/pauly/buildbot/Nightly/itk/v4.8.2/itk/Modules/Core/Common/src/itkMultiThreader.cxx:416:
itk::ERROR: MultiThreader(0x3c1c8b0): Exception occurred during SingleMethodExecute
/mnt/build/pauly/buildbot/Nightly/itk/v4.8.2/itk/Modules/Core/Common/src/itkMultiThreaderPThreads.cxx:241:
itk::ERROR: MultiThreader(0x3c1c8b0): Unable to join thread.

As you can see, re-running the same command line with the same input files and options can result in different random outcomes.

I was wondering if anyone has encountered similar issues with `greedy` command? Or can recommend some further tests to narrow down the list of causes?

Our server is running CentOS 7. The server has undergone a major upgrade recently, after which ASHS stopped working.

Thanks so much for any help on this matter!