condor_qsub issue

[Durai Arasan]

Hi All,

To run DTITK in parallel we are using HTCondor. HTCondor has its own version of qsub called condor_qsub. At line 73 shown below, the synchronization and holding does not work and as a result the next jobs are started prematurely.

Has anyone faced this issue with qsub synchronization option not working?

48 template=$1

49 subjects=$2

50 mask=$3

51 no_of_iter=$4

52 ftol=$5

53

54 jid=ddr_"$$"

55

56 for subj in `cat ${subjects}`

57 do

58 echo $subj

59 pref=`getTVPrefix $subj`

60 if [ "${DTITK_USE_QSUB}" -eq 1 ]

61 then

62 jname=${jid}_${subj}

63 jname=`echo $jname | sed -e 's/\//_/g'`

64 cmd="condor_qsub -cwd -o ${pref}.log -e ${pref}.err -b y -N ${jname} ${DTITK_ROOT}/scripts/dti_diffeomorphic_reg"

65 else

66 cmd="${DTITK_ROOT}/scripts/dti_diffeomorphic_reg"

67 fi

68 ${cmd} ${template} ${subj} ${mask} 1 ${no_of_iter} ${ftol}

69 done

70

71 if [ "${DTITK_USE_QSUB}" -eq 1 ]

72 then

73 condor_qsub -sync y -hold_jid ${jid}_* -o /dev/null -e /dev/null -b y echo "done" > /dev/null

74 echo "done"

75 fi

Thank you,

Durai