CMP without gui and qsub

[Pedro Costa del Amo]

Hello everyone,

I intend to carry out the cmp workflow in parallel over 200 subjects (obviously doing it one by one is not an option). What I am trying to do, is running cmp via the qsub command over a cluster of headless nodes (without graphic interface). As I expected, I cannot initially do it without a display, as it was already said in https://groups.google.com/forum/?fromgroups=#!searchin/cmtk-users/without$20gui/cmtk-users/wdhWiQpopFw/FasEGtimKigJ. I've thought of implementing a new gui class by deleting all those methods which require of a graphical interface.  That would permit one to create that cmpgui object which stores all the parameters for the workflow: 

from cmp.gui import CMPGUI
cmpgui = CMPGUI()
cmpgui.load_state('/path/to/your/pickle/state/LOG/cmp.pkl')

cmpgui.active_dicomconverter = True
cmpgui.project_name = '...'
cmpgui.project_dir = '.../'
cmpgui.subject_name = '...'
cmpgui.subject_timepoint = '...'
cmpgui.subject_workingdir = '.../'
cmp.connectome.mapit(cmpgui)

Would it be enough? Is there already an easier way to do it? 

Thank you in advance:

Pedro Costa 

[Pedro Costa del Amo]

Hello again,

I managed to make some modifications to the script with which you would start the pipeline without gui, so that it can be runed from computers without graphical interface. In my case I've used it to run the pipeline with qsub in parallel for several subjects in a cluster. The idea is quite simple. I've just created a new class which substitutes the gui one, making it possible to call the mapit method from the connectome.py file without starting the gui. The only modifications for its use would be then the initialization of the necessary parameters,  in the same way as it would be done with the file i  /example/default_project/map_connectome.py in the source code repository.

You can find the script attached.

Best,

Pedro

[cmpetty]

Hey Pedro ... I run the standard pipeline via SGE with headless machines all the time ... In my case i use Xvnc to open a fake display before launch the cmtk pipeline.  Here's some simple code to do it:

#generate a random integer between 100-500

RANDINT=$(( 100+(`od -An -N2 -i /dev/urandom` )%(500-100+1) ));

echo "my random interger is ${RANDINT}\n";

#start the Xvnc with that random-int

echo "running Xvnc\n";

Xvnc :${RANDINT} &

sleep 10

#test to see if the lock was created, obviously here you could

#check to see if one of your number already exists and generate a

#different number

if [ -e "/tmp/.X${RANDINT}-lock" ];then

    echo "lock file was created\n";

fi

export DISPLAY=:${RANDINT}

#then i run cmtk

python run_cmtk_all.py $SUBJ $VISIT ${DTIBXH} ${T1BXH} $RUNFS

This is done through a bash submission script, but it could just as easily be launched from python before loading CMPGUI

[Pedro Costa del Amo]

Thanks! That does the work as well

Best,

Pedro

[samantha...@vanderbilt.edu]

Hi guys,

I am new to cmtk and I am also trying to run the pipeline on hundreds of images simultaneously in order to cut down on computing time because I have a deadline in two weeks. It seems as if the above script is not really an option for me to use on the machine I currently have. Does anyone have any suggestions of another way to get around this or perhaps know of any remote computing power that may be available to rent to run all this on? 

Thanks so much,

Samantha  

[Erik Ziegler]

Try amazon ec2. The nitrc CE instance for cloud computation works very well. Just ftp the data back and forth and run something like a cc2.8xl instance.

Hope that helps,

Erik

--
You received this message because you are subscribed to the Google Groups "cmtk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cmtk-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Erik Ziegler]

Also, you might want to look at the nipype workflows for CMTK on lots of subjects.