juicer on SGE?

[Vera]

Hello,

I was wondering if juicer can be run on Sungrid Engine as well as UGE? Maybe you have a script for that?

I managed to run juicer on SGE until the "DEDUP" script is called  (using the UGER code with slightest modifications).

The problem seems to be that the qsub-command is used to submit a job to the queue, which in turn submits another qsub command (from the script "split_rmdups.awk").

This is the tail of my uger.out file:

(-: Mem align of /exports/eddie/scratch/vkaiser2/splits/Test_R2.fastq.sam done successfully

(-: Sort read 1 aligned file by readname completed.

(-: Sort read 2 aligned file by readname completed.

/exports/eddie/scratch/vkaiser2/splits/Test.fastq.sam created successfully.

(-: Finished sorting all sorted files into a single merge.

sh: qsub: command not found

sh: qsub: command not found

sh: qsub: command not found

sh: qsub: command not found

(-: Alignment and merge done, launching other jobs.

/var/spool/gridscheduler/execd/node1f01/job_scripts/597724: line 5: qsub: command not found

/var/spool/gridscheduler/execd/node1f01/job_scripts/597724: line 11: qsub: command not found

/var/spool/gridscheduler/execd/node1f01/job_scripts/597724: line 14: qsub: command not found

/var/spool/gridscheduler/execd/node1f11/job_scripts/597725: line 3: qsub: command not found

/var/spool/gridscheduler/execd/node1f11/job_scripts/597726: line 3: qsub: command not found

Thanks for any advice.. :-)

[Neva Durand]

Hi Vera,

The Sun Grid Engine was acquired by Univa and is now the Univa grid engine;  I don't know the details of the improvement Univa has made, but essentially Sun Grid Engine is a deprecated version of Univa grid engine.

 

Our UGER scripts are for the Univa grid engine. What modifications have you made to make it work on Sun Grid Engine?  The qsub command is the same in both, so it's surprising that you get a "command not found" in this part of the script.  Possibly this is because on the individual nodes, the environment is not set up.  In our scripts, we have to load the cluster before calling the split_rmdups.  Here's the relevant code:

 qsub -o ${topDir}/uger.out -j y -q ${queue} -r y -N ${groupname}_osplit $holdjobs <<- DEDUP

    source $usePath

    $load_cluster

    awk -v queue=$long_queue -v outfile=$topDir/uger.out -v juicedir=${juiceDir} -v dir=$outputdir -v groupname=$groupname -f ${juiceDir}/scripts/split_rmdups.awk $outputdir/merged_sort.txt

DEDUP

You've gotten the qsub command to work in general I take it?  

Best

Neva

--
You received this message because you are subscribed to the Google Groups "3D Genomics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 3d-genomics...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/c074f2cb-6911-4e7f-8461-71ce9c8eb2c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Neva Cherniavsky Durand, Ph.D.

Staff Scientist, Aiden Lab

www.aidenlab.org

[Vera]

Hi Neva,

yes, I also thought that Univa and Sun Grid engine were pretty much the same...

The main modification that I made to juicer.sh was to change the $queue and $long_queue variables because qsub wouldn't run with those arguments.

Essentially, I replaced instances, such as

"qsub -o ${topDir}/uger.out -j y -q ${queue}"

with

"qsub -o ${topDir}/uger.out -j y -l h_rt=03:00:00"

throughout the script.

juicer.sh is running fine (using qsub commands) until the DEDUP stage, i.e. my file "aligned/merged_sort.txt", which is the last file that is created, looks good.

The problem seems to be that ${juiceDir}/scripts/split_rmdups.awk  contains several qsub-commands that are initiated FROM a qsub command in the main juicer.sh script.

That is, a process running on one node on the SGE cluster cannot qsub another job to the cluster, but each job needs to be scheduled from the headnode.

At least that's how I understand the problem...

Also, I removed all instances of "source $usePath" in the juicer.sh script (because I cannot access usePath=/broad/software/scripts/useuse).

If you could let me know what's in the "useuse" file, that might help, though I think it's not the issue, really..

Cheers,

Vera

[Neva Durand]

Hi Vera,

The queue names would be unique to your cluster system.  Without knowing how your cluster is set up, it's difficult to help.  On ours, there are different queues set up that have different requirements (e.g., our short queue only runs for two hours but has higher priority than the long queue).

The useuse script is for initializing dotkit.  Again this depends on your cluster.  For ours, the dotkit is how we load different software onto individual nodes - essentially setting the library path and other environment variables needed for finding the path when software is launched.  There's lots of information on the web about dotkit if you search.

You will need to do something equivalent for the individual nodes to run qsub.  You'll need to set environmental variables within the script so it can call qsub - or perhaps refer to qsub by the absolute filename (though I'm not sure that will be enough).

You might want to talk to someone in charge of the cluster at your institution to see how you should go about this.

Best

Neva

To view this discussion on the web visit https://groups.google.com/d/msgid/3d-genomics/4452b4ef-321a-4c0e-96a9-57abfa2798b0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Vera]

Hi Neva,

thanks for your quick replies! I will look into it and, hopefully, find a way!

Vera

[Vera]

FYI,  there is the command 

"ssh headnode qsub script.sh"

that can submit commands from a job that is already running via qsub on our SGE cluster.  

(Did have to ask lots of people before I found it!)

Getting nice output files now :-)