[Singularity] : Sun Grid Engine and Singularity Container

[Shashi Ranjan]

I have recently started working on singularity container, and trying to find the way to use singularity with Sun grid engine. 

Actually we are currently using the Sun grid engine to schedule the job on amazon ec2 using autoscaling and it's generally time and cost consuming.

We want shift to docker containers but that is not supported through sun grid engine. When I started working singularity, I found that it possible.

Could you please guide or help me in achieving it.

With Regards,

Shashi Ranjan,
Mobile # 9008189000
in.linkedin.com/in/shashiranjanIndia

[vanessa s]

Hi Shashi,

Singularity containers can be run equivalently to scripts, so you could just add a line to run/exec a command to the container in any of your standard scripts, for example if I had run.sh:

#!/bin/sh
# other setup stuff
singularity exec /path/to/preproces.img  python /scripts/preprocess.py --input1 input1
singularity run /path/to/analysis.img --arg1 param1

and then submit with qsub run.sh as you normally would. 

How are you using sun grid engine + ec2 with autoscaling? Are you using something like StarCluster? That' s interesting, I have mostly thought about autoscaling in the context of servers/services. Do you find using the cloud to be affordable? It seems most institutions have (mostly) preference for things that need to scale because firing up instances adds up $$ quickly!

Best,

Vanessa

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[Shashi Ranjan]

Hi Vanessa,

Thanks . Yes I am using starcluster for it, regarding you question for affordability - yes it's affordable, but depends if you are working for any organization or personal work. It cheaper and faster in case of organization but you want to use extensively for personal work, it might be costly.

let me know if you need any more insight. I will try to implement the idea you suggested for the sge and singularity and will update you further.

With Regards,

Shashi Ranjan,
Mobile # 9008189000
in.linkedin.com/in/shashiranjanIndia

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[Paolo Di Tommaso]

Have a look to Nextflow. It allows the deployment of containerised workloads in the AWS cloud with the support of autoscaling and spot instances 

https://www.nextflow.io/blog/2016/deploy-in-the-cloud-at-snap-of-a-finger.html

p 

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.