Advanced SLURM options

[Marc Hoeppner]

Hi,

our local compute center just upgraded their cluster and for whatever reason require to specify the slurm option "--qos=<some_option>" for certain partitions... Currently, Bpipe does not seem to allow Slurm options outside of the standard Bpipe parameters. For now, I hard-coded the option into my Bpipe installation as I only run on a single partition, but I think something like

"slurm_request_options"

akin to what is available for LSF or SGE would be nice. Any chance of that making it into the next release?

Best,

Marc

[Marc Hoeppner]

Come to think of it, maybe just do away with the executor-specific options and implement a generic "request" option that works for all platforms? That way implementing additional systems wouldn't require their own, dedicated solution. 

[Simon Sadedin]

It seems like you are correct, there isn't a good way to shove something arbitrary into the Slurm options. I've added an issue to add that as a feature.

As a short term hack, you could customize it yourself just by editing bin/bpipe-slurm.sh in your Bpipe install to put whatever option you need in there.

Cheers,

Simon

On Tue, Jan 3, 2017 at 8:43 AM, Marc Hoeppner via bpipe-discuss <bpipe-discuss+APn2wQeVsPV16SULDy...@googlegroups.com> wrote:

Come to think of it, maybe just do away with the executor-specific options and implement a generic "request" option that works for all platforms? That way implementing additional systems wouldn't require their own, dedicated solution. 

--
You received this message because you are subscribed to the Google Groups "bpipe-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bpipe-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to bpipe-...@googlegroups.com.
Visit this group at https://groups.google.com/group/bpipe-discuss.
For more options, visit https://groups.google.com/d/optout.

[Simon Sadedin]

NB: did a quick implementation of this in a branch called custom_submit_opts.

See Issue #193. Any testing would be much appreciated!

On Wed, Jan 4, 2017 at 10:13 AM, Simon Sadedin <ssad...@gmail.com> wrote:

It seems like you are correct, there isn't a good way to shove something arbitrary into the Slurm options. I've added an issue to add that as a feature.

As a short term hack, you could customize it yourself just by editing bin/bpipe-slurm.sh in your Bpipe install to put whatever option you need in there.

Cheers,

Simon