Use cases of wlm-operator

[Taras Shapovalov]

Hey guys,

Can you help me to understand the purpose of wlm-operator (that integrates singularity with slurm and k8s). I have read already several related articles and README in the sources of this project, but still don't get what are real use cases of the operator. My current understanding is such that a user runs a pod, that in turn runs sbatch to submit a real job to Slurm, and then another pod collects the output of the job. But why not submit directly to Slurm?

Best regards,

Taras

[Ian Lumb]

One AI/DL/ML use case is drawn out in the article at https://medium.com/sylabs/introducing-hpc-affinities-to-the-enterprise-a-new-open-source-project-integrates-singularity-and-6461091c2626, Taras. 

We know of interest in use cases along these lines from a number of organizations.

We're collaborating with others to develop a Life Sciences use case example that is pure HPC in nature ... 

Not sure if that's what you're looking for in terms of a response ...

-Ian

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAJr6v3EUbzGkzbgTVHDXMLdeEhS%2Bk_jPFKyfRcefC0woO8TPhw%40mail.gmail.com.

--

Ian Lumb

Business Development, Sylabs, Inc.

www.sylabs.io

   

[Taras Shapovalov]

Hi Ian,

Thanks for the link. I've already read that article and did not find what I am looking for. The article provides example to run lolcow containers via Kubernetes and gives the link to another article about some demo with badge recognition (that does not provide any details though). What I need is to find such scenario(s) where submitting Slurm jobs through Kubernetes+wlm-operator gives enough benefits over submitting the jobs directly to Slurm. The wlm-operator and Kubernetes adds a lot of complexity to the "running containers on a cluster" needs, so I am trying to understand what value this complexity gives to the end users or cluster administrators.

Best regards,

Taras

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAE5%3D3a4qZDSN7ynsmHuN96Z7e9et9CJS6vKUX6c7mZgMtvZVpw%40mail.gmail.com.

[Ian Lumb]

Short answer: It doesn't!

Well, other than the ability to work via kubectl and YAML files ... 

The motivating use case combines (micro)services and HPC in a hybrid fashion ... in real time. Although you won't find a specific example detailed, our year-end white paper (please see the section "Emerging Community-Driven Use Cases" at https://sylabs.io/assets/white-papers/2018_Year-End_White_Paper_FINAL.pdf) attempts to paint the broad brushstrokes, Taras. 

-Ian

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAJr6v3EqEe0uuUfyxEsW%2BpSJUKPvbS6u3ZvN4usE7HKLY3e4ag%40mail.gmail.com.