[slurm-users] Site factor plugin example?

[Angel de Vicente]

Hello,

I want to experiment with writing our own site factor plugin. In the
documentation I found the API details
(https://slurm.schedmd.com/site_factor.html), but it would be much
easier for me if I had some example site factor plugin to start with.

Do you know of any examples that can set me in the right direction?

Thanks,
--
Ángel de Vicente
Research Software Engineer (Supercomputing and BigData)
Tel.: +34 922-605-747
Web.: http://research.iac.es/proyecto/polmag/

GPG: 0x8BDC390B69033F52

[Loris Bennett]

Hi Angel,

Angel de Vicente <angel.de...@iac.es> writes:

> Hello,
>
> I want to experiment with writing our own site factor plugin. In the
> documentation I found the API details
> (https://slurm.schedmd.com/site_factor.html), but it would be much
> easier for me if I had some example site factor plugin to start with.
>
> Do you know of any examples that can set me in the right direction?

Did you ever find an example or write your own plugin which you could
provide as a example?

Cheers,

Loris

--
Dr. Loris Bennett (Herr/Mr)
ZEDAT, Freie Universität Berlin

[Angel de Vicente]

Hello Loris,

"Loris Bennett" <loris....@fu-berlin.de> writes:

> Did you ever find an example or write your own plugin which you could
> provide as a example?

I'm afraid not (though I didn't persevere, because for the moment we are
trying to encourage our users not to waste resources with a different
approach).

But, in any case, I would still be interested in a site factor plugin
example, because I might revisit this in the future.

Cheers,

[Loris Bennett]

Hello Angel,

Angel de Vicente <angel.de...@iac.es> writes:

> Hello Loris,
>
> "Loris Bennett" <loris....@fu-berlin.de> writes:
>
>> Did you ever find an example or write your own plugin which you could
>> provide as a example?
>
> I'm afraid not (though I didn't persevere, because for the moment we are
> trying to encourage our users not to waste resources with a different
> approach).

Pray tell. I would also be interested in other approaches.

> But, in any case, I would still be interested in a site factor plugin
> example, because I might revisit this in the future.
>
> Cheers,

[Christopher Samuel]

On 10/13/23 10:10, Angel de Vicente wrote:

> But, in any case, I would still be interested in a site factor plugin
> example, because I might revisit this in the future.

I don't know if you saw, but there is a skeleton example in the Slurm
sources:

src/plugins/site_factor/none

Not sure if that helps?

All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA

[Reed Dier]

Hi Angel and Loris,

I hope this will be of at least some help, as I was tasked with trying to get site factor implemented in our cluster for the sake of making conformant, predictable priority values that were “pretty” and round, and I was not able to find any good documentation for it either.

I had originally hoped it could be set in slurm.conf or something, since I just wanted/needed a constant value, but this was the solution that worked for us.

That said, the only way I was able to make this work was by adding it to our existing job_submit.lua.

function slurm_job_submit(job_desc, part_list, submit_uid)
   job_desc.site_factor = {{ site_factor_num }}
   return slurm.SUCCESS
end

This is a heavily dialed down copy of our job_submit.lua script, but we set the site_factor_num as a constant because of what we were trying to achieve, and we stuck that line in the slurm_job_submit function.

Hope that helps send y’all down the correct rabbit hole.

Cheers,

Reed

[Loris Bennett]

Christopher Samuel <ch...@csamuel.org> writes:

> On 10/13/23 10:10, Angel de Vicente wrote:
>
>> But, in any case, I would still be interested in a site factor
>> plugin example, because I might revisit this in the future.
>
> I don't know if you saw, but there is a skeleton example in the Slurm
> sources:
>
> src/plugins/site_factor/none
>
> Not sure if that helps?

Thanks for the pointer, Chris. I couldn't find the folder 'none' on
Github at first, because it doesn't see to be on the 'master' branch,
but once I switched branches, I found it.

I'll have a go at creating a memory-wasted factor.

[Loris Bennett]

Loris Bennett <loris....@fu-berlin.de> writes:

> Christopher Samuel <ch...@csamuel.org> writes:
>
>> On 10/13/23 10:10, Angel de Vicente wrote:
>>
>>> But, in any case, I would still be interested in a site factor
>>> plugin example, because I might revisit this in the future.
>>
>> I don't know if you saw, but there is a skeleton example in the Slurm
>> sources:
>>
>> src/plugins/site_factor/none
>>
>> Not sure if that helps?
>
> Thanks for the pointer, Chris. I couldn't find the folder 'none' on

> Github at first, because it doesn't seem to be on the 'master' branch,

> but once I switched branches, I found it.
>
> I'll have a go at creating a memory-wasted factor.

OK, the structure of the plugin itself given here

https://github.com/SchedMD/slurm/blob/slurm-23.02/src/plugins/site_factor/none/site_factor_none.c

seems relatively straight forward. The crux seems to be what one would
need in

_update(void *x, void *ignored)

to get the data regarding the requested and used memory for each user
for a given period.

From my rather limited understanding, the multifactor plugin just has
access to information such as 'effective usage' and 'normalized shares'
for the association. Thus it is not possible to directly access the
amount of unused memory. Therefore it seems like I would have rely on
generating my own metric for memory-wasting and then reading that from
the plugin.

Or does anyone see an alternative?