[slurm-users] How to read job accounting data long output? `sacct -l`

[Chandler Sobel-Sorenson]

Is there a recommended way to read output from `sacct` involving `-l` or `--long` option?  I have dual monitors and shrunk the terminal's font down to 6 pt or so until I could barely read it, giving me 675 columns.  This was still not enough...

Perhaps there is a way of displaying it so the lines don't wrap and I can use left/right arrow keys to scroll the output, much like `systemctl` and `journalctl` can do?

Perhaps there is a way to import it into a spreadsheet?

This was with version 19.05 at least.  Apologies if the output has changed in newer versions...

Thanks

[Bjørn-Helge Mevik]

Chandler Sobel-Sorenson <chan...@genome.arizona.edu> writes:

> Perhaps there is a way to import it into a spreadsheet?

You can use `sacct -P -l`, which gives you a '|' separated output, which
should be possible to import in a spread sheet.

(Personally I only use `-l` when I'm looking for the name of an
attribute and am to lazy to read the man page. Then I use -o to specify
what I want returned.)

Also, in newer versions at least, there is --json and --yaml to give you
output which you can parse with other tools (or read, if you really want :).

--
Cheers,
Bjørn-Helge Mevik

[Will Furnass]

If you pipe output into 'less -S' then you get horizontal scrolling.

Will

[Davide DelVento]

It would be very useful if there were a way (perhaps a custom script
parsing the sacct output) to provide the information in the same
format as "scontrol show job"

Has anybody attempted to do that?

[Ross Dickson]

I wrote a simple Python script to transpose the output of sacct from a row into a column.  See if it meets your needs.

 https://github.com/ComputeCanada/slurm_utils/blob/master/sacct-all.py

- Ross Dickson

Dalhousie University  /  ACENET  /  Digital Research Alliance of Canada 

[Paul Edmon]

The seff utility (in slurm-contribs) also gives good summary info.

You can also you --parsable to make things more managable.

-Paul Edmon-

[Marcus Wagner]

Hi Bjørn-Helge,

it it important to know, that the json output seems to be broken.

First of all, it does not (compared to the normal output) obey to the truncate option -T.
But more important, I saw a job, where in a "day output" (-S <date> -E <date+1day>) no steps were recorded.
Using sacct -j <jobid> --json instead showed that job WITH steps.

Best
Marcus

Dipl.-Inf. Marcus Wagner

IT Center
Gruppe: Server, Storage, HPC
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
wag...@itc.rwth-aachen.de
www.itc.rwth-aachen.de

Social Media Kanäle des IT Centers:
https://blog.rwth-aachen.de/itc/
https://www.facebook.com/itcenterrwth
https://www.linkedin.com/company/itcenterrwth
https://twitter.com/ITCenterRWTH
https://www.youtube.com/channel/UCKKDJJukeRwO0LP-ac8x8rQ

[Bjørn-Helge Mevik]

Marcus Wagner <wag...@itc.rwth-aachen.de> writes:

> it it important to know, that the json output seems to be broken.
>
> First of all, it does not (compared to the normal output) obey to the truncate option -T.
> But more important, I saw a job, where in a "day output" (-S <date> -E <date+1day>) no steps were recorded.
> Using sacct -j <jobid> --json instead showed that job WITH steps.

It is hard to call it "broken" when it is documented behaviour:

--json Dump job information as JSON. All other formatting arguments will be ignored

--
Cheers,
Bjørn-Helge

[Marcus Wagner]

That depends on what is meant with formatting argument.
To me, formatting arguments are "-b", "-l", "-o", "-p|-P"
Instead, I can use filtering arguments with --json, like "-u", "-p" etc. And I would assume, that -S, -E and -T are filtering options, not formatting options.

But as I explained before, not obeying to -T is bad behaviour. That is nothing I would call "broken".


But getting sometimes no steps for a job (if in a larger JSON-output with many jobs) and then getting the steps, if one asks specifically for that jobid. That is something I would call broken.

Best Marcus

[Bjørn-Helge Mevik]

Marcus Wagner <wag...@itc.rwth-aachen.de> writes:

> That depends on what is meant with formatting argument.

Yes, they could surely have defined that.

> etc. And I would assume, that -S, -E and -T are filtering options, not
> formatting options.

I'd describe -T as a formatting option:

-T, --truncate
Truncate time. So if a job started before --starttime the start
time would be truncated to --starttime. The same for end time and
--endtime.

As I read this, it changes how a job is written, it does not select
jobs.

> But getting sometimes no steps for a job (if in a larger JSON-output
> with many jobs) and then getting the steps, if one asks specifically
> for that jobid. That is something I would call broken.

That sounds worse, yes.

--
B/H

[Chandler Sobel-Sorenson]

Awesome thanks!


Will Furnass wrote on 12/14/22 1:23 AM:
> *External Email*

[Chandler Sobel-Sorenson]

Bjørn-Helge Mevik wrote on 12/14/22 12:19 AM:

> Chandler Sobel-Sorenson <chan...@genome.arizona.edu> writes:
>> Perhaps there is a way to import it into a spreadsheet?
> You can use `sacct -P -l`, which gives you a '|' separated output, which
> should be possible to import in a spread sheet.

Awesome thanks!