whole suite timing?

[Jonny Williams]

hi there

is there an easy way to find out how long a whole suite took to run? i know i can look at the job.out files for info on individual cylc tasks but would like to know the whole suite duration.

thanks for any tips!

cheers

jonny

[Bruno P. Kinoshita]

Oh, good question! Couldn't find a straight answer looking at the cylc commands. Best I could come up with was:

bruno@ubuntu:$ cd ~/cylc-run/$my_suite/

bruno@ubuntu:$ sqlite3 cylc-suite.db "SELECT CAST(strftime('%s', MAX(time)) as integer) - CAST(strftime('%s', MIN(time)) as integer) FROM task_events"

Which is actually using the Ubuntu sqlite3 utility to inspect the sqlite database directly, subtracting the start and end dates from task_events.

There must be a better way. Let's wait and see what others suggest (so that I can learn too!)

Cheers

Bruno

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hilary Oliver]

Hi Jonny,

Bruno is correct - the best way is to manually query the suite run database, with the sqlite3 CLI. But he's incorrect that there must be a better way!

The DB is fairly simple; you can explore the content with sqlite3 or a GUI sqlite browser utility.

The suite log records suite start-up and shutdown time too, but logs typically get "rolled over" in a long-running suite.

At the moment the only built-in reporting tool we have is "cylc report-timings" which extracts task job timings from suite databases and presents basic summary stats on those. Beyond that it's up to users, for now.

Hilary

[Hilary Oliver]

> he's incorrect that there must be a better way!

Well, if Bruno meant, "there should be a better way", then he probably is right about that too :-)

[Jonny Williams]

thanks a lot @bruno and @hilary! i'll have a look at this in the morning!

cheers

jonny

[Matt Shin]

From Cylc 7.8.X onwards with https://github.com/cylc/cylc/pull/2781 the header entries in the suite log should be carried over each time it rolls. (Unfortunately, I don't think it is carried over if you do a stop/restart - so may not be useful for the overall time.)

[Whitcomb, Mr. Tim]

I’d do this by using the report-timings command with the “raw” output option to get the timings, at which point you can filter using your favorite tool to sort out the earliest start time and last succeed time to get the total suite timings.