revisit our model cold-start tasks with Cylc 6?

[Whitcomb, Mr. Tim]

Hi all -

I'm getting spun up on the new date capabilities that came with the new 6.x release and I'm wondering if it's time for me to revisit some earlier decisions we made.

To revisit the traditional way we did this: when cold-starting our data assimilation system, the non-cylc scripts (that our users are used to) set the start time to the initial cold-start cycle (e.g. 2014010100) and start in the middle of the cycle (i.e. skipping the obs processing and data assimilation and jumping ahead to the forecast model) and then proceed on to the next full cycle. In our cylc suite, the solution we decided on after some consultation here was to have the start time be the initial full cycle, then have some cold-start tasks that set the date to be 6 hours earlier and run the subset of tasks there.

With the new changes, I'm wondering if it would now be straightforward to modify things to better match what we were doing already, since I still get tripped up occasionally grabbing a set of 00Z initial conditions and then kicking cylc off with a 06Z start time! Essentially, this would be a graph like:

[[[R1]]] # repeat once at the initial cycle point
graph = "fcst -> output"
[[[T00,T06,T12,T18 ]]] # incomplete - want this at every cycle point *but* the initial one
graph = "output[-PT6H] -> obs -> assimilate -> fcst -> output"

In the various examples that are in the documentation, I don't think I saw one that specifically matched this - after some fumbling around, I was able to get the graph I wanted by setting that second graph to:

[[[+PT6H/P1D,+PT12H/P1D,+PT18H/P1D,+PT24H/P1D]]]

but that seems a bit obtuse - is there an easier way to do this?

Also, are the "R1" tasks skipped when warm-starting the suite?

Tim

[Fitzpatrick, Ben]

Hi Tim,

You can simplify the second graph section to:

[[[+PT6H/PT6H]]]

graph = "output[-PT6H] -> obs -> assimilate -> fcst -> output"

(start 6 hours later, repeat every 6 hours). The caveat is that this only gives 00:00, 06:00, 12:00, 18:00 cycle points when the initial cycle point is at 00:00, 06:00, 12:00, or 18:00.

Although R1 tasks do a similar job to the old start-up (and cold-start) tasks, they aren't special any more and won't behave any differently to other tasks when it comes to warm starting. Warm-starting is now effectively like taking a graph of the suite (preserving its original initial cycle point) and chopping off all the tasks older than a given warm start cycle point. If the warm start cycle point is later than that of the R1 tasks, they won't run.

I've added some more documentation in https://github.com/cylc/cylc/pull/1278.

Cheers,

Ben

--

---
You received this message because you are subscribed to the Google Groups "cylc-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cylc-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Whitcomb, Mr. Tim]

Thanks, Ben! Those additions to the docs are nice too.

The way I originally posed this would have the same caveat (i.e. the initial cycle point has to be at 0,6,12,18 in order for it to work properly), but I like the "Every 6 hours" part. Is there any way in a suite to check for valid initial cycle times (i.e. only T00,T06,T12,T18 are valid and it would refuse to start if given T03)?

Tim

[Hilary Oliver]

Hi Tim,

> Is there any way in a suite to check for valid initial cycle times (i.e. only T00,T06,T12,T18 are valid
>  and it would refuse to start if given T03)?

There isn't a built-in way (although that might not be a bad idea...) but it's easy enough to do using Jinja2; e.g. from an earlier post to the mailing list:

> You can abort suite parsing by using Jinja2 to print an error message that is not valid cylc syntax (i.e. any error message):
>

> #!Jinja2

> {% set ICP = "20140808T03" %}
> {% if ICP | get_hour not in ["00","12"] %}
>    {{"ERROR: ICP must end in 00 or 12"}}
> {% endif %}

> # (rest of suite definition...)
>

> here get_hour is a custom filter in <suite-dir>/Jinja2Filters/get_hour.py:
>
> def get_hour(ICP):
>    return ICP[-2:]
>
> Validation of this suite returns:
>
> ParseError: Invalid line 1: ERROR: ICP must end in 00 or 12

This of course requires that your initial cycle point is held in a Jinja2 variable, and it assumes a string format ending in two hour digits (you could avoid this by having the filter make a system call to the "cylc cyclepoint" command). Another options is an initial task that fails if the initial cycle point is not right.

Hilary

[Hilary Oliver]

Just to add to Ben's comments on warm starts and initial tasks - the new way is much simpler: an unqualified "R1" means "run once at the initial cycle point".  A warm start is now simply a start at some point later than the initial point, so the R1 tasks won't run then. Suite visualization conforms to this too.  If you plot the graph from the initial cycle point you'll see the R1 tasks; if you plot it from a later point you won't.

Hilary

[Hilary Oliver]

Another minor point: the START string in this graph recurrence form ```R[n]/START[+/-OFFSET]/PERIOD/``` constrains the start point of the sequence, e.g. ```R/T00/P1D``` means "cycle daily from 00 hours at or after the initial cycle point" (even if the initial cycle point ends in 03 hours, say).  However, you can't say within a single recurrence expression "cycle 6 hourly from the first of 00 or 06 or 12 or 18 hours at or after the initial cycle point".

Hilary