parameterised tasks

[jo...@bodekerscientific.com]

I'm setting up some ensemble sensitivity tests using parameterised tasks in Cylc but am getting the following error when I run the suite:

"Illegal parameter value: [cylc][parameters]tnuc = -40..0..10: -40..0..10: bad value"

The relevant bits from my suite.rc:

[cylc]

    [[parameters]]

        ensemble = {{ range(ENSEMBLE_SIZE) | join(', ') }}

        tnuc = -40..0..10

        ar = 1, 3.47

 [scheduling]

      [[dependencies]]

         graph = "fcm_make => scm<ensemble,ar,tnuc> "

[runtime]

     [[scm<ensemble,ar,tnuc>]]

             TNUC_EN = $CYLC_TASK_PARAM_tnuc

             AR_EN = $CYLC_TASK_PARAM_ar

I've looked in the docs (https://cylc.github.io/cylc/html/single/cug-html.html#9.6.1) and I've had success when using ar =  1..5,  so not sure what I am doing wrong here.

Basically, I want to run the model with combinations of tnuc = -40, -30, -20, -10 and 0, and ar = 1 and 3.47

Any help would be much appreciated.

Jono

[Hilary Oliver]

Hi Jono,

The online user guide is always for the latest release, and my guess is you're using an older version of Cylc??  For integer parameter ranges, I think negative values have only been supported since cylc-7.6.0.

You example validates and graphs fine for me at cylc-7.7.2, if I add in the missing [[[environment]]] section heading near the bottom, and replace "ar = 1, 3.47" with "ar = 1, 3p47".  The "3.47" problem is arguably a bug in that we intend to support integer and string valued parameters, but quoting the float value doesn't cause it to be interpreted as a string (that's probably because all values in the suite config file are initially parsed as strings, and then interpreted as integer, real, etc. later according to the string content) ... but should be easy enough for you to work around (e.g. "3p47" or whatever).

If you're using the NeSI or NIWA HPC facilities in NZ, note that cylc-7.7.2 is installed, but is not the default yet.

Cheers,

Hilary

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jono Conway]

Hi Hilary

Yes, I'm on the NIWA HPC, which has cylc 7.5.0 by default. Are there any work-arounds for negative values in older versions of cylc or should I see about using the latest version of cylc on the HPC?

Jono

You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jono Conway
Research Scientist
Bodeker Scientific
Alexandra, New Zealand
Tel: +64 3 448 8118
Web: http://www.bodekerscientific.com/people/jono-conway  

[Tom Coleman]

You can use jinja to make a quasi negative list, where "-" is replaced by something like "m".

{% set tnuc_min = -40 %}

tnuc = {{ range(tnuc_min, 1, 10) | join(", ") | replace("-", "m") }}

Upgrading to cylc 7.7.x would be better though as you aren't then hacking around in other areas to reconvert the "m40", "m30" to an integer again.

[Hilary Oliver]

$ source /opt/niwa/share_prelim/niwa_modules.sh

$ module load cylc/7.7.2

$ cylc --version

7.7.2

Now when you run your suite, cylc will arrange for suite jobs to invoke cylc-7.7.2 on job hosts as well (by setting CYLC_VERSION in job environments).

Hilary

[Jono Conway]

great, thanks.

That sorted using negative integers

The variable ar is still passed to the UM namelist as 3p47, so these ensemble members fail. Do I need to do something other than $AR_EN in the UM namelist to get this converted to a float? 

thanks,

Jono

[Tom Coleman]

You could change the environment section to have:

             AR_EN = ${CYLC_TASK_PARAM_ar/p/.}

So, just make it substitute "p" for ".".

[Hilary Oliver]

Hi Jono,

Yep, cylc passes the literal parameter value to the task job environment as-is, so at run time your job scripting needs to transform it (3p47) into the form you need (3.47).  The quickest way to do that is probably as part of the AR_EN variable assignment as Tom suggests.

Hilary

[Jono Conway]

Thanks Tom and Hilary

Those fixes worked a treat!

J

[jo...@bodekerscientific.com]

Hi again

I am trying to have several variables vary together at the same time. 

i.e. 

run 1

a = -10, b = 263.15, c = -20

run 2

a = -20, b = 253.15, c = -30

and so on.

Is there any way to set the values of B and C depend on A at run time?

e.g. 

[runtime]

     [[model<a>]]

             A = $CYLC_TASK_PARAM_a

             B = $CYLC_TASK_PARAM_a + 273.15

             C = $CYLC_TASK_PARAM_a - 10

Obviously the above does not work, but I hope it shows what I am trying to achieve. 

any ideas appreciated

Thanks,

Jono

great, thanks.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jono Conway
Research Scientist
Bodeker Scientific
Alexandra, New Zealand
Tel: +64 3 448 8118
Web: http://www.bodekerscientific.com/people/jono-conway  

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jono Conway
Research Scientist
Bodeker Scientific
Alexandra, New Zealand
Tel: +64 3 448 8118
Web: http://www.bodekerscientific.com/people/jono-conway  

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cylc+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Hilary Oliver]

Hi Jono,

Sorry, it's not clear to me what you mean/want.

If a, b, and c are supposed to be cylc task parameters, then model<a> gets expanded over values of a, and each instance knows its parameter value at run time (via $CYLC_TASK_PARAM_a) but it knows nothing about the other parameters b and c (maybe you want to generate all the tasks model<a, b, c>?). 

Hilary

great, thanks.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jono Conway
Research Scientist
Bodeker Scientific
Alexandra, New Zealand
Tel: +64 3 448 8118
Web: http://www.bodekerscientific.com/people/jono-conway  

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jono Conway
Research Scientist
Bodeker Scientific
Alexandra, New Zealand
Tel: +64 3 448 8118
Web: http://www.bodekerscientific.com/people/jono-conway  

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to a topic in the Google Groups "cylc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cylc/16DDvlA32AM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cylc+uns...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jono Conway
Research Scientist
Bodeker Scientific
Alexandra, New Zealand
Tel: +64 3 448 8118
Web: http://www.bodekerscientific.com/people/jono-conway  

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

[Tom Coleman]

Hi Jono,

Do you just mean you want A, B, C stored as environment variables in that task? If B and C are constant changes from A, then yes, that is doable. You can do something like
[[model<a>]]
[[[environment]]]
B = $((CYLC_TASK_PARAM_a + 273.15)) #works for ksh as it handles floats, otherwise use something like awk to get your results if decimals are involved
    C = $((CYLC_TASK_PARAM_a - 10))

[Jono Conway]

Hi Tom,

Yes, you had the gist of it. Thanks for the awk tip - that worked a treat.  I ended up running a prescript to do the maths, as the printf wasn't available when I tried to use awk to set the variable in [[[environment]]].

 21 [runtime]

127     [[scm<ensemble,ar,tnuc>]]

128         inherit = NIWA_XC50, NIWA_XC50_SL

129         pre-script = """

130         # add offsets to other variables

131         export STARTICETKELVIN_EN=$( awk "BEGIN {printf \"%.2f\n\", CYLC_TASK_PARAM_tnuc + 273.15 }" )

132         export ALLICETDEGC_EN=$( awk "BEGIN {printf \"%.2f\n\", CYLC_TASK_PARAM_tnuc - 10 }" )

133         """

Cheers,

Jono