math and formatting in cheetah templates

Vince Skahan

unread,

Jul 15, 2015, 11:39:54 PM7/15/15

to weewx-de...@googlegroups.com

Looking for opinions on best practices....

Last month, I started working on repackaging the wdSearchX3.py code that calculates the date+time of the last rain into an extension, based on a request or two from the -user list. Being the dry season here, I figured give it a try. Repackaging the weewx-wd routine into an extension was straightforward, but I also wanted to be able to say not just the date+time of the last rain, but how many days it's been since it rained.

Question is whether there's a best practice for getting the difference between two datestamps.

One way that works ok is to let cheetah do that calculation with its #set feature, ala:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<body>
<h2>Last measurable rain </h2>
<table>
<tr> <td> current time </td><td> $latest.dateTime</td> </tr>
<tr> <td> last rain </td><td> $last_rain </td></tr>
#set dayssince = "%.2f" % (($latest.dateTime.raw - $last_rain.raw) / (24*60*60.00))
<tr> <td> days since rain</td><td> $dayssince </td></tr>
</table>
</body>
</html>

Output of the above quickie template looks like:

Last measurable rain
current rain 15-Jul-2015 20:30
last rain 28-Jun-2015 09:50
days since rain 17.44

Since scenic Seattle/Tacoma has a no-rain and also a far longer always-rain season, I'm hoping to extend the weewx-wd stuff to also have a $last-no-rain kind of extension timestamp, and also be able to report how many days it's been since it "didn't" rain, so I was curious if doing the math in cheetah was good/bad/neither strategically.

Thomas Keffer

unread,

Jul 15, 2015, 11:53:11 PM7/15/15

to Vince Skahan, weewx-de...@googlegroups.com

I tend to do these things using a search list extension. There are lots of good reasons to go that way.

SLEs allow the calculation to be used elsewhere without having to recode.
SLEs can be packaged up as an extension, then installed to be used in templates other than the one you provide.
SLEs can easily return a ValueHelper and get all its goodies, like formatting and unit conversion options, for free.
It's good code practice to separate models from display.

By contrast, if you do everything in the template, the user has to manually copy and paste to new places, and fiddle directly with the formatting.

-tk

gjr80

unread,

Jul 16, 2015, 12:34:57 AM7/16/15

to weewx-de...@googlegroups.com

If I could add a few thoughts...not claiming to be any sort of expert but have used a lot of both with weewx-wd. Rules that I try to live by: (1) if it is fairly short and used once only I tend to put it I a template and (2) if I want formatting/unit conversion then use a SLE.
I do find following Python (SLE) code far, far easier than Python (Cheetah) so any thing complex is far better in a SLE. That being said if you look at weewx-wd you will find many instances where this has not been followed, testtags.php.tmpl is a good case. But then again, weewx-wd started early on with weewx v2 and weewx offers a lot more now, plus weewx-wd is a real behemoth anyway. Probably need to give some of the code a good cleanup/out.

Gary

Robin

unread,

Jul 16, 2015, 10:12:36 AM7/16/15

to weewx-de...@googlegroups.com

Vince,

Please could you post the working SLE here?

PRETTY PLEASE?

Regards

Robin

Vince Skahan

unread,

Jul 16, 2015, 11:41:37 AM7/16/15

to weewx-de...@googlegroups.com

coming soon....

Vince Skahan

unread,

Jul 18, 2015, 5:07:03 PM7/18/15

to weewx-de...@googlegroups.com

On Wednesday, July 15, 2015 at 8:53:11 PM UTC-7, Tom Keffer wrote:

I tend to do these things using a search list extension. There are lots of good reasons to go that way.

makes sense - and calculating the difference in seconds was very straightforward.

Where I'm a little lost in the SLE coding is how to return the additional value 'lastrain' with the right python structure so that it plugs into weewx's unit conversion magic where folks can set their units or decimal point accuracy etc. via skin.conf and/or formatting in their cheetah template.

The weewx-wd code I'm reusing looks like this (any cut/paste/tabs errors below are mine):

if lastrain_ts is not None:

try:

_row = db_lookup().getSql("SELECT MAX(dateTime) FROM archive WHERE rain > 0 AND dateTime > ? AND dateTime <= ?", (lastrain_ts, lastrain_ts + 86400))

lastrain_ts = _row[0]

except:

lastrain_ts = None

lastrain_vt = (lastrain_ts, 'unix_epoch', 'group_time')

lastrain_vh = ValueHelper(lastrain_vt, formatter=self.generator.formatter, converter=self.generator.converter)

search_list_extension = {'last_rain' : lastrain_vh}

So to get the difference since lastrain, I added in:

# days since last rain: might be no rain in a new db so handle that too

if lastrain_ts is None:

secs_since_lastrain = None

lastrain_days = None

else:

# delta time since last rain

secs_since_lastrain = ( time.time() - lastrain_ts )

lastrain_days = secs_since_lastrain / (60*60*24.00)

lastrain_days_vt = (lastrain_days, 'unix_epoch', 'group_time')

lastrain_days_vh = ValueHelper(lastrain_days_vt, formatter=self.generator.formatter, converter=self.generator.converter)

and modified the SLE at the bottom to add my lastrain_days parameter

search_list_extension = {'last_rain' : lastrain_vh, 'lastrain_days' : lastrain_days_vh }

So.....in my index.html.tmpl file if I reference $lastrain_days I get a date that looks like 1970, but $lastrain_days.raw is correctly the fractional days since the last rain was recorded.

What I *think* it should look like is $lastrain_days should be the days since last rain (not requiring .raw in the template), and that the formatting of how many decimal points it uses should be settable via skin.conf or alternately by formatting it in the .tmpl file

Does that make sense ? Suggestions what the easiest way to accomplish that might be, or where there are examples to borrow (more) code from ? I didn't see any obvious examples of a time-offset unit / group / whatever but I might be overthinking it at this point....

Thomas Keffer

unread,

Jul 18, 2015, 6:09:02 PM7/18/15

to Vince Skahan, weewx-de...@googlegroups.com

If I understand your problem correctly, you've got two problems.

First, there is a mix up in units. lastrain_days_vt is using measurement units of days, but you've put it in a value tuple where the unit is "unix_epoch," which is measured in seconds.

Everything should be in seconds. Perhaps it would help to keep things straight if you called it time_since_last_rain, or just lastrain, instead of lastrain_days.

The second problem is that the difference between two times is no longer a unix epoch time. It's just time. You can't use strftime() to format it. The "uptime" tag ($station.uptime) has the same problem, and machinery was put in units.py to deal with it. What you want is units 'second' and unit group 'group_deltatime':

lastrain_vt = (lastrain, 'second', 'group_deltatime')

When it comes time to convert into a string, it uses the (undocumented) option delta_time for the formatting, using a weird custom formatting, built on Python's "interpolation" facility. The file skin.conf doesn't list the option, so the default value is being used. If it were listed, it would look like this:

[[TimeFormats]]

# This section sets the string format to be used for each time scale.

# The values below will work in every locale, but may not look

# particularly attractive. See the Customizing Guide for alternatives.

day = %X

week = %X (%A)

month = %x %X

year = %x %X

rainyear = %x %X

current = %x %X

ephem_day = %X

ephem_year = %x %X

delta_time = %(day)d%(day_label)s, %(hour)d%(hour_label)s, %(minute)d%(minute_label)s

So, the tag

$last_rain

would give something that looks like the present "uptime" strings:

3 days, 1 hour, 48 minutes

Like any other ValueHelper, it can be overridden using the .format() suffix. So,

$last_rain.format("%(day)d%(day_label)s")

would give

3 days, 1 hour

Note that the option delta_time (in skin.conf) is shared with station "uptime," so if you specify it and change it, the formatting used for uptime will also change. So, perhaps it's best to use the .format() option. Or, maybe you like specifying the time since last rain down to the second!

Hope this helps.

-tk

Vince Skahan

unread,

Jul 18, 2015, 10:28:56 PM7/18/15

to weewx-de...@googlegroups.com

Thanks Tom - helped lots.

Example output is:

Last measurable rain
current time 18-Jul-2015 19:20
last db time 18-Jul-2015 19:20

last rain 28-Jun-2015 09:50

lastrain deltatime: 20 days, 9 hours, 31 minutes

Code wound up being pretty simple, I stole/borrowed/reused liberally from weewx-wd and your station.py uptime code ala the following. Still need to clean up the variable names a little for clarity, but the following looks like it works ok, also handles a new db with no rain data (yet) ok in brief simulator testing....

# Get ts for day of last rain from statsdb

# Value returned is ts for midnight on the day the rain occurred
_row = db_lookup().getSql("SELECT MAX(dateTime) FROM archive_day_rain WHERE sum > 0")

lastrain_ts = _row[0]
# Now if we found a ts then use it to limit our search on the archive
# so we can find the last archive record during which it rained. Wrap
# in a try statement just in case

if lastrain_ts is not None:
try:
_row = db_lookup().getSql("SELECT MAX(dateTime) FROM archive WHERE rain > 0 AND dateTime > ? AND dateTime <= ?", (lastrain_ts, lastrain_ts + 86400))
lastrain_ts = _row[0]

print lastrain_ts
except:
lastrain_ts = None
else:
lastrain_ts = None

# Wrap our ts in a ValueHelper

lastrain_vt = (lastrain_ts, 'unix_epoch', 'group_time')
lastrain_vh = ValueHelper(lastrain_vt, formatter=self.generator.formatter, converter=self.generator.converter)

# next idea stolen with thanks from weewx station.py
# note this is delta time from 'now' not the last weewx db time
delta_time = time.time() - lastrain_ts if lastrain_ts else None

# Wrap our ts in a ValueHelper
delta_time_vt = (delta_time, 'second', 'group_deltatime')
delta_time_vh = ValueHelper(delta_time_vt, formatter=self.generator.formatter, converter=self.generator.converter)

# Create a small dictionary with the tag names (keys) we want to use
search_list_extension = {'last_rain' : lastrain_vh, 'lastrain_delta_time' : delta_time_vh }

return [search_list_extension]

gjr80

unread,

Jul 19, 2015, 1:21:55 AM7/19/15

to weewx-de...@googlegroups.com

At the risk of being labelled a pedant, might I suggest that timespan.stop might be a better timestamp to use rather than time.time() when calculating secs_since_lastrain. I know it makes little difference in this case but the SLE will be used in a report that will be as of timespan.stop. I have found a few places where I have used time.time() and depending on the situation it can be an issue.

Gary

Thomas Keffer

unread,

Jul 19, 2015, 9:22:03 AM7/19/15

to Vince Skahan, weewx-de...@googlegroups.com

That looks pretty good. I think you've got it.

My only (small) suggestion is to keep the names a little more consistent. Use "last_rain", or "lastrain", but don't mix in the SLE dictionary.

-tk

Vince Skahan

unread,

Jul 19, 2015, 11:02:52 AM7/19/15

to weewx-de...@googlegroups.com, vince...@gmail.com

On Sunday, July 19, 2015 at 6:22:03 AM UTC-7, Tom Keffer wrote:

That looks pretty good. I think you've got it.

My only (small) suggestion is to keep the names a little more consistent. Use "last_rain", or "lastrain", but don't mix in the SLE dictionary.

Thanks Tom (and Gary) for the suggestions - I'll try to do the cleanup today if time permits....

Reply all

Reply to author

Forward

current rain	15-Jul-2015 20:30
last rain	28-Jun-2015 09:50
days since rain	17.44