Aggregated Values in MQTT Output

[Karen K]

If I recall it right, sometimes there were requests for the possibility to output aggregated values to MQTT. 

What I was thinking about, was the problem of the Belchertown skin showing wrong rain values due to the calculation of the observation type dayRain. See Belchertown issue #685

There is a comment in restx.py saying:

                # NB: The WU considers the archive with time stamp 00:00
                # (midnight) as (wrongly) belonging to the current day
                # (instead of the previous day). But, it's their site,
                # so we'll do it their way.  That means the SELECT statement
                # is inclusive on both time ends:

That is also used for MQTT, but it seems not appropriate in the context of MQTT. So I looked for a solution.

My result is a more general solution allowing the user to define arbitrary aggregated values for output with MQTT by using tags similar to them used for Cheetah generator. I sent a PR (#31) for Matthew Wall's MQTT extension in the hope it will be useful und get merged.

With a simple dayRain = day.rain.sum the observation type dayRain will be re-calculated (for MQTT only) with the normal interval that is open at the left side.

Other aggregated values can be defined in the same simple way.

[Greg Troxel]

Karen K <kk44...@gmail.com> writes:

> There is a comment in restx.py saying:
>
> # NB: The WU considers the archive with time stamp 00:00
> # (midnight) as (wrongly) belonging to the current day
> # (instead of the previous day). But, it's their site,
> # so we'll do it their way. That means the SELECT statement
> # is inclusive on both time ends:
>
> That is also used for MQTT, but it seems not appropriate in the context of
> MQTT. So I looked for a solution.

Assuming 5m archive interval, without loss of generality.

Most observations in weewx are a value at a time, with temperature being
the canonical example. 0000 is indeed part of the same day as 0001, so
the temperature at 0000 should be used in min/max for "next day" rather
than "previous day".

But, rain is different. The rain value at 0000 is the rain that fell
between 2355 and 0000. So that value belongs in the sum of the previous
day.

More pedantically, one can thing of rain as the number of clicks
arriving

>= 235500 and < 000000

or perhaps it's > and <=. It's hard to know, and it of course doesn't
really matter.

All of this is a long way of saying that I think considering rain that
is reported at 0000 to belong to the previous day is sensible, always.

I also don't really care personally, as whether daily rain is
[0000,0000) or [0005,0005) doesn't matter to me, or the plants.

[Karen K]

May be I should give an additional example regarding rain. And this is the reason people were aware of that detail:

Actually if you show yesterday rain ($yesterday.rain.sum) and today rain (dayRain) then the rain between 23:55 and 00:00 ist included in the yesterday rain as well as in the today rain. If you add it together, then you get a higher value than the rain of those 2 days together.

The general way of WeeWX is to think of an archive interval to be open at the left end and closed at the right end. This is because it records the readings of the last 5 minutes.

[Greg Troxel]

Karen K <kk44...@gmail.com> writes:

> The general way of WeeWX is to think of an archive interval to be open at
> the left end and closed at the right end. This is because it records the
> readings of the last 5 minutes.

Thanks; I was unclear on that and (] seems sensible.

> Actually if you show yesterday rain ($yesterday.rain.sum) and today rain (
> dayRain) then the rain between 23:55 and 00:00 ist included in the
> yesterday rain as well as in the today rain. If you add it together, then
> you get a higher value than the rain of those 2 days together.

That sounds like a bug. Surely it should be in one or the other, and I
think it's 99% clear it belons in $yesterday.

[Tom Keffer]

In WeeWX, intervals are consistently open on the left, closed on the right. That means that the first archive interval of the day runs from a split second after 0000 and ends precisely at 0005. A temperature timestamped at 0000 will be included in the previous day's statistics. Same with rain: rain that is timestamped 0000 will be included in the previous day's total.

Same with days: a day starts a split second after midnight, and ends precisely at the next midnight. Unfortunately, with the WU, if you timestamp something 0000 it includes it in the current day, not the previous day. It's as if the day started 5 minutes before midnight, then ends 5 minutes before the next midnight. 

I'm not keen on making this behavior optional because so much of the code depends on the convention of open on the left, closed on the right. At the very least, if the option was exercised, all database queries would have to be reformatted. 

At the end of the day (pun intended), I decided to indulge the WU, but I don't think it's a good idea in general.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/rmi5yn3q0nn.fsf%40s1.lexort.com.

[Karen K]

So I hope the topic of intervals is settled with Tom's post.

Let us then go back to the original topic of this thread: 

• to make MQTT use the standard kind of intervals even for dayRain and 
  
• to provide a means to send aggregated values by MQTT.
  

That is the goal of the PR #31 to Matthew Wall's MQTT extension.

[Karen K]

@Matthew Wall: Could you have a look at that PR?