How to Access summaries in a restful driver

[Karen K]

I searched the topics, and I did not found anything about that topic, so I try to ask.

I am new to Weewx, and I am new to Python. So have mercy, if I do something stupid.

I wanted to upload data to Wetternetz-Sachsen. There was no driver, so I tried to adapt an existing one. The example I used was the "Windy" driver. And I am glad to say it works. 

All is fine so far.

But they want the user to upload some aggregate values such as minimum temperature and maximum temperature, too. The record variable contains the data of the "archive" table, only. But there are additional tables such as "archive_day_outTemp". They are used in reports, but I found no way to access them.

What did not work was that:

def get_record(self, record, dbmanager): 

    # run parent class
    _datadict = super(WnsThread,self).get_record(record,dbmanager)
    # actual time stamp 
    _time_ts = _datadict['dateTime'] 

    try:

        if 'outTempDayMax' not in _datadict:

            _result = dbmanager.getSql(

                "SELECT min,max FROM %s "
                "WHERE dateTime = ?"
                % "archive_day_outTemp", (_time_ts))
            if _result is not None:
                _datadict['outTempDayMin']=_result[0]
                _datadict['outTempDayMax']=_result[1]
    except weedb.OperationalError as e:
        log.debug("%s: Database OperationalError '%s'",self.protocol_name,e)
    return _datadict

If I replace "archive_day_outTemp" by dbmanager.table_name, then no exception is raised any more, but the "archive" table is read instead of the table I want to read. So I see the problem is within the getSql() statement. But I have no clue.

[gjr80]

Hi,

You are on the right track but making things difficult for yourself by trying to use the daily summaries to obtain day max/min aggregates. Whilst it’s true that the day max/min values can be found in the daily summaries they are also available by querying the archive table. In this case querying the archive will not take too long, if you were after max/min for a month or year then you might consider going to the extra trouble of using the daily summaries(or maybe not). The get_record() method has everything there that you need to query the archive, have a look at the get_record() method in weewx.restx.RESTThread, in particular the part where dayRain is calculated. In this case SUM() is used but it could just as easily be MAX() or MIN() to give you the max/min for the day for the field you are interested in..

Gary

[Karen K]

Hi Gary,

thank you very much for your reply. I did as you suggested, and it worked.

I only wonder why those summeries are calculated, then. To no purpose? I don't know.

Karen

[Vince Skahan]

On Friday, October 30, 2020 at 7:46:14 AM UTC-7, Karen K wrote:

I only wonder why those summeries are calculated, then. To no purpose? I don't know.

Pre-calculated info you can query much faster than aggregating the low-level archive data.

Think of the NOAA reports.  If you wanted to look back to see the hi/lo/ave info for a date in time, would you just look at the pre-computed info in those reports (which already has the info you want), or would you do a complicated db query to figure it out and do totals/averages/min/max calculations ?

[Tom Keffer]

There is an API for calculating these things: XTypes. It will automatically calculate using the daily summaries if it can. No need to do the SQL queries directly.

It would look something like this. (NOT TESTED; highly schematic, parts missing):

import weeutil.weeutil
import weewx.units
import weewx.xtypes
import weewx.restx


class WnsThread(weewx.restx.RESTThread):
   
    ...

   
    def get_record(self, record, dbmanager):
        # run parent class
        _datadict = super(WnsThread, self).get_record(record, dbmanager)
        # actual time stamp
        _time_ts = _datadict['dateTime']

        # Get the midnight-to-midnight time span:
        timespan = weeutil.weeutil.archiveDaySpan(_time_ts)

        # Use XTypes to calculate the aggregate. The result will be a ValueTuple
        temp_min_vt = weewx.xtypes.get_aggregate('outTemp', timespan, 'min', dbmanager)
        temp_max_vt = weewx.xtypes.get_aggregate('outTemp', timespan, 'max', dbmanager)

        # Convert to the same unit system as the incoming record. The result of the conversion
        # will be a ValueTuple. The first element of the tuple (subscript '0') will be the actual
        # value.
        _datadict['outTempDayMin'] = weewx.units.convertStd(temp_min_vt, record['usUnits'])[0]
        _datadict['outTempDayMax'] = weewx.units.convertStd(temp_max_vt, record['usUnits'])[0]

        return _datadict

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/1729b30f-48cd-47c7-bfdc-c1efc7505f23o%40googlegroups.com.

[Karen K]

Thank you all for your support. 

What Tom Keffer suggested sounds short and simple. I did not test it by now, because it already worked, but I think, I will try that solution at a later time.

I uploaded a first version of the driver to github:

https://github.com/roe-dl/weewx-wns

[Karen K]

I was curious about the XTypes. So I could not wait and tried them for some values I had not included by now. 

It works great.

Some time spans are not to be aligned at boundaries. For them I tried:

from weeutil.weeutil import TimeSpan
h3timespan = TimeSpan(_time_ts-10800,_time_ts)

I hope that won't be aligned otherwise.