How to tell weewx to expect metric or US when importing from a text file?

[Ashley Hinton]

Hey everyone.

I'm having a bit of a brain freeze over how to tell Weewx what units (metric or US) to expect when importing a temperature figure from a text file.

My set up uses Weewx version 3.9.2 running under Python2 (setup.py install method) on a Raspberry Pi 4, OS is Raspian Buster (Debian 10)

I have two data sources at the moment:

The main weather station is an Aercus Instruments Weathersleuth which is basically a fine offset observer. That uses the Intercepter plugin /home/weewx/bin/user/interceptor.py

I also have two extra sensors feeding from another pi on the network using the One Wire (OWFS) plugin /home/weewx/bin/user/owfs.py - these are mapped as extraTemp1 & 2 and report as Greenhouse and Garage temperatures.

As far as I'm aware all of those report the temperature as metric when polled by weewx and I don't recall having to modify anything.

What I want to do is map a third, extraTemp3, to data from a Temper USB temperature sensor I've got in the cupboard where the Raspberry Pi sits (the one running Weewx) 

All well and good, i've installed a python application which when called will report the temperature from the Temper.

> temper-poll

Found 1 devices

Device #0: 39.7°C 103.4°F

I'm sure I can write a bash script to run temper-poll, strip out unwanted data, and overwrite the result to a file in /tmp/ (such as cupboard.txt) - once happy I'll set up a cron for the script and then a service in /home/weewx/bin/user as per the example: https://github.com/weewx/weewx/wiki/add-sensor  - modifying as needed.

As per notes within weewx.conf [StdConvert] has always remained as US units.

What I can't figure out is how Weeex knows whether the incoming data, from any of the sensors across the system, are metric or US? is it specified within the service(s) themselves?

I notice further down in the example of adding an extra sensor::

That is, whatever unit system you use in pond.txt, must match the unit system used by the incoming record. The unit system used by the incoming record will be given by event.record['usUnits']

Does that mean I need to add the above to my new service?

In short:

Both Interceptor and Owfs are presenting temperature data in degrees C, but the Weewx [stdConvert] is untouched from default (US)

My reports are in metric, but of course that's just the reports. I could change it at any time without affecting the database or the incoming sensor data to Weewx.

Do I need to do anything to my new service in /weewx/bin/user/ to specify that the data is metric or US depending on what value gets presented from /tmp/cupboard.txt ?

I'm reluctant to just try it as I don't want incorrect data, even though this is just so I an easily see the temperature of the cupboard where my main "server" raspberry pi, router, network switch and a few other bits of kit reside. Ignore the alarmingly high temp as the stick is currently plugged directly into the pi and its a hot day here - a USB extension cable will give a more accurate reading of the ambient temp.

Thanks for any help and guidance and of course apologies in advance if it's in the instructions (I did try a search - honest!)

Regards

Ashley

[gjr80]

Hi,

There is one simple rule when adding/altering fields in a loop packet/archive record; always take note of the unit system used by the packet/record. Loop packets are emitted by the driver using a unit system of the drivers choice, it does not matter what unit system is used and it need bear no resemblance to the unit system used in the database or in any reports (the StdConvert service takes care of converting packets/records to the unit system used by the database and there are mechanisms within the StdReport service for converting to whatever display units the user wishes to use). If a WeeWX service wants to add a field to the loop packet that service should check the unit system used in the packet and then convert the data to be added to the appropriate units before adding the data to the packet. In some cases it does not matter (eg humidity, radiation, direction all use the same units across all unit systems) but obviously it matters for temperatures, speeds, pressures etc. You can certainly have your external source provide temperatures in C if your driver is emitting temperatures in C but what happens if you get a new station/driver that emits F? Such a chnage would result in your C data being treated as if it was in F.

So back to your system. The service that you write will need to do a number of things. First it needs to read your data and know what units are being used (units could be hard coded in your service or read from the external source - its your source so you control it). Then you need to check the unit system used in the record you are augmenting and convert your data to the units used in the record. Finally you add the converted data to the record. Using the example you linked your code might look something like (untested):

import syslog
import weewx
import weewx.units
from weewx.wxengine import StdService

class PondService(StdService):
    def __init__(self, engine, config_dict):
        super(PondService, self).__init__(engine, config_dict)      
        d = config_dict.get('PondService', {})
        self.filename = d.get('filename', '/var/tmp/pond.txt')
        syslog.syslog(syslog.LOG_INFO, "pond: using %s" % self.filename)
        self.bind(weewx.NEW_ARCHIVE_RECORD, self.read_file)

    def read_file(self, event):
        try:
            with open(self.filename) as f:
                value = f.read()
            value = float(value)            
            syslog.syslog(syslog.LOG_DEBUG, "pond: found value of %s" % value)
        except Exception as e:
            syslog.syslog(syslog.LOG_ERR, "pond: cannot read value: %s" % e)
        else
            value_vt = weewx.units.ValueTuple(value, 'degree_C', 'group_temperature')
            conv_value_vt = weewx.units.convertStd(value_vt, event.record['usUnits'])
            event.record['extraTemp1'] = conv_value_vt.value

I have highlighted the changes in yellow. Firstly we import weewx.units as we are going ot use some of the unit conversion functions that reside in the WeeWX units module. The changed use of float() lets us make use of the existing try..except to catch the error that would occur if the data read cannot be converted to a number. The code indented under 'else' is executed if the try..except does not encounter an error, in other words the data is read and successfully converted to a number. Looking at the 'else' code in detail, first we express the imported data as a ValueTuple, this is a three element tuple that defines a value, its units and the unit group it belongs to. This is necessary to use the WeeWX unit conversion functions. In this case we know our imported data is in C and is a temperature. We then call the convertStd function to convert that ValueTuple into the units used by that particualr unit group in that particular packet. There may be no conversion needed, but we know now that irrespective of the unit system used in event.record, conv_value_vt contains our imported data in the same temperature units used in event.record. So we can just simply add the value to the record.

Hopefully that gives you some hints on a way ahead.

Gary

[Ashley Hinton]

Hi

Thanks so much for your help, I have a better understanding and have managed to get it working by modifying your example as needed. 

Initially WeeWX crashed with a syntax error on else but guessed it needed to say else: after noticing try: (without googling it)

In my weewx.conf I have an entry under the generic labels section:

extraTemp3 = The Core Temperature (we nickname the cupboard where the router, etc sits as "the core")

Under [engine] [[service]] I added:

user.cupboardtemp.CupboardService

I'm not bothering to plot any graphs as I just want to quickly see how the cupboard is doing heat-wise on the main report:

http://www.achinton.co.uk

It turns out that temper-poll (which polls the Temper USB device) has a -c option which just returns the value in degrees C, I didn't need to filter anything before writing to the text file which the WeeWX service then reads.

At the moment I have a cron which runs every 10 minutes executing a bash script which contains:

/usr/bin/python2 /usr/local/bin/temper-poll -c > /tmp/cupboardtemp.txt

I should imagine there are more elegant ways of getting the data from the Temper USB directly within a WeeWX service, the temper-poll program was written in python as well so certainly something I'll have a play around with. Plus learning some Python is something I keep promising myself.

Thanks again,

Ashley