How to handle mixed US and metric units in a driver?

Bill Burton

unread,

Mar 8, 2020, 3:57:43 PM3/8/20

to weewx-development

Hello,

I'm implementing a driver for the Columbia Weather Systems MicroServer that supports a variety of their weather stations including the Pulsar 600. So far I have the driver polling the station reliably under WeeWX 3.9.2 under Python 2.7 and 4.0.0b13 under Python 3.6.

However, the issue I'm trying to resolve is the station can return a mix of US and metric units for the different fields yet a loop packet can only return one unit type. So, what is the best way to return a record that could be a mix of unit types? Note that I'm requesting the extended XML format which returns the unit type for each field. The following are examples of these records, first US/Imperial followed by an all metric record:

My first inclination is to split up the fields into separate loop packets - one packet for US/Imperial measurements and a second packet for metric measurements then yield each one depending on availability. Does this sound like a reasonable approach or is there a better one?

However, some conversions will be required for metric as some measurements such as wind speed can be returned in different units being one of kph or meters per second. Otherwise, wind speed could be returned as mph or knots.

One issue is I haven't been able to find any definitive documentation as to all the fields a driver is supposed to output and what units are expected for US vs. metric packets. So any pointers on fields and their definitions would be helpful.

If there are any existing drivers that handle a mixture of units in a similar way that I could reference, that would be very helpful to know.

Thanks for any input,

-Bill

John Kline

unread,

Mar 8, 2020, 4:01:53 PM3/8/20

to Bill Burton, weewx-development

Since you are writing the driver, have you considered performing the necessary conversions so that the driver returns US or metric (i.e., one or the other, not a mix)?

On Mar 8, 2020, at 12:57 PM, Bill Burton <bburt...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/1142ae3a-c6c4-47d6-af01-33de21b2d5fd%40googlegroups.com.

Thomas Keffer

unread,

Mar 8, 2020, 10:33:06 PM3/8/20

to Bill Burton, weewx-development

All units within a packet must use one of the three weewx unit systems, US, METRIC, or METRICWX. You can change from packet to packet, but not within a packet.

You could emit a different packet for each observation type, but you could miss out on some calculations. For example, to calculate dewpoint, you need outside temperature plus outside humidity. If they are in separate packets, StdWXCalculate can't perform the calculation.

Make sure you read the section Porting to new hardware in the Customization Guide.

-tk

--

Bill Burton

unread,

Mar 8, 2020, 11:32:33 PM3/8/20

to weewx-development

Hello,

On Sunday, March 8, 2020 at 4:01:53 PM UTC-4, John Kline wrote:

Since you are writing the driver, have you considered performing the necessary conversions so that the driver returns US or metric (i.e., one or the other, not a mix)?

Yes, good question. I suppose it could be done that way picking one unit type for the packet and then converting all fields not in the same unit type. However, if the weather station is configured to return metric units but the driver is written to convert all fields to US/imperial but the archive database is configured to be metric, then there's a double conversion, metric -> US -> metric. So I was thinking if the fields returned by the driver are the original units returned by the weather station, then there's zero or one conversions to store archive records, never two conversions.

-Bill

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-de...@googlegroups.com.

Thomas Keffer

unread,

Mar 8, 2020, 11:45:15 PM3/8/20

to Bill Burton, weewx-development

As you concluded, it's generally best to keep the unit system as close to the "native" unit system of the hardware as possible.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/0d252600-cedb-4d9a-8902-b14028684e99%40googlegroups.com.

John Kline

unread,

Mar 9, 2020, 12:03:06 AM3/9/20

to Thomas Keffer, Bill Burton, weewx-development

This isn’t the general case. I find it difficult to see the harm in converting to and fro for some cases. Are you worried about loss of precision?

On Mar 8, 2020, at 8:45 PM, Thomas Keffer <tke...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/CAPq0zEAD_5VZ%2B54oLvhkZ7m1XeqDDJ0YHDdvvxvrXpL0J3FrOg%40mail.gmail.com.

Thomas Keffer

unread,

Mar 9, 2020, 12:17:38 AM3/9/20

to weewx-development

Yes. A good driver makes minimal interpretations about the data it is seeing. If possible, it should just pass it on.

John Kline

unread,

Mar 9, 2020, 12:26:31 AM3/9/20

to Thomas Keffer, weewx-development

I agree that, all things being equal, a driver should just pass it on; but this is not that case.

There won’t be any meaningful loss of precision for the double conversions. As such, the alternative, splitting into separate packets, is an unnecessary complication even without factoring in that StdWXCalculate might not be able to perform some calculations.

On Mar 8, 2020, at 9:17 PM, Thomas Keffer <tke...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/CAPq0zEB4b%3DKjhmSL5A52OakiPURH8k1JzpijSCCrY9vNxJ83bg%40mail.gmail.com.

Thomas Keffer

unread,

Mar 9, 2020, 9:48:04 AM3/9/20

to John Kline, weewx-development

Well, I did say, "if possible!"

In your case, it does make sense to convert to a common unit system and keep everything together in one packet.

Bill Burton

unread,

Mar 9, 2020, 9:54:14 AM3/9/20

to weewx-development

Hello Tom,

On Sunday, March 8, 2020 at 10:33:06 PM UTC-4, Tom Keffer wrote:

All units within a packet must use one of the three weewx unit systems, US, METRIC, or METRICWX. You can change from packet to packet, but not within a packet.

Yes, that's what I understand from looking at many drivers.

You could emit a different packet for each observation type, but you could miss out on some calculations. For example, to calculate dewpoint, you need outside temperature plus outside humidity. If they are in separate packets, StdWXCalculate can't perform the calculation.

Thanks for mentioning that. The way the station supports changing units is only for one unit type, not generally for individual fields. Here's a screen shot of the units selection admin screen:

So for example, if temperature is changed to degrees C, then all temperature fields with return that unit.

Here's a list of all the possible units available:

Rain: inches or mm
Temperature: degrees F or degrees C
Wind speed: mph, kph, meters/second, knots
Pressure: inches Hg, Millibar, Kilopascals, Hectopascals
Altitude: feet or meters
Visibility: miles, meters, kilometers, feet
Rain rate: inches/hour or mm/hour

Based on the different metric units this station returns, it looks like the METRICWX type is better most of the time. An exception would be wind in kph would best use the METRIC unit type.

Make sure you read the section Porting to new hardware in the Customization Guide.

Thanks for mentioning that page. I had reviewed it several times before I started work on the driver but there were a few things I'd missed.

-Bill

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-de...@googlegroups.com.

mwall

unread,

Mar 9, 2020, 10:26:56 AM3/9/20

to weewx-development

bill,

for hardware that can be configured to send data in different unit types, you have to either:

a) in the driver initialization, set the hardware to the units that you want, then hope that no one changes it while the station is running

or:

b) in every loop packet check the unit type for each observation, then convert (if necessary) to the units in the usUnits system of your loop packet

option b requires more calculation, but it is much more robust. and unless your hardware sends data more frequently than 10 times a second (or even 100 times a second?) it will have no affect on performance.

m

Bill Burton

unread,

Mar 9, 2020, 12:57:08 PM3/9/20

to weewx-development

Hello Matthew,

On Monday, March 9, 2020 at 10:26:56 AM UTC-4, mwall wrote:

bill,

for hardware that can be configured to send data in different unit types, you have to either:

a) in the driver initialization, set the hardware to the units that you want, then hope that no one changes it while the station is running

That's what I'm doing now except that it's not possible to configure the station via an API so the driver is just trusting the returned units are the right ones.

or:

b) in every loop packet check the unit type for each observation, then convert (if necessary) to the units in the usUnits system of your loop packet

option b requires more calculation, but it is much more robust. and unless your hardware sends data more frequently than 10 times a second (or even 100 times a second?) it will have no affect on performance.

Actually, since I'm writing the driver to interface with the Columbia MicroServer computer which in turn interfaces with the Pulsar 600 hardware, that computer is doing all the heavy lifting. So this driver just polls the MicroServer computer at an arbitrary interval and downloads the data. Since there's no configuration API, the driver can't easily be used to configure the MicroServer, So I'm trying to make the driver as robust as possible so no matter what units of data is thrown at it, the results will be correct and predictable.

Right now I'm using a polling interval of 10 seconds to download the XML data but only with the expectation that eventually the skin I'm using will be able to display frequent updates for wind data.

Since wind data is the only type that can benefit from very frequent updates, my thought at this point is to return one packet each polling interval just containing windSpeed, windDir, windGust and windGustDir and specify the units for the loop packet to whatever unit type is returned in the XML for the windSpeed. For the rest of the data, return it every 60 seconds. I have an idea on how to implement this using two packets -- one for US units and a second for metric units. The logic for this approach should be only slightly more complex than converting all the data to a common unit type.

m

Also, using your IP-100 driver as a starting point turned out to be a good decision. This driver is structurally the same as that one as I was able to make most of the changes within a few of the existing methods. The hard parts were porting it to also work with v4 and Python 3 along with updates to the exception handling as the hardware returns corrupt XML from time to time.