Adding density altitude

[Jacob Wood]

I would like to show density altitude and display an image graph on the weewx html page.

I'm running a raspberry pi with an i2c sensor.

I'm guessing the best way would be to have it calculated in software (formula here), how dewpoint is calculated?  I just don't understand when and where weewx calculates dewpoint and records it to a database.

I do have very little experience debian/python and everything else, so any help would be appreciated!

[gjr80]

Hi Jacob,

What you want to do with density altitude is straight forward though there are a few steps involved and you will need to get your hands into some python. Firstly some background.

Displaying a current obs value on a page/in a report merely needs us to add the obs to the loop packets and/or archive records generated by WeeWX. For density altitude this would be best done by writing a custom service that adds density altitude to the WeeWX generated loop packets. WeeWX will then accumulate the density altitude data over an archive period and include density altitude in the WeeWX generated archive record. It won't be saved to the database but it will allow you to use the current value (using the tag $current.densityAltitude - assumes the density altitude data observation is named densityAltitude) in a WeeWX generated report. To be able to plot density altitude or include aggregates such as min/max etc you need to save density altitude to the database. This can be done in a number of ways but is best done by customizing the WeeWX database schema.

If you have a read through the sections Customizing the weeWX service engine and Customizing the database you will get a bit of an idea of the concepts involved. The examples in these sections are similar to what you will need to do.

Creating a data service is quite easy, it can be as simple as something like the following (untested):

import weewx
from weewx.engine import StdService

class DensityAltitude(StdService):

    def __init__(self, engine, config_dict):

      # Initialize my superclass first:
      super(DensityAltitude, self).__init__(engine, config_dict)

      # Bind to any new archive record events:
      self.bind(weewx.NEW_LOOP_PACKET, self.new_loop_packet)

      self.last_total = None

    def new_loop_packet(self, event):

        if 'outTemp' in event.packet and 'outHumidity' in event.packet:
            try:
                da = event.packet['outTemp'] + 2.0 * event.packet['outHumidity']
            except TypeError:
                da = None
            event.packet['densityAltitude'] = da

Whilst this code uses a nonsense formula, it shows the basics of what you need to do. It creates a new service class based on class StdService, initialises the class and importantly binds itself to the NEW_LOOP_PACKET event which means we can intercept the WeeWX generated loop packet, use the data contained in it and add our new observation to it. The work is done in the new_loop_packet() method, this does a simple calculation and adds the value to the loop packet. There is a bit of checking to make sure the pre-requisites exist, the try..except is there to pickup if any of the pre-requisites exist but are None. Note that the WeeWX approach adding obs is if a source does not exist then don't add the obs, if the source exists but is not working (eg a failed sensor) then add the obs but set it to None, if the source exists and is functioning then add the obs value. Translated to a derived obs we would omit the derived obs if it's pre-requisites don't exist, add it as None if the pre-requisites exist but we cannot calculate a value and of course add the calculated value if all pre-requisites exist and the calculation can be completed. The loop packet is accessible via event.packet which is a python dictionary containing the loop packet data. You can view the fields available in your loop packets by running WeeWX directly and observing the LOOP: lines (REC: lines show the archive records).

Having the code is great but to get WeeWX to use it you need to save it to a file, say /home/weewx/bin/user/da.py (user code is best saved to the /home/weewx/bin/user directory as this directory is protected during a WeeWX upgrade). You then add the new service to the WeeWX service list in weewx.conf as follows:

[Engine]
   
    [[Services]]
        # This section specifies the services that should be run. They are
        # grouped by type, and the order of services within each group
        # determines the order in which the services will be run.
        prep_services = weewx.engine.StdTimeSynch
        data_services = user.da.DensityAltitude
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport

Restart WeeWX (run it directly) and you should see field densityAltitude appear in both loop packets and archive records.

You can modify the database schema so that density altitude is stored in the database by following steps in Modifying the database, just change the names to suit.

There are a few other gotchas that I am sure will come up, for example units (eg what if the loop packet temperature is in F and you need C for a formula - easily dealt with but not necessary for this walk through) but we can deal with them later, just get the concepts clear in your mind for now.

In any case, see how you go digesting the above and feel free to ask more questions.

Gary

[Jacob Wood]

Thank you for the reply Gary.  Managed to get the driver to calculate the density altitude and output it to the same csv file with all the other readings.  Would I still need to add the service, or what would be the best way to add my custom data field into the loop packets and archive?

Thank You

Jacob

[gjr80]

Jacob,

Not sure where the csv file comes into it, does your RPi produce a csv file that the file parse driver then reads? Or are we talking about something else? Also not sure what you refer to when you use the term 'driver', is that the driver for your weather station (file parse?) or the code for the service I mentioned above? Perhaps a brief description of your system/WeeWX config would help.

In any case, the standard approach taken by WeeWX is that drivers read data from sensors, do any raw processing needed to decode sensor readings then package up and emit data as either a WeeWX format loop packet and/or archive record. Any calculated/derived obs are added later via a WeeWX service. At the end of the day nothing is going to break if a driver adds derived obs, its just the convention that has been adopted (largely to keep a consistent approach with how derived obs are calculated).

If it were me I would have the driver I am using emit only the direct sensor obs, ie temperature from a temperature sensor, rain from a rain gauge. Derived obs such as dewpoint, windchill and density altitude I would add later via a service. WeeWX already adds a number of derived obs by default (eg windchill and dewpoint) to loop packets and archive records if they do not exist in the packet/record. This is done via the StdWXCalculate service. To add density altitude I would use a custom service like I outlined above, if you add services rather than modifying core WeeWX code your customisations remain across upgrades, otherwise they will be lost. I would then customise my db schema to add density altitude, once you have density altitude in your db schema and once you have density altitude in loop packets/archive records WeeWX will automatically save density altitude to the db. You can then use density altitude and aggregates thereof in your reports and you can also use density altitude in plots.

Gary

[Yann_M]

Hi Gary,

Following your previous replies & weewx 1st class documentation, I managed to get 'density' in my reports & database. I thank you for that !

But as the calculation for 'density' needs pressure (absolute in Pa) and my hardware (WS2812) is only returning a relative (to altitude) 'pressure' value, I needed to introduce an intermediate 'abspressure' value to my loop/archive records :

def new_loop_packet(self, event):

        if 'pressure' in event.packet :
            try:
                altitude_metric = 1326         # Need to get the '$Station.altitude' value (in meter) but how ?
                abspr = 100 * (event.packet['pressure'] * (1 - (0.0065 * altitude_metric / 288.15)) ** 5.255)

            except TypeError:
                abspr = None
            event.packet['abspressure'] = abspr

The above works fine, but instead of adding the new 'altitude_metric' variable, I'd prefer to use the elevation already set in weewx.conf, without knowing how to do so ?

Sorry if my question looks dumb, but as you can guess I have very little knowledge in Python, I understand 'station.elevation' beeing a Tuple (value, units, format) from which I have no idea on how to 1) call it in the above script and 2) isolate only its value for my calculation ?

Your help would be much appreciated ...

Yann

[gjr80]

Yann,

In your code you will be able to access the station altitude through the property self.engine.stn_info.altitude_vt. This property returns altitude as a ValueTuple which contains the altitude value, its units and unit group. The problem is you need altitude in metres and engine.stn_info.altitude_vt could be in metres or it could be in feet. To obtain the altitude in metres we need to run engine.stn_info.altitude_vt through a converter and ask for the result in metres. That will return us a ValueTuple that contains the altitude in metres, we can then access the actual numeric value by accessing the ValueTuples .value property. The following (untested) code should work:

altitude_metric = weewx.units.convert(engine.stn_info.altitude_vt, 'meter').value

I expect you will need to add the follwoing import to your import list if it is not already there:

import weewx.units

In terms of needing absolute pressure that should be what is appearing in the WeeWX pressure field. WeeWX uses three different pressures; pressure, altimeter and barometer - they are explained in this Wiki article. The driver for your station should know which fof the three pressures your station emits and it should populate the correct field. What driver are you using? If the WeeWX pressure field actually contains an altitude corrected pressure (ie WeeWX field altimeter) then somethign is not configured correctly or the driver is not working correctly.

Gary

[Yann_M]

Thank you for your last reply Gary,

Without much understanding why, this worked  :

altitude_metric = weewx.units.convert(self.engine.stn_info.altitude_vt, 'meter').value

Regarding the 2nd part of your answer / question, I'm using a LaCrosse WS2812 station (ws28xx driver), with this setting in weewx.conf :

pressure = hardware
barometer = hardware
altimeter = hardware

I get these outputs in the log:

REC:    2020-06-09 01:10:00 CEST (1591657800) abspressure: 86243.0903176, appTemp: 3.58493191965, cloudbase: 1383.6687016, dateTime: 1591657800, density: 1.0759016949, dewpoint: 5.6507119535, ET: None, heatindex: 6.09090909091, humidex: 6.09090909091, inDewpoint: 7.58780896445, inHumidity: 56.0, inTemp: 16.3909090909, inTempBatteryStatus: 0.0, interval: 5, maxSolarRad: 0.0, outHumidity: 97.0, outTemp: 6.09090909091, outTempBatteryStatus: 0.0, pressure: 1012.14545455, rain: 0.258, rainBatteryStatus: 0.0, rainRate: 1.19545454545, rxCheckPercent: 100.0, usUnits: 17, windBatteryStatus: 0.0, windchill: 6.09090909091, windDir: 18.6782954436, windGust: 4.90000000392, windGustDir: 22.5, windrun: 0.64909090961, windSpeed: 2.16363636537
LOOP:   2020-06-09 01:10:56 CEST (1591657856) abspressure: 86247.7380374, appTemp: 3.2397243331, cloudbase: 1383.62871855, dateTime: 1591657856, density: 1.07631006422, dewpoint: 5.56013080839, heatindex: 6.0, humidex: 6.0, inDewpoint: 7.59628096224, inHumidity: 56, inTemp: 16.4, inTempBatteryStatus: 0, maxSolarRad: 0.0, outHumidity: 97, outTemp: 6.0, outTempBatteryStatus: 0, pressure: 1012.2, rain: 0.0, rainBatteryStatus: 0, rainRate: 1.29, rxCheckPercent: 100, usUnits: 17, windBatteryStatus: 0, windchill: 6.0, windDir: 22.5, windGust: 3.10000000248, windGustDir: 22.5, windSpeed: 2.500000002

=> 'pressure' (REC:1012.14 & LOOP:1012.2) is wrong : due to station altitude being at 1330 meter, absolute pressure should be around the 860-865hPa mark. And as it's probably temperature compensated by the hardware (not explicitly written in the LaCrosse datasheet), it should then be called 'barometer' in WeeWX language.
=> 'barometer' is NOT provided by hardware.

=> 'altimeter' is NOT provided by hardware.

So, YES the hardware is emiting the 'pressure' instead of the 'barometer' value. Of course I also tried other settings in the .conf file, all without success, 'pressure' always being the only usable value ('barometer' and 'altimeter' went either missing or wrong).

This was the reason behind the need for my 'abspressure' calculation.

Y

[gjr80]

Some comments below.

On Wednesday, 10 June 2020 08:16:27 UTC+10, Yann_M wrote:

Regarding the 2nd part of your answer / question, I'm using a LaCrosse WS2812 station (ws28xx driver), with this setting in weewx.conf :

pressure = hardware
barometer = hardware
altimeter = hardware

The above settings (presumably under [StdWXCalculate] [[Calculations]]) mean that WeeWX will only accept pressure, barometer and altimeter from the driver/hardware and if any of the fields are missing WeeWX will not attempt to calculate the missing field(s). Putting aside for the moment whether the driver is truly emitting (absolute) pressure or some other value, that explains why you only ever see pressure in loop packets/archive records; pressure is the only pressure related field emitted by the WS28XX driver. The available settings for the [[Calculations]] stanza are described here in the User's Guide, you may find prefer_hardware is a better setting to use.

 

I get these outputs in the log:

REC:    2020-06-09 01:10:00 CEST (1591657800) abspressure: 86243.0903176, appTemp: 3.58493191965, cloudbase: 1383.6687016, dateTime: 1591657800, density: 1.0759016949, dewpoint: 5.6507119535, ET: None, heatindex: 6.09090909091, humidex: 6.09090909091, inDewpoint: 7.58780896445, inHumidity: 56.0, inTemp: 16.3909090909, inTempBatteryStatus: 0.0, interval: 5, maxSolarRad: 0.0, outHumidity: 97.0, outTemp: 6.09090909091, outTempBatteryStatus: 0.0, pressure: 1012.14545455, rain: 0.258, rainBatteryStatus: 0.0, rainRate: 1.19545454545, rxCheckPercent: 100.0, usUnits: 17, windBatteryStatus: 0.0, windchill: 6.09090909091, windDir: 18.6782954436, windGust: 4.90000000392, windGustDir: 22.5, windrun: 0.64909090961, windSpeed: 2.16363636537
LOOP:   2020-06-09 01:10:56 CEST (1591657856) abspressure: 86247.7380374, appTemp: 3.2397243331, cloudbase: 1383.62871855, dateTime: 1591657856, density: 1.07631006422, dewpoint: 5.56013080839, heatindex: 6.0, humidex: 6.0, inDewpoint: 7.59628096224, inHumidity: 56, inTemp: 16.4, inTempBatteryStatus: 0, maxSolarRad: 0.0, outHumidity: 97, outTemp: 6.0, outTempBatteryStatus: 0, pressure: 1012.2, rain: 0.0, rainBatteryStatus: 0, rainRate: 1.29, rxCheckPercent: 100, usUnits: 17, windBatteryStatus: 0, windchill: 6.0, windDir: 22.5, windGust: 3.10000000248, windGustDir: 22.5, windSpeed: 2.500000002

=> 'pressure' (REC:1012.14 & LOOP:1012.2) is wrong : due to station altitude being at 1330 meter, absolute pressure should be around the 860-865hPa mark. And as it's probably temperature compensated by the hardware (not explicitly written in the LaCrosse datasheet), it should then be called 'barometer' in WeeWX language.
=> 'barometer' is NOT provided by hardware.

=> 'altimeter' is NOT provided by hardware.

So, YES the hardware is emiting the 'pressure' instead of the 'barometer' value. Of course I also tried other settings in the .conf file, all without success, 'pressure' always being the only usable value ('barometer' and 'altimeter' went either missing or wrong).

 

If you are certain the value in the pressure field is in fact what WeeWX terms barometer then you should be able to make use of the WeeWX StdCalibrate and StdWXCalculate services to correctly obtain the three pressures without the need to alter the driver or add any other code. Let's say the driver is actually emitting what WeeWX knows as barometer in the pressure field. First up you can use StdCalibrate to assign the value in field pressure to field barometer and then clear the pressure field. To do this (not tested):

[StdCalibrate]
    barometer = pressure
    pressure = None

With these settings after the StdCalibrate service is run you will have the old pressure value in field barometer and the pressure field should be None. You can now have the StdWXCalculate calculate pressure and altimeter with the following settings:

[StdWXCalculate]
    ....

    [[Calculations]]
        pressure = prefer_hardware
        altimeter = prefer_hardware

There are a couple of variations you could use, you could omit the pressure = None setting and use pressure = software to force pressure to be calculated every time. The downside of this approach is that any service acting on your data after StdCalibrate is run and before StdWXCalculate would see the erroneous pressure field (only StdQC runs between StdCalibrate and StdWXCalculate on a default install so probably not much of a risk). You could also set pressure = software and altimeter = software since you know you will never use these values from your driver.

In any case (provided barometer is correct) you should now have correctly calculated pressure and altimeter fields in your loop packets and archive records. One thing to remember, calculation of pressure from barometer requires the temperature from 12 hours ago, if the temperature 12 hours ago is not available (eg for the first 12 hours of a new install) then pressure will not be calculated until the temperature 12 hours ago is available. Since altimeter is calculated from pressure altimeter will not be available either until the temperature from 12 hours ago is available.

Gary