Database Schema: Signals

[Craig Young]

What units group do the observation type Signals fall under?   Or, if I use a signal do I need to update a configuration file to place it into a units group.

[Craig Young]

In the wview_extended.py schema there is a group of types for signals (signal1, signal2, .. signal8) and stored in the DB as reals.  I looked in units.py but did not see signals listed.

[Tom Keffer]

Signals are for ill-defined measurements. 

Unit groups exist for two reasons:

1. To pick an appropriate unit for a type of measurement. For example, ºC for temperatures.
2. To pick an appropriate format and label.

Signals don't fit neatly into these reasons. They don't take a unit, and it's not obvious what format and what label they should use. So, they were left out of units.py and defaults.py. 

You can use the signal types without adding them to a unit group. You just won't be able to convert them to a different unit (which they don't have anyway), and there will be no automatic formatting and labelling. If you need a format, use a .format() suffix. If you need a format, just add it on. For example:

$current.signal1.format("%d") widgets

Alternatively, if your signal is actually some kind of counter, you could assign them to group_count. Then they would use "%d" for the format, and an empty string for the label.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b9fa5024-38c9-4d95-8e4b-5c9bb4f0ddb8n%40googlegroups.com.

[Craig Young]

Thanks Tom.  Signal1 for my station is the signal voltage from a pyrgometer.  Signal2 is the temperature of the pyrgometer sensor (C) and Signal3 is the long wave intensity (W/m2) calculated by the datalogger.  So if I understand correctly, the weewx engine will pass these values untouched through the various services and add to the DB as real numbers.  I can then deal with them manually when creating the report.

[Tom Keffer]

Exactly.

Or, alternatively, you can assign them to appropriate unit groups in the file user/extensions.py:

import weewx.units

weewx.units.obs_group_dict['signal1'] = 'group_radiation'

weewx.units.obs_group_dict['signal2'] = 'group_temperature'

weewx.units.obs_group_dict['signal3'] = 'group_radiation'

Then you would not need to specify a format and label. It would also allow you to do things like 

The temperature is $current.signal2.degree_C ($current.signal2.degree_F)

to publish the temperature in both ºC and ºF.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2dd9fe34-ffde-4e8f-aaf6-b205009b76a7n%40googlegroups.com.

[gjr80]

The other other variation on Tom's advice to use extensions.py, particularly if you are (still keen on) writing your own driver, is to include the unit group assignments in the driver file. They statements only need to be somewhere where they are executed each time WeeWX starts. If the fields are inextricably linked to the driver having everything in one place can be of benefit. I've used this approach with some of the drivers I have written.

Gary

[Tom Keffer]

Good advice!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/bfef0c77-5e59-4d78-b1a8-f557b0dddaa6n%40googlegroups.com.

[Craig Young]

I will add the assignments in the driver.

[Craig Young]

Signal4 and Signal 5 are sensor tilt measurements in degrees.  For example, if the sensor is tilted 1.5 degrees in the N/S direction the value for Signal4 = 1.5 degrees.  Looking at units.py and defaults.py I see groups for degrees temperature, degrees direction, but not degrees tilt.  Should a new group "group-tilt" be added through the driver or should this be added to weewx files to be made available to everyone?

[gjr80]

Remember, unit groups are for picking an appropriate unit for a field/observation (including allowing for conversion between applicable units) and providing a suitable format and label. Your description brings to mind two possible suitable unit groups; group_direction and group_angle. Both of these unit groups support fields/observations in degrees (degree_compass for group_direction and degree_angle for group_angle. Both provide (default) unit labels of the degree symbol, both offer similar (default) formats (%03.0f v %02.0f). group_angle facilitates unit conversion between degrees and radians, which may or may not be something you use now or in the future, group_direction does not support this conversion.

We tend to re-use unit groups where we can, only creating new unit groups where there is no other suitable unit group. Remember, default unit group formats and labels are only defaults and can be altered or overridden in a tag using .format() (eg for tilt you might want to always display a sign and format to one decimal place so you might decide to alter the default or override the default format using .format() with the format string '%+02.1').

Gary

[Craig Young]

Because tilt can be a negative number I think we have to rule out group_direction which I think has a range of 0 to 360.  Not sure about group_angle, do you know if it allows for positive and negative angles?

Craig

[gjr80]

Unit groups pose no limitations on what data can be stored in any database field. All WeeWX numeric fields in the db schemas shipped with WeeWX are of type real so can handle positive and negative numbers. What the unit groups do is set the default formatting and default unit label when the field is used in a WeeWX report tag. The unit group also sets what units the value can be converted to/from. The default formatting and labels associated with a unit group can be (and are) changed by the user either via config file or programatically. Default formatting can be overridden on a tag by tag basis through use of the .format() tag. Unit conversion options available with a given unit group can also be changed (eg more available units added), though this is normally done  programmatically and is not something that can be done via tags in reports.

You will find that group_direction and group_angle will both allow you to store your positive/negative angle values; the difference will be in the default formatting, default unit label and available unit conversion options. Since you have positive/negative values I expect you will be, at the very least, wanting to use a format in reports that is different to the default used by both unit groups (refer to my previous post for the default formats). Depending on requirements this might be as simple as using the .format() tag in reports.

Sorry if I seem to be belabouring the point, but I think it important to be clear that unit groups have no effect on what/how data is stored in the database; unit groups only affect the presentation of data.

If it was me I would be use group_angle , mainly because the unit group name is a better fit and the availability of conversion to radians if required. I would use either .format() to change tag formatting in reports or override the default formatting in skin.conf/weewx.conf if I frequently use the field in tags.

Gary

[Craig Young]

I inserted this line into my driver:

 weewx.units.obs_group_dict['signal1'] = 'group_angle'

when I run weewx I get this error in syslog:

Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__: Caught unrecoverable exception:
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****  'group_angle'
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****  Traceback (most recent call last):
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 154, in main
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      engine.run()
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 210, in run
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 245, in dispatchEvent
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      callback(event)
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 363, in new_loop_packet
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      converted_packet = self.converter.convertDict(event.packet)
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/units.py", line 952, in convertDict
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      target_dict[obs_type] = self.convert(as_value_tuple(obs_dict, obs_type))[0]
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/units.py", line 917, in convert
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      new_unit_type = self.group_unit_dict.get(val_t[2], USUnits[val_t[2]])
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/lib/python3.7/collections/__init__.py", line 914, in __getitem__
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      return self.__missing__(key)            # support subclasses that define __missing__
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****    File "/usr/lib/python3.7/collections/__init__.py", line 906, in __missing__
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****      raise KeyError(key)
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****  KeyError: 'group_angle'
Sep 26 16:48:49 CLOWeatherStation weewx[5736] CRITICAL __main__:     ****  Exiting.

if I change the line to this:

weewx.units.obs_group_dict['signal1'] = 'group_rain'

and run weewx I do not get the error.

Is group_angle a recent addition and not in my dictionary?

[Karen K]

Craig Young schrieb am Dienstag, 26. September 2023 um 06:03:11 UTC+2:

I inserted this line into my driver:

 weewx.units.obs_group_dict['signal1'] = 'group_angle'

You cannot use group names that you invent on the fly. The group for observation types measured in degrees is group_direction. 

If you want to define a new unit group you have to look into weewx.units.std_groups. It references 3 dicts, and all of them need an entry for your unit group.

[gjr80]

On Tuesday, 26 September 2023 at 14:03:11 UTC+10 craig.y...@gmail.com wrote:

Is group_angle a recent addition and not in my dictionary?

Indeed it is, commit 2aba36a of 26 May refers. I was certain I was looking at master, I must have been looking at one of the V5 branches. No matter, there are a few things you can do. If developing your driver on v4.x you could define group_angle yourself and just make a note to remove the definition once v5 is released and you are using v5, or you could use another unit group in the meantime and again make a note to change in group_angle once v5 is released and you are using v5. Or you could cut your development over to one of the v5 beta releases, though if you are a new WeeWX user it might be easier to continue with v4.x for the time being.

Gary 

[Craig Young]

Since I am mostly in the development/test stage I will use another unit for now and switch once V5 is released.

Craig