Trouble viewing new data column(s)

[James Runser]

I'm trying to integrate a home-built tide and temperature monitor into weewx (configured and working without issue with an Acurite Atlas). I'm using a MySQL DB and tried simply inserting my data (waterTemp, tideHeight) into archive. I use current timestamp, usUnits and 10 for the required interval values and let the remaining columns default to null ( except - duh - waterTemp and tideHeight). 

The data are written without issue but I'm never seeing the results in the Seasons report - I chose waterTemp as the learning value since the skin.conf already had a reference to water temp.

My thinking is that my data is not returned by the report engine - based, perhaps - on timestamp? Is there a better way to hook net new data into the flow? I do in fact see the waterTemp tag in the current conditions box that was enabled via current.inc. - the issue is that the value is always N/A. I haven't dug deeply enough to figure out where the actual native Weewx insert is done but am wary of the effort required to piggy-back on that process.

Advice or suggestions appreciated but a solid answer would not be kicked out of bed for eating crackers ;)

 

[James Runser]

Incremental progress. As described above but with the addition of a restart I have data in the statistics section but still N/A for Water Temp in "Current Conditions". Further, the data in statistics is formatted as "00.000000" instead of the hoped-for "00.00  °F". I've walked through the template and inc files but still can't find the right building blocks. I even tried changing the waterTemp column from a double to a double(4,2) but am still getting all the extra zeros.

[gjr80]

If you add a new observation to the database you need to tell WeeWX what unit group it belongs to in order to be able to effectively use the formatting and unit conversion capabilities of the WeeWX tag system in reports. For example, if you add a field that is a pressure once WeeWX knows the new field is a pressure WeeWX knows what default formatting to apply, the correct unit label and you can use the formatting and unit conversion aspects of the tag system to change the format/convert units in a WeeWX report template. If WeeWX does not know what unit group the observation belongs to you just get the raw value from the database (which is typically a float with many decimal places - ie what you are seeing now).

There are a number of ways to assign an observation to a unit group. In your case the easiest approach is to add a few lines of code to /home/weewx/bin/user/extensions.py (or /usr/share/weewx/user/extensions.py if you installed WeeWX as a package). Try adding the following to the bottom of extensions.py:

import weewx.units

weewx.units.obs_group_dict['waterTemp'] = 'group_temperature'

weewx.units.obs_group_dict['tideHeight'] = 'group_length'

(group_temperature was the obvious choice for waterTemp; tideHeight could be group_length, group_altitude, group_rain or group_distance. The deciding factor here is the available units for each group - if you look at the Units appendix to the Customisation Guide you will see what I mean)

You will need to restart WeeWX for the changes to take effect. The extensions.py code is run at WeeWX startup and the above lines will make the appropriate unit group assignments for your new fields. You should now be able to use tags based on waterTemp and tideHeight in your reports. You will find some further info on assigning unit groups in the Customising units and unit groups section of the Customisation Guide. You will also find information on the WeeWX tag system in the Tags section of the Customisation Guide.

A couple of notes about units.

For the WeeWX tag system to correctly and consistently display observation values and units you need to ensure the data you are adding to the database is in the correct units. How you do this depends on how you are inserting data into the database. If you are using a WeeWX service to augment loop packets (the preferred approach) then your service needs to take cognisance of the unit system (ie WeeWX field usUnits) of the loop packet being augmented and add waterTemp and tideHeight to the loop packet using the appropriate units for the unit group each observation belongs to for the unit system used by the loop packet. For example, if the loop packet uses US customary units (ie usUnits = 0) then waterTemp would need to be in Fahrenheit and tideHeight in inches. Once you do this WeeWX takes care of everything else. You will find information on the unit systems, unit groups and units used by WeeWX in the Units appendix to the Customisation Guide.

If you are directly inserting data in the WeeWX database (not the preferred approach) then it is up to you to ensure that the values are inserted using units applicable to the database unit system and the unit group used for each observation.

Apologies for the long response but the WeeWX tag/unit/formatting system has a lot of moving parts and you need to have them all correctly configured or you will have problems.

Gary

[Ian Millard]

Gary,

Follow up question from me in a similar vein. What is the correct way to add a completely new measurement group to WeeWX, for example part_per_million_per_cubic_meter?

Ian

Sent from my iPad.

On 17 Jan 2023, at 05:20, gjr80 <gjrod...@gmail.com> wrote:

If you add a new observation to the database you need to tell WeeWX what unit group it belongs to in order to be able to effectively use the formatting and unit conversion capabilities of the WeeWX tag system in reports. For example, if you add a field that is a pressure once WeeWX knows the new field is a pressure WeeWX knows what default formatting to apply, the correct unit label and you can use the formatting and unit conversion aspects of the tag system to change the format/convert units in a WeeWX report template. If WeeWX does not know what unit group the observation belongs to you just get the raw value from the database (which is typically a float with many decimal places - ie what you are seeing now).

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f7048253-f8ff-4c85-a838-369dcd22db25n%40googlegroups.com.

[gjr80]

Ian,

When you say measurement group I presume you mean unit - part_per_million_per_cubic_meter is a unit whereas what you call a 'measurement group' sounds to me to be more like a unit group, eg group_temperature or group_pressure which are groups that contain details of all of the units used for temperature and pressure respectively. Not trying to be pedantic, but it's important to know what part of the WeeWX unit system we are dealing with - there are lots of moving parts. I'll assume your question is about adding a new unit.

Same basic process but instead of manipulating weewx.units.obs_group_dict you might need to manipulate weewx.units.USUnits, weewx.units.MetricUnits and weewx.units.MetricWXUnits unit systems if your new unit will be the default unit used for one or more of the three unit systems. If it is not to be a default but rather another unit in an existing unit group then you would need to add it to the weewx.units.conversionDict so that WeeWX knows how to convert to and from your new unit in it's unit group. If your new unit is not a default unit in a unit system or is not a unit to be converted to/from then you essentially won't be using your new unit. You may also need to define default unit formatting and unit labels for your new unit in units.default_unit_format_dict and units.default_unit_label_dict respectively.

Ignoring the existing WeeWX group_concentration, let's say you want a new unit group, 'group_new_concentration' and it will be the only unit in that unit group (ie no unit conversions required). Default unit formatting will be 1 decimal place and the default unit label will be 'ppm/m3'. So putting that altogether you might end up with something like:

import weewx.units

# set the unit system units

weewx.units.USUnits['group_new_concentration'] = 'part_per_million_per_cubic_meter'

weewx.units.MetricUnits['group_new_concentration'] = 'part_per_million_per_cubic_meter'

weewx.units.MetricWXUnits['group_new_concentration'] = 'part_per_million_per_cubic_meter'

# define default formatting

weewx.units.default_unit_format_dict['part_per_million_per_cubic_meter'] = '%.1f'

# define default unit label

weewx.units.default_unit_label_dict['part_per_million_per_cubic_meter'] = ' ppm/m3'

Let's say you want to add part_per_million_per_cubic_meter to the existing group_concentration unit group. part_per_million_per_cubic_meter will not be the default group_concentration unit in any of the unit systems. Same default formatting and unit label. For the sake of argument lets say to convert from microgram_per_meter_cubed to part_per_million_per_cubic_meter you divide by 10, ie 10 microgram_per_meter_cubed = 1 part_per_million_per_cubic_meter (this is a nonsense of course, it is just to illustrate a point). In this case you might use:

 import weewx.units

# define conversion functions for group_concentration

weewx.units.conversionDict['microgram_per_meter_cubed'] = {'part_per_million_per_cubic_meter': lambda x: x / 10.0}

weewx.units.conversionDict['part_per_million_per_cubic_meter'] = {'microgram_per_meter_cubed': lambda x: x * 10.0}

# define default formatting

weewx.units.default_unit_format_dict['part_per_million_per_cubic_meter'] = '%.1f'

# define default unit label

weewx.units.default_unit_label_dict['part_per_million_per_cubic_meter'] = ' ppm/m3'

If you wanted to set part_per_million_per_meter_cubed as the the default US customary unit then you would add in the following line:

weewx.units.USUnits['group_concentration'] = 'part_per_million_per_cubic_meter'

Finally, where to put the code. Of course you could put it in extensions.py as per my previous post. If you were using this as part of a driver or service you have written (and that is the only place it will be used, ie someone who does not have your driver will unlikely use these settings) then you can put it in your driver or service file with zero indent, that way the code will be read and executed when your driver or service is loaded. My preference is the latter if it is applicable.

Gary

[Cameron D]

I hope this is not a real example you are suggesting -  part_per_million  would be a concentration, which already has an implied volume component according to whether it is in air, water etc.   A unit of   part_per_million_per_cubic_meter does not make any sense to me. </pedantic>

[Steeple Ian]

Gary,

Thank you for your very extensive and detailed reply which confirmed my thoughts that it is not a simple process. However it has given a greater understanding of what is required.

Ian