Error with NOAA reports, aggregate type meanmax

[mihec]

Hi,

I'm having issues with NOAA report generation ever since I installed the station but never minded enough to fix it.

I use Weatherflow Air & Sky, weewx version is 3.9.2.

This is the error message in the /var/log/syslog:

Jun  5 18:52:37 raspberrypi weewx[451]: cheetahgenerator: Generate failed with exception '<class 'weewx.ViolatedPrecondition'>'
Jun  5 18:52:37 raspberrypi weewx[451]: cheetahgenerator: **** Ignoring template /etc/weewx/skins/neowx/NOAA/NOAA-YYYY.txt.tmpl
Jun  5 18:52:37 raspberrypi weewx[451]: cheetahgenerator: **** Reason: Invalid aggregation type 'meanmax'
Jun  5 18:52:37 raspberrypi weewx[451]: ****  Traceback (most recent call last):
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 332, in generate
Jun  5 18:52:37 raspberrypi weewx[451]: ****      fd.write(str(compiled_template))
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__
Jun  5 18:52:37 raspberrypi weewx[451]: ****      rc = getattr(self, mainMethName)()
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "_etc_weewx_skins_neowx_NOAA_NOAA_YYYY_txt_tmpl.py", line 741, in respond
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "_etc_weewx_skins_neowx_NOAA_NOAA_YYYY_txt_tmpl.py", line 315, in __errorCatcher28
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "<string>", line 1, in <module>
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/tags.py", line 329, in __getattr__
Jun  5 18:52:37 raspberrypi weewx[451]: ****      return self._do_query(aggregate_type)
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/tags.py", line 343, in _do_query
Jun  5 18:52:37 raspberrypi weewx[451]: ****      val=val, **self.option_dict)
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/wxmanager.py", line 60, in getAggregate
Jun  5 18:52:37 raspberrypi weewx[451]: ****      return weewx.manager.DaySummaryManager.getAggregate(self, timespan, obs_type, aggregateType, **option_dict)
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/manager.py", line 1268, in getAggregate
Jun  5 18:52:37 raspberrypi weewx[451]: ****      **option_dict)
Jun  5 18:52:37 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/manager.py", line 450, in getAggregate
Jun  5 18:52:37 raspberrypi weewx[451]: ****      raise weewx.ViolatedPrecondition("Invalid aggregation type '%s'" % aggregate_type)
Jun  5 18:52:37 raspberrypi weewx[451]: ****  ViolatedPrecondition: Invalid aggregation type 'meanmax'
Jun  5 18:52:37 raspberrypi weewx[451]: cheetahgenerator: Generate failed with exception '<class 'weedb.NoColumnError'>'
Jun  5 18:52:37 raspberrypi weewx[451]: cheetahgenerator: **** Ignoring template /etc/weewx/skins/neowx/NOAA/NOAA-YYYY-MM.txt.tmpl
Jun  5 18:52:38 raspberrypi weewx[451]: cheetahgenerator: **** Reason: no such column: wind
Jun  5 18:52:38 raspberrypi weewx[451]: ****  Traceback (most recent call last):
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 332, in generate
Jun  5 18:52:38 raspberrypi weewx[451]: ****      fd.write(str(compiled_template))
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__
Jun  5 18:52:38 raspberrypi weewx[451]: ****      rc = getattr(self, mainMethName)()
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "_etc_weewx_skins_neowx_NOAA_NOAA_YYYY_MM_txt_tmpl.py", line 496, in respond
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "_etc_weewx_skins_neowx_NOAA_NOAA_YYYY_MM_txt_tmpl.py", line 308, in __errorCatcher35
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "<string>", line 1, in <module>
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/tags.py", line 329, in __getattr__
Jun  5 18:52:38 raspberrypi weewx[451]: ****      return self._do_query(aggregate_type)
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/tags.py", line 343, in _do_query
Jun  5 18:52:38 raspberrypi weewx[451]: ****      val=val, **self.option_dict)
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/wxmanager.py", line 60, in getAggregate
Jun  5 18:52:38 raspberrypi weewx[451]: ****      return weewx.manager.DaySummaryManager.getAggregate(self, timespan, obs_type, aggregateType, **option_dict)
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/manager.py", line 1268, in getAggregate
Jun  5 18:52:38 raspberrypi weewx[451]: ****      **option_dict)
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/manager.py", line 459, in getAggregate
Jun  5 18:52:38 raspberrypi weewx[451]: ****      _row = self.getSql(select_stmt % interpolate_dict)
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weewx/manager.py", line 395, in getSql
Jun  5 18:52:38 raspberrypi weewx[451]: ****      _cursor.execute(sql, sqlargs)
Jun  5 18:52:38 raspberrypi weewx[451]: ****    File "/usr/share/weewx/weedb/sqlite.py", line 41, in guarded_fn
Jun  5 18:52:38 raspberrypi weewx[451]: ****      raise weedb.NoColumnError(e)
Jun  5 18:52:38 raspberrypi weewx[451]: ****  NoColumnError: no such column: wind

Is the problem related to the missing aggregation type or something else?

Thanks.

[gjr80]

Something appears to be forcing some of your aggregates that should be calculated on the daily summaries to be instead calculated on the archive. is there any chance you have some future dated records in your archive? Try the following to check:

1. open your database with sqlite3:

$ sqlite3 /var/lib/weewx/weewx.sdb

(if sqlite3 is not found you will need to install it using something like sudo apt install sqlite3 and then try again)

2. obtain the timestamp of the last record in the archive in human readable form:

sqlite> SELECT datetime(MAX(dateTime), 'unixepoch', 'localtime') FROM archive;

This should display something like:

2023-06-05 03:05:00

3. exit sqlite3:

sqlite> .q

If the date at step 2 above is in the future you have some future dated records in your archive. To remove these:

1. stop WeeWX

$ sudo systemctl stop weewx

2. make a backup of your WeeWX database:

$ cp /var/lib/weewx/weewx.sdb /var/lib/weewx/weewx_backup.sdb

3. open your database with sqlite3:

$ sqlite3 /var/lib/weewx/weewx.sdb

4. delete all future dated archive records:

sqlite> DELETE FROM archive WHERE dateTime>STRFTIME('%s');

5. check there are no more future dated records:

sqlite> SELECT datetime(MAX(dateTime), 'unixepoch', 'localtime') FROM archive;

6. if there are no more future dated records exit sqlite3:

sqlite> .q

7. rebuild the daily summaries:

$ wee_database --rebuild-daily

8. restart WeeWX:

$ sudo systemctl restart WeeWX

Depending on your system some of the above linux commands may need to be prefixed with sudo.

Gary

[mihec]

Hi Gary,

thanks for you quick response!

I checked for the latest date in my database and it seems I have no such records:

pi@raspberrypi:~/database $ sqlite3 weewx.sdb
SQLite version 3.16.2 2017-01-06 16:32:41
Enter ".help" for usage hints.

sqlite> SELECT datetime(MAX(dateTime), 'unixepoch', 'localtime') FROM archive;

2023-06-05 20:43:00
sqlite> .q

I tried to do a rebuild-daily:

pi@raspberrypi:~/database $  wee_database --rebuild-daily

Traceback (most recent call last):

  File "/usr/bin/wee_database", line 18, in <module>
    import user.extensions      #@UnusedImport
  File "/usr/share/weewx/user/extensions.py", line 18, in <module>
    locale.setlocale(locale.LC_ALL, '')
  File "/usr/lib/python2.7/locale.py", line 581, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

Something in my configuration doesn't seem ok... Probably not related to original post.

Thanks for any further hint.

ponedeljek, 5. junij 2023 ob 20:02:53 UTC+2 je oseba gjr80 napisala:

[gjr80]

I am away from home so cannot help with the locale issue, Tom or others may have some insights. It does not appear to be related to the meanmax/wind issue.

I still think that something is causing WeeWX to see future dated records, or more precisely, the current record during a report cycle is not being seen as the most recent record. If 20:43 was the time of your most recent archive record I am guessing you are using a very short archive period, say one minute? From the short log extract it appears that WeeWX is handling this OK, but it would be better to see a longer log extract, preferably following a WeeWX restart so we can see the full WeeWX startup. Could you restart WeeWX and then post a log extract showing the full WeeWX startup and then say, at least 10 or so archive intervals.

Gary

[mihec]

I agree with you, maybe there was some date misalignment after fresh install. I've attached the syslog just after the weewx start.

I do have 1-minute archiving interval.

Is it possible to simply remove first few entries in the database perhaps? I don't mind losing first few hours.

Thanks.

torek, 6. junij 2023 ob 13:04:00 UTC+2 je oseba gjr80 napisala:

[gjr80]

It is possible the error is to do with the first records in your archive rather than the latest, though this would only likely be the case if your NOAA format files have never been generated (otherwise the 2023 NOAA year would currently be being generated, and since you are using WeeWX v3.9.2 I assume you have data from 2022 or earlier). Let's hold of making changes to your data just yet.

Could you stop WeeWX, edit weewx.conf, set debug = 1 then save weewx.conf. Then delete all of the neowx generated NOAA format files (*.txt) on your WeeWX system, they should be somewhere under /var/www/html/weewx, possibly under neowx/NOAA (this depends on the neowx skin). This will force WeeWX to regenerate all of the neowx NOAA format files. Finally, restart WeeWX and take a log extract showing the full WeeWX startup and the first few report generation cycles - capturing the full WeeWX startup is important, it contains a lot of WeeWX config info. This wiki page has tips on how to get a good (and complete) log extract if you are having problems.

Gary