Error Message when generating Archive

[joh....@googlemail.com]

Hi,

After two tries with cheaper Weather Stations which failed due to Quality I bought now a Davis Vantage Pro2 Wireless with a Weatherlink Live Adapter

I have setup now a new Weewx Installation and had it running for some days with "simulator" without any issues.

Now I got my davis and I have configured it immediatly. 

But unfortunatley I am getting now the following error:

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y.txt.tmpl

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'month.wind.count.raw'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y_txt_tmpl.py", line 862, in respond

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'count' while searching for 'month.wind.count.raw'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/rss.xml.tmpl

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'max' while searching for 'day.wind.max'

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_rss_xml_tmpl.py", line 268, in respond

Sep 17 19:55:17 weewx-test weewx[23751] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'max' while searching for 'day.wind.max'

Sep 17 19:55:17 weewx-test weewx[23751] INFO weewx.cheetahgenerator: Generated 5 files for report SeasonsReport in 0.66 seconds

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine: Caught unrecoverable exception in generator 'weewx.imagegenerator.ImageGenerator'

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  no such table: archive_day_wind

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weedb/sqlite.py", line 29, in guarded_fn

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      return fn(*args, **kwargs)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weedb/sqlite.py", line 211, in execute

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      return sqlite3.Cursor.execute(self, *args, **kwargs)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  sqlite3.OperationalError: no such table: archive_day_wind

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  During handling of the above exception, another exception occurred:

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 197, in run

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      obj.start()

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 280, in start

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      self.run()

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/imagegenerator.py", line 41, in run

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      self.genImages(self.gen_ts)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/imagegenerator.py", line 180, in genImages

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      aggregate_interval=aggregate_interval)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/xtypes.py", line 86, in get_series

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      aggregate_interval)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/xtypes.py", line 628, in get_series

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      aggregate_interval)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/xtypes.py", line 140, in get_series

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      agg_vt = get_aggregate(obs_type, stamp, do_aggregate, db_manager)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/xtypes.py", line 102, in get_aggregate

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      **option_dict)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/xtypes.py", line 796, in get_aggregate

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      row = db_manager.getSql(sql, timespan)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/manager.py", line 426, in getSql

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      _cursor.execute(sql, sqlargs)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weedb/sqlite.py", line 37, in guarded_fn

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****      raise weedb.NoTableError(e)

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  weedb.NoTableError: no such table: archive_day_wind

Sep 17 19:55:18 weewx-test weewx[23751] ERROR weewx.reportengine:         ****  Generator terminated

Any ideas?

[Tom Keffer]

The reason why this can occur is a rather technical one, but has to do with mixing the older, V3.x, way of specifying a schema, and the newer, V4.x way.

1. Stop weewx

2. Assuming there is nothing valuable in it, delete the archive /var/lib/weewx/weewx.sdb

3. Make sure your [DataBindings] section looks like this. Pay attention to the highlighted areas.

[DataBindings]

    [[wx_binding]]
        # The database must match one of the sections in [Databases].
        # This is likely to be the only option you would want to change.
        database = archive_sqlite
        # The name of the table within the database
        table_name = archive
        # The manager handles aggregation of data for historical summaries
        manager = weewx.manager.DaySummaryManager
        # The schema defines the structure of the database.
        # It is *only* used when the database is created.
        schema = schemas.wview_extended.schema

4. Restart weewx.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8cddf9c5-ac26-4bbf-bc3b-e21d0bc5e513n%40googlegroups.com.

[joh....@googlemail.com]

Many thanks, that was the issue!!!

[Alessandro Del Prete]

Hi,

I have the same issue, it's a brand new installation of weewx 4.1.1, working since a month with a belchertown skin and a weatherflow tempest.

Today I was checking weewx.log and I noticed this:

Oct 27 21:39:24 ubuntu weewx[59297] DEBUG weewx.manager: Daily summary version is 2.0
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Traceback (most recent call last):
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** unicode_string = compiled_template.respond()
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Traceback (most recent call last):
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** unicode_string = compiled_template.respond()
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'

Then I found your reply to solve the problem. I checked the DataBindings section, and everything's like you suggested, no mixing (it was a new installation), so I don't know why the issue came up only after some time.

Is it possible to solve the problem without deleting the database? 

Thanks for any advice on this.

Alessandro

[vince]

On Tuesday, October 27, 2020 at 2:20:33 PM UTC-7, Alessandro Del Prete wrote:

I have the same issue, it's a brand new installation of weewx 4.1.1, working since a month with a belchertown skin and a weatherflow tempest.

Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'
Oct 27 21:39:24 ubuntu weewx[59297] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl

If you're running the Belchertown skin, why are you 'also' still running the default Seasons skin ?

I'd just disable Seasons in weewx.conf and restart weewx if you don't need it. 

[Alessandro Del Prete]

Because I used it as the "default" skin for now, I'm still testing things, and it was useful to double check things.

I can disable it, but this does not solve the problem, it bypasses it. I still don't know what caused the problem, maybe there's a bad configuration and that was only a symptom, that's why I asked.

Thanks for the suggestion, if there's no solution I'll disable that skin.

[Tom Keffer]

Can you take a look inside your database? What is the result of:

sqlite3 /var/lib/weewx/weewx.sdb

sqlite> SELECT tbl_name FROM sqlite_master WHERE type='table';

sqlite> .quit

This will tell us whether the table archive_day_wind exists in your database. 

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9653844c-9165-4cac-a075-e1117984736en%40googlegroups.com.

[Alessandro Del Prete]

The table exists. Could it be corrupted in some way? That would explain why the problem came up only recently.

SQLite version 3.31.1 2020-01-27 19:55:54

Enter ".help" for usage hints.

sqlite> SELECT tbl_name FROM sqlite_master WHERE type='table';

archive

archive_day_altimeter

archive_day_appTemp

archive_day_appTemp1

archive_day_barometer

archive_day_batteryStatus1

archive_day_batteryStatus2

archive_day_batteryStatus3

archive_day_batteryStatus4

archive_day_batteryStatus5

archive_day_batteryStatus6

archive_day_batteryStatus7

archive_day_batteryStatus8

archive_day_cloudbase

archive_day_co

archive_day_co2

archive_day_consBatteryVoltage

archive_day_dewpoint

archive_day_dewpoint1

archive_day_ET

archive_day_extraHumid1

archive_day_extraHumid2

archive_day_extraHumid3

archive_day_extraHumid4

archive_day_extraHumid5

archive_day_extraHumid6

archive_day_extraHumid7

archive_day_extraHumid8

archive_day_extraTemp1

archive_day_extraTemp2

archive_day_extraTemp3

archive_day_extraTemp4

archive_day_extraTemp5

archive_day_extraTemp6

archive_day_extraTemp7

archive_day_extraTemp8

archive_day_forecast

archive_day_hail

archive_day_hailBatteryStatus

archive_day_hailRate

archive_day_heatindex

archive_day_heatindex1

archive_day_heatingTemp

archive_day_heatingVoltage

archive_day_humidex

archive_day_humidex1

archive_day_inDewpoint

archive_day_inHumidity

archive_day_inTemp

archive_day_inTempBatteryStatus

archive_day_leafTemp1

archive_day_leafTemp2

archive_day_leafWet1

archive_day_leafWet2

archive_day_lightning_distance

archive_day_lightning_disturber_count

archive_day_lightning_energy

archive_day_lightning_noise_count

archive_day_lightning_strike_count

archive_day_luminosity

archive_day_maxSolarRad

archive_day_nh3

archive_day_no2

archive_day_noise

archive_day_o3

archive_day_outHumidity

archive_day_outTemp

archive_day_outTempBatteryStatus

archive_day_pb

archive_day_pm10_0

archive_day_pm1_0

archive_day_pm2_5

archive_day_pressure

archive_day_radiation

archive_day_rain

archive_day_rainBatteryStatus

archive_day_rainRate

archive_day_referenceVoltage

archive_day_rxCheckPercent

archive_day_signal1

archive_day_signal2

archive_day_signal3

archive_day_signal4

archive_day_signal5

archive_day_signal6

archive_day_signal7

archive_day_signal8

archive_day_snow

archive_day_snowBatteryStatus

archive_day_snowDepth

archive_day_snowMoisture

archive_day_snowRate

archive_day_so2

archive_day_soilMoist1

archive_day_soilMoist2

archive_day_soilMoist3

archive_day_soilMoist4

archive_day_soilTemp1

archive_day_soilTemp2

archive_day_soilTemp3

archive_day_soilTemp4

archive_day_supplyVoltage

archive_day_txBatteryStatus

archive_day_UV

archive_day_uvBatteryStatus

archive_day_windBatteryStatus

archive_day_windchill

archive_day_windDir

archive_day_windGust

archive_day_windGustDir

archive_day_windrun

archive_day_windSpeed

archive_day_wind

archive_day__metadata

sqlite>

[Tom Keffer]

This error could happen if a query was made for aggregation periods that do not run midnight-to-midnight, but I don't know why the Belchertown skin would do that.

One thing to try: rebuilding the daily summaries.

wee_database --drop-daily

wee_database --rebuild-daily

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c8a13b8e-c377-47dd-9477-21372e4b0261n%40googlegroups.com.

[Alessandro Del Prete]

Done. And restarted weewx.

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond

Oct 28 15:32:21 ubuntu weewx[611964] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'

[Tom Keffer]

The only other thing I can think of is that the Belchertown skin is making a query on something other than midnight-to-midnight.

We can check that. Please replace your copy of xtypes.py (you should find it in /usr/share/weewx/weewx/xtypes.py) with the attached. It has been instrumented to log the aggregation interval. 

Run with debug=1 from startup through the first reporting cycle. Post the full log. It could be rather voluminous, so you may want to post the log as an attachment, or upload to a service such as pastebin.com.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a80c017d-070d-4c90-b71b-8273d74cb5efn%40googlegroups.com.

[Alessandro Del Prete]

The full log is here: https://pastebin.com/f62DRnHL

Luckily the reporting cycle started almost immediately, I noticed this " Daily summary cannot be used." error :

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.reportengine: Found configuration file /etc/weewx/skins/Seasons/skin.conf for report 'SeasonsReport'

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']

Oct 28 18:17:27 ubuntu weewx[694317] INFO weewx.restx: Hubitat: Published record 2020-10-28 18:17:17 CET (1603905437)

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.manager: Daily summary version is 2.0

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: obs_type=wind; aggregate_type=count

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: timespan=[1972-10-01 00:00:00 CET (86742000) -> 1972-10-02 00:00:00 CET (86828400)]

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: first_timestamp=1970-01-01 01:01:00 CET (60); last_timestamp=2020-10-28 18:17:00 CET (1603905420)

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: Daily summary cannot be used.

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: obs_type=wind; aggregate_type=count

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: timespan=[1978-10-01 00:00:00 CET (276044400) -> 1978-10-02 00:00:00 CET (276130800)]

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: first_timestamp=1970-01-01 01:01:00 CET (60); last_timestamp=2020-10-28 18:17:00 CET (1603905420)

Oct 28 18:17:27 ubuntu weewx[694317] DEBUG weewx.xtypes: Daily summary cannot be used.

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Seasons/NOAA/NOAA-%Y-%m.txt.tmpl

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Seasons_NOAA_NOAA__Y__m_txt_tmpl.py", line 431, in respond

Oct 28 18:17:27 ubuntu weewx[694317] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'count' while searching for 'day.wind.count.raw'

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: obs_type=wind; aggregate_type=count

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: timespan=[2020-10-01 00:00:00 CEST (1601503200) -> 2020-10-02 00:00:00 CEST (1601589600)]

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: first_timestamp=1970-01-01 01:01:00 CET (60); last_timestamp=2020-10-28 18:17:00 CET (1603905420)

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: Daily summary will be used.

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: obs_type=wind; aggregate_type=count

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: timespan=[2020-10-02 00:00:00 CEST (1601589600) -> 2020-10-03 00:00:00 CEST (1601676000)]

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: first_timestamp=1970-01-01 01:01:00 CET (60); last_timestamp=2020-10-28 18:17:00 CET (1603905420)

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: Daily summary will be used.

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: obs_type=wind; aggregate_type=count

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: timespan=[2020-10-03 00:00:00 CEST (1601676000) -> 2020-10-04 00:00:00 CEST (1601762400)]

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: first_timestamp=1970-01-01 01:01:00 CET (60); last_timestamp=2020-10-28 18:17:00 CET (1603905420)

Oct 28 18:17:28 ubuntu weewx[694317] DEBUG weewx.xtypes: Daily summary will be used.

[Tom Keffer]

It looks like the Belchertown skin is asking for aggregations over dates back in 1978. Seeing as how WeeWX wasn't even invented until 2008, you probably have some bad data in there, which is likely causing the problem.

What is the earliest timestamp you expect in your database? For purposes of illustration, let's assume it is 12-May 2018. Use a tool like the website epochconverter.com to figure out what this is in Unix epoch time. The answer for the example is 1589241600.

Now use the sqlite3 tool to see what records predate that. 

sqlite3 /var/lib/weewx/weewx.sdb

sqlite> select datetime(dateTime, 'unixepoch', 'localtime'), dateTime from archive where dateTime < 1589241600;

If it's all garbage, then it's safe to delete these data:

sqlite> delete from archive where dateTime <1589241600;

Then drop and rebuild the daily summaries, just like you did before.

Make a backup before you start.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0c24713c-71a7-4d9d-986d-7bee1272c9e9n%40googlegroups.com.

[Alessandro Del Prete]

Tom, thanks a lot for the reply, I will try this and report back.

Just one question: in the log I see the problem is on the Seasons skin, not Belchertown. Am I missing something?

[Alessandro Del Prete]

The first timestamp is Oct. 12 2020, here are the top rows of the query (the list is huge).

sqlite> select datetime(dateTime, 'unixepoch', 'localtime'), dateTime from archive where dateTime < 1602536552;

1970-01-01 01:01:00|60

2020-10-12 16:05:00|1602511500

2020-10-12 16:06:00|1602511560

2020-10-12 16:07:00|1602511620

2020-10-12 16:08:00|1602511680

2020-10-12 16:09:00|1602511740

2020-10-12 16:10:00|1602511800

2020-10-12 16:11:00|1602511860

2020-10-12 16:12:00|1602511920

2020-10-12 16:13:00|1602511980

2020-10-12 16:14:00|1602512040

2020-10-12 16:15:00|1602512100

2020-10-12 16:16:00|1602512160

2020-10-12 16:17:00|1602512220

2020-10-12 16:18:00|1602512280

So I deleted that record in bold, dropped and rebuilt the daily summaries, and restarted weewx: monitored the log for a few minutes, no errors found.

Now the question is: where did that record come from? :)

Thanks for all the help Tom, greatly appreciated.

Alessandro

[Tom Keffer]

It's a very common problem with computers that do not have a real-time clock. The computer reboots, and the clock resets to epoch time zero, which is 1-jan-1970, just like you saw.

Do you know whether your computer has an RTC? If not, get one.

-tl

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fbf5e636-f72e-4e7b-9821-25e409e05bd3n%40googlegroups.com.

[Alessandro Del Prete]

It's an Intel NUC7i3BNK. It has RTC, and I don't have errors on reboot, during the POST, regarding the battery.

[Tom Keffer]

As I look through the driver code for the Tempest, it looks like it gets the timestamp from the hardware. So, there must be an issue with the clock of the Tempest.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5ae293a3-7aee-464a-9da1-7eb033fc4648n%40googlegroups.com.

[Alessandro Del Prete]

Can I configure weewx to always use its timestamp instead of the PWS hw? Or getting it from the pws is better?

Is there any way I can track this event to make sure it's the Tempest clock? I checked the Tempest right now and I have no errors, the clock is ok.

[Tom Keffer]

Yes, you could change the driver to use the PC's timestamp, instead of the Tempest's, but, let me warn you, the Tempest has a bad habit of delaying packets for very long periods of time. See this exchange I had with their support staff.

It's what ultimately led me to return my Tempest. Its API needs some work.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/cad676bf-9277-4eca-a933-54f4503ce3a5n%40googlegroups.com.

[Alessandro Del Prete]

I'll read the exchange, thanks. I'm very interested in this problem, I have an ongoing conversation with support because the lightning sensor is not sending data via UDP, I just receive events on the app, but via UDP I received no evt_strike events. I know there's a specific discussion about this lightining issue and the UDP raw data vs enriched API data, and I hope they understand that locally, via UDP, we want to see ALL sensors' data, it's not acceptable that they decide what I can read from the system and what not.

When you refer to the delays you mean also with UDP broadcast data? Because I don't see these delays, and I monitor it quite often. I'm using this driver: WeeWX UDP driver for WeatherFlow station - Owners / Third-Party Integrations - Tempest Weather System

PS: I asked WF support also about the timestamp issue, if there's a possibility that the Tempest sends epoch time zero on reboot or in any other case.

[Alessandro Del Prete]

Tom, that was a very interesting discussion with WF. I hope they will follow your advices about the random backoff/frequency hopping.

Do you still have that small program you developed to monitor the packets? It would be really useful, I'd like to use it to monitor how it goes here.

Thanks again.

[Tom Keffer]

I looked around and can't seem to find it. Sorry.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/13a7cc6b-c287-451f-92b6-258cb3f4d8fbn%40googlegroups.com.

[Alessandro Del Prete]

No problem, thanks anyway. Looked really useful for debugging.

[vince]

On Wednesday, October 28, 2020 at 4:11:45 PM UTC-7, Alessandro Del Prete wrote:

I'll read the exchange, thanks. I'm very interested in this problem, I have an ongoing conversation with support because the lightning sensor is not sending data via UDP, I just receive events on the app, but via UDP I received no evt_strike events. I know there's a specific discussion about this lightining issue and the UDP raw data vs enriched API data, and I hope they understand that locally, via UDP, we want to see ALL sensors' data, it's not acceptable that they decide what I can read from the system and what not.

You already lost that argument.

[...disclaimer - past WF user for 24 months including being a field tester of Sky, Solar panel, and Tempest...]

WF is moving in the opposite direction of what you want, toward 'increasing' need for continuous internet connectivity to the WF servers, and toward making the UDP messages notionally useless.  There have been many complaints from the original testers in the forums (including me).   Some have voted with their feet and gotten rid of all their WF gear (including me).  My opinion is you can't fix bad hardware with software.

My guess - WF is very much in the weather forecasting business, and is focussing their efforts there, more so than somebody like Davis who is in the 'instrumentation' business.

Re: sensors and UDP....

The rain data is 'fixed up' via RainCheck before noon the next day local time, available only on their servers via the ReST interface.   How they determine how to (badly, for me) 'fix' your rain data is not released to the consumer.  A lot of people just disable RainCheck for their stations to avoid confusion.  My opinion is the haptic sensor is fatally flawed and they'll come out with a cheap tipping bucket eventually, if EcoWitt doesn't undercut their price to the point where it's not a good idea to even compete there.

The lightning data is now totally synthetic and crowd-sourced, presented on their app only via an algorithm that is not released.  Your sensed lightning readings make up 'part' of what they report as lightning for your station, but the UDP info says only what your sensor saw, which frequently is nothing even though you're ducking for cover due to lightning nearby.   The lightning sensor is the weakest part of the system, it's very shaky.

What they include from where in 'fixing' up your sensor readings is not released by WF.  There is no way to opt out of your data being altered short of blocking your Hub from connecting to Internet, but at that point (a) you can't get continuous learning tuning adjustments, (b) you can't get firmware updates, and (c) it is unclear if a previously-tuned WF station would even work without connectivity to the WF servers.

 

PS: I asked WF support also about the timestamp issue, if there's a possibility that the Tempest sends epoch time zero on reboot or in any other case.

I don't believe it resets the clock on reboot, mine rebooted all the time and I never saw odd data in weewx or in the UDP.

I've always wondered what the Hub does on boot up, other than establishing essentially a MQTT session to the WF servers.  There has to be something related to setting a system clock in there somewhere under the hood, but it's never been documented that I recall.

Definitely worth a test.  Block your hub from all Internet access.  Power it down.  Power it back up.  What does it do ?  Does it block startup if it can't phone home for network time ?  What are the timestamps in the UDP broadcasts ?

You can also tell if your system has rebooted by looking at uptime in the UDP broadcasts.  See the API docs for where.

Re: the UDP monitor program, if you run my UDP listener with the --raw --decoded options you will see a reasonably nicely formatted output of what the Hub is emitting.   You 'can' run this on the same box as weewx and the weatherflow UDP driver, but be sure to set 'shared_socket = true' in weewx.conf so that the driver coexists with other things also listening for udp/50222 on the same box.

My listener is at https://github.com/vinceskahan/weatherflow-udp-listener

Feel free to email offline if you need help with my listener...

 

[Alessandro Del Prete]

You already lost that argument.

Don Quijote is my hero since I was a little kid. I'm used to it. :)

 

 

WF is moving in the opposite direction of what you want, toward 'increasing' need for continuous internet connectivity to the WF servers, and toward making the UDP messages notionally useless.  There have been many complaints from the original testers in the forums (including me).   Some have voted with their feet and gotten rid of all their WF gear (including me).  My opinion is you can't fix bad hardware with software.

My guess - WF is very much in the weather forecasting business, and is focussing their efforts there, more so than somebody like Davis who is in the 'instrumentation' business.

It makes sense from a certain point of view, but the two things are not in contrast: you can gather all data you want, but users want to receive the raw data from the sensors, knowing it is not "enriched" (and that's a plus for some). 

Re: sensors and UDP....

The rain data is 'fixed up' via RainCheck before noon the next day local time, available only on their servers via the ReST interface.   How they determine how to (badly, for me) 'fix' your rain data is not released to the consumer.  A lot of people just disable RainCheck for their stations to avoid confusion.  My opinion is the haptic sensor is fatally flawed and they'll come out with a cheap tipping bucket eventually, if EcoWitt doesn't undercut their price to the point where it's not a good idea to even compete there.

In the thread I replied to on their forum, DSJ explicitly said that he was finally convinced of having the same Raincheck enable/disable feature also for Lightning: https://community.weatherflow.com/t/lightning-reports-tempest/6262/133

We'll se if they keep the promise. I replied at the end of that thread.

The lightning data is now totally synthetic and crowd-sourced, presented on their app only via an algorithm that is not released.  Your sensed lightning readings make up 'part' of what they report as lightning for your station, but the UDP info says only what your sensor saw, which frequently is nothing even though you're ducking for cover due to lightning nearby.   The lightning sensor is the weakest part of the system, it's very shaky.

Vince, I NEVER received a single evt_strike through UDP, not even during a big storm that lasted for hours during the night, I was monitoring debug log to finally wait for at least 1 event but nothing. Meanwhile I was receiving a lot of lightning notifications on the android app. They say I should see raw data of the sensors through UDP, but that was not the case, I guess I have a faulty station/sensor.

What they include from where in 'fixing' up your sensor readings is not released by WF.  There is no way to opt out of your data being altered short of blocking your Hub from connecting to Internet, but at that point (a) you can't get continuous learning tuning adjustments, (b) you can't get firmware updates, and (c) it is unclear if a previously-tuned WF station would even work without connectivity to the WF servers.

Well, if all raw data of the sensors is sent via UDP, you don't have altered data, but the one coming from the sensors. And that's what I want. To compare it with my other stations (ecowitt and davis).

 

I don't believe it resets the clock on reboot, mine rebooted all the time and I never saw odd data in weewx or in the UDP.

I've always wondered what the Hub does on boot up, other than establishing essentially a MQTT session to the WF servers.  There has to be something related to setting a system clock in there somewhere under the hood, but it's never been documented that I recall.

Definitely worth a test.  Block your hub from all Internet access.  Power it down.  Power it back up.  What does it do ?  Does it block startup if it can't phone home for network time ?  What are the timestamps in the UDP broadcasts ?

You can also tell if your system has rebooted by looking at uptime in the UDP broadcasts.  See the API docs for where.

Thanks for the suggestions, if it happens again I'll dig into it.

 

Re: the UDP monitor program, if you run my UDP listener with the --raw --decoded options you will see a reasonably nicely formatted output of what the Hub is emitting.   You 'can' run this on the same box as weewx and the weatherflow UDP driver, but be sure to set 'shared_socket = true' in weewx.conf so that the driver coexists with other things also listening for udp/50222 on the same box.

My listener is at https://github.com/vinceskahan/weatherflow-udp-listener

Feel free to email offline if you need help with my listener...

I had already bookmarked your listener, in case I needed to debug things. Thank you very much for the support, I'll contact you if needed. :)

Alessandro