Sync'd with development now getting database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime

[Leon Shaner]

Hey, WeeWX'ers.

I just sync'd my local copy of the development branch.
Now I'm getting these errors. How to fix?


Nov 16 13:51:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:51:00 EST (1573930260) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:52:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:52:00 EST (1573930320) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:53:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:53:00 EST (1573930380) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:54:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:54:00 EST (1573930440) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:55:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:55:00 EST (1573930500) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:56:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:56:00 EST (1573930560) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:57:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:57:00 EST (1573930620) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:58:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:58:00 EST (1573930680) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 13:59:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:59:00 EST (1573930740) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:00:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:00:00 EST (1573930800) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:01:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:01:00 EST (1573930860) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:02:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:02:00 EST (1573930920) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:03:15 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:03:00 EST (1573930980) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:04:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:04:00 EST (1573931040) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:05:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:05:00 EST (1573931100) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Nov 16 14:06:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:06:00 EST (1573931160) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime



Regards,
\Leon
--
Leon Shaner :: Dearborn, Michigan (iPad)

[Thomas Keffer]

Could I see a copy of weewx.conf, appropriately obfuscated? You can use the utility wee_debug to do this: 

wee_debug --info

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/1FA52BCB-AD9B-4C38-98B6-4E55EA7D66ED%40isylum.org.

[Leon Shaner]

Tom,

Here you go, attached below...

BTW, wee_debug is not obfuscating the new WU apikey property.

Also for some strange reason the error is no longer being reported.  :-/

Did you happen to fix anything in the last 6 months related to time zones or even daylight savings vs. standard time?

Seems to me that the archive entries are trying to record records that are off by about an hour, and so are colliding with records record before the upgrade.

Then after an hour went by there was no longer any such overlap and the errors went away.

Here is just before I stopped weewx to do the upgrade.

Note the timestamps...

Nov 16 13:15:14 nixie weewx[401]: Added record 2019-11-16 13:15:00 EST (1573928100) to daily summary in 'weewx.sdb'

Nov 16 13:15:25 nixie systemd[1]: weewx.service: Killing process 401 (weewxd) with signal SIGKILL.

Nov 16 13:15:25 nixie systemd[1]: weewx.service: Killing process 401 (weewxd) with signal SIGKILL.

Nov 16 13:15:25 nixie systemd[1]: Stopped weewx weather system.

And after the upgrade:

Nov 16 13:21:01 nixie weewx[4545] INFO weewx.manager: Added record 2019-11-16 14:19:00 EST (1573931940) to database 'weewx.sdb'

Nov 16 13:21:01 nixie weewx[4545] INFO weewx.manager: Added record 2019-11-16 14:19:00 EST (1573931940) to daily summary in 'weewx.sdb'

Nov 16 13:21:02 nixie weewx[4545] INFO weewx.drivers.wmr300: get history complete: count=64 last_index=293 history_end_index=294

Nov 16 13:21:02 nixie weewx[4545] INFO weewx.drivers.wmr300: finish history completed after attempt 1 of 3

Nov 16 13:21:02 nixie weewx[4545] INFO weewx.engine: Starting main packet loop.

Nov 16 13:21:04 nixie weewx[4545] INFO weewx.drivers.wmr300: history buffer at 0.80% (294)

Nov 16 13:22:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 13:22:00 EST (1573928520) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime

Then the last UNIQUE constraint error:

Nov 16 14:19:14 nixie weewx[4545] ERROR weewx.manager: Unable to add record 2019-11-16 14:19:00 EST (1573931940) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime

[gjr80]

Would appreciate some more info on this, I just used 4.0 wee_debug it it obfuscated a new WU API key as expected. The obfuscation code is pretty basic, for WU it is if key name == 'api_key' then value with 'XXX obfuscated by wee_debug XXX'. I seem to remember you changing WU config option names some time ago, what is the config option name that failed, I'm betting it was not 'api_key'.

Gary

[Leon Shaner]

Gary,

I had just done a new sync this afternoon from the latest development.

That introduced a disagreement as to whether the property should be api_key as I had it, or apikey as the newly sync'd code was expecting.

I figured Tom had taken a different path and decided to use apikey afterall, because that's what restx.py was expecting.

So I changed from api_key to apikey in the weewx.conf to make the code work.

Then in wunderfixer I also had to change where it expected api_key to use apikey.

In retrospect, I probably should have fixed restx.py instead since I can now see that wee_debug is also expecting api_key, not apikey.  :-/

Ugg and now I see that restx.py expects apikey in my local copy, but when viewing it on github it expects api_key.

So I think github failed me in some regard w.r.t. the pull from development branch to my fork, to my local copy.  :-(   Likely user error, but it pulled over 200 changes then told me I was up to date.  :-/

Net net, sorry for the noise about api_key not being obfuscated.

I'm off to sort out why I'm not actually running the latest sync I thought I was running.  :S

I suppose some other part of the code could have been out of sync too, leading to the timestamps being off by an hour / overlapping with data in the same hour from prior to the upgrade.  :-/

Regards,

\Leon

--

Leon Shaner :: Dearborn, Michigan (iPad)

On Nov 16, 2019, at 9:19 PM, gjr80 <gjrod...@gmail.com> wrote:



--

You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/b762f6cd-7e78-444e-b80c-e8009f003b7c%40googlegroups.com.

[gjr80]

Are you sure you have picked up development? I just searched through restx.py and wunderfixer in the development branch and there are no apikey entries (well apiKey is used in a URL but this has nothing to do with the config option name), everything is api_key. I would be very surprised to see this config option change names, it would break backwards compatibility for a start.

Gary

Gary,

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-de...@googlegroups.com.

[Leon Shaner]

Gary,

I already did the same validation and came to the same doubt as stated below...

Something went wrong with the sync.

Regards,

\Leon

--

Leon Shaner :: Dearborn, Michigan (iPad)

On Nov 16, 2019, at 9:39 PM, gjr80 <gjrod...@gmail.com> wrote:



To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/184f59d7-eca8-4f60-bc44-68d3edceebb3%40googlegroups.com.