MySQL DB issue, connection was killed error, which casues weewx to stop

[bdf0506]

Hello all. I ran into an issue where MySQL was shut down, and as a result it kills the connection to weewx. But weewx doesn't seem to have logic to understand when the database comes back. It seems we could implement a sleep and retry.

This is similar to this thread from 2018 that I reported:

https://groups.google.com/forum/#!msg/weewx-user/GN5lCvsQ-sU/2Ukh8ftfCAAJ

It looks like there is no reference to reason code 1927 as I see in my logs.

https://github.com/weewx/weewx/blob/development/bin/weedb/NOTES.md

Log file:

Jun 14 20:00:19 weewx weewx[1294]: manager: Added record 2020-06-14 20:00:00 EDT (1592179200) to database 'weewx'
Jun 14 20:00:19 weewx weewx[1294]: manager: Added record 2020-06-14 20:00:00 EDT (1592179200) to daily summary in 'weewx'
Jun 14 20:00:19 weewx weewx[1294]: restx: Wunderground-PWS: Published record 2020-06-14 20:00:00 EDT (1592179200)
Jun 14 20:00:20 weewx weewx[1294]: cheetahgenerator: Generated 5 files for report StandardReport in 1.28 seconds
Jun 14 20:00:20 weewx weewx[1294]: copygenerator: copied 0 files to /opt/weewx/public_html
Jun 14 20:02:50 weewx weewx[1294]: restx: Influx: Unexpected exception of type <class 'weedb.DatabaseError'>
Jun 14 20:02:50 weewx weewx[1294]: restx: MQTT: Unexpected exception of type <class 'weedb.DatabaseError'>
Jun 14 20:02:51 weewx weewx[1294]: restx: Influx: Thread exiting. Reason: (1927, 'Connection was killed')
Jun 14 20:02:51 weewx weewx[1294]: restx: MQTT: Thread exiting. Reason: (1927, 'Connection was killed')
Jun 14 20:05:23 weewx weewx[1294]: engine: Shutting down StdReport thread
Jun 14 20:05:23 weewx weewx[1294]: sdr: MainThread: shutdown process rtl_433 -q -U -F json -R40
Jun 14 20:05:36 weewx weewx[1294]: sdr: MainThread: timed out waiting for stderr-thread
Jun 14 20:05:36 weewx weewx[1294]: engine: Caught unrecoverable exception in engine:
Jun 14 20:05:36 weewx weewx[1294]: **** (1927, 'Connection was killed')
Jun 14 20:05:36 weewx weewx[1294]: **** Traceback (most recent call last):
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/engine.py", line 871, in main
Jun 14 20:05:36 weewx weewx[1294]: **** engine.run()
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/engine.py", line 201, in run
Jun 14 20:05:36 weewx weewx[1294]: **** self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/engine.py", line 223, in dispatchEvent
Jun 14 20:05:36 weewx weewx[1294]: **** callback(event)
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/engine.py", line 567, in post_loop
Jun 14 20:05:36 weewx weewx[1294]: **** self._catchup(self.engine.console.genArchiveRecords)
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/engine.py", line 614, in _catchup
Jun 14 20:05:36 weewx weewx[1294]: **** lastgood_ts = dbmanager.lastGoodStamp()
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/manager.py", line 206, in lastGoodStamp
Jun 14 20:05:36 weewx weewx[1294]: **** _row = self.getSql("SELECT MAX(dateTime) FROM %s" % self.table_name)
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weewx/manager.py", line 394, in getSql
Jun 14 20:05:36 weewx weewx[1294]: **** _cursor.execute(sql, sqlargs)
Jun 14 20:05:36 weewx weewx[1294]: **** File "/opt/weewx/bin/weedb/mysql.py", line 49, in guarded_fn
Jun 14 20:05:36 weewx weewx[1294]: **** raise klass(e)
Jun 14 20:05:36 weewx weewx[1294]: **** DatabaseError: (1927, 'Connection was killed')
Jun 14 20:05:36 weewx weewx[1294]: **** Exiting.

Anything we can do to help remediate this situation?

[bdf0506]

Looks like this was already solved, just needed to update my code. All set. Thanks for listening. :)

https://github.com/weewx/weewx/commit/d1b15032f743c4f35205a2b273731ae49b318db9