weewx crash again because: OperationalError: database is locked

Yves Martin

unread,

Oct 22, 2014, 12:22:52 PM10/22/14

to weewx...@googlegroups.com

Hi,

weewx crash again because I have this error: OperationalError: database is locked

Initial, I was said it was because of the flash memory to slow... I've discover the I/O was a bit slow, so I've change for something faster, but I get the same error and I have to restart weewx manually after 4 days.

Here is the last log :

Oct 22 08:15:24 raspberrypi weewx[2815]: reportengine: Caught unrecoverable exception in generator weewx.cheetahgenerator.CheetahGenerator

Oct 22 08:15:24 raspberrypi weewx[2815]: **** database is locked

Oct 22 08:15:24 raspberrypi weewx[2815]: **** Traceback (most recent call last):

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 137, in run

Oct 22 08:15:24 raspberrypi weewx[2815]: **** obj.start()

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 304, in start

Oct 22 08:15:24 raspberrypi weewx[2815]: **** self.run()

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/cheetahgenerator.py", line 129, in run

Oct 22 08:15:24 raspberrypi weewx[2815]: **** self.generate(time_period, self.gen_ts)

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/cheetahgenerator.py", line 223, in generate

Oct 22 08:15:24 raspberrypi weewx[2815]: **** start_ts = archivedb.firstGoodStamp()

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 127, in firstGoodStamp

Oct 22 08:15:24 raspberrypi weewx[2815]: **** _row = self.getSql("SELECT MIN(dateTime) FROM %s" % self.table)

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 286, in getSql

Oct 22 08:15:24 raspberrypi weewx[2815]: **** _cursor.execute(sql, sqlargs)

Oct 22 08:15:24 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weedb/sqlite.py", line 148, in execute

Oct 22 08:15:24 raspberrypi weewx[2815]: **** raise weedb.OperationalError(e)

Oct 22 08:15:24 raspberrypi weewx[2815]: **** OperationalError: database is locked

Oct 22 08:15:24 raspberrypi weewx[2815]: **** Generator terminated...

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: Caught unrecoverable exception in generator weewx.imagegenerator.ImageGenerator

Oct 22 08:15:30 raspberrypi weewx[2815]: **** database is locked

Oct 22 08:15:30 raspberrypi weewx[2815]: **** Traceback (most recent call last):

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 137, in run

Oct 22 08:15:30 raspberrypi weewx[2815]: **** obj.start()

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 304, in start

Oct 22 08:15:30 raspberrypi weewx[2815]: **** self.run()

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 40, in run

Oct 22 08:15:30 raspberrypi weewx[2815]: **** self.genImages(self.gen_ts)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 73, in genImages

Oct 22 08:15:30 raspberrypi weewx[2815]: **** archivedb = self._getArchive(plot_options['archive_database'])

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 324, in _getArchive

Oct 22 08:15:30 raspberrypi weewx[2815]: **** self.archive_cache[archive_name] = weewx.archive.Archive.open(archive_dict)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 75, in open

Oct 22 08:15:30 raspberrypi weewx[2815]: **** return Archive(_connect, table)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 50, in __init__

Oct 22 08:15:30 raspberrypi weewx[2815]: **** self.sqlkeys = self._getTypes()

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 642, in _getTypes

Oct 22 08:15:30 raspberrypi weewx[2815]: **** column_list = self.connection.columnsOf(self.table)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weedb/sqlite.py", line 125, in columnsOf

Oct 22 08:15:30 raspberrypi weewx[2815]: **** column_list = [row[1] for row in self.genSchemaOf(table)]

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weedb/sqlite.py", line 114, in genSchemaOf

Oct 22 08:15:30 raspberrypi weewx[2815]: **** for row in self.connection.execute("""PRAGMA table_info(%s);""" % table):

Oct 22 08:15:30 raspberrypi weewx[2815]: **** OperationalError: database is locked

Oct 22 08:15:30 raspberrypi weewx[2815]: **** Generator terminated...

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: copied 0 files to /var/www/weewx

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: Running report FTP

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report FTP

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./index.html

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./year.html

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./month.html

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: Caught unrecoverable exception in generator weewx.imagegenerator.ImageGenerator

Oct 22 08:15:30 raspberrypi weewx[2815]: **** database is locked

Oct 22 08:15:30 raspberrypi weewx[2815]: **** Traceback (most recent call last):

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 137, in run

Oct 22 08:15:30 raspberrypi weewx[2815]: **** obj.start()

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/reportengine.py", line 304, in start

Oct 22 08:15:30 raspberrypi weewx[2815]: **** self.run()

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 40, in run

Oct 22 08:15:30 raspberrypi weewx[2815]: **** self.genImages(self.gen_ts)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 199, in genImages

Oct 22 08:15:30 raspberrypi weewx[2815]: **** aggregate_interval, aggregate_type)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 464, in getSqlVectorsExtended

Oct 22 08:15:30 raspberrypi weewx[2815]: **** aggregate_interval, aggregate_type)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 400, in getSqlVectors

Oct 22 08:15:30 raspberrypi weewx[2815]: **** for _rec in _cursor.execute(sql_str, (startstamp, stopstamp)):

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 400, in getSqlVectors

Oct 22 08:15:30 raspberrypi weewx[2815]: **** for _rec in _cursor.execute(sql_str, (startstamp, stopstamp)):

Oct 22 08:15:30 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weedb/sqlite.py", line 148, in execute

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./week.html

Oct 22 08:15:30 raspberrypi weewx[2815]: **** raise weedb.OperationalError(e)

Oct 22 08:15:30 raspberrypi weewx[2815]: **** OperationalError: database is locked

Oct 22 08:15:30 raspberrypi weewx[2815]: **** Generator terminated...

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: copied 0 files to /var/www/weewx

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: Running report FTP

Oct 22 08:15:30 raspberrypi weewx[2815]: reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report FTP

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./almanac.html

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./about.html

Oct 22 08:15:30 raspberrypi weewx[2815]: ftpupload: Uploaded file ./yday.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file NOAA/NOAA-2014.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./index.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./year.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./month.html

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: ftp'd 8 files in 1.13 seconds

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: Running report RSYNC

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report RSYNC

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: rsync upload not requested. Skipped.

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./week.html

Oct 22 08:15:31 raspberrypi weewx[2815]: wxengine: Shut down StdReport thread.

Oct 22 08:15:31 raspberrypi weewx[2815]: restx: Shut down StationRegistry thread.

Oct 22 08:15:31 raspberrypi weewx[2815]: restx: Shut down CWOP thread.

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./almanac.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./about.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file ./yday.html

Oct 22 08:15:31 raspberrypi weewx[2815]: ftpupload: Uploaded file NOAA/NOAA-2014.html

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: ftp'd 8 files in 1.17 seconds

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: Running report RSYNC

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report RSYNC

Oct 22 08:15:31 raspberrypi weewx[2815]: reportengine: rsync upload not requested. Skipped.

Oct 22 08:15:31 raspberrypi weewx[2815]: vantage: successfully woke up console

Oct 22 08:15:31 raspberrypi weewx[2815]: wxengine: Caught unrecoverable exception in wxengine:

Oct 22 08:15:31 raspberrypi weewx[2815]: **** database is locked

Oct 22 08:15:31 raspberrypi weewx[2815]: **** Traceback (most recent call last):

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 963, in main

Oct 22 08:15:31 raspberrypi weewx[2815]: **** engine.run()

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 189, in run

Oct 22 08:15:31 raspberrypi weewx[2815]: **** engine.run()

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 189, in run

Oct 22 08:15:31 raspberrypi weewx[2815]: **** self.dispatchEvent(weewx.Event(weewx.POST_LOOP))

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 210, in dispatchEvent

Oct 22 08:15:31 raspberrypi weewx[2815]: **** callback(event)

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 562, in post_loop

Oct 22 08:15:31 raspberrypi weewx[2815]: **** self._catchup(self.engine.console.genArchiveRecords)

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 625, in _catchup

Oct 22 08:15:31 raspberrypi weewx[2815]: **** origin='hardware'))

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 210, in dispatchEvent

Oct 22 08:15:31 raspberrypi weewx[2815]: **** callback(event)

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/wxengine.py", line 574, in new_archive_record

Oct 22 08:15:31 raspberrypi weewx[2815]: **** self.archive.addRecord(event.record)

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weewx/archive.py", line 192, in addRecord

Oct 22 08:15:31 raspberrypi weewx[2815]: **** e))

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weedb/__init__.py", line 130, in __exit__

Oct 22 08:15:31 raspberrypi weewx[2815]: **** self.connection.commit()

Oct 22 08:15:31 raspberrypi weewx[2815]: **** File "/usr/share/weewx/weedb/__init__.py", line 97, in commit

Oct 22 08:15:31 raspberrypi weewx[2815]: **** self.connection.commit()

Oct 22 08:15:31 raspberrypi weewx[2815]: **** OperationalError: database is locked

Oct 22 08:15:31 raspberrypi weewx[2815]: **** Exiting.

Oct 22 08:15:35 raspberrypi weewx[2815]: genimages: Generated 22 images for StandardReport in 131.21 seconds

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: copied 0 files to /var/www/weewx

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: Running report FTP

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report FTP

Oct 22 08:15:35 raspberrypi weewx[2815]: ftpupload: Uploaded file ./yearwinddir.png

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: ftp'd 1 files in 0.43 seconds

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: Running report RSYNC

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report RSYNC

Oct 22 08:15:35 raspberrypi weewx[2815]: reportengine: rsync upload not requested. Skipped.

Oct 22 08:17:01 raspberrypi /USR/SBIN/CRON[9142]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)

Oct 22 09:17:01 raspberrypi /USR/SBIN/CRON[9149]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)

Oct 22 10:17:01 raspberrypi /USR/SBIN/CRON[9156]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)

Oct 22 11:17:01 raspberrypi /USR/SBIN/CRON[9163]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)

Oct 22 12:09:10 raspberrypi weewx[9197]: wxengine: Initializing weewx version 2.7.0

any idea?

Thomas Keffer

unread,

Oct 22, 2014, 12:49:37 PM10/22/14

to weewx-user

If weewx gets a database locked error in the reporting thread, it will crash the thread. This is what happened with the first three errors. However, a new thread is launched with every archive interval, so later reports should be fine.

However, in the fourth error (starts at 08:15:31) the locking error happened in the main thread. Normally, weewx catches this error, waits 2 minutes, then restarts. However, your locking error just happened to occur during the transaction commit, which is unguarded, so the program crashed. This is a bug. I'll try to come up with a patch for this in the next few days.

This is really tough for me to debug remotely because I have never had a locking error. I have to rely on user reports like yours!

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Andrew Milner

unread,

Oct 22, 2014, 1:25:42 PM10/22/14

to weewx...@googlegroups.com

I have never seen a weewx locking error on my RPI either - but I will admit to running mySQL rather than sqlite since I believe it to be possibly more resilient and able to handle locks and transactions better (just a hunch) - even though it is only used for weewx and as a print server and the (occasional) web visitor.

Yves Martin

unread,

Oct 22, 2014, 10:29:04 PM10/22/14

to weewx...@googlegroups.com

Hi Tom,

Thanks for the answer and I'm happy to make this soft stronger ;)

weewx never crashed the during the first 6 months of use... I believed the database were not very big at this time.

YM

Le mercredi 22 octobre 2014 12:22:52 UTC-4, Yves Martin a écrit :

Hi,

weewx crash again because I have this error: OperationalError: database is locked

Initial, I was said it was because of the flash memory to slow... I've discover the I/O was a bit slow, so I've change for something faster, but I get the same error and I have to restart weewx manually after 4 days.

Here is the last log :

Oct 22 08:15:24 raspberrypi weewx[2815]: reportengine: Caught unrecoverable exception in generator weewx.cheetahgenerator.CheetahGenerator
Oct 22 08:15:24 raspberrypi weewx[2815]: **** database is locked
Oct 22 08:15:24 raspberrypi weewx[2815]: **** Traceback (most recent call last):

(...)

Steve

unread,

Oct 23, 2014, 11:29:36 AM10/23/14

to weewx...@googlegroups.com

I also run MySQL rather than sqlite on my RPi and have never had a lock error. (Weeks away from 12 months now)

My Pi also monitors my energy consumption that writes to another MySQL database. (This is why I ran Weewx on MySQL too).

I let the MySQL databases get to 1.5GB+ before I did something to stop my energy consumption from making it grow so big, but every night via a cron job at 2am I have mysqldump all my databases to my NAS without weewx or measureit complaining about it, or stopping them while it happens (a 20 minute + operation). And more than once I've broken my installation and have been able to restore the last backup from my NAS.

My personal (with no credentials) opinion is to use MySQL on the RPi to avoid DB lock-ups as it works for me without any drama.

Regards,

Steve.

Thomas Keffer

unread,

Oct 23, 2014, 11:53:50 AM10/23/14

to weewx-user

That's a useful data point and recommendation. But, I believe that sqlite can be made to work reliably on the RPi. It's just a matter of covering all the failure modes.

-tk

Fat-fingered from my Android

Andrew Milner

unread,

Oct 23, 2014, 12:46:24 PM10/23/14

to weewx...@googlegroups.com

My lay view is that the old weewx was fine with sqlite, but once you went to multithreads you effectively became multi user and thus I would contend that mysql is better. It does not really seem to impact anything to use mysql - which seems to run fine with the RPi - although I dont run apache as the web server - just lighttpd instead!! Did not want to strain the RPi tooooo much!!!! Am currently intrigued to see if I continue to have days with no FOusb errors - because since I ut in my new, better, newer disk I do not seem to have the same errors from the driver. Am not yet convinced as not many days have passed - but am keeping a beedy eye on it as maybe the errors are coming from a slowed down usb bus - and now I have a more efficient usb disk I am clearing the bus quicker. Don't know - but it sure seems to be running much better on the new installation!!!! Am also not yet running meso which does increase the RPi workload a bit.

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/CeikzAs8hPg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

Thomas Keffer

unread,

Oct 23, 2014, 1:18:59 PM10/23/14

to weewx-user

Weewx has always been multi-threaded.

-tk

Tom Keffer

unread,

Oct 24, 2014, 2:29:54 PM10/24/14

to weewx...@googlegroups.com

OK, this has now been fixed in R2422. Any database locking exceptions that happen during commits should be caught. Weewx will then sleep for 2 minutes and try again.

I don't know what release this will appear in. I'm really trying to focus on V3, instead of doing yet another dot-dot release!

-tk

Yves Martin

unread,

Oct 26, 2014, 9:57:36 AM10/26/14

to weewx...@googlegroups.com

Great! (weewx crashed again twice yesterday... if you need the logs, I can send them to you.)

Is there a tool to update sqllite to mysql with weewx?

Another question, perhaps off-topic, but is weewx will support native multi-language on next version (v3)?

YM

Thomas Keffer

unread,

Oct 26, 2014, 10:49:03 AM10/26/14

to weewx-user

1. No need to send the logs. I think I understand the problem now.

2. See sqlite3-to-mysql.py in the User Contributions area of SourceForge. I have not tried it. Let us know if it works for you.

3. I'm not sure what you mean by "native multi-language." Right now, weewx is fully internationalized --- there are users who have adapted it to Japanese. However, if you are looking for separate language skins, one for French, one for German, etc., no, there are no plans to do that. Too big a maintenance headache!

-tk

Yves Martin

unread,

Oct 27, 2014, 11:24:18 AM10/27/14

to weewx...@googlegroups.com

Hi Tom,

When I say multi-languages, I mean at least 2 languages on the same interface. Here it will be English and French... using metric and imperial values...

I've never tried it in fact to use both and I don't know if it could have some limitations.

Actually, my web site is fully translated in French, but we also use English here in Canada ;)

YM

Le mercredi 22 octobre 2014 12:22:52 UTC-4, Yves Martin a écrit :

Thomas Keffer

unread,

Oct 27, 2014, 11:44:47 AM10/27/14

to weewx-user

That should be possible. Remember: it's possible to override the units:

$day.outTemp.max.degree_C

-tk

Yves Martin

unread,

Oct 27, 2014, 1:11:51 PM10/27/14

to weewx...@googlegroups.com

This will be my next update ;) I will let you know.

I will also try the sqlite3-to-mysql.py script, and of course also let you know.

Thanks.

YM

Yves Martin

unread,

Oct 29, 2014, 1:30:52 PM10/29/14

to weewx...@googlegroups.com

Hi Tom,

About the script (2), do you have an ABC very basic step doc to make the script working ?

YM

Le mercredi 22 octobre 2014 12:22:52 UTC-4, Yves Martin a écrit :

Andrew Milner

unread,

Oct 29, 2014, 3:21:37 PM10/29/14

to weewx...@googlegroups.com

22 imges in131 seconds says you have an rpi problem and nothing to do with weewx.

get a decent sd card and a good power supply

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/CeikzAs8hPg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

Yves Martin

unread,

Oct 31, 2014, 7:30:07 AM10/31/14

to weewx...@googlegroups.com

?

Message has been deleted

Yves Martin

unread,

Nov 5, 2014, 11:49:28 AM11/5/14

to weewx...@googlegroups.com

Hi Andrew,

What is your suggestion about the SD card? I've tried industrial SLC flash card, with fast transfer but very poor I/O, then 2 different kind of SANDISK flash card. This last one is a Sandisk Extreme 16GB class 10 45MB/s.

Power supply is connected directly on USB of UPS (very stable and DC only)

YMartin.com/meteo

Le mercredi 22 octobre 2014 12:22:52 UTC-4, Yves Martin a écrit :

Thomas Keffer

unread,

Nov 5, 2014, 12:00:24 PM11/5/14

to weewx-user

I've been running a 16GB SanDisk Extreme Plus for over 2 months now on an RPi B+, with zero problems. No attempt to minimize disk writes.

I intend to continue until it finally fails.

-tk

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

Dave Webb KB1PVH

unread,

Nov 5, 2014, 12:08:54 PM11/5/14

to weewx...@googlegroups.com

I have been using a 16g SanDisk Ultra on my Pi for 15 months without any problems. I would make positive that your power supply is not fluctuating. I had a problem when my wife would plug her phone in to charge off the same supply that the Pi was plugged into and it made the Pi do bizarre things.

qsl.net/kb1pvh/weather

Dave-KB1PVH

Sent from my Samsung S4

Andrew Milner

unread,

Nov 5, 2014, 1:16:27 PM11/5/14

to weewx...@googlegroups.com

I think the current draw is more significant than the voltage maintenance with the RPi. Regardless of the whether you have a UPS or not, make sure you are using an adequate (3amp) power supply plugged into the UPS - especially if you are using WiFi or network or ther USB devices. Hence why I originally suggested a powered USB hub for your USB devices.

I generate (just checked) with times as follows:

Nov 5 12:00:44 raspberrypi weewx[11611]: archive: added record 2014-11-05 12:00:00 EET (1415181600) to database 'archive'; table 'archive'

Nov 5 12:00:44 raspberrypi weewx[11611]: restx: PWSWeather: Published record 2014-11-05 12:00:00 EET (1415181600)

Nov 5 12:00:44 raspberrypi weewx[11611]: restx: Wunderground-PWS: Published record 2014-11-05 12:00:00 EET (1415181600)

Nov 5 12:01:08 raspberrypi weewx[11611]: genimages: Generated 18 images for SmallImages in 23.75 seconds

Nov 5 12:01:25 raspberrypi weewx[11611]: genimages: Generated 18 images for BigImages in 17.33 seconds

Nov 5 12:01:26 raspberrypi weewx[11611]: GaugeGenerator: Generated 6 images for HTMLPages in 0.45 seconds

Nov 5 12:01:26 raspberrypi weewx[11611]: cheetahgenerator: generated 1 'SummaryByMonth' files for HTMLPages in 0.72 seconds

Nov 5 12:01:32 raspberrypi weewx[11611]: cheetahgenerator: generated 2 'SummaryByYear' files for HTMLPages in 5.18 seconds

Nov 5 12:01:33 raspberrypi weewx[11611]: cheetahgenerator: generated 11 'ToDate' files for HTMLPages in 1.39 seconds

so any times for any item in excess of 30 seconds I regard as wrong and excessive. I picked 1200 for the log snippet as that forces most things to be regenerated!!

vds

unread,

Nov 5, 2014, 2:05:10 PM11/5/14

to weewx...@googlegroups.com

On Wednesday, November 5, 2014 9:00:24 AM UTC-8, Tom Keffer wrote:

I've been running a 16GB SanDisk Extreme Plus for over 2 months now on an RPi B+, with zero problems. No attempt to minimize disk writes.

My Raspbian Pi running 'motion' to do timelapse weather movies via a USB webcam is starting to show signs of occasional stupidity again.

Symptoms are that my cron job that uploads a snapshot to Wunderground every few minutes starts failing (a clue for me to check it) and I see the typical failures after logging in when I try to 'su' or do 'ls -al' or anything like that, with a message saying the filesystem is mounted read-only. Checking the logs I see definite disk error type things. There's nothing I can do without privs at that point other than yank the power and cross my fingers. It's recovered four or five times thus far.

All the webcam stuff runs to a tmpfs partition so that's not an issue. I did notice this morning that there's a huge amount of syslogging from cron - I have a cron job that runs once/minute to self-heal the wifi when it goes nutso occasionally. The Raspbian defaults log every time a cron job runs, so I added a '-L 0' option to /etc/default/cron to quiet that noise down which should set the system to a more reasonable logging behavior. We'll see how that works out.

Again, I'm running a USB webcam powered off the Pi itself, which is powered by a lousy power supply plugged in without any surge suppression or the like, so perhaps it's a bit of a torture test from a power perspective. Rather than change two things at once, I'll let this run for a while and see how it goes. If it goes whacked again I'll leave the software the same and see if I can make the power situation a bit better.

Andrew Milner

unread,

Nov 5, 2014, 2:12:11 PM11/5/14

to weewx...@googlegroups.com

Hi vds - I ultimately lost my RPi installation by ignoring the warning signs of an impending disk problem. I was not using an sd - but an external old usb hard disk .. and eventually it died and so did all my data and software become irrecoverabe. Don't ignore the warning signs and keep your fingers crossed else you will wake up one morning with no fingers!!

Yves Martin

unread,

Nov 10, 2014, 8:57:01 AM11/10/14

to weewx...@googlegroups.com

In fact, the USB plug powered by the UPS has just been used. Before, it was a 1A. USB plug, plugged directly on the UPS.

I'm crossing my fingers, 11 days without any crash now ;)

This USB plug is not hosted by a hub or used to host a hub, it is just a direct plug powered by the UPS. For info, the model is : CyberPower CP1350PFCLCD PFC Sinewave UPS 1350VA 810W (http://www.cyberpowersystems.com/products/ups-systems/pfc-sinewave-series/CP1350PFCLCD.html?selectedTabId=specifications&imageI=#tab-box)

There are 2 USB ports 1A regulated on each port, and for what I did, 1A. is enough.

I've noted a slightly more stable signal since (average of 98%) ...

YM

YMartin.com/meteo

Le mercredi 22 octobre 2014 12:22:52 UTC-4, Yves Martin a écrit :

Message has been deleted

Yves Martin

unread,

Nov 14, 2014, 10:23:17 AM11/14/14

to weewx...@googlegroups.com

I cross my fingers. 15 days without any issue since I've plugged my RPi on the UPS directly (1A.). Signal seems also slightly more stable (from 97 to 98.5% ...) better)

Yves,
YMartin.com/meteo

Andrew Milner

unread,

Nov 14, 2014, 11:40:18 AM11/14/14

to weewx...@googlegroups.com

Hope it's solved then!! Most power supplies for the RPi are usually 1.5A though. This is one ad I found as to why theirs is a 2A supply .. may be of interest ...https://www.modmypi.com/5v-2A-modmypi-raspberry-pi-power-supply

Liz

unread,

Nov 14, 2014, 2:34:26 PM11/14/14

to weewx...@googlegroups.com

On Fri, 14 Nov 2014 08:40:18 -0800 (PST)
Andrew Milner <andrew.s...@gmail.com> wrote:

> Hope it's solved then!! Most power supplies for the RPi are usually
> 1.5A though. This is one ad I found as to why theirs is a 2A
> supply .. may be of interest
> ...https://www.modmypi.com/5v-2A-modmypi-raspberry-pi-power-supply

Often discussed in ham radio circles fro the same reason.
The RPi is more stable on 5.25V, even with a powered USB hub.

Yves Martin

unread,

Nov 26, 2014, 8:53:20 AM11/26/14

to weewx...@googlegroups.com

Unfortunately, I had to restart the RPi because my iNternet provider change, but since this time, the RPi creashed a few times, so I don't think it is only related to the stability of the power-supply of the RPi.

Yves

Andrew Milner

unread,

Nov 26, 2014, 10:13:05 AM11/26/14

to weewx...@googlegroups.com

But you are still not using a 1.5 amp power supply. You may have a UPS that provides a steady voltage - but is it able to provide enough amps under all conditions?

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/CeikzAs8hPg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

Yves Martin

unread,

Nov 27, 2014, 9:49:58 AM11/27/14

to weewx...@googlegroups.com

The RPi use under 1A. and still connected on the UPS with a very stable voltage. I'm waiting a Volt-Amp-meter to see exactly what is the real current/voltage used, but it was working fine during 22 days without crash and now it crashed a few times in a week... I do not lost data because the temporary memory of the Vantage doing his job very well, but we can see empty data in the voltage loop of the VP2. It is just annoying to restart the weewx daemon regularly ...

Yves

Thomas Keffer

unread,

Nov 27, 2014, 11:08:17 AM11/27/14

to weewx-user

Hello, Yves

Could you please post the relevant section of the log for when weewx crashed? Preferably with debug=1.

-tk

Andrew Milner

unread,

Nov 27, 2014, 11:47:48 AM11/27/14

to weewx...@googlegroups.com

Yves

I refer you to this link - different from the last one I gave you - from the RPi forum where once again it is the amps that are just as important as the voltage. The UPS guarantees the voltage I know - but what amps?? What is your loading on USB and ethernet ports? These all have an effect on the supply requirements.

http://www.raspberrypi.org/forums/viewtopic.php?f=28&t=40290

My RPi is perfectly stable - as are many others running weewx with RPi.

Andrew.

Liz

unread,

Nov 27, 2014, 2:25:59 PM11/27/14

to weewx...@googlegroups.com

On Thu, 27 Nov 2014 08:47:48 -0800 (PST)
Andrew Milner <andrew.s...@gmail.com> wrote:

> My RPi is perfectly stable - as are many others running weewx with
> RPi.

My RPi was stable for months. I upgraded last weekend and it crashed
within 2 days.

Liz

Thomas Keffer

unread,

Nov 27, 2014, 3:15:38 PM11/27/14

to weewx-user

How about a few more details, Liz?

Weewx may struggle with something outside of its control, but the goal is for it to never crash.

-tk

Liz

unread,

Nov 28, 2014, 4:14:43 AM11/28/14

to weewx...@googlegroups.com

On Thu, 27 Nov 2014 12:15:35 -0800
Thomas Keffer <tke...@gmail.com> wrote:

> How about a few more details, Liz?
>
> Weewx may struggle with something outside of its control, but the
> goal is for it to never crash.
>
> -tk

Weewx isn't running on the the RPi
The RPi crashed.... not Weewx
Sorry for giving the wrong impression.

RPi runs ham radio stuff.

Yves Martin

unread,

Dec 9, 2014, 9:26:29 AM12/9/14

to weewx...@googlegroups.com

I've put a 2A power-supply directly plugged on my UPS... same issue.

My weewx crash now a few times a day, sometime it runs during a couple of days... but definitively it is unstable here. I will wait for a stable version 3 and try it after all.

YM

mwall

unread,

Dec 9, 2014, 10:24:08 AM12/9/14

to weewx...@googlegroups.com

On Tuesday, December 9, 2014 9:26:29 AM UTC-5, Yves Martin wrote:

I've put a 2A power-supply directly plugged on my UPS... same issue.

My weewx crash now a few times a day, sometime it runs during a couple of days... but definitively it is unstable here. I will wait for a stable version 3 and try it after all.

YM

yves,

what you posted is not a crash. crash is when weewx stops running.

you are experiencing 'database locked' failures. weewx is still running.

we see 'database locked' most often on systems with slow/defective disks.

go ahead and try v3 - i am curious to see whether it resolves any of the 'database locked' issues. v3 will not fix a slow/defective drive, but it manages the database connections differently than v2.

in terms of data capture and storage, v3 is already quite stable - i have been running it on a few different systems with different hardware for 3 months now. so far i have not experienced any 'database locked' failures, but ymmv.

m

Andrew Milner

unread,

Dec 9, 2014, 10:46:26 AM12/9/14

to weewx...@googlegroups.com

One thing I can assure you is that it is unlikely to be a problem with weewx, so V3 is unlikely to fix the root cause of your problems. Have you tried using mysql rather than sqlite - it is possibly more resilient?? My weewx, with FineOffset and using MySQL with lighttpd never crashes on my RPi model B - although I admit to using an external hard drive rather than SD card. Other weewx users do not have the same problems that you are experiencing. Have you tried to ditch your UPS entirely - in case that is somehow causing the problem? I guess at the end of the day it is even possible that you have a dodgy RPi!! Another thing I have done is to use turbo mode on the RPi to try and make sure all IO is handled as rapidly as possible, It almost looks as though your database is locking because something is taking an eternity to execute - and you need to try and identify what exactly is causing that. As far as I know there are many RPi users of weewx with a variety of different weather stations. The one thing that is 'unique' in your installation is your UPS. Do you have other devices plugged in to it - I see it is only 800watt. and the available 1A on the 2 USB ports is actually shared between them, so I wonder if the 800watt capability is possibly being exceeded if other devices are also connected to it. Just more food for thought .....

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/CeikzAs8hPg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

Yves Martin

unread,

Dec 11, 2014, 11:23:40 PM12/11/14

to weewx...@googlegroups.com

No, weewx stop running.

ps -aux or top for example, do not show weewx running, I have to (re)start it.

YM

Yves Martin

unread,

Dec 11, 2014, 11:27:55 PM12/11/14

to weewx...@googlegroups.com

Hi Andrew,

I did not install mysql, because I don't know how to use the script :(

Anyway, I will be very very unlucky is it was my flash memory again, it is the 4th memory I try with same results... I don't think it is also a power problem, I've also tried different solutions with same issue. It is not a mémory (size) issue, I just use 22% of the disk ... I'm a bit out of ideas.

Yves

Andrew Milner

unread,

Dec 12, 2014, 12:01:25 AM12/12/14

to weewx...@googlegroups.com

1. try

ps aux

or

ps -A

or

ps -e

rather than

ps -aux

--

Thomas Keffer

unread,

Dec 12, 2014, 7:45:59 AM12/12/14

to weewx-user

Yves,

1. What version of weewx are you running now? V3 has the additional protections to keep weewx from crashing. It won't fix your problem, but it should keep the program from crashing.

2. You still have not posted the log, It's really tough to debug remotely without the log.

-tk

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

Yves Martin

unread,

Dec 12, 2014, 9:05:03 AM12/12/14

to weewx...@googlegroups.com

I use your last version 2.7.0 (not 3 yet because I've read it was alpha or beta...). I will take time to track the crashes and copy the logs... In one day, I had 4 crashed at the top! I've noted, when I restart weewx after a long crash, it usually crash just after it and I have to restart again. You can easily see the crashes in the weekly logs of "État de la batterie solaire" and "Qualité du signal de la station", the 2 last graphs.

I will let you know soon about the logs. Thanks.

YM

Thomas Keffer

unread,

Dec 12, 2014, 10:44:44 AM12/12/14

to weewx-user

V3.0 has been out for about a week. Everyone seems to be having good results from it.

-tk

Yves Martin

unread,

Dec 31, 2014, 11:32:17 AM12/31/14

to weewx...@googlegroups.com

Hi Tom,

Here are the last 2 crashes... It happen a few times every week, sometime 2 or 3 times a day.

Dec 31 10:22:14 raspberrypi weewx[14922]: reportengine: ftp'd 42 files in 8.93 seconds

Dec 31 10:22:14 raspberrypi weewx[14922]: reportengine: Running report RSYNC

Dec 31 10:22:14 raspberrypi weewx[14922]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report RSYNC

Dec 31 10:22:14 raspberrypi weewx[14922]: reportengine: rsync upload not requested. Skipped.

Dec 31 10:22:16 raspberrypi weewx[14922]: vantage: Getting archive packets since 2014-12-31 10:21:00 EST (1420039260)

Dec 31 10:22:16 raspberrypi weewx[14922]: cheetahgenerator: generated 11 'ToDate' files for StandardReport in 27.42 seconds

Dec 31 10:22:16 raspberrypi weewx[14922]: vantage: successfully woke up console

Dec 31 10:22:16 raspberrypi weewx[14922]: vantage: Retrieving 1 page(s); starting index= 3

Dec 31 10:22:16 raspberrypi weewx[14922]: forecast: MainThread: WU: not yet time to do the forecast

Dec 31 10:22:16 raspberrypi weewx[14922]: archive: added record 2014-12-31 10:22:00 EST (1420039320) to database 'weewx.sdb'; table 'archive'

Dec 31 10:22:27 raspberrypi weewx[14922]: reportengine: Caught unrecoverable exception in generator weewx.imagegenerator.ImageGenerator

Dec 31 10:22:27 raspberrypi weewx[14922]: **** database is locked

Dec 31 10:22:27 raspberrypi weewx[14922]: **** Traceback (most recent call last):

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/reportengine.py", line 137, in run

Dec 31 10:22:27 raspberrypi weewx[14922]: **** obj.start()

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/reportengine.py", line 304, in start

Dec 31 10:22:27 raspberrypi weewx[14922]: **** self.run()

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 40, in run

Dec 31 10:22:27 raspberrypi weewx[14922]: **** self.genImages(self.gen_ts)

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 199, in genImages

Dec 31 10:22:27 raspberrypi weewx[14922]: **** aggregate_interval, aggregate_type)

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/archive.py", line 464, in getSqlVectorsExtended

Dec 31 10:22:27 raspberrypi weewx[14922]: **** aggregate_interval, aggregate_type)

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/archive.py", line 400, in getSqlVectors

Dec 31 10:22:27 raspberrypi weewx[14922]: **** for _rec in _cursor.execute(sql_str, (startstamp, stopstamp)):

Dec 31 10:22:27 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weedb/sqlite.py", line 148, in execute

Dec 31 10:22:27 raspberrypi weewx[14922]: **** raise weedb.OperationalError(e)

Dec 31 10:22:27 raspberrypi weewx[14922]: **** OperationalError: database is locked

Dec 31 10:22:27 raspberrypi weewx[14922]: **** Generator terminated...

Dec 31 10:22:27 raspberrypi weewx[14922]: reportengine: copied 0 files to /var/www/weewx

Dec 31 10:22:27 raspberrypi weewx[14922]: reportengine: Running report FTP

Dec 31 10:22:27 raspberrypi weewx[14922]: reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report FTP

Dec 31 10:22:27 raspberrypi weewx[14922]: ftpupload: Uploaded file ./year.html

Dec 31 10:22:27 raspberrypi weewx[14922]: ftpupload: Uploaded file ./month.html

Dec 31 10:22:27 raspberrypi weewx[14922]: ftpupload: Uploaded file ./week.html

Dec 31 10:22:27 raspberrypi weewx[14922]: ftpupload: Uploaded file ./yday.html

Dec 31 10:22:28 raspberrypi weewx[14922]: reportengine: ftp'd 4 files in 1.18 seconds

Dec 31 10:22:28 raspberrypi weewx[14922]: reportengine: Running report RSYNC

Dec 31 10:22:28 raspberrypi weewx[14922]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report RSYNC

Dec 31 10:22:28 raspberrypi weewx[14922]: reportengine: rsync upload not requested. Skipped.

Dec 31 10:22:28 raspberrypi weewx[14922]: wxengine: Shut down StdReport thread.

Dec 31 10:22:28 raspberrypi weewx[14922]: restx: Shut down StationRegistry thread.

Dec 31 10:22:28 raspberrypi weewx[14922]: restx: Shut down CWOP thread.

Dec 31 10:22:28 raspberrypi weewx[14922]: vantage: successfully woke up console

Dec 31 10:22:28 raspberrypi weewx[14922]: wxengine: Caught unrecoverable exception in wxengine:

Dec 31 10:22:28 raspberrypi weewx[14922]: **** database is locked

Dec 31 10:22:28 raspberrypi weewx[14922]: **** Traceback (most recent call last):

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 963, in main

Dec 31 10:22:28 raspberrypi weewx[14922]: **** engine.run()

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 189, in run

Dec 31 10:22:28 raspberrypi weewx[14922]: **** self.dispatchEvent(weewx.Event(weewx.POST_LOOP))

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 210, in dispatchEvent

Dec 31 10:22:28 raspberrypi weewx[14922]: **** callback(event)

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 562, in post_loop

Dec 31 10:22:28 raspberrypi weewx[14922]: **** self._catchup(self.engine.console.genArchiveRecords)

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 625, in _catchup

Dec 31 10:22:28 raspberrypi weewx[14922]: **** origin='hardware'))

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 210, in dispatchEvent

Dec 31 10:22:28 raspberrypi weewx[14922]: **** callback(event)

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/wxengine.py", line 574, in new_archive_record

Dec 31 10:22:28 raspberrypi weewx[14922]: **** self.archive.addRecord(event.record)

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weewx/archive.py", line 192, in addRecord

Dec 31 10:22:28 raspberrypi weewx[14922]: **** e))

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weedb/__init__.py", line 130, in __exit__

Dec 31 10:22:28 raspberrypi weewx[14922]: **** self.connection.commit()

Dec 31 10:22:28 raspberrypi weewx[14922]: **** File "/usr/share/weewx/weedb/__init__.py", line 97, in commit

Dec 31 10:22:28 raspberrypi weewx[14922]: **** self.connection.commit()

Dec 31 10:22:28 raspberrypi weewx[14922]: **** OperationalError: database is locked