Backup sqlite file

[Jeff Allison]

Hello,

Just recently installed weewx and I'm looking for the best way to make regular backups. My plan is to create a Windows batch file that uses pscp to copy all the relevant files to a folder on my Windows system and schedule that to run daily. Pretty straightforward, but I'm concerned about getting a valid copy of the sqlite database file. The docs make this warning:

For a SQLite configuration, do not make the copy of the database file while in the middle of a transaction! Schedule the backup for immediately after an archive record is written, and then make sure the backup completes before the next archive record arrives.

I don't know how I can make sure the database is not in the middle of a transaction when I'm attempting the copy. Shutting down and restarting weewx I guess is possible using ssh commands in the batch file, but that's getting more complicated. How worried do I need to be about getting a bad backup file if I just copy the file over?

Thanks!

[p q]

I run a cron job at 2am that shuts down Weewx, copies the database and restarts it. I used to then copy the backup copy to google drive, but google made the authentication impossible, so I stopped doing that part. I just manually copy it from time to time - perhaps I should do what you're doing with pscp too. Anyway, copying the copy would protect you from a conflict.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a7ffec0e-81ac-453c-b113-1c497e36783dn%40googlegroups.com.

--

Peter Quinn
(415)794-2264

[Jeff Allison]

Cron job and copy of the copy sounds like a good plan. Thanks!

- Jeff 

[John Kline]

> I don't know how I can make sure the database is not in the middle of a transaction when I'm attempting the copy.

Do a sha1sum of the database:

    sha1sum /home/weewx/archive/weewx.sdb

Make the copy.

Do another sha1sum of the database:

    sha1sum /home/weewx/archive/weewx.sdb

If the sha1 hasn’t changed, you have a good copy.

If it has changed, sleep and try again.

On Jun 21, 2024, at 2:14 PM, Jeff Allison <jall...@amate.us> wrote:

Cron job and copy of the copy sounds like a good plan. Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f0f5983d-f11f-471e-9f46-bce458053fa2n%40googlegroups.com.

[vince]

My script for Linux is (here) if you want to see how I do it.   A less cleaned up version has been in use here for something like 15 years if not longer.   As the other folks said, copy your db to a scratch dir and work off the copy.  At one point quite a few years ago this FAQ came up and several folks did checks for their backups being good and they were always fine.

[Jeff Allison]

Thank you for the script! This coupled with a Windows job to squirrel away the backup separately will do the trick nicely.

As a PS, my thanks to the whole community that has developed weewx. For years I've been stuck on Davis's creaky old Weatherlink software to archive the data and upload to Weather Underground. Having a dedicated little device to do it is so much better.

- Jeff 

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/j25FlOyup8U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/74f5d3ca-e52e-4e9b-a9dd-52237f5ced6fn%40googlegroups.com.