Back up of weexwx.sdb...

[Lorin Tremblay]

Hi!

Was wondering If this bash script is an acceptable method to back up the weewx database on the daily..

Here is my bash script to do so and I have a cron job that triggers It at 00:01 everyday...

# Stop weewx

sudo systemctl stop weewx

# Define the date format for backup filename

DATE=$(date +"%Y%m%d_%H%M%S")

# Backup database (assuming the database is at /var/lib/weewx/weewx.sdb; adjust if different)

sudo zip -r "/var/lib/weewx/weewxdb_$DATE.zip" /var/lib/weewx/weewx.sdb

# Introduce a short delay as a buffer (for example, 10 seconds). Adjust as needed.

sleep 10

# Start weewx again

sudo systemctl start weewx

is this wrong or can It be improved?

[vince]

This has been asked and answered literally dozens of times here over the years. Please do a little searching of the old posts for so many threads and methods that it's impossible to reiterate here.  There are also long discussions of how to validate your backup is good for using to restore your system in the future.

Short answer is 'copy' to a temporary file.  Compress the temporary file.  No need to stop/restart weewx if you use sqlite3 typically.
 

[Lorin Tremblay]

Thanks for the info…

Here is my update bash script that run with a cron job.

Any comment is more than appreciated I’m stating to code here and there, so I appreciate any input…

And yes in the future I’ll do more research before posting anything!!

 

#!/bin/bash

# Set the script name prefix as a variable

sc_name="BackupDB_Weewx"

# Check if sqlite3 is installed

if ! command -v sqlite3 &>/dev/null; then

    logger -t "$sc_name" "Error: sqlite3 is not installed on this system."

    exit 1

fi

    logger -t "$sc_name" "Starting backup of the Weewx Database."

make_backup_copy() {

    db="/var/lib/weewx/weewx.sdb"  # Specify the WeeWX database path

    backup_dir="/var/lib/weewx/"  # Specify the directory where backups will be stored

    timestamp="$(date +"%Y%m%d_%H%M%S")"  # Generate a timestamp

    # Create the full path for the temporary copy with the timestamp

    tmp_db_copy="$backup_dir/weewxdb_${timestamp}.sdb"

    check_sum="`/usr/bin/cksum "$db" | cut -d ' ' -f 1`"

    cp "$db" "$tmp_db_copy"

    check_sum_after="`/usr/bin/cksum "$tmp_db_copy" | cut -d ' ' -f 1`"  # Calculate checksum for the copied file

    if [ "$check_sum" != "$check_sum_after" ]; then

        logger -t "$sc_name" "$db changed during copy process!"

        return 1

    fi

    integrity_check="`echo "pragma integrity_check;" | sqlite3 "$tmp_db_copy"`"

    if [ "$integrity_check" != "ok" ]; then

        logger -t "$sc_name" "$tmp_db_copy failed integrity check!"

        return 2

    fi

        logger -t "$sc_name" "$tmp_db_copy integrity check passed!"

    return 0

}

while true; do

    make_backup_copy

    retval="$?"

    if [ "$retval" -eq 0 ]; then

        logger -t "$sc_name" "Backup successful."

        sudo zip -r "$tmp_db_copy.zip" "$tmp_db_copy"

        logger -t "$sc_name" "Zipping succesful :)"

        sleep 1

        sudo rm -r "$tmp_db_copy"

        logger -t "$sc_name" "Removal of un zipped copy done"

        break  # Exit the loop if the backup is successful

    else

        logger -t "$sc_name" "Backup failed. Retryingin 10 seconds..."

        sleep 10

    fi

done

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1b6472b5-c52e-426b-abf1-1829719469c3n%40googlegroups.com.

[Ton Karsten]

A question,
Has the script been tested yet?
Can you also make it a downloadable file?
can you make it possible to save the copy of the db on google drive?

Op zondag 29 oktober 2023 om 22:53:43 UTC+1 schreef Lorin Tremblay:

[p q]

can you make it possible to save the copy of the db on google drive? --- Not so easy after Google made some changes to their APIs. I used to do this nightly but haven't been able to figure out how to do it anymore. 

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0521ea33-a5f6-490d-b8c3-5ca13a2e91aen%40googlegroups.com.

--

Peter Quinn
(415)794-2264

[Lorin Tremblay]

Yes it ran yesterday as planned on one of my weewx setup and the other loop forever hence this revised version stops after 10 attempts..

#!/bin/bash

# Set the script name prefix as a variable

sc_name="BackupDB_Weewx"

attempt=1

# Check if sqlite3 is installed

if ! command -v sqlite3 &>/dev/null; then

    logger -t "$sc_name" "Error: sqlite3 is not installed on this system."

    exit 1

fi

logger -t "$sc_name" "Starting backup of the Weewx Database."

make_backup_copy() {

    db="/var/lib/weewx/weewx.sdb"  # Specify the WeeWX database path

    backup_dir="/var/lib/weewx/"  # Specify the directory where backups will be stored

    timestamp="$(date +"%Y%m%d_%H%M%S")"  # Generate a timestamp

    # Create the full path for the temporary copy with the timestamp

    tmp_db_copy="$backup_dir/weewxdb_${timestamp}.sdb"

    check_sum="`/usr/bin/cksum "$db" | cut -d ' ' -f 1`"

    cp "$db" "$tmp_db_copy"

    check_sum_after="`/usr/bin/cksum "$tmp_db_copy" | cut -d ' ' -f 1`"  # Calculate checksum for the copied file

    integrity_check="`echo "pragma integrity_check;" | sqlite3 "$tmp_db_copy"`"

    if [ "$integrity_check" != "ok" ] && [ "$attempt" -lt 10 ]; then

        logger -t "$sc_name" "$tmp_db_copy failed integrity check!"

        return 2

    fi

    logger -t "$sc_name" "$tmp_db_copy integrity check passed!"

    return 0

}

while true; do

    make_backup_copy

    retval="$?"

    if [ "$retval" -eq 0 ]; then

        logger -t "$sc_name" "Backup successful."

        sudo zip -r "$tmp_db_copy.zip" "$tmp_db_copy"

        logger -t "$sc_name" "Zipping successful :)"

        sleep 1

        sudo rm -r "$tmp_db_copy"

        logger -t "$sc_name" "Removal of unzipped copy done"

        break  # Exit the loop if the backup is successful

    else

        logger -t "$sc_name" "Backup failed. Retrying in 10 seconds (Attempt $attempt)..."

        attempt+=1

        sleep 10

        if [ "$attempt" -ge 10 ]; then

            logger -t "$sc_name" "Maximum retry attempts reached. Exiting."

            break  # Exit the loop if maximum retry attempts are reached

        fi

    fi

done

For remote transfer I use the Ftp skin, like this:

    # backing in up the database to an ftp server.

    [[backup_DB_FTP]]

        skin = Ftp

        # Override HTML_ROOT:

        HTML_ROOT = /var/lib/weewx

        server = www.yourdomain.com

        path = /path/of/the/remote/server/location

        user = Username

        password = password

The script can be triggered via a cron job to the interval that you desire.

My crontab is the following.

1 0 * * * /home/pi/Desktop/backup_DB.sh > /home/pi/Desktop/cron

Run everyday at 00:01

FYI I’m not a programmer just dabble with code and starting to learn how to code, but nonetheless it ran and gave the expected results.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0521ea33-a5f6-490d-b8c3-5ca13a2e91aen%40googlegroups.com.