Version 5 Database behaviour

97 views
Skip to first unread message

Steeple Ian

unread,
May 20, 2023, 4:03:37 PM5/20/23
to weewx-development
I have been trying out 5.0.0b1 to ensure that the weewx-`weather34 skin I maintain is compatible and have noticed an oddity when moving my existing weewx.sdb database file across.

I installed using the pip installer method which was impressively smooth, ran for a while with the simulator, then installed the GW1000 Gateway driver. I then made a copy of my existing database file, stopped the new install and moved the copy across, restarted. 

On looking at the syslog, I noticed that there was an error writing to the database as the database file was reported as read only. At this point, I updated the ownership and permissions from the command line but got the same error report. I took another copy of the database file, same result. Eventually the only way I could get the database file into a writable condition was to use wee_database —drop-daily and —rebuild.

I have never had a problem of copying a mature database file across to a new installation before. It was doubly baffling as I assume that it would not be possible to use —drop-daily and —rebuild if the the database file was truly read only.

Ian

Tom Keffer

unread,
May 20, 2023, 5:34:41 PM5/20/23
to Steeple Ian, weewx-development
It would help if we could see the exact commands you were running, In particular, did you use an explicit path to a weewx.conf? What was in that configuration file?

Nothing has changed in the database file, but how it is found is different.

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/6f3eb1c5-a7d7-4086-a24a-05d941812511n%40googlegroups.com.

Steeple Ian

unread,
May 21, 2023, 11:31:19 AM5/21/23
to weewx-development
Thank you Tom,

I will be back home tomorrow evening and planning a fresh install on a clean OS so I will document as I go along and feedback.

Ian

Steeple Ian

unread,
May 22, 2023, 5:13:32 PM5/22/23
to weewx-development
Tom,

This is a complete report of the process I used

New development server up and running on RaspberryPi B+
For the sake of this exercise the user ID is igm
OS Raspberry PI OS Lite (64-Bit)
installed applications: -
    Webmin 2.021 (open source control panel-like appication)
   Apache 2

installed WeeWX 5.0.0b1: -
    sudo apt update && sudo apt upgrade
    sudo apt -y install gcc
    sudo apt -y install python3-dev
    sudo apt -y install python3-pip
    python3 -m pip install wheel
    python3 -m pip install weewx --user
    python3 -m pip install --upgrade ephem (the version of ephem that wheel installs 3.7.7.0 has a known bug, this upgrades to ephem 4.1.4)
    cd /home/igm/.local
    python3 ./weectl station create (selected simulator as driver and edited weewx.conf to set debug =2, HTML_ROOT = /var/www/html/weewx set in StdReport stanza. Complete weewx.conf  at the foot of this posting)
    cd ~/weewx-data sudo
    cp util/systemd/weewx.service /etc/systemd/system
    sudo systemctl daemon-reload
    sudo systemctl enable weewx
    sudo systemctl start weewx

After the completion of the first achive cycle, Seasons skin created perfectly and subsequently updating every 5min

Next stage was to stop WeeWX on my main server and make a copy of my working database file
Stopped WeeWX on my development server, moved the working database file copy to my development server in place of the existing database file.
To match the ownership, group and permissions: -
    cd /
    sudo
chown igm:igm /home/igm/weewx-data/archive/weewx.sdb
    sudo chmod 644 /home/igm/weewx-data/archive/weewx.sdb
    (checked in my control panel that attributes had been changed correctly)
    restarted weewx on the devlopment server and logged the below error messages
    (I tried sudo chmod 777 but still got the same errors)

I stopped WeeWX again and tried: -
   wee_database --update (but still the same errors on restart)

I then tried: -
    wee_database --drop-daily
    wee_database --rebuild

This combination then cleared the errors and the database started updating

May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Replace failed for database weewx.sdb: attempt to write a readonly database May 22 21:00:14 igm weewx[13207] ERROR weewx.manager: Unable to add record 2023-05-22 21:00:00 BST (1684785600) to database 'weewx.sdb': attempt to write a readonly database 


My weewx.conf file: -

# WEEWX CONFIGURATION FILE
#
# Copyright (c) 2009-2022 Tom Keffer <tke...@gmail.com>
# See the file LICENSE.txt for your rights.

##############################################################################

# This section is for general configuration information.

# Set to 1 for extra debug info, otherwise comment it out or set to zero
debug = 2

# Root directory of the weewx data file hierarchy for this station
WEEWX_ROOT = /home/igm/weewx-data

# Whether to log successful operations. May get overridden below.
log_success = True

# Whether to log unsuccessful operations. May get overridden below.
log_failure = True

# Do not modify this. It is used when installing and updating weewx.
version = 5.0.0b1

##############################################################################

#   This section is for information about the station.

[Station]
   
    # Description of the station location, such as your town:
    location = "Steeple Claydon, UK"
   
    # Latitude in decimal degrees. Negative for southern hemisphere
    latitude = 51.940
    # Longitude in decimal degrees. Negative for western hemisphere.
    longitude = -0.987
   
    # Altitude of the station, with the unit it is in. This is used only
    # if the hardware cannot supply a value.
    altitude = 88, meter    # Choose 'foot' or 'meter' for unit
   
    # Set to type of station hardware. There must be a corresponding stanza
    # in this file, which includes a value for the 'driver' option.
    station_type = Simulator
   
    # If you have a website, you may specify an URL. This is required if you
    # intend to register your station.
    #station_url = https://www.example.com
   
    # The start of the rain year (1=January; 10=October, etc.). This is
    # downloaded from the station if the hardware supports it.
    rain_year_start = 1
   
    # Start of week (0=Monday, 6=Sunday)
    week_start = 0

##############################################################################

[Simulator]
    # This section is for the weewx weather station simulator
   
    # The time (in seconds) between LOOP packets.
    loop_interval = 2.5
   
    # The simulator mode can be either 'simulator' or 'generator'.
    # Real-time simulator. Sleep between each LOOP packet.
    mode = simulator
    # Generator.  Emit LOOP packets as fast as possible (useful for testing).
    #mode = generator
   
    # The start time. Format is YYYY-mm-ddTHH:MM. If not specified, the default
    # is to use the present time.
    #start = 2011-01-01T00:00
   
    # The driver to use:
    driver = weewx.drivers.simulator

##############################################################################

#   This section is for uploading data to Internet sites

[StdRESTful]
   
    # Uncomment and change to override logging for uploading services.
    # log_success = True
    # log_failure = True
   
    [[StationRegistry]]
        # To register this weather station with weewx, set this to true,
        # then fill out option 'station_url', located in the [Station] section above.
        register_this_station = False
   
    [[AWEKAS]]
        # This section is for configuring posts to AWEKAS.
       
        # If you wish to post to AWEKAS, set the option 'enable' to true, then specify a username
        # and password. To guard against parsing errors, put the password in quotes.
        enable = false
        username = replace_me
        password = replace_me
   
    [[CWOP]]
        # This section is for configuring posts to CWOP.
       
        # If you wish to post to CWOP, set the option 'enable' to true,
        # then specify the station ID (e.g., CW1234).
        enable = false
        station = replace_me
        # If this is an APRS (radio amateur) station, specify the
        # passcode (e.g., 12345). Otherwise, ignore.
        passcode = replace_me
   
   
    [[PWSweather]]
        # This section is for configuring posts to PWSweather.com.
       
        # If you wish to post to PWSweather.com, set the option 'enable' to true, then specify a
        # station and password. To guard against parsing errors, put the password in quotes.
        enable = false
        station = replace_me
        password = replace_me
   
    [[WOW]]
        # This section is for configuring posts to WOW.
       
        # If you wish to post to WOW, set the option 'enable' to true, then specify a station and
        # password. To guard against parsing errors, put the password in quotes.
        enable = false
        station = replace_me
        password = replace_me
   
    [[Wunderground]]
        # This section is for configuring posts to the Weather Underground.
       
        # If you wish to post to the Weather Underground, set the option 'enable' to true,  then
        # specify a station (e.g., 'KORHOODR3') and password. To guard against parsing errors, put
        # the password in quotes.
        enable = false
        station = replace_me
        password = replace_me
       
        # If you plan on using wunderfixer, set the following
        # to your API key:
        api_key = replace_me
       
        # Set the following to True to have weewx use the WU "Rapidfire"
        # protocol. Not all hardware can support it. See the User's Guide.
        rapidfire = False

##############################################################################

#   This section specifies what reports, using which skins, to generate.

[StdReport]
   
    # Where the skins reside, relative to WEEWX_ROOT
    SKIN_ROOT = skins
   
    # Where the generated reports should go, relative to WEEWX_ROOT
    HTML_ROOT = /var/www/html/weewx
   
    # Uncomment and change to override logging for reports
    # log_success = True
    # log_failure = True
   
    # The database binding indicates which data should be used in reports.
    data_binding = wx_binding
   
    # Each of the following subsections defines a report that will be run.
    # See the customizing guide to change the units, plot types and line
    # colors, modify the fonts, display additional sensor data, and other
    # customizations. Many of those changes can be made here by overriding
    # parameters, or by modifying templates within the skin itself.
   
    [[SeasonsReport]]
        # The SeasonsReport uses the 'Seasons' skin, which contains the
        # images, templates and plots for the report.
        skin = Seasons
        enable = true
   
    [[SmartphoneReport]]
        # The SmartphoneReport uses the 'Smartphone' skin, and the images and
        # files are placed in a dedicated subdirectory.
        skin = Smartphone
        enable = false
        HTML_ROOT = public_html/smartphone
   
    [[MobileReport]]
        # The MobileReport uses the 'Mobile' skin, and the images and files
        # are placed in a dedicated subdirectory.
        skin = Mobile
        enable = false
        HTML_ROOT = public_html/mobile
   
    [[StandardReport]]
        # This is the old "Standard" skin. By default, it is not enabled.
        skin = Standard
        enable = false
   
    [[FTP]]
        # FTP'ing the results to a webserver is treated as just another report,
        # albeit one with an unusual report generator!
        skin = Ftp
       
        # If you wish to use FTP, set "enable" to "true", then
        # fill out the next four lines.
        # Use quotes around passwords to guard against parsing errors.
        enable = false
        user = replace_me
        password = replace_me
        server = replace_me    # The ftp server name, e.g, www.myserver.org
        path = replace_me    # The destination directory, e.g., /weather
       
        # Set to True for an FTP over TLS (FTPS) connection. Not all servers
        # support this.
        secure_ftp = False
       
        # To upload files from something other than what HTML_ROOT is set
        # to above, specify a different HTML_ROOT here.
        #HTML_ROOT = public_html
       
        # Most FTP servers use port 21
        port = 21
       
        # Set to 1 to use passive mode, zero for active mode
        passive = 1
   
    [[RSYNC]]
        # rsync'ing to a webserver is treated as just another report
        skin = Rsync
       
        # If you wish to use rsync, you must configure passwordless ssh using
        # public/private key authentication from the user account that weewx
        # runs to the user account on the remote machine where the files
        # will be copied.
        #
        # If you wish to use rsync, set "enable" to "true", then
        # fill out server, user, and path.
        # The server should appear in your .ssh/config file.
        # The user is the username used in the identity file.
        # The path is the destination directory, such as /var/www/html/weather.
        # Be sure that the user has write permissions on the destination!
        enable = false
        server = replace_me
        user = replace_me
        path = replace_me
       
        # To upload files from something other than what HTML_ROOT is set
        # to above, specify a different HTML_ROOT here.
        #HTML_ROOT = public_html
       
        # Rsync can be configured to remove files from the remote server if
        # they don't exist under HTML_ROOT locally. USE WITH CAUTION: if you
        # make a mistake in the remote path, you could could unintentionally
        # cause unrelated files to be deleted. Set to 1 to enable remote file
        # deletion, zero to allow files to accumulate remotely.
        delete = 0
   
    # Options in the [[Defaults]] section below will apply to all reports.
    # What follows are a few of the more popular options you may want to
    # uncomment, then change.
    [[Defaults]]
       
        # Which language to use for all reports. Not all skins support all languages.
        # You can override this for individual reports.
        lang = en
       
        # Which unit system to use for all reports. Choices are 'us', 'metric', or 'metricwx'.
        # You can override this for individual reports.
        unit_system = metricwx
       
        [[[Units]]]
           
            # Option "unit_system" above sets the general unit system, but overriding specific unit
            # groups is possible. These are popular choices. Uncomment and set as appropriate.
            # NB: The unit is always in the singular. I.e., 'mile_per_hour',
            # NOT 'miles_per_hour'
            [[[[Groups]]]]
                # group_altitude     = meter              # Options are 'foot' or 'meter'
                # group_pressure     = mbar               # Options are 'inHg', 'mmHg', 'mbar', or 'hPa'
                # group_rain         = mm                 # Options are 'inch', 'cm', or 'mm'
                # group_rainrate     = mm_per_hour        # Options are 'inch_per_hour', 'cm_per_hour', or 'mm_per_hour'
                # group_temperature  = degree_C           # Options are 'degree_C', 'degree_F', or 'degree_K'
                # The following line is used to keep the above lines indented properly.
                # It can be ignored.
                unused = unused
           
            # Uncommenting the following section frequently results in more
            # attractive formatting of times and dates, but may not work in
            # your locale.
            [[[[TimeFormats]]]]
                # day        = %H:%M
                # week       = %H:%M on %A
                # month      = %d-%b-%Y %H:%M
                # year       = %d-%b-%Y %H:%M
                # rainyear   = %d-%b-%Y %H:%M
                # current    = %d-%b-%Y %H:%M
                # ephem_day  = %H:%M
                # ephem_year = %d-%b-%Y %H:%M
                # The following line is used to keep the above lines indented properly.
                # It can be ignored.
                unused = unused
       
        [[[Labels]]]
            # Users frequently change the labels for these observation types
            [[[[Generic]]]]
                # inHumidity     = Inside Humidity
                # inTemp         = Inside Temperature
                # outHumidity    = Outside Humidity
                # outTemp        = Outside Temperature
                # extraTemp1     = Temperature1
                # extraTemp2     = Temperature2
                # extraTemp3     = Temperature3
                # The following line is used to keep the above lines indented properly.
                # It can be ignored.
                unused = unused

##############################################################################

#   This service acts as a filter, converting the unit system coming from
#   the hardware to a unit system in the database.

[StdConvert]
   
    # The target_unit affects only the unit system in the database. Once
    # chosen it cannot be changed without converting the entire database.
    # Modification of target_unit after starting weewx will result in
    # corrupt data - the database will contain a mix of US and METRIC data.
    #
    # The value of target_unit does not affect the unit system for
    # reporting - reports can display US, Metric, or any combination of units.
    #
    # In most cases, target_unit should be left as the default: US
    #
    # In particular, those migrating from a standard wview installation
    # should use US since that is what the wview database contains.
   
    # DO NOT MODIFY THIS VALUE UNLESS YOU KNOW WHAT YOU ARE DOING!
    target_unit = US    # Options are 'US', 'METRICWX', or 'METRIC'

##############################################################################

#   This section can adjust data using calibration expressions.

[StdCalibrate]
   
    [[Corrections]]
        # For each type, an arbitrary calibration expression can be given.
        # It should be in the units defined in the StdConvert section.
        # Example:
        foo = foo + 0.2

##############################################################################

#   This section is for quality control checks. If units are not specified,
#   values must be in the units defined in the StdConvert section.

[StdQC]
   
    [[MinMax]]
        barometer = 26, 32.5, inHg
        pressure = 24, 34.5, inHg
        outTemp = -40, 120, degree_F
        inTemp = 10, 120, degree_F
        outHumidity = 0, 100
        inHumidity = 0, 100
        windSpeed = 0, 120, mile_per_hour
        rain = 0, 10, inch

##############################################################################

#   This section controls the origin of derived values.

[StdWXCalculate]
   
    [[Calculations]]
        # How to calculate derived quantities.  Possible values are:
        #  hardware        - use the value provided by hardware
        #  software        - use the value calculated by weewx
        #  prefer_hardware - use value provide by hardware if available,
        #                      otherwise use value calculated by weewx
       
        pressure = prefer_hardware
        altimeter = prefer_hardware
        appTemp = prefer_hardware
        barometer = prefer_hardware
        cloudbase = prefer_hardware
        dewpoint = prefer_hardware
        ET = prefer_hardware
        heatindex = prefer_hardware
        humidex = prefer_hardware
        inDewpoint = prefer_hardware
        maxSolarRad = prefer_hardware
        rainRate = prefer_hardware
        windchill = prefer_hardware
        windrun = prefer_hardware

##############################################################################

#   For hardware that supports it, this section controls how often the
#   onboard clock gets updated.

[StdTimeSynch]
   
    # How often to check the weather station clock for drift (in seconds)
    clock_check = 14400
   
    # How much it can drift before we will correct it (in seconds)
    max_drift = 5

##############################################################################

#   This section is for configuring the archive service.

[StdArchive]
   
    # If the station hardware supports data logging then the archive interval
    # will be downloaded from the station. Otherwise, specify it (in seconds).
    archive_interval = 300
   
    # If possible, new archive records are downloaded from the station
    # hardware. If the hardware does not support this, then new archive
    # records will be generated in software.
    # Set the following to "software" to force software record generation.
    record_generation = hardware
   
    # Whether to include LOOP data in hi/low statistics
    loop_hilo = True
   
    # Uncomment and change to override logging for archive operations
    # log_success = True
    # log_failure = True
   
    # The data binding used to save archive records
    data_binding = wx_binding

##############################################################################

#   This section binds a data store to a database.

[DataBindings]
   
    [[wx_binding]]
        # The database must match one of the sections in [Databases].
        # This is likely to be the only option you would want to change.
        database = archive_sqlite
        # The name of the table within the database
        table_name = archive
        # The manager handles aggregation of data for historical summaries
        manager = weewx.manager.DaySummaryManager
        # The schema defines the structure of the database.
        # It is *only* used when the database is created.
        schema = schemas.wview_extended.schema

##############################################################################

#   This section defines various databases.

[Databases]
   
    # A SQLite database is simply a single file
    [[archive_sqlite]]
        database_name = weewx.sdb
        database_type = SQLite
   
    # MySQL
    [[archive_mysql]]
        database_name = weewx
        database_type = MySQL

##############################################################################

#   This section defines defaults for the different types of databases.

[DatabaseTypes]
   
    # Defaults for SQLite databases
    [[SQLite]]
        driver = weedb.sqlite
        # Directory in which the database files are located, relative to WEEWX_ROOT:
        SQLITE_ROOT = archive
   
    # Defaults for MySQL databases
    [[MySQL]]
        driver = weedb.mysql
        # The host where the database is located
        host = localhost
        # The user name for logging in to the host
        user = weewx
        # The password (use quotes to guard against parsing errors)
        password = weewx

##############################################################################

#   This section configures the internal weewx engine.

[Engine]
   
    # The following section specifies which services should be run and in what order.
    [[Services]]
        prep_services = weewx.engine.StdTimeSynch
        data_services = ,
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
        xtype_services = weewx.wxxtypes.StdWXXTypes, weewx.wxxtypes.StdPressureCooker, weewx.wxxtypes.StdRainRater, weewx.wxxtypes.StdDelta
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport



Ian


   

Tom Keffer

unread,
May 22, 2023, 5:22:51 PM5/22/23
to Steeple Ian, weewx-development
This is surely a straightforward permissions and ownership issue.

What do you see with

ls -l /home/igm/weewx-data/archive

What user is the daemon running as?

cat /etc/systemd/system/weewx.service

They should match.

-tk

Steeple Ian

unread,
May 22, 2023, 5:40:35 PM5/22/23
to weewx-development
Yes they do, that is what confuses me.

Tom Keffer

unread,
May 22, 2023, 6:00:29 PM5/22/23
to Steeple Ian, weewx-development
Humor me. Please post the output.

Steeple Ian

unread,
May 22, 2023, 7:38:14 PM5/22/23
to weewx-development
This is from before I used wee_database --drop-daily and wee_database --rebuild

igm@igm:~/weewx-data/archive $ ls -l /home/igm/weewx-data/archive
total 279980
-rw-r--r-- 1 igm igm 203984896 May 22 19:05 weewx.sdb
igm@igm:~/weewx-data/archive $ cat /etc/systemd/system/weewx.service
# systemd unit configuration file for WeeWX
#
# For information about running WeeWX under systemd,
# be sure to read https://github.com/weewx/weewx/wiki/systemd
#
[Unit]
Description=WeeWX weather system
Documentation=https://weewx.com/docs

Requires=time-sync.target
After=time-sync.target
RequiresMountsFor=/home

[Service]
ExecStart=/usr/bin/python3 /home/igm/.local/lib/python3.9/site-packages/weewxd.py /home/igm/weewx-data/weewx.conf
StandardOutput=null
# To run as a non-root user, uncomment and set username and group here:
User=igm
Group=igm

[Install]
WantedBy=multi-user.target

and this is after running the database  I used wee_database --drop-daily and wee_database --rebuild which rendered the database writeable

igm@igm:~ $ ls -l /home/igm/weewx-data/archive
total 279980
-rw-r--r-- 1 igm igm 203984896 May 23 00:30 weewx.sdb
igm@igm:~ $ cat /etc/systemd/system/weewx.service
# systemd unit configuration file for WeeWX
#
# For information about running WeeWX under systemd,
# be sure to read https://github.com/weewx/weewx/wiki/systemd
#
[Unit]
Description=WeeWX weather system
Documentation=https://weewx.com/docs

Requires=time-sync.target
After=time-sync.target
RequiresMountsFor=/home

[Service]
ExecStart=/usr/bin/python3 /home/igm/.local/lib/python3.9/site-packages/weewxd.py /home/igm/weewx-data/weewx.conf
StandardOutput=null
# To run as a non-root user, uncomment and set username and group here:
User=igm
Group=igm

[Install]
WantedBy=multi-user.target

As far as I can see they are both the same output

Tom Keffer

unread,
May 22, 2023, 7:44:18 PM5/22/23
to Steeple Ian, weewx-development
They do look the same.

I should have asked earlier, but how about the log from a case where it failed? Set debug=1, put the new database in place, run to failure. Post the log.

It is also possible that it's not using the database you think it's using.

Steeple Ian

unread,
May 22, 2023, 8:13:27 PM5/22/23
to weewx-development
I have to roll everything back to the state I had the development server in when the error occured. Its very late in the UK now so will do that tomorrow if that is OK?

Do you require the log of the entire 5min archive cycle or just the snippet of the error?

Graham Eddy

unread,
May 22, 2023, 11:11:01 PM5/22/23
to weewx-development
write perm on directory for sqlite3 updates?
⊣GE⊢

Tom Keffer

unread,
May 23, 2023, 7:24:18 AM5/23/23
to Graham Eddy, weewx-development
Not sure what you mean by that, Graham. Permissions on a sqlite database follow file permissions --- there's no special "update" permission like there is on MySQL.  Or, do you mean something else?

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

Graham Eddy

unread,
May 23, 2023, 7:26:50 AM5/23/23
to weewx-development
i have a vague memory that to update db file via sqlite3 API you need to have write perm on the file’s directory
⊣GE⊢

Tom Keffer

unread,
May 23, 2023, 7:43:30 AM5/23/23
to Graham Eddy, weewx-development
That's true. 

Ian: check the permissions and ownership of the parent directory, /home/igm/weewx-data/archive.

In the end, this problem will almost surely be due to some overlooked permissions issue.

Steeple Ian

unread,
May 23, 2023, 5:00:10 PM5/23/23
to weewx-development
Tom, Graham,

Yes that was the issue, with sqlite3, you have to change the permissions on the containing directory.

Armed with that information I have found a way which appears to be robust. Instead of copying the compressed file across from my main server, I crompressed the archive folder/contents, renamed the archive folder on my development server and extracted the compressed folder in its place. Did not even have to set any permissions as it took up the recursive permissions of the weewx-data folder. I have tried this a dozen times and it worked every time.

I wonder whether it is worth making a note of this in the documentation as I am sure I won't be last person wanting to move across an existing database file to a new install of WeeWX 5 when it is released.

As always thanks very much for your assistance.

Ian

Steeple Ian

unread,
May 23, 2023, 5:26:08 PM5/23/23
to weewx-development

Peter Fletcher

unread,
May 24, 2023, 2:15:42 PM5/24/23
to weewx-development
FWIW, sqlite3 needs write permissions to the directory because it writes certain (very) temporary files there during database write and update operations.
Reply all
Reply to author
Forward
0 new messages