Help with rsync setup for weewx user

[Shane Burkhardt]

I apologize if similar questions have been posted before, but I have not found anything recent and am really banging my head against the wall. I have rsync set-up but it still prompts for passwords for weewx or root users, although the key pair works fine for me as user. I have run keygen as weewx and as root. It looks like it saves root in the /root/.ssh directory but it saves weewx in the /var/lib/weewx/.ssh directory. I have copied the relevant public keys to the authorized keys file for the user on the remote server I am connecting to. Still no luck. I saw very old postings about a config file, but cannot figure out what that is or find recent documentation of what that might be or where that should go.

Any help would be much appreciated!

I am running weewx on Ubuntu 24.02.02 VM installed via APT. It is using the weewx user to run. My weather station is a WS5000-IP connected through the GX1000 driver. The Weewx version is 5.10.

Here is a snippet from the log:

May 13 15:00:23 asok weewxd[81319]: ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '-e', 'ssh', '/var/www/weewx>
May 13 15:00:23 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
May 13 15:00:23 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
May 13 15:00:23 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** XX...@my.host.com: Permission denied (publickey,password).
May 13 15:00:23 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
May 13 15:00:23 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(232) [sender=3.2.7]
May 13 15:00:25 asok weewxd[81319]: DEBUG user.gw1000: Next update in 5 seconds

[John Kline]

I assume weewx is running as the weewx user.  What is the remote user specified in your RSYNC section?  It might be best to post your RSYNC section of weewx.conf.  It is unlikely to contain a password.

On May 13, 2025, at 5:47 PM, Shane Burkhardt <ing...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/b17c5f1d-74fd-4890-bb70-76c0023966fdn%40googlegroups.com.

[vince]

[....short answer...]

There is nothing special.  It is ssh-101.  You need passwordless ssh set up as a prerequisite for rsync-over-ssh to work.  Nothing weewx-specific at all there.

[...longer answer...]

Simplest test is to ssh into the remote host using the private key of the user you want weewx to rsync as. The incantation should be

      "ssh -i /var/www/weewx/.ssh/weewx_account_private_key_filename_here my.host.com date"  

to try to ssh in and run the date command on the far side, possibly adding the -v switch to provide more debugging if it doesn't work right away.

(obviously edit in your actual filename and remote hostname or ip address above)

I don't run the apt variant, so if you run this as other than user 'weewx' you should (should) get a permission denied even trying to read the weewx user's private key as any other non-privileged user.  If you run "sudo bash" to open a root shell first and 'then'  the test above, it should work hopefully if your keys are set up correctly in weewx's .ssh tree.   You'll likely get a prompt asking you to accept the remote side's host key into that .ssh tree's known_hosts file.

I don't remember what weewx's rsync defaults are but I personally always fully specify everything just to be sure I know what it's going to do rather than relying on app defaults.

So from weewx.conf:

        # If you wish to use rsync, set "enable" to "true", then
        # fill out server, user, and path.
        # The server should appear in your .ssh/config file.

Meaning.....you might need a /var/lib/weewx/.ssh/config file entry for your remote system.

Mine looks like the following, FWIW:

Host myhostname.domain.com nnn.nnn.nnn.nnn          <= edit
 IdentityFile ~/.ssh/my_private_key_filename        <= edit
 user remote_user_on_server                         <= edit
 hostname nnn.nnn.nnn.nnn                           <= edit

I put the FQDN and ip addresses in there mainly for historical reasons and use the ip address in weewx.conf so I don't need to rely on DNS working.

   [[RSYNC]]
        delete = 0
        skin = Rsync
        enable = true
        server = nnn.nnn.nnn.nnn                           <= edit (ip address or FQDN here)
        user = remote_user_on_server                       <= edit
        path = /server_side/full/path/to/rsync/into        <= edit
        log_success = false
        log_failure = true

 

Hope this helps.  Basically if you can ssh in as the weewx user using "its" private key you specified in its .ssh/config file, rsync should work too.

[Shane Burkhardt]

The remote user is my account on the remote server.

    [[RSYNC]]
        # rsync'ing to a webserver is treated as just another report.
        skin = Rsync

        # If you wish to use rsync, you must configure passwordless ssh using
        # public/private key authentication from the user account that weewx
        # runs to the user account on the remote machine where the files
        # will be copied.
        #

        # If you wish to use rsync, set "enable" to "true", then
        # fill out server, user, and path.
        # The server should appear in your .ssh/config file.

        # The user is the username used in the identity file.
        # The path is the destination directory, such as /var/www/html/weather.
        # Be sure that the user has write permissions on the destination!
        enable = true
        server = my.host.com
        user = shane
        path = /var/www/weather

        # To upload files from something other than what HTML_ROOT is set
        # to above, specify a different HTML_ROOT here.
        HTML_ROOT = /var/www/weewx

        # Rsync can be configured to remove files from the remote server if
        # they don't exist under HTML_ROOT locally. USE WITH CAUTION: if you
        # make a mistake in the remote path, you could could unintentionally
        # cause unrelated files to be deleted. Set to 1 to enable remote file
        # deletion, zero to allow files to accumulate remotely.
        delete = 0

I am sure this is an rsync setup issue as I can "sudo -u shane weectl report run RSYNC" and it works likt it is supposed to when it is run under my username. But I cannot get passwordless to work for either root or the weewx user if I was to run "sudo -u weewx weectl report run RSYNC". I think I am at a loss as to whether /var/lib/weewx/.ssh is the correct place that the keys should go (this is where keygen put them) and even though I installed the public key on my remote server under my user "shane", it does not work and still prompts for a password.

[Shane Burkhardt]

There is nothing special.  It is ssh-101.  You need passwordless ssh set up as a prerequisite for rsync-over-ssh to work.  Nothing weewx-specific at all there.

Yup I understand that. I can get passwordless to work with my account "shane" but I cannot get passwordless to work with either root or weewx.

Simplest test is to ssh into the remote host using the private key of the user you want weewx to rsync as. The incantation should be

      "ssh -i /var/www/weewx/.ssh/weewx_account_private_key_filename_here my.host.com date"  

So that did uncover that the permissions for /var/lib/weewx/.ssh were not set appropriately. I did set them as chmod 0700 following other online websites on how to set up rsync, but apparently that was not permissive enough. After resolving the permissions issue, I was able to see the date, but then when executing:

"sudo -u weewx weectl report run RSYNC". it still prompted for the password. This is leading me to think that Weewx is not looking for the key files where keygen put them. I don't know where Weewx is looking for them. This is one of the undocumented mysteries (or if it is documented, i don't know where.).

I don't run the apt variant, so if you run this as other than user 'weewx' you should (should) get a permission denied even trying to read the weewx user's private key as any other non-privileged user.  If you run "sudo bash" to open a root shell first and 'then'  the test aboveshane, it should work hopefully if your keys are set up correctly in weewx's .ssh tree.   You'll likely get a prompt asking you to accept the remote side's host key into that .ssh tree's known_hosts file.

lol yes permissions were an issue. Yes I think the problem is with user "weewx" .ssh tree. I don't think where keygen puts the files, and where Weewx is looking for the files is correct. Or maybe it is, I don't know. If I run keygen as the weewx user, it puts the files in "/var/lib/weewx/.ssh". This is where I pulled to install that public key onto the remote server so those keys match. running the command above that actually points to the private key works.

So from weewx.conf:

        # If you wish to use rsync, set "enable" to "true", then
        # fill out server, user, and path.
        # The server should appear in your .ssh/config file.

This is the part I was struggling with. It was "what is this config file thing and where does it go". Apparently I did not know what to search for online to find documentation on this as any search for config immediately pulled up crap for running rsync in daemon mode with rsyncd.conf and not what this config file was.

Mine looks like the following, FWIW:

Host myhostname.domain.com nnn.nnn.nnn.nnn          <= edit
 IdentityFile ~/.ssh/my_private_key_filename        <= edit
 user remote_user_on_server                         <= edit
 hostname nnn.nnn.nnn.nnn                           <= edit

Ok so I created the config file as "/var/lib/weewx/.ssh/config"

Here is the file content:

Host my.host.com
   IdentityFile /var/lib/weewx/.ssh/weewx
   user shane
   hostname my.host.com

In trying to test this with "sudo -u weewx weectl report run RSYNC" it is still prompting for a password. Is there supposed to be the indentation on the last 3 lines? I tried it with, and without.

I put the FQDN and ip addresses in there mainly for historical reasons and use the ip address in weewx.conf so I don't need to rely on DNS working.

   [[RSYNC]]
        delete = 0
        skin = Rsync
        enable = true
        server = nnn.nnn.nnn.nnn                           <= edit (ip address or FQDN here)
        user = remote_user_on_server                       <= edit
        path = /server_side/full/path/to/rsync/into        <= edit
        log_success = false
        log_failure = true

 

This is from my weewx.conf:

        skin = Rsync
        enable = true

        server = my.host.com
        user = shane
        path = /var/www/weather

        HTML_ROOT = /var/www/weewx
        delete = 0

I went ahead and added the log_success and log_failure, but it was doing that anyway as I have debugging set. Unfortunately, still getting the same error in the log:

May 13 20:10:20 asok weewxd[81319]: DEBUG weeutil.rsyncupload: rsyncupload: cmd: [['rsync', '--archive', '--stats', '-e', 'ssh', '/var/www/weewx/', 'sh...@my.host>
May 13 20:10:21 asok weewxd[81319]: ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '-e', 'ssh', '/var/www/weewx>
May 13 20:10:21 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
May 13 20:10:21 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
May 13 20:10:21 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** sh...@my.host.com: Permission denied (publickey,password).
May 13 20:10:21 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
May 13 20:10:21 asok weewxd[81319]: ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(232) [sender=3.2.7]

[vince]

I emailed you.

In the interim try ‘ssh-copy-id’ to get the weewx user public key correctly into the remote system’s account authorized_keys file.

[Shane Burkhardt]

Thanks everyone for your help... sorry on this tardy post as well. Thank you @vince for your offline help! c

It seemed to be a combination of factors I was dealing with. Although I had followed some great online instructions on setting up rsync, none of them really discussed config files. A combination of getting that set up properly, plus not screwing up the permissions on the file (which of course I did) was what it took to get rsync working properly. I found out:

That the weewx user, being a special user, was still not getting permissions right when i ran tests using sudo -u... I ended up having to give weewx bash access, and then run su - weewx.

Running keygen and copy with sudo -u was not writing keys properly... needed to be run under su - weewx or copy paste manually to the proper locations.

The weewx user defaulted to /var/lib/weewx for its home directory.

I had to make sure the permissions and ownership of the config file was for weewx (duh!).

 

On Tuesday, May 13, 2025 at 3:47:48 PM UTC-7 Shane Burkhardt wrote: