Rsync issue

[Benjamin Trey Blue]

Hey all,

I installed (DEB package install) and configured weewx on RPi last week and after enjoying it on my LAN, I am now trying to set-up rsync to send it to a website.

I called myself following the rsync guide at http://www.weewx.com/docs/usersguide.htm#config_RSYNC and setting up passwordless SSH using the link provided there (https://www.tecmint.com/ssh-passwordless-login-using-ssh-keygen-in-5-easy-steps/) but I cannot get it to successfully sync when wee_reports runs automatically. In syslog I get the following:

May 28 07:41:31 raspberrypi weewx[698] ERROR weeutil.rsyncupload: [['rsync', '--archive', '--stats', '-e ssh', '/var/www/html/', 'US...@mydomain.com:/home4/USER/public_html']] reported errors: Permission denied, please try again.. Received disconnect from XX.XX.XX.XX port 22:2: Too many authentication failures for USER. Disconnected from XX.XX.XX.XX port 22. rsync: connection unexpectedly closed (0 bytes received so far) [sender]. rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.3]

My weewx.conf is as follows:

   [[RSYNC]]

        skin = Rsync

        enable = true

        server = mydomain.com

        user = USER

        path = /home4/USER/public_html/

        HTML_ROOT = /var/www/html/

        delete = 0

Strangely, whenever I run the following commands manually, rsync works and my site updates:

 

sudo wee_reports 

rsync --archive -e ssh /var/www/html/ US...@mydomain.com:/home4/USER/public_html/ 

sudo rsync --archive -e ssh /var/www/html/ US...@mydomain.com:/home4/USER/public_html/ 

 

sudo wee_reports simply generates

May 28 11:15:39 raspberrypi wee_reports[8119] INFO weeutil.rsyncupload: rsync'd 81 files (1,174,239 bytes) in 5.17 seconds

Also, I haven't made any changes to the default configuration of weewx in terms of the user account it runs with.

I appreciate any advice or troubleshooting steps that you all can point me towards.

Thank you,
Trey 

 

[John Kline]

Sorry, didn’t mean to send that yet:

Should read:

Does it work if you add the following to the SSH configuration in weewx.conf?

[[SSH]]

    ssh_options = -o IdentitiesOnly=yes

On May 28, 2020, at 8:34 AM, John Kline <jo...@johnkline.com> wrote:



Does it work if you add:

[[SSH]]

ssh_options = -o IdentitiesOnly=yes

-o IdentitiesOnly=yes

[Benjamin Trey Blue]

Thanks for the reply - I do not see a section for [[SSH]] in my weewx.conf - do I nest that under the same [StdReport] section where [[RSYNC]] is found? I pasted it just below the end of [[RSYNC]] but can't see where the option was passed along:

May 28 11:45:58 raspberrypi weewx[8853] ERROR weeutil.rsyncupload: [['rsync', '--archive', '--stats', '-e ssh', '/var/www/html/', 'US...@mydomain.com:/home4/USER/public_html']] reported errors: Permission denied, please try again.. Received disconnect from XX.XX.XX.XX port 22:2: Too many authentication failures for USER. Disconnected from XX.XX.XX.XX port 22. rsync: connection unexpectedly closed (0 bytes received so far) [sender]. rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.3]

[John Kline]

Oops, I meant the [[RSYNC]] section.

[[RSYNC]]

    ssh_options = -o IdentitiesOnly=yes

[[SSH]]

    ssh_options = -o IdentitiesOnly=yes

On May 28, 2020, at 8:56 AM, Benjamin Trey Blue <tblue...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/76ef08ae-203e-4caf-9cfa-616faa16be67%40googlegroups.com.

[Benjamin Trey Blue]

Ahhh, got it - thank you. Added it there and I can see it being passed, but the result is the same:

May 28 12:51:17 raspberrypi weewx[10689] ERROR weeutil.rsyncupload: [['rsync', '--archive', '--stats', '-e ssh -o IdentitiesOnly=yes', '/var/www/html/', 'US...@mydomain.com:/home4/USER/public_html']] reported errors: Permission denied, please try again.. Received disconnect from XX.XX.XX.XX port 22:2: Too many authentication failures for USER. Disconnected from XX.XX.XX.XX port 22. rsync: connection unexpectedly closed (0 bytes received so far) [sender]. rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.3]

Manually invoked with sudo wee_reports again and it worked:

May 28 12:57:23 raspberrypi wee_reports[10914] INFO weeutil.rsyncupload: rsync'd 112 files (1,461,848 bytes) in 7.24 seconds 

 

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[John Kline]

That’s too bad.  I believe you are getting this error because it is trying too many SSH keys.  IdentitiesOnly should have fixed that.

On May 28, 2020, at 10:01 AM, Benjamin Trey Blue <tblue...@gmail.com> wrote:



To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/215768b7-1c70-447b-9f14-013db031ee89%40googlegroups.com.

[Tom Keffer]

Are you running wee_reports as the same user as weewxd? Perhaps the latter does not have access to the keys?

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/215768b7-1c70-447b-9f14-013db031ee89%40googlegroups.com.

[Benjamin Trey Blue]

Thanks for the effort, anyway, John!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/215768b7-1c70-447b-9f14-013db031ee89%40googlegroups.com.

[Benjamin Trey Blue]

Hey Tom,

I haven't made any changes to the user account that weewx uses by default - I had assumed from my reading that it defaults to root. The only two non-system accounts on the RPi are root and pi.

Sudo wee_reports works from the terminal though, so there has to be some discrepancy with my assumptions, but I'm too Linux-dumb to have figured it out yet and I'm not really sure how to go about even troubleshooting it further. 

Should I maybe consider setting-up weewx to run as a non-root user and then reconfigure the passwordless SSH, assuming I can get that bit restored to a blank slate?

Thank you,

Trey

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/215768b7-1c70-447b-9f14-013db031ee89%40googlegroups.com.

[Tom Keffer]

That's basically what I do. The process weewxd gets run as user 'weewx', with the .ssh subdirectory in its home directory. See the wiki article Run as a non-root user.

But, you can do it as root. Just make sure the proper credentials are in the home directory for root.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/cffaa2cc-5041-4a9c-9120-4e0cb7a2b335%40googlegroups.com.

[vince]

On Thursday, May 28, 2020 at 3:16:04 PM UTC-7, Benjamin Trey Blue wrote:

...but I'm too Linux-dumb to have figured it out yet and I'm not really sure how to go about even troubleshooting it further. 

Should I maybe consider setting-up weewx to run as a non-root user and then reconfigure the passwordless SSH, assuming I can get that bit restored to a blank slate?

If you consider yourself "Linux-dumb" then perhaps you might want to just run the defaults.   Just make sure root's .ssh directory has the right setup for rsync to be able to log into the remote computer. 

[Benjamin Trey Blue]

Thank you, Tom - I decided to clean-up all the SSH configs on the webhost's server and my RPi and then worked back through the process - it's working now!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/cffaa2cc-5041-4a9c-9120-4e0cb7a2b335%40googlegroups.com.

[Ossian]

Hi All,

I am having a similar issue. I keep seeing mention of the need or possibility to provide correct keys etc for root to use, but have not found anything concrete to go on.

default installation of weewx:

weewx running as root (ps aux) shows this     root       17898  0.3  1.0 123032 40900 ?        S    15:41   0:01 python3 /usr/share/weewx/weewxd --daemo

ssh passwordless works perfectly for my user

i have tried by coping id_rsa to /root/.ssh

i have modified /etc/ssh/ssh_conf to point to /home/me/.ssh/id_rsa

i copied id_rsa to /etc/ssh

i pointed /etc/ssh/ssh_conf to use /etc/ssh/id_rsa

i even tried to paste id_rsa.pub contents to /root/.ssh/authorized_keys

im going mad. I keep seeing it stated that it is possible to tell root to use other keys, but so far no joy.

i have searched and searched, 

Running on Ubuntu 20.4 server, local and target

Jun 14 15:56:16 local weewx[17898] DEBUG weeutil.rsyncupload: rsyncupload: cmd: [['rsync', '--archive', '--stats', '-e ssh', '/var/www/html/weewx/', 'us...@192.168.0.252:var/www/html/weather']]

Jun 14 15:56:16 local weewx[17898] ERROR weeutil.rsyncupload: [['rsync', '--archive', '--stats', '-e ssh', '/var/www/html/weewx/', 'us...@192.168.0.252:var/www/html/weather']] reported errors: Host key verification failed.. rsync: connection unexpectedly closed (0 bytes received so far) [sender]. rsync error: error in rsync protocol data stream (code 12) at io.c(235) [sender=3.1.3]

Jun 14 15:56:16 local weewx[17898] INFO weeutil.rsyncupload: rsync executed in 0.03 seconds

[[RSYNC]]

skin = Rsync

enable = true

        server = 192.168.0.252

        user = user

        path = /var/www/html/weather/

I have changed the username and server name to user and local for this post

I can create content in the target folder using that user

any help very gratefully received, Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/cffaa2cc-5041-4a9c-9120-4e0cb7a2b335%40googlegroups.com.

[mwall]

On Sunday, June 14, 2020 at 12:18:28 PM UTC-4, Ossian wrote:

I am having a similar issue. I keep seeing mention of the need or possibility to provide correct keys etc for root to use, but have not found anything concrete to go on.

there are many ways to do this, for example, one approach is with command-line arguments, while another approach is to use the ssh config file.

lets say that you connect to the server 'server.example.com' via rsync as the user 'weewx' using the credentials in these files:

/home/pi/.ssh/id_rsa_weewx

/home/pi/.ssh/id_rsa_weewx.pub

lets say that you are running weewx as the the user 'root', whose home directory is /root

1) ensure that root has an ssh directory

sudo mkdir -p /root/.ssh

2) copy the credential files into root's ssh space

sudo cp /home/pi/.ssh/id_rsa_weewx /root/.ssh

sudo cp /home/pi/.ssh/id_rsa_weewx.pub /root/.ssh

3) create the ssh configuration file /home/root/.ssh/config with these contents:

Host server.example.com

    User weewx  # this is the username used to connect to the server

    IdentityFile /home/root/.ssh/id_rsa_weewx  # credentials used to connect to the server

4) finally, ensure that permissions are correct:

sudo chown root.root /home/root/.ssh

sudo chmod 700 /home/root/.ssh

sudo chmod 600 /home/root/.ssh/config

sudo chmod 600 /home/root/.ssh/id_rsa*

now when the user 'root' attempts a connection to server.example.com, it will use the correct credentials, no matter how ssh, scp, or rsync is invoked

if you run weewx as some other, user say 'pi' or 'weewx', then put the files into that user's .ssh directory instead of root's .ssh directory.

m

[Ossian]

Awesome, thanks so much for being so patient and providing crystal clear instrustions. I believe I was 99% there, but had not reset the permissions on roots ssh folder.

In the meantime I have found that following the instructions for systemd to https://github.com/weewx/weewx/wiki/systemd

with a bit of additional permsission changing got me running weewx as a non root user, then the next part of creating the ssh keys and ssh-copy-id etc was very easy and it just worked.

I have been doing this on a test virtual machine to make sure all is good before I touch the live system.

Thanks again!

Oz