rsync: host key verification failed - problem
735 views
Skip to first unread message
Tomasz Lewicki
Jul 20, 2023, 4:06:54 PM7/20/23
to weewx-user
rsync: host key verification failed
Dear Weewx users.
I'm trying to send files generated by Weewx to external webserver (paid hosting) with rsync. I'm familiar with rsync, ssh and Linux. Weewx is running on Raspberry Pi as 'pi' user. Here are my configs:
/home/pi/.ssh/config
Host CF
# HostName external.domain.com <- here of course is real web address
HostName 10.20.30.40 <- here of course is real IP
Port 222
User acetone <- this is my username on remote webserver (hosting)
/etc/weewx/weewx.conf
(...)
[[RSYNC]]
# rsync'ing to a webserver is treated as just another report
skin = Rsync
# If you wish to use rsync, you must configure passwordless ssh using
# public/private key authentication from the user account that weewx
# runs to the user account on the remote machine where the files
# will be copied.
#
# If you wish to use rsync, set "enable" to "true", then
# fill out server, user, and path.
# The server should appear in your .ssh/config file.
# The user is the username used in the identity file.
# The path is the destination directory, such as /var/www/html/weather.
# Be sure that the user has write permissions on the destination!
enable = true
server = external.domain.com
port = 222
user = acetone
path = domains/external.domain.com/private_html/meteo/
# To upload files from something other than what HTML_ROOT is set
# to above, specify a different HTML_ROOT here.
#HTML_ROOT = /var/www/html/weewx
# Rsync can be configured to remove files from the remote server if
# they don't exist under HTML_ROOT locally. USE WITH CAUTION: if you
# make a mistake in the remote path, you could could unintentionally
# cause unrelated files to be deleted. Set to 1 to enable remote file
# deletion, zero to allow files to accumulate remotely.
delete = 0
(...)
I can log in to external server with ssh (passwordless) with command:
$ ssh -p 222 ace...@external.domain.com
or just
$ ssh CF <- this is host from .ssh/config
I got error message from weewx:
Jul 20 21:36:04 WeewxFR24 weewx[28079] DEBUG weeutil.rsyncupload: rsyncupload: cmd: [['rsync', '--archive', '--stats', '-e', 'ssh -p 222', '/var/www/html/weewx/', 'ace...@external.domain.com:~/domains/external.domain.com/private_html/meteo']]
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '-e', 'ssh -p 222', '/var/www/html/weewx/', 'ace...@external.domain.com:~/domains/external.domain.com/private_html/meteo']
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: **** Host key verification failed.
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.3]
I can send single file or whole directory with this command applied directly from CLI:
$ rsync --archive --stats -e 'ssh external.domain.com -p 222 -l acetone' /var/www/html/weewx/ CF:domains/external.domain.com/private_html/meteo/
I can also send files with simpler command (no additional options for -e ssh because they are in .ssh/config):
$ rsync --archive --stats -e ssh /var/www/html/weewx/ CF:domains/external.domain.com/private_html/meteo/
But *not* with:
rsync --archive --stats -e ssh /var/www/html/weewx/ ace...@external.domain.com:domains/external.domain.com/private_html/meteo
And *not* with:
rsync --archive --stats -e ssh /var/www/html/weewx/ external.domain.com:domains/external.domain.com/private_html/meteo
As you can see, if .ssh/config is properly filled with data, it is much simpler to 'do' rsync. But why it works when executed manually but not from inside Weewx? Keys are valid if I can log in without password from CLI. Maybe you could do [[RSYNC]] option more flexible and use .ssh/config?
Dear Weewx users.
I'm trying to send files generated by Weewx to external webserver (paid hosting) with rsync. I'm familiar with rsync, ssh and Linux. Weewx is running on Raspberry Pi as 'pi' user. Here are my configs:
/home/pi/.ssh/config
Host CF
# HostName external.domain.com <- here of course is real web address
HostName 10.20.30.40 <- here of course is real IP
Port 222
User acetone <- this is my username on remote webserver (hosting)
/etc/weewx/weewx.conf
(...)
[[RSYNC]]
# rsync'ing to a webserver is treated as just another report
skin = Rsync
# If you wish to use rsync, you must configure passwordless ssh using
# public/private key authentication from the user account that weewx
# runs to the user account on the remote machine where the files
# will be copied.
#
# If you wish to use rsync, set "enable" to "true", then
# fill out server, user, and path.
# The server should appear in your .ssh/config file.
# The user is the username used in the identity file.
# The path is the destination directory, such as /var/www/html/weather.
# Be sure that the user has write permissions on the destination!
enable = true
server = external.domain.com
port = 222
user = acetone
path = domains/external.domain.com/private_html/meteo/
# To upload files from something other than what HTML_ROOT is set
# to above, specify a different HTML_ROOT here.
#HTML_ROOT = /var/www/html/weewx
# Rsync can be configured to remove files from the remote server if
# they don't exist under HTML_ROOT locally. USE WITH CAUTION: if you
# make a mistake in the remote path, you could could unintentionally
# cause unrelated files to be deleted. Set to 1 to enable remote file
# deletion, zero to allow files to accumulate remotely.
delete = 0
(...)
I can log in to external server with ssh (passwordless) with command:
$ ssh -p 222 ace...@external.domain.com
or just
$ ssh CF <- this is host from .ssh/config
I got error message from weewx:
Jul 20 21:36:04 WeewxFR24 weewx[28079] DEBUG weeutil.rsyncupload: rsyncupload: cmd: [['rsync', '--archive', '--stats', '-e', 'ssh -p 222', '/var/www/html/weewx/', 'ace...@external.domain.com:~/domains/external.domain.com/private_html/meteo']]
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '-e', 'ssh -p 222', '/var/www/html/weewx/', 'ace...@external.domain.com:~/domains/external.domain.com/private_html/meteo']
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: **** Host key verification failed.
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
Jul 20 21:36:04 WeewxFR24 weewx[28079] ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.3]
I can send single file or whole directory with this command applied directly from CLI:
$ rsync --archive --stats -e 'ssh external.domain.com -p 222 -l acetone' /var/www/html/weewx/ CF:domains/external.domain.com/private_html/meteo/
I can also send files with simpler command (no additional options for -e ssh because they are in .ssh/config):
$ rsync --archive --stats -e ssh /var/www/html/weewx/ CF:domains/external.domain.com/private_html/meteo/
But *not* with:
rsync --archive --stats -e ssh /var/www/html/weewx/ ace...@external.domain.com:domains/external.domain.com/private_html/meteo
And *not* with:
rsync --archive --stats -e ssh /var/www/html/weewx/ external.domain.com:domains/external.domain.com/private_html/meteo
As you can see, if .ssh/config is properly filled with data, it is much simpler to 'do' rsync. But why it works when executed manually but not from inside Weewx? Keys are valid if I can log in without password from CLI. Maybe you could do [[RSYNC]] option more flexible and use .ssh/config?
Warren Gill
Jul 20, 2023, 4:58:39 PM7/20/23
to weewx...@googlegroups.com
I took all the file transfer bits out of weewx and just run the rsync command in a crontab. It seems more reliable, and easier to troubleshoot.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4616524f-f3dd-4485-aa98-5ba3a97a7adcn%40googlegroups.com.
vince
Jul 20, 2023, 5:24:20 PM7/20/23
to weewx-user
Unless you did something custom, weewx runs as root, not as user 'pi'.
You need 'root' to be able to ssh into the remote system with the user+key you specified. Same procedure you did to get 'pi' to work, just do it after sudo(ing) to root so you set the 'root' account up similiarly. Two minute thing to do.
Tomasz Lewicki
Jul 21, 2023, 6:14:24 AM7/21/23
to weewx-user
Thank you for replies.
@Warren Gill: yes, I was thinking about crontab but wanted to use built-in solution. Of course crontab is good as backup solution.
@vince: you're right, I didn't think about it from this side. Just for record for future seekers:
1. I logged as root ('sudo -i' from 'pi' account)
2. generated SSH keys ('ssh-keygen')
3. copied them to external server ('ssh-copy-id ace...@external.domain.com -p 222')
4. copied /home/pi/.ssh/config to /root/.ssh/config
5. changed owner of 'config' ('chown root:root /root/.ssh/config')
6. waited for next synchronization
7. smiled because everything worked as expected :)
Kalju
Feb 8, 2024, 3:33:14 AM2/8/24
to weewx-user
Hi all,
Facing same issue with ERROR weeutil.rsyncupload: **** Host key verification failed error.
Facing same issue with ERROR weeutil.rsyncupload: **** Host key verification failed error.
However, it's really strange...
I can ssh to remote site as both root as well as pi user
I can do manual rsync with no problems but weewx builtin rsync returns above error
At the moment my weewx installation is syncing using cron with no problems but I want to understand why builtin rsync fails
Problem started after upgrade from weewx 5.0.0 to 5.0.1
Problem started after upgrade from weewx 5.0.0 to 5.0.1
Any ideas?
Thank you forward :-)
Tomasz Lewicki
Feb 8, 2024, 3:44:06 AM2/8/24
to weewx-user
In my weewx.conf I have a section:
[[RSYNC]]
enable = true
server = XYZ # It's only identifier needed for file described below. Can be anything, for example MY_SECRET_SERVER
path = /path/to/files/on/my/remote/server # for example: /domains/my-domain/public_html/pws
And in /root/.ssh I have file named simply 'config':
Host XYZ
HostName my.remote.server.com # HostName can be also direct IP address
Port 45678 # Port to SSH if not standard 22
User me # name of your user on remote server
And rsync from inside of Weewx works in my case.
[[RSYNC]]
enable = true
server = XYZ # It's only identifier needed for file described below. Can be anything, for example MY_SECRET_SERVER
path = /path/to/files/on/my/remote/server # for example: /domains/my-domain/public_html/pws
And in /root/.ssh I have file named simply 'config':
Host XYZ
HostName my.remote.server.com # HostName can be also direct IP address
Port 45678 # Port to SSH if not standard 22
User me # name of your user on remote server
And rsync from inside of Weewx works in my case.
Dominic Reich
Feb 8, 2024, 4:49:48 AM2/8/24
to weewx...@googlegroups.com
Kalju <kalju....@gmail.com> wrote:
>Hi all,
>
>Facing same issue with ERROR weeutil.rsyncupload: **** Host key verification
>failed error.
>However, it's really strange...
>I can ssh to remote site as both root as well as pi user
>I can do manual rsync with no problems but weewx builtin rsync returns above
>error
>At the moment my weewx installation is syncing using cron with no problems but
>I want to understand why builtin rsync fails
>Problem started after upgrade from weewx 5.0.0 to 5.0.1
>
>Any ideas?
Adding the debug info should output the actual rsync command initiated by
>Hi all,
>
>Facing same issue with ERROR weeutil.rsyncupload: **** Host key verification
>failed error.
>However, it's really strange...
>I can ssh to remote site as both root as well as pi user
>I can do manual rsync with no problems but weewx builtin rsync returns above
>error
>At the moment my weewx installation is syncing using cron with no problems but
>I want to understand why builtin rsync fails
>Problem started after upgrade from weewx 5.0.0 to 5.0.1
>
>Any ideas?
weewx to the logs. That's what I assume when looking at line 104 in
`src/weeutil/rsyncupload.py` from the sources of 5.0.1.
Maybe that gives a clue in which the actual rsync commands differ and
why it might fail within Weewx.
Somewhere around 5.0.1 or 5.1.x the user got switched to weewx (not root
anymore) so maybe there is a problem? (I can't remember when the switch
was (it was mentioned on the list somwhere) and haven't found it in the
docs yet)
I'd always suggest looking as what weewx is running so you may not have
to play the guessing game on which user to look at.
sudo ps ux | grep weewxd
should return the user that actually runs the (python) daemon like on my
old Raspberry this runs still as root with 4.10.2.
root 690 2.8 1.5 647360 60428 ? Sl Feb06 75:38 python3 /usr/share/weewx/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf
As it was a package-installation I never messed with it, though I
welcome the change to a non-root package installation per default :)
In case host keys have changed on the server a deletion of the known_hosts
file within ~/.ssh could also help, probably.
-dominic
>Thank you forward :-)
>On Friday, July 21, 2023 at 1:14:24 PM UTC+3 Tomasz Lewicki wrote:
>
> Thank you for replies.
>
> @Warren Gill: yes, I was thinking about crontab but wanted to use built-in
> solution. Of course crontab is good as backup solution.
>
> @vince: you're right, I didn't think about it from this side. Just for
> record for future seekers:
>
> 1. I logged as root ('sudo -i' from 'pi' account)
> 2. generated SSH keys ('ssh-keygen')
> 3. copied them to external server ('ssh-copy-id ace...@external.domain.com
> -p 222')
> 4. copied /home/pi/.ssh/config to /root/.ssh/config
> 5. changed owner of 'config' ('chown root:root /root/.ssh/config')
> 6. waited for next synchronization
> 7. smiled because everything worked as expected :)
>
Educating the mind without educating the heart is no education at all.
- Aristotle
Kalju
Feb 8, 2024, 5:22:46 AM2/8/24
to weewx-user
Hi Dominic!
Thanks for the tip. Indeed, the user was changed from root to weewx
That solved the issue
Thanks!
Thanks!
Mario Wesolek
Feb 8, 2024, 6:32:48 AM2/8/24
to weewx-user
some problem.. weewx homedir is /var/lib/weewx ... copy the hostkey and config file to /var/lib/weewx/.ssh and chown it to weewx:weewx
Ben W.
Jun 1, 2024, 8:34:36 PM6/1/24
to weewx-user
Greetings!
Thank you for logging these steps last year:
1. I logged as root ('sudo -i' from 'pi' account)
2. generated SSH keys ('ssh-keygen')
3. copied them to external server ('ssh-copy-id ace...@external.domain.com -p 222')
4. copied /home/pi/.ssh/config to /root/.ssh/config
5. changed owner of 'config' ('chown root:root /root/.ssh/config')
6. waited for next synchronization
7. smiled because everything worked as expected :)
I'm stuck between 6 and 7 and unsure what I'm doing wrong. My logs are below. I've copied the public key from the 'root' user and verified on my paid web hosting site that they have the same key even with 'root'. I can SSH in, but the host is still asking for a password - only if I enter the password am I able to authenticate. It's as-if either side isn't seeing the key. I'm assuming that's my problem with rsync. The logs:
_______
Jun 01 19:18:18 rpi systemd[1]: Started weewx.service - WeeWX.
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Initializing weewxd version 5.0.2
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Command line: /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Using Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Located at /usr/bin/python3
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Platform Linux-6.6.20+rpt-rpi-2712-aarch64-with-glibc2.36
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Locale: 'en_GB.UTF-8'
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Entry path: /usr/share/weewx/weewxd.py
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: WEEWX_ROOT: /etc/weewx
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Configuration file: /etc/weewx/weewx.conf
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: User module: /etc/weewx/bin/user
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Debug: 0
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Loading station type GW1000 (user.gw1000)
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayDriver: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayService: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: StdConvert target unit is 0x1
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Archive will use data binding wx_binding
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Record generation will be attempted in 'software'
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Using archive interval of 300 seconds (software record generation)
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: StationRegistry: Station will be registered.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: Wunderground: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: PWSWeather: Data for station PROOF0FHUMBOLDT will be posted
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: CWOP: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: WOW: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: AWEKAS: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.mqtt: service version is 0.24
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: MQTT: No config info. Skipped.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: 'pyephem' detected, extended almanac data is available
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Starting up weewx version 5.0.2
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.manager: Starting backfill of daily summaries
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.manager: Daily summaries up to date
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Starting main packet loop.
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: Using 't_rainyear' for rain total
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: skipping rain measurement of 282.6: no last rain
Jun 01 19:18:19 rpi weewxd[200894]: INFO user.gw1000: Using 't_rainyear' for rain total
Jun 01 19:18:19 rpi weewxd[200894]: INFO user.gw1000: skipping rain measurement of 282.6: no last rain
Jun 01 19:20:19 rpi weewxd[200894]: INFO weewx.manager: Added record 2024-06-01 19:20:00 CDT (1717287600) to database 'weewx.sdb'
Jun 01 19:20:19 rpi weewxd[200894]: INFO weewx.manager: Added record 2024-06-01 19:20:00 CDT (1717287600) to daily summary in 'weewx.sdb'
Jun 01 19:20:20 rpi weewxd[200894]: INFO weewx.restx: PWSWeather: Published record 2024-06-01 19:20:00 CDT (1717287600)
Jun 01 19:20:20 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 0.58 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 18 images for report SeasonsReport in 0.23 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.reportengine: Copied 5 files to /var/www/html/weewx
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 6 files for report SmartphoneReport in 0.05 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 6 images for report SmartphoneReport in 0.05 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.reportengine: Copied 6 files to /var/www/html/weewx/smartphone
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 1 files for report MobileReport in 0.02 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 4 images for report MobileReport in 0.03 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.reportengine: Copied 2 files to /var/www/html/weewx/mobile
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 13 files for report StandardReport in 0.50 seconds
Jun 01 19:20:22 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 12 images for report StandardReport in 0.11 seconds
Jun 01 19:20:22 rpi weewxd[200894]: INFO weewx.reportengine: Copied 13 files to /var/www/html/weewx
Jun 01 19:20:22 rpi weewxd[200894]: INFO user.belchertown: version 1.3.1
Jun 01 19:20:23 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 12 files for report Belchertown in 1.06 seconds
Jun 01 19:20:23 rpi weewxd[200894]: INFO weewx.reportengine: Copied 39 files to /var/www/html/weewx/belchertown
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '--delete', '-e', 'ssh -p 65002', '/var/www/html/weewx/belchertown/', 'USERidREDACTED@IP_ADDRESS:public_html']
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: **** Host key verification failed.
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(231) [sender=3.2.7]
^C
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Initializing weewxd version 5.0.2
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Command line: /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Using Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Located at /usr/bin/python3
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Platform Linux-6.6.20+rpt-rpi-2712-aarch64-with-glibc2.36
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Locale: 'en_GB.UTF-8'
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Entry path: /usr/share/weewx/weewxd.py
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: WEEWX_ROOT: /etc/weewx
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Configuration file: /etc/weewx/weewx.conf
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: User module: /etc/weewx/bin/user
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Debug: 0
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Loading station type GW1000 (user.gw1000)
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayDriver: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayService: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: StdConvert target unit is 0x1
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Archive will use data binding wx_binding
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Record generation will be attempted in 'software'
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Using archive interval of 300 seconds (software record generation)
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: StationRegistry: Station will be registered.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: Wunderground: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: PWSWeather: Data for station PROOF0FHUMBOLDT will be posted
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: CWOP: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: WOW: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: AWEKAS: Posting not enabled.
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.mqtt: service version is 0.24
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.restx: MQTT: No config info. Skipped.
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: 'pyephem' detected, extended almanac data is available
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Starting up weewx version 5.0.2
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.manager: Starting backfill of daily summaries
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.manager: Daily summaries up to date
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Starting main packet loop.
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: Using 't_rainyear' for rain total
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: skipping rain measurement of 282.6: no last rain
Jun 01 19:18:19 rpi weewxd[200894]: INFO user.gw1000: Using 't_rainyear' for rain total
Jun 01 19:18:19 rpi weewxd[200894]: INFO user.gw1000: skipping rain measurement of 282.6: no last rain
Jun 01 19:20:19 rpi weewxd[200894]: INFO weewx.manager: Added record 2024-06-01 19:20:00 CDT (1717287600) to database 'weewx.sdb'
Jun 01 19:20:19 rpi weewxd[200894]: INFO weewx.manager: Added record 2024-06-01 19:20:00 CDT (1717287600) to daily summary in 'weewx.sdb'
Jun 01 19:20:20 rpi weewxd[200894]: INFO weewx.restx: PWSWeather: Published record 2024-06-01 19:20:00 CDT (1717287600)
Jun 01 19:20:20 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 0.58 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 18 images for report SeasonsReport in 0.23 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.reportengine: Copied 5 files to /var/www/html/weewx
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 6 files for report SmartphoneReport in 0.05 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 6 images for report SmartphoneReport in 0.05 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.reportengine: Copied 6 files to /var/www/html/weewx/smartphone
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 1 files for report MobileReport in 0.02 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 4 images for report MobileReport in 0.03 seconds
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.reportengine: Copied 2 files to /var/www/html/weewx/mobile
Jun 01 19:20:21 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 13 files for report StandardReport in 0.50 seconds
Jun 01 19:20:22 rpi weewxd[200894]: INFO weewx.imagegenerator: Generated 12 images for report StandardReport in 0.11 seconds
Jun 01 19:20:22 rpi weewxd[200894]: INFO weewx.reportengine: Copied 13 files to /var/www/html/weewx
Jun 01 19:20:22 rpi weewxd[200894]: INFO user.belchertown: version 1.3.1
Jun 01 19:20:23 rpi weewxd[200894]: INFO weewx.cheetahgenerator: Generated 12 files for report Belchertown in 1.06 seconds
Jun 01 19:20:23 rpi weewxd[200894]: INFO weewx.reportengine: Copied 39 files to /var/www/html/weewx/belchertown
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '--delete', '-e', 'ssh -p 65002', '/var/www/html/weewx/belchertown/', 'USERidREDACTED@IP_ADDRESS:public_html']
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: **** Host key verification failed.
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
Jun 01 19:20:23 rpi weewxd[200894]: ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(231) [sender=3.2.7]
^C
___________
Manually coping the html files work after entering the password, but who wants to do that every five minutes :)
"Host key verification failed" is obviously my problem but cannot figure it out. Any pointers, directions or well-wishes would be greatly appreciated! Almost there......
Thanks!
vince
Jun 1, 2024, 8:59:44 PM6/1/24
to weewx-user
If you google your exact error "Host key verification failed." and it will return what's going on....
https://askubuntu.com/questions/45679/ssh-connection-problem-with-host-key-verification-failed-error
The weewx-related answer is that you're using old notes that are still expecting the 'root' user on the weewx system to be the local user, which is no longer accurate. In v5 it is the 'weewx' user (upgrade guide link). So it is very likely the host key of the remote computer is not known in the /home/weewx/.ssh/known_hosts file. Simplest way around this would be do append whatever is in your legacy /root/.ssh/known_hosts file to your /home/weewx/.ssh/known_hosts file.
gjr80
Jun 1, 2024, 9:44:39 PM6/1/24
to weewx-user
Nothing to do with your rsync issue, but I notice in your log extract that you are running the Ecowitt gateway driver as both a driver and service in the same WeeWX instance against the same Ecowitt gateway device:
Jun 01 19:18:18 rpi weewxd[200894]: INFO __main__: Debug: 0
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Loading station type GW1000 (user.gw1000)
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayDriver: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayService: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: StdConvert target unit is 0x1
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: Loading station type GW1000 (user.gw1000)
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayDriver: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: GatewayService: version is 0.6.1
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: device address is 192.168.7.206:45000
Jun 01 19:18:18 rpi weewxd[200894]: INFO user.gw1000: poll interval is 20 seconds
Jun 01 19:18:18 rpi weewxd[200894]: INFO weewx.engine: StdConvert target unit is 0x1
I am hard pressed to think of a use case needing this configuration, it was certainly not something I ever planned for when writing the driver. The recommended use of the Ecowitt gateway driver in a given WeeWX instance is to run it as a driver unless you have multiple gateway devices or another device type that must be run with its own driver. Running as both a driver and service against the same device on the same WeeWX instance will impact performance (the gateway device is polled twice as often and the entire sensor data and metadata is decoded twice irrespective of and field filtering that may be applied).
At best you are just increasing the system (and WeeWX) load, at worst you are increasing the system load and may experience unknown 2nd order effects on the data emitted by WeeWX.
I would suggest dropping back to running the Ecowitt gateway driver as a driver only.
Gary
Ben W.
Jun 1, 2024, 11:29:41 PM6/1/24
to weewx-user
Hi Gary,
Thank you for this information. To be frank, I'm unsure how I did that, but I'll research how to resolve it. I have my weather station plus indoor sensors all feeding through the Ecowitt GW hub. My intent was to have WeeWX capture everything from the GW hub data.
Thank you!
Ben W.
Jun 1, 2024, 11:32:07 PM6/1/24
to weewx-user
Thanks, Vince!
I'm pretty sure I saw that link in my results, but the Ubuntu reference resulted in my continue scrolling. I will try your suggestion when I get back home!
PBudmark
Jun 2, 2024, 9:07:15 AM6/2/24
to weewx-user
As correctly stated in https://groups.google.com/g/weewx-user/c/50VCTyL-Tpw/m/DgScCzIDAQAJ, the home directory for weewx is
/var/lib/weewx and not /home/weewx, so .ssh need to go into
/var/lib/weewx/.ssh
On Sunday, June 2, 2024 at 4:32:07 AM UTC+1 Ben W. wrote:
Thanks, Vince!I'm pretty sure I saw that link in my results, but the Ubuntu reference resulted in my continue scrolling. I will try your suggestion when I get back home!
On Saturday, June 1, 2024 at 7:59:44 PM UTC-5 vince wrote:
If you google your exact error "Host key verification failed." and it will return what's going on....https://askubuntu.com/questions/45679/ssh-connection-problem-with-host-key-verification-failed-errorThe weewx-related answer is that you're using old notes that are still expecting the 'root' user on the weewx system to be the local user, which is no longer accurate. In v5 it is the 'weewx' user (upgrade guide link). So it is very likely the host key of the remote computer is not known in the /home/weewx/.ssh/known_hosts file. Simplest way around this would be do append whatever is in your legacy /root/.ssh/known_hosts file to your /home/weewx/.ssh/known_hosts file.
On Saturday, June 1, 2024 at 5:34:36 PM UTC-7 Ben W. wrote:
Greetings!::
vince
Jun 2, 2024, 11:39:00 AM6/2/24
to weewx-user
Agree. My error there. I keep forgetting the weewx accounts $HOME for packaged installs is in an odd location. Thanks !
James B
May 20, 2025, 11:46:01 PMMay 20
to weewx-user
I ran RSYNC with 4.10 with no problems. I've read and re-read the RSYNC docs for 4.10, 5.0, and 5.1
No success so I really appreciate any help. Ironically it looks like from the 5.0 documentation, if I had upgraded
from 4.10 rather than a clean 5.1 install, the installer would have kept my user "pi" and not "weewx", and i think i wouldn't
have these problems. I think my problem is the daemon user is "weewx" and I can't set up permissions for ssh with "weewx".
What I've done so far:
___________________________________________________
1. made fresh SD card with bookworm
2. logged in as user "pi"
3. fresh weewx install using apt install
4. set up weewx.conf:
[[RSYNC]]
enable = true
server = ZZZ.1and1-data.host
user = XXX
path = YYY/
5. ps aux | grep "weewx" (shows user weewx running weewxd)
weewx 6460 1.9 8.8 364336 81720 ? Ssl 12:46 10:30 python3 /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
6. ssh-keygen
7. ssh-copy-id X...@ZZZ.1and1-data.host
8. ssh'd to remote_server (success)
9. rsync'd test files to remote_server (success)
10. cp ~/.ssh/*.* /var/lib/weewx/.ssh
11. sudo chown weewx:weewx /var/lib/weewx/.ssh/*.*
-rwxrwx--- 1 weewx weewx 560 May 20 12:23 id_rsa.pub
-rw------- 1 weewx weewx 364 May 19 22:18 known_hosts
-rwxrwx--- 1 weewx weewx 142 May 20 12:23 known_hosts.old
12. weectl report run RSYNC (WORKS from user pi without error YAY)
13. running weewx via systemd, NO JOY. Here's the error message:
weewxd[6460]: ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '-e', 'ssh', '/var/www/html/weewx/', 'X...@ZZZ.1and1-data.host:YYY']
weewxd[6460]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
weewxd[6460]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
weewxd[6460]: ERROR weeutil.rsyncupload: **** X...@ZZZ.1and1-data.host: Permission denied (publickey,password).
weewxd[6460]: ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
weewxd[6460]: ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(232) [sender=3.2.7]
From reading the above posts, and looking everywhere i can, i am pretty sure i haven't set up the weewx ssh key properly.
I can't find a config or .config file in my ~/.ssh directory, but rsync works as user pi, so that doesn't seem to be the problem.
How do I set up ssh for the weewx user? I copied my pi ssh files as instructed above (and as mentioned by vince) but that didn't work.
Thank you so much for any help.
No success so I really appreciate any help. Ironically it looks like from the 5.0 documentation, if I had upgraded
from 4.10 rather than a clean 5.1 install, the installer would have kept my user "pi" and not "weewx", and i think i wouldn't
have these problems. I think my problem is the daemon user is "weewx" and I can't set up permissions for ssh with "weewx".
What I've done so far:
___________________________________________________
1. made fresh SD card with bookworm
2. logged in as user "pi"
3. fresh weewx install using apt install
4. set up weewx.conf:
[[RSYNC]]
enable = true
server = ZZZ.1and1-data.host
user = XXX
path = YYY/
5. ps aux | grep "weewx" (shows user weewx running weewxd)
weewx 6460 1.9 8.8 364336 81720 ? Ssl 12:46 10:30 python3 /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
6. ssh-keygen
7. ssh-copy-id X...@ZZZ.1and1-data.host
8. ssh'd to remote_server (success)
9. rsync'd test files to remote_server (success)
10. cp ~/.ssh/*.* /var/lib/weewx/.ssh
11. sudo chown weewx:weewx /var/lib/weewx/.ssh/*.*
-rwxrwx--- 1 weewx weewx 560 May 20 12:23 id_rsa.pub
-rw------- 1 weewx weewx 364 May 19 22:18 known_hosts
-rwxrwx--- 1 weewx weewx 142 May 20 12:23 known_hosts.old
12. weectl report run RSYNC (WORKS from user pi without error YAY)
13. running weewx via systemd, NO JOY. Here's the error message:
weewxd[6460]: ERROR weeutil.rsyncupload: rsync reported errors. Original command: ['rsync', '--archive', '--stats', '-e', 'ssh', '/var/www/html/weewx/', 'X...@ZZZ.1and1-data.host:YYY']
weewxd[6460]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
weewxd[6460]: ERROR weeutil.rsyncupload: **** Permission denied, please try again.
weewxd[6460]: ERROR weeutil.rsyncupload: **** X...@ZZZ.1and1-data.host: Permission denied (publickey,password).
weewxd[6460]: ERROR weeutil.rsyncupload: **** rsync: connection unexpectedly closed (0 bytes received so far) [sender]
weewxd[6460]: ERROR weeutil.rsyncupload: **** rsync error: unexplained error (code 255) at io.c(232) [sender=3.2.7]
From reading the above posts, and looking everywhere i can, i am pretty sure i haven't set up the weewx ssh key properly.
I can't find a config or .config file in my ~/.ssh directory, but rsync works as user pi, so that doesn't seem to be the problem.
How do I set up ssh for the weewx user? I copied my pi ssh files as instructed above (and as mentioned by vince) but that didn't work.
Thank you so much for any help.
John Kline
May 21, 2025, 12:00:39 AMMay 21
to weewx...@googlegroups.com, weewx-user
Steps 6 and 7 need to be performed as user weewx.
The test in step 8 should be performed as user weewx.
I believe you are correct that this would have just worked had you upgraded from 4.1; but I believe you would be running as root, not pi.
On May 20, 2025, at 8:46 PM, James B <jbar...@gmail.com> wrote:
I ran RSYNC with 4.10 with no problems. I've read and re-read the RSYNC docs for 4.10, 5.0, and 5.1
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/d910463d-ab6a-4158-84fc-5edc1bb6e5e2n%40googlegroups.com.
James Barber
May 21, 2025, 1:09:53 AMMay 21
to 'John Kline' via weewx-user, weewx-user
Thank you so much for a quick reply. I have tried logging in ..... su weewx..... and it asks for a password. This weewx user was created by the apt install so I don't know the password and "weewx" ain't it hah. I searched for the password to no avail :(. I could try making a new user weewx and set my own password ?
matthew wall
May 21, 2025, 7:18:01 AMMay 21
to weewx...@googlegroups.com
you can become the weewx user without entering a password:
sudo su weewx
or you can set a password for the weewx account:
sudo passwd weewx
> To view this discussion visit https://groups.google.com/d/msgid/weewx-user/3BC3255B-0B08-4533-B3F3-A3CA86463758%40gmail.com.
sudo su weewx
or you can set a password for the weewx account:
sudo passwd weewx
vince
May 21, 2025, 1:10:30 PMMay 21
to weewx-user
I always forget 'sudo su'. Thanks.
Just tested this on debian12 - the shell needs to be temporarily changed from nologin to bash first....
sudo usermod -s /usr/bin/bash weewx # temporarily change the shell to a valid one
sudo su weewx # now this will work
(do anything requiring running as user weewx)
sudo usermod -s /usr/bin/nologin weewx # reverse the temporary change
On debian12 it was 'not' necessary to set a weewx password.
vince
May 21, 2025, 1:12:12 PMMay 21
to weewx-user
oops - that is /usr/sbin/nologin of course. Sorry.
sudo usermod -s /usr/bin/bash weewx # temporarily change the shell to a valid one
sudo su weewx # now this will work
(do anything requiring running as user weewx)
sudo usermod -s /usr/sbin/nologin weewx # reverse the temporary change
James B
May 21, 2025, 8:04:00 PMMay 21
to weewx-user
Thank you so much, John, Matthew, Vince. This totally worked. Problem solved. Here's notes to remind me in the future, with your fixes:
Settinng up RSYNC as weewx user for weewx 5.+
1. ps aux | grep "weewx" (confirm user weewx running weewxd)
2. log in as weewx
2. log in as weewx
sudo usermod -s /usr/bin/bash weewx
sudo su weewx
3. ssh-keygen
4. ssh-copy-id your_remote_server_goes_here
5. ssh to your_remote_server_goes_here (see if you can successfully ssh without password)
6. rsync test files to your_remote_server_goes_here (see if rsync is working from cmd line)
7. exit weewx user
4. ssh-copy-id your_remote_server_goes_here
5. ssh to your_remote_server_goes_here (see if you can successfully ssh without password)
6. rsync test files to your_remote_server_goes_here (see if rsync is working from cmd line)
7. exit weewx user
8. sudo usermod -s /usr/sbin/nologin weewx
9. check weewx log files -- should be working!
vince
May 21, 2025, 8:35:07 PMMay 21
to weewx-user
Add this one to the wiki or the FAQ !
Warren Gill
May 21, 2025, 11:05:34 PMMay 21
to weewx...@googlegroups.com
No need to change the user login shell even (usermod)... just use the shell option on su.
sudo su - weewx --shell=/usr/bin/bash
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/7cb21a34-4535-4bca-be7e-d84fdad9c0a5n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages