SFTP upload error ftplib.py

Stefan Ferber

unread,

Dec 27, 2020, 4:47:10 PM12/27/20

to weewx-user

Cannot find the defect in my sftp configuration. Could you please check?

[[SmartphoneReport]]

# The SmartphoneReport uses the 'Smartphone' skin, and the images and

# files are placed in a dedicated subdirectory.

skin = Smartphone

enable = true

HTML_ROOT = /var/www/html/weewx/smartphone

[[MobileReport]]

# The MobileReport uses the 'Mobile' skin, and the images and files

# are placed in a dedicated subdirectory.

skin = Mobile

enable = true

HTML_ROOT = /var/www/html/weewx/mobile

[[StandardReport]]

# This is the old "Standard" skin. By default, it is not enabled.

skin = Standard

enable = false

[[FTP]]

# FTP'ing the results to a webserver is treated as just another report,

# albeit one with an unusual report generator!

skin = Ftp

# If you wish to use FTP, set "enable" to "true", then

# fill out the next four lines.

# Use quotes around passwords to guard against parsing errors.

enable = true

user = <username-removed-here>

password = "<password-removed-here>"

server = home15604585.1and1-data.host # The ftp server name, e.g, www.myserver.org

path = /SmartFerber/weewx # The destination directory, e.g., /weather

# Set to True for an FTP over TLS (FTPS) connection. Not all servers

# support this.

secure_ftp = true

# To upload files from something other than what HTML_ROOT is set

# to above, specify a different HTML_ROOT here.

#HTML_ROOT = /var/www/html/weewx

# Most FTP servers use port 21

port = 22

# Set to 1 to use passive mode, zero for active mode

passive = 1

getting multiple errors:

Dec 27 22:35:51 raspberrypi weewx[24064] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 28.09 seconds

Dec 27 22:35:58 raspberrypi weewx[24064] INFO weewx.imagegenerator: Generated 15 images for report SeasonsReport in 6.71 seconds

Dec 27 22:35:58 raspberrypi weewx[24064] INFO weewx.reportengine: Copied 5 files to /var/www/html/weewx

Dec 27 22:36:01 raspberrypi weewx[24064] INFO weewx.cheetahgenerator: Generated 6 files for report SmartphoneReport in 2.09 seconds

Dec 27 22:36:03 raspberrypi weewx[24064] INFO weewx.imagegenerator: Generated 6 images for report SmartphoneReport in 2.40 seconds

Dec 27 22:36:03 raspberrypi weewx[24064] INFO weewx.reportengine: Copied 6 files to /var/www/html/weewx/smartphone

Dec 27 22:36:04 raspberrypi weewx[24064] INFO weewx.cheetahgenerator: Generated 1 files for report MobileReport in 0.78 seconds

Dec 27 22:36:06 raspberrypi weewx[24064] INFO weewx.imagegenerator: Generated 4 images for report MobileReport in 1.65 seconds

Dec 27 22:36:06 raspberrypi weewx[24064] INFO weewx.reportengine: Copied 2 files to /var/www/html/weewx/mobile

Dec 27 22:36:26 raspberrypi weewx[24064] ERROR weewx.reportengine: ftpgenerator: (0): caught exception '<class 'socket.timeout'>': timed out

Dec 27 22:36:26 raspberrypi weewx[24064] ERROR weewx.reportengine: **** Traceback (most recent call last):

Dec 27 22:36:26 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/reportengine.py", line 331, in run

Dec 27 22:36:26 raspberrypi weewx[24064] ERROR weewx.reportengine: **** n = ftp_data.run()

Dec 27 22:36:26 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/share/weewx/weeutil/ftpupload.py", line 127, in run

Dec 27 22:36:26 raspberrypi weewx[24064] ERROR weewx.reportengine: **** ftp_server.connect(self.server, self.port)

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/lib/python3.7/ftplib.py", line 155, in connect

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** self.welcome = self.getresp()

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/lib/python3.7/ftplib.py", line 236, in getresp

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** resp = self.getmultiline()

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/lib/python3.7/ftplib.py", line 226, in getmultiline

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** nextline = self.getline()

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/lib/python3.7/ftplib.py", line 204, in getline

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** line = self.file.readline(self.maxline + 1)

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** File "/usr/lib/python3.7/socket.py", line 589, in readinto

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** return self._sock.recv_into(b)

Dec 27 22:36:27 raspberrypi weewx[24064] ERROR weewx.reportengine: **** socket.timeout: timed out

I can do manual sftp as user pi

$ sftp -P22 <username-removed-here> @home15604585.1and1-data.host

<username-removed-here> @home15604585.1and1-data.host's password:

Connected to <username-removed-here>@home15604585.1and1-data.host.

sftp>

and user root

sudo sftp -P22 <username-removed-here>@home15604585.1and1-data.host

The authenticity of host 'home15604585.1and1-data.host (2001:8d8:1000:a9:7f57:3133:3ca8:a8cd)' can't be established.

ECDSA key fingerprint is SHA256:J4oM+B2g7zZWAI3DolXR2e4vdIMrGO301kEN15/slsQ.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'home15604585.1and1-data.host,2001:8d8:1000:a9:7f57:3133:3ca8:a8cd' (ECDSA) to the list of known hosts.

<username-removed-here>@home15604585.1and1-data.host's password:

Connected to <username-removed-here>@home15604585.1and1-data.host.

sftp>

Next I updated pysftp:

$ sudo pip3 install pysftp

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple

Collecting pysftp

Downloading https://www.piwheels.org/simple/pysftp/pysftp-0.2.9-py3-none-any.whl

Collecting paramiko>=1.17 (from pysftp)

Using cached https://files.pythonhosted.org/packages/95/19/124e9287b43e6ff3ebb9cdea3e5e8e88475a873c05ccdf8b7e20d2c4201e/paramiko-2.7.2-py2.py3-none-any.whl

Requirement already satisfied: cryptography>=2.5 in /usr/lib/python3/dist-packages (from paramiko>=1.17->pysftp) (2.6.1)

Collecting pynacl>=1.0.1 (from paramiko>=1.17->pysftp)

Downloading https://www.piwheels.org/simple/pynacl/PyNaCl-1.4.0-cp37-cp37m-linux_armv6l.whl (605kB)

100% |████████████████████████████████| 614kB 93kB/s

Collecting bcrypt>=3.1.3 (from paramiko>=1.17->pysftp)

Downloading https://www.piwheels.org/simple/bcrypt/bcrypt-3.2.0-cp37-cp37m-linux_armv6l.whl (57kB)

100% |████████████████████████████████| 61kB 374kB/s

Requirement already satisfied: six in /usr/lib/python3/dist-packages (from pynacl>=1.0.1->paramiko>=1.17->pysftp) (1.12.0)

Collecting cffi>=1.4.1 (from pynacl>=1.0.1->paramiko>=1.17->pysftp)

Downloading https://www.piwheels.org/simple/cffi/cffi-1.14.4-cp37-cp37m-linux_armv6l.whl (346kB)

100% |████████████████████████████████| 348kB 133kB/s

Collecting pycparser (from cffi>=1.4.1->pynacl>=1.0.1->paramiko>=1.17->pysftp)

Using cached https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl

Installing collected packages: pycparser, cffi, pynacl, bcrypt, paramiko, pysftp

Successfully installed bcrypt-3.2.0 cffi-1.14.4 paramiko-2.7.2 pycparser-2.20 pynacl-1.4.0 pysftp-0.2.9

Now I am out of ideas what to do. Any hints?

Tom Keffer

unread,

Dec 27, 2020, 7:45:46 PM12/27/20

to weewx-user

WeeWX supports FTPS, which is FTP over TLS. It does not support SFTP, which uses a Secure Socket Shell protocol. I'm guessing your FTP server simply does not support FTPS.

Can you use rsync? It's better in almost every way.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6581b3da-a717-4f9e-b263-ce6a7d456cc8n%40googlegroups.com.

Stefan Ferber

unread,

Dec 28, 2020, 5:47:01 AM12/28/20

to weewx-user

Thx for your quick answer - 100% correct: although my provider IONOS supports SFTP and FTPS, looks like my account supports only SFTP.

Stefan Ferber

unread,

Dec 28, 2020, 6:23:06 AM12/28/20

to weewx-user

Just installed this weewx extension

https://github.com/matthewwall/weewx-sftp
works great ;-)

Stefan Ferber

unread,

Dec 28, 2020, 6:24:06 AM12/28/20

to weewx-user

issue is closed.

Anton vanNwnhzn@GMail

unread,

Dec 28, 2020, 6:35:41 AM12/28/20

to 'Stefan Ferber' via weewx-user

This message in the forums Domoticz/Stackoverflow explains why ftplib is not suitable for sftp:
https://www.domoticz.com/forum/viewtopic.php?p=259314#p259314
As explained in the Domoticz-thread myself also looking for an 'easy' alternative.
Tried paramiko-variants like pysftp, and also rsync.
The basic scriptlines no problem, but stumble at the preparation/handling/inclusion for ssh of username & password.
Somebody found a compatible&short 'ssh-password function for dummies'?

Op 28-12-2020 om 11:47 schreef 'Stefan Ferber' via weewx-user:

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4cc49dd1-65bf-46f1-8d13-2be64001a47en%40googlegroups.com.

-- 

===============================================================
Contactinfo voor Anton van Nieuwenhuijzen:
Email    = ton...@gmail.com
Fax2Mail = (+31/0)84.8397303 [ook Voice2Mail]   
===============================================================
Deze E-mail en eventuele aanhangende files zijn 
alleen bestemd voor de geadresseerde(n). 
Als je deze E-mail ten onrechte hebt ontvangen, 
dan aub verwijderen en de afzender informeren.

Anton vanNwnhzn@GMail

unread,

Dec 28, 2020, 6:38:14 AM12/28/20

to 'Stefan Ferber' via weewx-user

Crossing communications:
Stefan's response is clear (and closing)

Op 28-12-2020 om 12:33 schreef Anton vanNwnhzn@GMail:

This message in the forums Domoticz/Stackoverflow explains why ftplib is not suitable: https://www.domoticz.com/forum/viewtopic.php?p=259314#p259314
Have myself tried to find an 'easy' alternative.

Tried paramiko-variants like pysftp, and also rsync.

The basic scriptlines no problem, but stumble at the preparation/handling for ssh of username & password.
Somebody found a 'function for dummies'?

Op 28-12-2020 om 11:47 schreef 'Stefan Ferber' via weewx-user:

Thx for your quick answer - 100% correct: although my provider IONOS supports SFTP and FTPS, looks like my account supports only SFTP.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4cc49dd1-65bf-46f1-8d13-2be64001a47en%40googlegroups.com.

-- 

===============================================================
Contactinfo voor Anton van Nieuwenhuijzen:
Email    = ton...@gmail.com
Fax2Mail = (+31/0)84.8397303 [ook Voice2Mail]   
===============================================================
Deze E-mail en eventuele aanhangende files zijn 
alleen bestemd voor de geadresseerde(n). 
Als je deze E-mail ten onrechte hebt ontvangen, 
dan aub verwijderen en de afzender informeren.

Reply all

Reply to author

Forward