Fail to upload data via ftp to online server

[Gorka Aurre]

Dear community, 

This is my first post in this group, so forgive me if I miss some pieces in the information, I'll do my best. I managed all the road by myself to be able to get the data from sensors, drive it to weewx input and upload it to Wunderground (still don't know how -I am beginner in this- but I did it). But, this post is because I'm having problems to upload weewx data to an online server via ftp (the standard ftp service weewx has by default). 

I've tried connecting directly with Filezilla to the cPanel to see if connection problems were originated in the cPanel and it worked OK. Even I tried to connect manually with ftp client from Shell directly from the RPi and worked fine. But when it comes to weewx working on it, it gives me "socket.timeout: timeout" error. Let me copy error log obtained (due to security reasons I have changed addresses by -MYSERVER.COM-):

Jan  7 22:36:10 raspberrypi weewx[10348] DEBUG weeutil.ftpupload: Attempting secure connection to ftp.MYSERVER.COM
Jan  7 22:36:10 raspberrypi weewx[10348] DEBUG weeutil.ftpupload: Secure data connection to ftp.MYSERVER.COM
Jan  7 22:36:10 raspberrypi weewx[10348] DEBUG weeutil.ftpupload: 0 ./daytempfeel.png efffd6c4f0395079d7355d145f1f8908a591f54ac06b9d2d1c42bd3ba263bbb6
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine: ftpgenerator: (2): caught exception '<class 'socket.timeout'>': timed out
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 437, in run
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 175, in run
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      ftp_server.connect(self.server, self.port)
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 160, in connect
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      self.welcome = self.getresp()
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 242, in getresp
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      resp = self.getmultiline()
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 232, in getmultiline
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      nextline = self.getline()
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 210, in getline
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      line = self.file.readline(self.maxline + 1)
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/socket.py", line 704, in readinto
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****      return self._sock.recv_into(b)
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine:         ****  socket.timeout: timed out
Jan  7 22:36:26 raspberrypi weewx[6258] ERROR weewx.reportengine: ftpgenerator: Upload failed

Weewx version I am using is 4.9.1, recently installed. All other services work OK, writing in database, reporting to Wunderground, creating html reports, etc, but I want to let it upload this html reports to my server. 

Hope you guys could help with this issue, it would be very appreciated. 

Thanks in advance!

Gorka

[Tom Keffer]

The symptoms are normally a "Errno 0" error, but some FTP servers have a problem with a secure connection.

Here are two things you can try.

First, under section [[FTP]] in weewx.conf, set reuse_ssl to True. 

[[FTP]]

        ...

        # Set to True for an FTP over TLS (FTPS) connection. Not all servers
        # support this.
        secure_ftp = False
        reuse_ssl = True

        ...

See if that helps. If it does not, then remove it and try a different option, this time secure_data, set to False:

[[FTP]]

        ...

        # Set to True for an FTP over TLS (FTPS) connection. Not all servers
        # support this.
        secure_ftp = False
        secure_data = False

        ...

See if that helps. If not, we'll go to the next level.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6b66d067-7a71-4d35-9954-a591a8bb616bn%40googlegroups.com.

[Gorka Aurre]

Thanks TK for your answer.

I've tried both and continues giving same error:

Jan  8 11:46:04 raspberrypi weewx[2020] DEBUG weeutil.ftpupload: Attempting connection to ftp.MYSERVER.COM
Jan  8 11:46:04 raspberrypi weewx[2020] DEBUG weeutil.ftpupload: Connected to ftp.MYSERVER.COM
Jan  8 11:46:04 raspberrypi weewx[2020] DEBUG weeutil.ftpupload: 0 ./daytempfeel.png e436b5dc907088a83dd88f0553b320a579ea6eeadb49d6ae4f3fc5e912a852c8
Jan  8 11:46:08 raspberrypi kernel: [  965.119655] hwmon hwmon1: Voltage normalised
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/daytempfeel.png to server ftp.hurran.eus. Reason: 'timed out'
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine: ftpgenerator: (1): caught exception '<class 'socket.timeout'>': timed out
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 437, in run
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 220, in run
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 491, in storbinary
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      with self.transfercmd(cmd, rest) as conn:
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 386, in transfercmd
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      return self.ntransfercmd(cmd, rest)[0]
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 347, in ntransfercmd
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      conn = socket.create_connection((host, port), self.timeout,
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/socket.py", line 843, in create_connection
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      raise err
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/socket.py", line 831, in create_connection
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****      sock.connect(sa)
Jan  8 11:46:10 raspberrypi weewx[2410] ERROR weewx.reportengine:         ****  socket.timeout: timed out

What does "go to the next level" mean for you? 

Thanks in advance, have a great day.

Gorka

[Tom Keffer]

It means you send me your credentials in an email and I debug.

One thing I've noticed: is your server in Russia by any chance? We've had several users having problems serving from ru servers.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/33921983-8fb0-4253-b5fe-05aa6bd06afcn%40googlegroups.com.

[Gorka Aurre]

Hey!

Good news. I continued trying, creating new users and experimenting with different FTP clients and got the point where the server blocked the access from my actual IP for too many unsuccessful attempts. So I contacted with hosting support and they unblocked all the users/IP. So started trying from less invasive, trying to get to weewx. First of all, with the default setup per user that the server gives to you in a recommended FTP client (Cyberduck). After getting successful connection, I tried with FileZilla and had to change few settings in FTP/TLS (configuring default settings to FTP over TLS in defined port), so got successful connection.

The thing in weewx, where previously I had all settings OK, was that the connection directory was a defined "/public_html/weewx". The user in the server had authorized only "/public_html/weewx", so connection was stablished directly in that directory (like the main root of the user). When weewx was trying to connect, it looked for "/public_html/weewx/public_html/weewx"; it was looking for something that didn't exist. So changed directory defined in weewx to "/", and IT WORKED!

Thank you for your help. Diving into weewx code I was surprised what good everything is programmed. Congrats to everyone that contributed to the open source code.

I will try to collaborate uploading new language in Basque. 

Thanks for your help!

Gorka

[Tom Keffer]

Glad that worked out.

May I suggest using rsync? Not all servers offer it, but if your provider does, you'll find it much faster, and not as prone to these kinds of problems.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/69de8e4a-3dcb-4fc7-bbe7-140aced27c71n%40googlegroups.com.