FTP Upload Failed - 0kb File on Remote Server

[Rory Gillies]

Hi, and Happy New Year. I am experiencing an issue with FTP upload to my servers, occasionally (say around once a week) WeeWX fails to upload and there is a single HTML file on the remote server with 0kb file size (it's not a specific file, however it is usually one of the year or month reports, and never been the index.html). Deleting the file restores the upload. I have two FTP skins as I am in the process of porting my site to a new server, but the 0kb file can happen on either server so it's not server-side specific. The syslog section is shown below. The files on the local server are all correct, it's during the FTP upload that something goes wrong.

Has anyone experienced a similar issue?

Jan  1 23:45:18 weather weewx[469313] INFO weewx.manager: Added record 2023-01-01 23:45:00 GMT (1672616700) to database 'weewx.sdb'
Jan  1 23:45:18 weather weewx[469313] INFO weewx.manager: Added record 2023-01-01 23:45:00 GMT (1672616700) to daily summary in 'weewx.sdb'
Jan  1 23:45:19 weather weewx[469313] INFO weewx.restx: Windy: Published record 2023-01-01 23:45:00 GMT (1672616700)
Jan  1 23:45:19 weather weewx[469313] INFO weewx.restx: PWSWeather: Published record 2023-01-01 23:45:00 GMT (1672616700)
Jan  1 23:45:19 weather weewx[469313] INFO weewx.restx: WOW: Published record 2023-01-01 23:45:00 GMT (1672616700)
Jan  1 23:45:19 weather weewx[469313] INFO weewx.restx: AWEKAS: Published record 2023-01-01 23:45:00 GMT (1672616700)
Jan  1 23:45:20 weather weewx[469313] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 1.21 seconds
Jan  1 23:45:21 weather weewx[469313] INFO weewx.imagegenerator: Generated 15 images for report SeasonsReport in 0.59 seconds
Jan  1 23:45:21 weather weewx[469313] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx/seasons
Jan  1 23:45:21 weather weewx[469313] INFO user.historygenerator: historygenerator.py: Generated 8 tables in 0.08 seconds
Jan  1 23:45:38 weather weewx[469313] INFO weewx.cheetahgenerator: Generated 13 files for report StandardReport in 17.47 seconds
Jan  1 23:45:38 weather weewx[469313] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx
Jan  1 23:45:41 weather weewx[469313] INFO weewx.cheetahgenerator: Generated 2 files for report Emkubed in 2.99 seconds
Jan  1 23:45:41 weather weewx[469313] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx/emkubed
Jan  1 23:45:49 weather weewx[469313] INFO weewx.reportengine: ftpgenerator: Ftp'd 33 files in 7.85 seconds
Jan  1 23:45:57 weather weewx[469313] INFO weewx.reportengine: ftpgenerator: Ftp'd 33 files in 8.32 seconds
Jan  1 23:50:17 weather weewx[469313] INFO weewx.manager: Added record 2023-01-01 23:50:00 GMT (1672617000) to database 'weewx.sdb'
Jan  1 23:50:17 weather weewx[469313] INFO weewx.manager: Added record 2023-01-01 23:50:00 GMT (1672617000) to daily summary in 'weewx.sdb'
Jan  1 23:50:17 weather weewx[469313] INFO weewx.restx: Windy: Published record 2023-01-01 23:50:00 GMT (1672617000)
Jan  1 23:50:18 weather weewx[469313] INFO weewx.restx: PWSWeather: Published record 2023-01-01 23:50:00 GMT (1672617000)
Jan  1 23:50:18 weather weewx[469313] INFO weewx.restx: WOW: Published record 2023-01-01 23:50:00 GMT (1672617000)
Jan  1 23:50:18 weather weewx[469313] INFO weewx.restx: AWEKAS: Published record 2023-01-01 23:50:00 GMT (1672617000)
Jan  1 23:50:19 weather weewx[469313] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 1.13 seconds
Jan  1 23:50:19 weather weewx[469313] INFO weewx.imagegenerator: Generated 15 images for report SeasonsReport in 0.58 seconds
Jan  1 23:50:19 weather weewx[469313] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx/seasons
Jan  1 23:50:19 weather weewx[469313] INFO user.historygenerator: historygenerator.py: Generated 8 tables in 0.08 seconds
Jan  1 23:50:37 weather weewx[469313] INFO weewx.cheetahgenerator: Generated 13 files for report StandardReport in 17.37 seconds
Jan  1 23:50:37 weather weewx[469313] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx
Jan  1 23:50:40 weather weewx[469313] INFO weewx.cheetahgenerator: Generated 2 files for report Emkubed in 3.00 seconds
Jan  1 23:50:40 weather weewx[469313] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx/emkubed
Jan  1 23:51:00 weather weewx[469313] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/month-2023-01.html to server www.360shetland.co.uk. Reason: 'timed out'
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine: ftpgenerator: (0): caught exception '<class 'socket.timeout'>': timed out
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 436, in run
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 210, in run
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 502, in storbinary
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      return self.voidresp()
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 257, in voidresp
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      resp = self.getresp()
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 242, in getresp
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      resp = self.getmultiline()
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 228, in getmultiline
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      line = self.getline()
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 210, in getline
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      line = self.file.readline(self.maxline + 1)
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/socket.py", line 704, in readinto
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****      return self._sock.recv_into(b)
Jan  1 23:51:00 weather weewx[469313] ERROR weewx.reportengine:         ****  socket.timeout: timed out
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine: ftpgenerator: (1): caught exception '<class 'socket.timeout'>': timed out
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 436, in run
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 210, in run
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 491, in storbinary
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      with self.transfercmd(cmd, rest) as conn:
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 386, in transfercmd
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      return self.ntransfercmd(cmd, rest)[0]
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 352, in ntransfercmd
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      resp = self.sendcmd(cmd)
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 279, in sendcmd
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      return self.getresp()
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 242, in getresp
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      resp = self.getmultiline()
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 228, in getmultiline
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      line = self.getline()
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 210, in getline
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      line = self.file.readline(self.maxline + 1)
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/socket.py", line 704, in readinto
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****      return self._sock.recv_into(b)
Jan  1 23:51:21 weather weewx[469313] ERROR weewx.reportengine:         ****  socket.timeout: timed out
Jan  1 23:51:41 weather weewx[469313] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/month-2023-01.html to server www.360shetland.co.uk. Reason: 'timed out'
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine: ftpgenerator: (2): caught exception '<class 'socket.timeout'>': timed out
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 436, in run
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 210, in run
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 491, in storbinary
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      with self.transfercmd(cmd, rest) as conn:
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 386, in transfercmd
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      return self.ntransfercmd(cmd, rest)[0]
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 352, in ntransfercmd
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      resp = self.sendcmd(cmd)
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 279, in sendcmd
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      return self.getresp()
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 242, in getresp
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      resp = self.getmultiline()
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 228, in getmultiline
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      line = self.getline()
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/ftplib.py", line 210, in getline
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      line = self.file.readline(self.maxline + 1)
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.9/socket.py", line 704, in readinto
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****      return self._sock.recv_into(b)
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine:         ****  socket.timeout: timed out
Jan  1 23:51:41 weather weewx[469313] ERROR weewx.reportengine: ftpgenerator: Upload failed
Jan  1 23:51:50 weather weewx[469313] INFO weewx.reportengine: ftpgenerator: Ftp'd 37 files in 8.63 seconds

[Tom Keffer]

I wanted to clarify something you said:

there is a single HTML file on the remote server with 0kb file size 

Does that mean it is the only file on the remote server? Or, it's the only file of 0kb size?

I would not rule out a server-side problem. Are the two servers hosted by the same company? Can you check the type of ftp servers they use (the unix command-line interface "ftp" is useful for this).

I can't think of any reason why the presence (or absence) of a file on the server would make any difference as far as weewx is concerned. It is ignorant of the contents on the server and simply uploads anything that has changed.

Try setting debug=2 in weewx.conf. It will create voluminous logging, but it will also give more detail about what's happening.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/40222478-2273-4f85-901b-11ef0d138194n%40googlegroups.com.

[Rory Gillies]

Hi, yes to clarify all the files are there, it's just a single file that has 0kb size. The two servers are completely different, one is my own virtual private server running Ubuntu and the other is a hosted Linux package. I'll try the level 2 debug and see if it shows up anything. Thanks!

[Tom Keffer]

Do you know what kind of FTP server they are?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/70d8ad9e-7820-4773-9e28-d9593851ccaen%40googlegroups.com.

[Rory Gillies]

Hi, both servers are running vsftpd.