FTP upload Problem

[PeterGill]

Hi,

Suddenly and out of the blue my ftp upload stopped working. The html files are generated as expected in /var/www/html/weewx and the file "celestial.html" referenced in the log extract below certainly exist in the library and opens correctly in Chrome.

Of course reboot has been tried.

Any help would be highly appreciated!

Sep 15 14:46:14 Homeserver weewx[573] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/celestial.html to server ftp.myurl.se. Reason: '550 Requested action not taken. File unavailable (e.g., file not found, no access).'
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine: ftpgenerator: (2): caught exception '<class 'ftplib.error_perm'>': 550 Requested action not taken. File unavailable (e.g., file not found, no access).
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 331, in run
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 178, in run
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 504, in storbinary
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      with self.transfercmd(cmd, rest) as conn:
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 399, in transfercmd
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      return self.ntransfercmd(cmd, rest)[0]
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 365, in ntransfercmd
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      resp = self.sendcmd(cmd)
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 273, in sendcmd
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      return self.getresp()
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 246, in getresp
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****      raise error_perm(resp)
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine:         ****  ftplib.error_perm: 550 Requested action not taken. File unavailable (e.g., file not found, no access).
Sep 15 14:46:14 Homeserver weewx[573] ERROR weewx.reportengine: ftpgenerator: Upload failed
Sep 15 14:46:14 Homeserver weewx[573] DEBUG weewx.reportengine: Report 'RSYNC' not enabled. Skipping.
Sep 15 14:46:16 Homeserver weewx[573] DEBUG user.MQTTSubscribe: (Service) MessageCallbackProvider data-> incoming topic: Jettystation/Watertemp, QOS: 0, retain: 0, payload: b'14.50'
Sep 15 14:46:16 Homeserver weewx[573] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> incoming Jettystation/Watertemp: extraTemp1: 14.5

[Tom Keffer]

I don't know why it would have changed, but make sure you have access permissions to whatever directory you are trying to upload to. 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ab759762-1149-47e8-bb73-911d61611c23n%40googlegroups.com.

[PeterGill]

Thanks for the quick answer. Sorry for a stupid follow up question - how do I make sure the right access permissions are set?

[Steve2Q]

Peter..I had the exact same problem with DreamHost. For some reason they blocked my IP. They unblocked it yesterday afternoon, and FTP started working a few hours later.

[PeterGill]

Thanks Steve, I´m talking to my provider right now and I´ll update here if the problem is with them

[PeterGill]

Steve, it turned out you were right, the problem was at my provider side. Problem solved. Thanks for engaging!

[Steve2Q]

Peter..you are very welcome. I hope I saved you saved some time and aggrevation.

[PeterGill]

Hi again,

Now the problem is back but in a different way, Suddenly and again without any change from my side files are not uploaded with FTP to my provider. After a lengthy discussion with their support they claim: 

" Our Tech Team was able to fetch the logs for FTP & SFTP yesterday and today. They found that you already have a 'weather'' subdirectory in your webspace and as per logs, you are doing "mkdir" for the 'weather' subdirectory and it fails due to it already exists."

I have tried with deleting the "weather" subdirectory on my FTP site to enable a full rebuild. The directory is recreated and "celestial.html" is uploaded and present on the FTP-site but then it stops. 

Error log is as follows:

Sep 26 23:30:28 Homeserver weewx[568] DEBUG weeutil.ftpupload: Attempting connection to ftp.myftp.se
Sep 26 23:30:28 Homeserver weewx[568] DEBUG weeutil.ftpupload: Connected to ftp.myftp.se
Sep 26 23:30:28 Homeserver weewx[568] DEBUG weeutil.ftpupload: 0 ./celestial.html c006bdaaa5145a308e2b5a1904f810fda268f86b0518f179d1f15f05f56b4526
Sep 26 23:30:28 Homeserver weewx[568] DEBUG weeutil.ftpupload: Uploaded file /var/www/html/weewx/celestial.html to /customers/d/4/7/myftp.se/httpd.www/weather/celestial.html
Sep 26 23:30:28 Homeserver weewx[568] DEBUG weeutil.ftpupload: 1 ./weekvolt.png 3de9d29fe23433bab4b9e084292883e5ac2914786a3d8673b78125d757266714
Sep 26 23:30:28 Homeserver weewx[568] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/weekvolt.png to server ftp.myftp.se. Reason: ''
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine: ftpgenerator: (2): caught exception '<class 'ftplib.error_proto'>':
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 331, in run
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 178, in run
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 503, in storbinary
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****      self.voidcmd('TYPE I')
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 278, in voidcmd
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****      return self.voidresp()
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 251, in voidresp
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****      resp = self.getresp()
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 247, in getresp
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****      raise error_proto(resp)
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine:         ****  ftplib.error_proto
Sep 26 23:30:28 Homeserver weewx[568] ERROR weewx.reportengine: ftpgenerator: Upload failed

Any help would be highly appreciated!

[Tom Keffer]

A MKD command for an existing subdirectory should simply produce a 521 error. WeeWX expects this and, if it occurs, moves on. If the FTP server is behaving properly, it certainly won't terminate the session.

Besides, the error is occurring during a file upload, not while making a directory.

What FTP server is your host using?

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0595bda6-a9c4-4013-b656-ee05f775c142n%40googlegroups.com.

[PeterGill]

Thanks again TK and again you were right.  The problem was apparently on the provider side. After reaching out to their support it suddenly started to work again. I´ll see if I can get a response from them on the reasons why. If so  I´ll post it here for reference.

/PG

 

[Tom Keffer]

Glad it ended well!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/069906c8-2d96-4958-a495-a2855605d615n%40googlegroups.com.

[PeterGill]

TK,

 if you have time I would just like to get your comment on the reply I got from my providers tech support quoted below. They claim a MKD command for an existing directory produces a 550 error, not a 521. If that´s true I assume WeeWX handles that as well. Anyway, the uploads suddenly started to work at 2pm the 27th of Sep without me doing anything at all. I have asked the tech support again if they have any idea on why.

Here is the response from the support
------------------------------------------------------------------------------------------
We actually got an update from our Technical Team. According to them, the relevant team tried to recheck this and they have even used Pure-FTPd and it's giving the same error message as we do and the old proxy:
 
```ftp> mkdir somefile
550 Can't create directory: File exists```
 
The RFC for FTP says the codes mkd can give is
 
MKD
257
500, 501, 502, 421, 530, 550
550 if it failed to create
 
Reference: https://datatracker.ietf.org/doc/html/rfc959
 
And just to make sure, they have  tested it in one of our server which still has the old setup

-------------------------------------------------------------------------------------------

As said, just look at this if you have time

[Tom Keffer]

If you look a little further down the RFC 959 standard that your support team linked to, you'll see:

      The prior existence of a subdirectory with the same name is an
      error, and the server must return an "access denied" error reply
      in that case.

         CWD /usr/dm
         200 directory changed to /usr/dm
         MKD pathname
         521-"/usr/dm/pathname" directory already exists;
         521 taking no action.

      The failure replies for MKD are analogous to its file  creating
      cousin, STOR.  Also, an "access denied" return is given if a file
      name with the same name as the subdirectory will conflict with the
      creation of the subdirectory (this is a problem on UNIX, but
      shouldn't be one on TOPS-20).

But it doesn't matter because WeeWX looks for either a 550 or 521.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d55d4351-6a33-4e85-9273-0272776f0a65n%40googlegroups.com.

[PeterGill]

Thought so. Thanks!!