FTP problems

96 views
Skip to first unread message

Richard Cropper

unread,
May 27, 2020, 8:50:02 AM5/27/20
to weewx-user
Hi
Having secured a connection, I still don't  seem to be able to upload files

I get the following error message on the log

May 27 13:30:36 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #1. Failed uploading public_html/weewx/monthtempfeel.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:36 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #2. Failed uploading public_html/weewx/monthtempfeel.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:37 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #3. Failed uploading public_html/weewx/monthtempfeel.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:37 raspberrypi weewx[899] ERROR weeutil.ftpupload: Failed to upload file public_html/weewx/monthtempfeel.png

May 27 13:30:37 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #1. Failed uploading public_html/weewx/yearwind.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:38 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #2. Failed uploading public_html/weewx/yearwind.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:38 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #3. Failed uploading public_html/weewx/yearwind.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:38 raspberrypi weewx[899] ERROR weeutil.ftpupload: Failed to upload file public_html/weewx/yearwind.png

May 27 13:30:39 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #1. Failed uploading public_html/weewx/statistics.html to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:39 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #2. Failed uploading public_html/weewx/statistics.html to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:40 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #3. Failed uploading public_html/weewx/statistics.html to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

May 27 13:30:40 raspberrypi weewx[899] ERROR weeutil.ftpupload: Failed to upload file public_html/weewx/statistics.html

May 27 13:30:41 raspberrypi weewx[899] ERROR weeutil.ftpupload: Attempt #1. Failed uploading public_html/weewx/weekbarometer.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory


Here's the relevant bit of my weewx.conf:


 [[FTP]]

        # FTP'ing the results to a webserver is treated as just another report,

        # albeit one with an unusual report generator!

        skin = Ftp


        # If you wish to use FTP, set "enable" to "true", then

        # fill out the next four lines.

        # Use quotes around passwords to guard against parsing errors.

        enable = true

        user =[user name]

        password = [password]

        server =  [server name]    # The ftp server name, e.g, www.myserver.org

        path = public_html/weewx/    # The destination directory, e.g., /weather


        # Set to True for an FTP over TLS (FTPS) connection. Not all servers

        # support this.

        secure_ftp = True

        secure_data = True


       # To upload files from something other than what HTML_ROOT is set

        # to above, specify a different HTML_ROOT here.

         HTML_ROOT = /var/www/html/weewx/


        # Most FTP servers use port 21

        port = 21


        # Set to 1 to use passive mode, zero for active mode

        passive = 1


The connection is FTP over TLS.


My weather data is from an Aercus Weather Sleuth using interceptor on a Raspberry pi.


It's as if FTP is looking for the wrong path on my  pi but maybe I misunderstand the error message. That's why I tried removing the hash before HTML_ROOT but I got the same result with the hash.


Otherwise very pleased with weewx  which I can view on my pi using nginx


Any help gratefully received.


Richard


Tom Keffer

unread,
May 27, 2020, 10:17:35 AM5/27/20
to weewx-user
Hello, Richard

Unfortunately, the error message is misleading (and should be changed). When it says

Failed uploading public_html/weewx/monthtempfeel.png to ftp.tq7weather.uk. Reason: 553 Can't open that file: No such file or directory

what it means is it could not upload to the target public_html/weewx/monthtempfeel.png. The unfortunate part is that the error message doesn't give the source.

Take a look in /var/www/html/weewx and make sure the files you expect to be in there are actually in there.

-tk


--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/35d77ba3-e9a1-4919-9b2e-bb0680490b8a%40googlegroups.com.

Tom Keffer

unread,
May 27, 2020, 10:20:50 AM5/27/20
to weewx-user
Fixed in commit 7ac6eaf.
Message has been deleted

Richard Cropper

unread,
May 27, 2020, 1:02:51 PM5/27/20
to weewx-user
Thanks Tom

Yes all the files are in /var/www/html/weewx 

If I understand you correctly, this has more to do with the file path of the remote host server.

I am at a loss to understand what is going on at that end.

I can at least connect but I either get a message as you have seen or a 'broken pipe' message if I play around with the file path.

I guess this is more to do with the remote server than weewx but thanks for your advice.

Regards

Richard
To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

Tom Keffer

unread,
May 27, 2020, 2:56:28 PM5/27/20
to weewx-user
Two things to try:

1. Make sure the FTP username has write permissions on the server.

2. Try different variations on the FTP path. For example,

path = public_html/weewx/    # What you have
path = public_html/weewx
path = /public_html/weewx/
path = /public_html/weewx
path = weewx
path = /weewx






To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/65f68c3f-31e6-4e6f-95aa-0268a0a82f78%40googlegroups.com.

Richard Cropper

unread,
May 28, 2020, 12:36:12 PM5/28/20
to weewx-user
Thanks Tom

I had in fact tried those steps but did so again, without success.

Suspecting that the directory might be the cause even though its permissions looked good, I deleted the weewx directory and then created it again.

This had made a difference but not solved the problem.

Here's an extract from the log:

May 28 17:24:35 raspberrypi weewx[723] ERROR weeutil.ftpupload: Attempt #1. Failed uploading /public_html/weewx/daytempfeel.png to polaris.servers.prgn.misp.co.uk. Reason: [Errno 32] Broken pipe

May 28 17:24:35 raspberrypi weewx[723] ERROR weeutil.ftpupload: Attempt #2. Failed uploading /public_html/weewx/daytempfeel.png to polaris.servers.prgn.misp.co.uk. Reason: [Errno 32] Broken pipe

May 28 17:24:35 raspberrypi weewx[723] ERROR weeutil.ftpupload: Attempt #3. Failed uploading /public_html/weewx/daytempfeel.png to polaris.servers.prgn.misp.co.uk. Reason: [Errno 32] Broken pipe

May 28 17:24:35 raspberrypi weewx[723] ERROR weeutil.ftpupload: Failed to upload file /public_html/weewx/daytempfeel.png


I have tried all possible file paths again, as you suggest but I always get this broken pipe message. If I extend the file path any higher, I get the 'cannot open this file' message.

I have been able to upload the index file from /var/www/html/weewx to the weewx directory on my server using curl.

Richard

Tom Keffer

unread,
May 28, 2020, 3:51:31 PM5/28/20
to weewx-user
Hard to say, but I think you're getting hit by issue #284. Try adding an option secure_data and setting it to false:

[[FTP]]
   ...
   secure_ftp = True
   secure_data = False

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/cc51fa47-91a9-4a06-838b-18a3ed7b3c2f%40googlegroups.com.

Tom Keffer

unread,
May 28, 2020, 4:28:49 PM5/28/20
to weewx-user
I just noticed that in refactoring the ftp uploader, the function _make_remote_dir() never gets called! Try this version of weeutil/ftpupload.py.

-tk


ftpupload.py

Richard Cropper

unread,
May 29, 2020, 7:19:38 AM5/29/20
to weewx-user
Hi Tom

This what I get with the new ftpupload.py

May 29 12:01:40 raspberrypi weewx[2431] INFO weewx.reportengine: Copied 5 files to /var/www/html/weewx

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine: Caught unrecoverable exception in generator 'weewx.reportengine.FtpGenerator'

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****  __init__() got an unexpected keyword argument 'max_tries'

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 197, in run

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****      obj.start()

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 280, in start

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****      self.run()

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 320, in run

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****      secure_data=to_bool(self.skin_dict.get('secure_data', True)))

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****  TypeError: __init__() got an unexpected keyword argument 'max_tries'

May 29 12:01:40 raspberrypi weewx[2431] ERROR weewx.reportengine:         ****  Generator terminated


I get the same message whether secure_data is set to True or False.

Regards

Richard




Richard Cropper

unread,
May 29, 2020, 7:22:19 AM5/29/20
to weewx-user
Incidentally, I've been nowhere near reportengine.py

Tom Keffer

unread,
May 29, 2020, 7:40:44 AM5/29/20
to weewx-user
The file I posted was to fix a problem introduced in V4.1, which I had assumed you were using. Apparently not. Disregard the file. Sorry for any confusion.

Did you try setting secure_data to false?

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c53bbd3e-00ee-4291-872d-6de13557e08c%40googlegroups.com.

Richard Cropper

unread,
May 29, 2020, 9:15:50 AM5/29/20
to weewx-user
I'm on v 4.0.0 - it can't have been more than a week since I installed it

Yes I did try setting secure_data to false. With the previous version of ftpupload.py I then get

May 29 14:10:56 raspberrypi weewx[4780] ERROR weeutil.ftpupload: Attempt #1. Failed uploading /public_html/weewx/weekwind.png to polaris.servers.prgn.misp.co.uk. Reason: 521 Data connection cannot be opened with this PROT setting.

May 29 14:10:57 raspberrypi weewx[4780] ERROR weeutil.ftpupload: Attempt #2. Failed uploading /public_html/weewx/weekwind.png to polaris.servers.prgn.misp.co.uk. Reason: 521 Data connection cannot be opened with this PROT setting.

May 29 14:10:57 raspberrypi weewx[4780] ERROR weeutil.ftpupload: Attempt #3. Failed uploading /public_html/weewx/weekwind.png to polaris.servers.prgn.misp.co.uk. Reason: 521 Data connection cannot be opened with this PROT setting.

May 29 14:10:57 raspberrypi weewx[4780] ERROR weeutil.ftpupload: Failed to upload file /public_html/weewx/weekwind.png



Regards

Richard

Tom Keffer

unread,
May 29, 2020, 8:33:20 PM5/29/20
to weewx-user
Well then, let's try setting the PROT setting, even though the data will not be secure. Try this version.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/67c65342-a58e-440b-a92a-8ed9dd6b3395%40googlegroups.com.
ftpupload.py

Richard Cropper

unread,
May 30, 2020, 6:26:07 AM5/30/20
to weewx-user
Sorry

This looks like the same error as with your first replacement:

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine: Caught unrecoverable exception in generator 'weewx.reportengine.FtpGenerator'

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****  __init__() got an unexpected keyword argument 'max_tries'

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 197, in run

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****      obj.start()

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 280, in start

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****      self.run()

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 320, in run

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****      secure_data=to_bool(self.skin_dict.get('secure_data', True)))

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****  TypeError: __init__() got an unexpected keyword argument 'max_tries'

May 30 11:20:54 raspberrypi weewx[21515] ERROR weewx.reportengine:         ****  Generator terminated


I've double checked to make sure I've used your latest.


I've tried with secure_data set to both true and false.


Incidentally, are capital letters critical in the true/false settings?


Regards


Richard

Tom Keffer

unread,
May 30, 2020, 8:29:57 AM5/30/20
to weewx-user
Hello,
  1. You don't want the latest. You are still on V4.0.0, so you want to revert to that version of ftpupload.py. You can find a copy here: https://github.com/weewx/weewx/blob/v4.0.0/bin/weeutil/ftpupload.py
  2. Once you get that sorted, try again with secure_data set to false.
  3. The case of true/false does not matter.
-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6dd53e7f-7f1a-434c-8d44-66b2ff6619e4%40googlegroups.com.

Tom Keffer

unread,
May 30, 2020, 9:06:04 AM5/30/20
to weewx-user
Wow. I just mislead you. My fault. What you need is a copy of V4.0.0, but with a patch that allows a secure authorization connection to be used, but with insecure data. Try the attached.

My apologies!

-tk
ftpupload.py

Richard Cropper

unread,
May 30, 2020, 9:22:53 AM5/30/20
to weewx-user
No problem , Tom

Unfortunately, still not there. Whether secure_data is set to true or false, I get the following:

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Attempt #2. Failed uploading /public_html/weewx/daywindvec.png to polaris.servers.prgn.misp.co.uk. Reason: [Errno 32] Broken pipe

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Attempt #3. Failed uploading /public_html/weewx/daywindvec.png to polaris.servers.prgn.misp.co.uk. Reason: [Errno 32] Broken pipe

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Failed to upload file /public_html/weewx/daywindvec.png

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Got error while attempting to make remote directory /public_html/weewx/NOAA

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload:      ****  Error: [Errno 32] Broken pipe

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Got error while attempting to make remote directory /public_html/weewx/NOAA

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload:      ****  Error: [Errno 32] Broken pipe

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Got error while attempting to make remote directory /public_html/weewx/NOAA

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload:      ****  Error: [Errno 32] Broken pipe

May 30 14:16:39 raspberrypi weewx[24128] ERROR weeutil.ftpupload: Unable to create remote directory /public_html/weewx/NOAA

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine: ftpgenerator: Caught exception '<type 'exceptions.IOError'>': Unable to create remote directory /public_html/weewx/NOAA

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 326, in run

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****      n = ftp_data.run()

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 143, in run

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****      self._make_remote_dir(ftp_server, remote_dir_path)

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 240, in _make_remote_dir

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****      raise IOError("Unable to create remote directory %s" % remote_dir_path)

May 30 14:16:39 raspberrypi weewx[24128] ERROR weewx.reportengine:         ****  IOError: Unable to create remote directory /public_html/weewx/NOAA



Would it help if I installed v4.1?

Richard

Tom Keffer

unread,
May 30, 2020, 10:12:40 AM5/30/20
to weewx-user
I doubt it. The logic is pretty much the same.

Would you be comfortable sending me the user login and password via private email? tke...@gmail.com

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9bfd90d4-0e7c-4366-816a-4a67e069b8d4%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages