New internet provider - now ftp doesn't work

146 views
Skip to first unread message

Rich Strle

unread,
Jan 16, 2021, 10:11:56 PM1/16/21
to weewx-user
I changed out my internet provider to t-mobile home internet service and now I'm getting ftp errors. I tried connecting to the ftp from another computer and that worked. When my raspberry pi tries to connect I get errors. This was working fine before I switched providers. Any ideas?

Jan 16 21:01:25 pi-weather weewx[30706] DEBUG weeutil.ftpupload: Attempting connection to www.cliffandbuster.com
Jan 16 21:01:26 pi-weather weewx[30706] DEBUG weeutil.ftpupload: Connected to www.cliffandbuster.com
Jan 16 21:01:26 pi-weather weewx[30706] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/seasons.css to server www.cliffandbuster.com. Reason: '229 Extended Passive Mode Entered (|||50207|)'
Jan 16 21:01:27 pi-weather weewx[30706] ERROR weewx.reportengine: ftpgenerator: (1): caught exception '<class 'ftplib.error_reply'>': 229 Extended Passive Mode Entered (|||50207|)
Jan 16 21:01:27 pi-weather weewx[30706] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan 16 21:01:27 pi-weather weewx[30706] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 331, in run
Jan 16 21:01:27 pi-weather weewx[30706] ERROR weewx.reportengine:         ****      n = ftp_data.run()

David Levine

unread,
Jan 17, 2021, 7:21:02 AM1/17/21
to weewx-user
Check out http://weewx.com/docs/usersguide.htm#config_FTP and look to add/adjust the passive setting in your weewx.conf config. 

David 

Rich Strle

unread,
Jan 17, 2021, 10:46:29 AM1/17/21
to weewx-user
Thanks for your response David but that didn't fix my issue. 

It was set at 1. I stopped the weewx process, set the ftp to 0, restarted weewx and it did not help. I changed it back to 1.

Tom Keffer

unread,
Jan 17, 2021, 11:17:33 AM1/17/21
to weewx-user
You definitely want passive mode (passive=1).

Two things:

1. By any chance, is your new provider using IPV6 (instead of the traditional IPV4)?

2. Set debug=2 in weewx.conf, then retry. This will put lots of information in the log about what FTP is doing.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ba3b1d83-b48b-4578-b7a5-85b3fe2e4fden%40googlegroups.com.

Rich Strle

unread,
Jan 17, 2021, 11:41:25 AM1/17/21
to weewx-user
I did as instructed. Attached is the log file after I made the change. I have a call into my provider to see if they are IPV6.
log_file.txt

galfert

unread,
Jan 17, 2021, 11:50:19 AM1/17/21
to weewx-user
Anyone care to google "T-Mobile FTP" ???? You'll see that this is an issue for everyone! My recommendation is to try to instead use SFTP.

Rich Strle

unread,
Jan 17, 2021, 12:41:18 PM1/17/21
to weewx-user
I spoke with t-mobile again. They didn't say they did anything but all of a sudden ftp started working. I will continue to monitor.

Rich Strle

unread,
Jan 17, 2021, 1:04:45 PM1/17/21
to weewx-user
My bad, the ftp in whew still doesn't work. The Access Point only has an IPv6 number listed, it doesn't have an IPv4 number listed. My website provider only allows 1 sftp account at the root level and I was trying to avoid using that.

Tom Keffer

unread,
Jan 17, 2021, 8:24:45 PM1/17/21
to weewx-user
Can you FTP using Filezilla? The goal here is to see if it's some firewall / network/ provider issue, or a problem peculiar to the Python FTP library.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/08999a97-41a4-4436-b59d-2f8793532fb6n%40googlegroups.com.

Rich Strle

unread,
Jan 17, 2021, 10:28:38 PM1/17/21
to weewx-user
I installed FileZilla and was able to use the same user name and password to upload files from /var/www/html/weewx/ to the same spot on my webserver. I did get an error message Hostname does not match certificate but I've gotten that before...

Rich Strle

unread,
Jan 17, 2021, 10:33:56 PM1/17/21
to weewx-user
Jan 17 21:30:44 pi-weather weewx[429] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/yearradiation.png to server www.cliffandbuster.com. Reason: '229 Extended Passive Mode Entered (|||50458|)'
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine: ftpgenerator: (2): caught exception '<class 'ftplib.error_reply'>': 229 Extended Passive Mode Entered (|||50458|)
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 331, in run
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 163, in run
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 504, in storbinary
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      with self.transfercmd(cmd, rest) as conn:
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 399, in transfercmd
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      return self.ntransfercmd(cmd, rest)[0]
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 359, in ntransfercmd
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      host, port = self.makepasv()
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 337, in makepasv
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      host, port = parse227(self.sendcmd('PASV'))
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 845, in parse227
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****      raise error_reply(resp)
Jan 17 21:30:45 pi-weather weewx[429] ERROR weewx.reportengine:         ****  ftplib.error_reply: 229 Extended Passive Mode Entered (|||50458|)
Jan 17 21:30:45 pi-weather weewx[429] DEBUG weewx.reportengine: Report 'RSYNC' not enabled. Skipping.

Tom Keffer

unread,
Jan 18, 2021, 8:16:26 AM1/18/21
to weewx-user
Good to hear that Filezilla is working. We can use it to understand what the server is doing.

Using Filezilla, go into its "Settings" (under the "Edit" menu). Then the "Debug" tab. There's a drop down list of message levels. Default is "0 - None". Set it to "3- Verbose". Then click "OK".

image.png

Then use Filezilla again to transfer some files. This time, watch the window at top that tells you what Filezilla is doing. See if the client (Filezilla, in this case) is negotiating with the server. Tell us what you see, or, better, copy and paste it into a post.



To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/74ddbc8e-2a66-4ff4-b7d0-712939105d15n%40googlegroups.com.

Rich Strle

unread,
Jan 18, 2021, 10:21:34 AM1/18/21
to weewx-user
Command: TYPE I
Trace: CFtpControlSocket::OnReceive()
Response: 200 TYPE is now 8-bit binary
Trace: CFtpRawTransferOpData::ParseResponse() in state 1
Trace: CControlSocket::SendNextCommand()
Trace: CFtpRawTransferOpData::Send() in state 2
Command: PASV
Trace: CFtpControlSocket::OnReceive()
Response: 200 TYPE is now 8-bit binary
Trace: CFtpRawTransferOpData::ParseResponse() in state 1
Trace: CControlSocket::SendNextCommand()
Trace: CFtpRawTransferOpData::Send() in state 2
Command: PASV
Trace: CFtpControlSocket::OnReceive()
Response: 227 Entering Passive Mode (107,180,51,77,196,142)
Trace: CFtpRawTransferOpData::ParseResponse() in state 2
Trace: CControlSocket::SendNextCommand()
Trace: CFtpRawTransferOpData::Send() in state 4
Trace: Binding data connection source IP to control connection source IP 192.168.12.117
Command: STOR yearbarometer.png
Trace: CTransferSocket::OnConnect
Trace: CTlsSocketImpl::Handshake()
Trace: Trying to resume existing TLS session.
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CFtpControlSocket::OnReceive()
Response: 227 Entering Passive Mode (107,180,51,77,197,217)
Trace: CFtpRawTransferOpData::ParseResponse() in state 2
Trace: CControlSocket::SendNextCommand()
Trace: CFtpRawTransferOpData::Send() in state 4
Trace: Binding data connection source IP to control connection source IP 192.168.12.117
Command: STOR yearhilow.png

Tom Keffer

unread,
Jan 18, 2021, 11:23:24 AM1/18/21
to weewx-user
What we want to see is the earlier chatter: where the client negotiates with the server. 

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7915453a-fa1a-4aa1-b04b-2e102626c35bn%40googlegroups.com.

Les Niles

unread,
Jan 18, 2021, 11:34:38 AM1/18/21
to weewx...@googlegroups.com
The server is responding to the PASV command by entering extended passive mode EPSV, which apparently is a legal response but which weewx/ftplib is not handling correctly. (I don’t have access to the code right now so can’t be more precise about where to fix.) The difference is that in EPSV mode the server doesn’t return an IP, just a port number. (The line “host, port = self.makepasv()” is not going to work.) 

Apparently passive mode has been the default for ftplib since python 2.1, so maybe just skip trying to change to PASV entirely?

  -Les


On Jan 18, 2021, at 7:21 AM, Rich Strle <ri...@strle.com> wrote:



Tom Keffer

unread,
Jan 18, 2021, 11:37:24 AM1/18/21
to weewx-user
That's pretty much what I'm thinking, but I'm trying to get the evidence from Filezilla. It should say that as it negotiates with the server. 

FTP is 40 year old technology. It's amazing that there are still so many incompatible versions out there.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/E07ACE72-CDA3-47AB-9C4B-FDA3293777C6%40gmail.com.

Rich Strle

unread,
Jan 18, 2021, 12:39:11 PM1/18/21
to weewx-user
Status: Resolving address of www.cliffandbuster.com
Status: Connecting to 107.180.51.77:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing of "/"...
Status: Directory listing of "/" successful
Status: Resolving address of www.cliffandbuster.com
Status: Connecting to 107.180.51.77:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Starting upload of /var/www/html/weewx/image.jpg
Status: File transfer successful, transferred 49,150 bytes in 1 second
Status: Retrieving directory listing of "/"...
Status: Directory listing of "/" successful

I disconnected and reconnected. Does filezilla trust the site now so we're not seeing all of the connection comments?

Tom Keffer

unread,
Jan 18, 2021, 6:46:12 PM1/18/21
to weewx-user
[Rich sent me his credentials via a private email.]

As soon as I tried FTP to Rich's server, I recognized the problem. The server is using Pure-FTPd, which reuses SSL connections. The Python library has a bug that prevents this. WeeWX V4.1.0 and greater have a workaround for the bug. To activate, add option reuse_ssl and set it to true.

    [[FTP]]
        ...
        reuse_ssl = True

See if that helps.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2bc3700f-0a95-4fa4-904f-d5f974159befn%40googlegroups.com.

Rich Strle

unread,
Jan 18, 2021, 8:24:36 PM1/18/21
to weewx-user
No joy

I tried  reuse_ssl = True here:

        # To upload files from something other than what HTML_ROOT is set
        # to above, specify a different HTML_ROOT here.
        #HTML_ROOT = /var/www/html/weewx

        reuse_ssl = True

        # Most FTP servers use port 21
        port = 21

        # Set to 1 to use passive mode, zero for active mode
        passive = 1

And closer to the bottom of the FTP section.

Jan 17 23:50:27 pi-weather weewx[429] DEBUG weewx.reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report 'FTP'
Jan 17 23:50:28 pi-weather weewx[429] DEBUG weeutil.ftpupload: Attempting connection to www.cliffandbuster.com
Jan 17 23:50:33 pi-weather weewx[429] DEBUG weeutil.ftpupload: Connected to www.cliffandbuster.com
Jan 17 23:50:33 pi-weather weewx[429] ERROR weeutil.ftpupload: Failed uploading /var/www/html/weewx/yearradiation.png to server www.cliffandbuster.com. Reason: '229 Extended Passive Mode Entered (|||50223|)'
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine: ftpgenerator: (0): caught exception '<class 'ftplib.error_reply'>': 229 Extended Passive Mode Entered (|||50223|)
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 331, in run
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****      n = ftp_data.run()
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weeutil/ftpupload.py", line 163, in run
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****      ftp_server.storbinary(stor_cmd, fd)
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 504, in storbinary
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****      with self.transfercmd(cmd, rest) as conn:
Jan 17 23:50:33 pi-weather weewx[429] ERROR weewx.reportengine:         ****    File "/usr/lib/python3.7/ftplib.py", line 399, in transfercmd


Tom Keffer

unread,
Jan 18, 2021, 8:50:26 PM1/18/21
to weewx-user
Well, that sucks. It works nicely on my end. 

So, it must be something on your end. Perhaps your firewall, or some IPv6 issue, which I am totally unqualified to diagnose. The only thing I can suggest is to try testing your connection to see if it is IPv6 ready: https://test-ipv6.com/

Other than that, I'm out of ideas.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ce09ba1d-7933-44b0-b5e8-242d302766e7n%40googlegroups.com.

Karen K

unread,
Jan 19, 2021, 4:53:40 PM1/19/21
to weewx-user
What catched my eyes is the "229 Extended Passive Mode Entered" in the log. There is passive mode and extended passive mode, and it's not the same. I googled a littled bit, and all the discussions about that message involve firewall and router configuration. 

From what I read, extended passive mode sends less information to the client how to transfer data, and the client (in this case weewx) is to assume what's missing. So it could be that weewx is not ready for extended passive mode but for standard passive mode, only. But more likely it is a firewall or router issue.

Rich Strle

unread,
Jan 19, 2021, 6:22:26 PM1/19/21
to weewx-user
Thank you Karen, Tom, and the others who tried to help. I think Karen could be on to something.

The t-mobile home internet router does not have an advanced interface and no bridge mode. The connection speed is slow at my location. The device is going back and I'm going to stick with my current provider.

Reply all
Reply to author
Forward
0 new messages