Failing to upload to web server

Neil S

unread,

Dec 20, 2019, 2:15:59 PM12/20/19

to weewx-user

So it appears that my ISP has done something to the server that now requires all connections to be secure (having problems with mail client as well).

However at the same time my Weewx stopped uploading to the server and the logs report...

Dec 20 18:35:48 pi-Weather weewx[5982]: ftpupload: Unable to connect or log into server : 421-Sorry, cleartext sessions and weak ciph... server.

421 Please reconnect using TLS security mechanisms.

Dec 20 18:35:48 pi-Weather weewx[5982]: ftpupload: Unable to connect or log into server : 421-Sorry, cleartext sessions and weak ciph... server.

421 Please reconnect using TLS security mechanisms.

Dec 20 18:35:48 pi-Weather weewx[5982]: ftpupload: Unable to connect or log into server : 421-Sorry, cleartext sessions and weak ciph... server.

421 Please reconnect using TLS security mechanisms.

Dec 20 18:35:48 pi-Weather weewx[5982]: ftpupload: Attempted 3 times to connect to server ftp.<removed for privacy>.co.uk. Giving up.

having searched the forum I have tried altering adding

secure_ftp = True

and also

secure_ftp = True

secure_data = False

With suitable weewx restart.

If I do either of the above I get

Dec 20 18:21:16 pi-Weather weewx[5845]: ftpupload: Unable to connect or log into server : [Errno 104] Connection reset by peer

Dec 20 18:21:16 pi-Weather weewx[5845]: ftpupload: Attempted 3 times to connect to server ftp.<removed for privacy>.co.uk. Giving up.

Dec 20 18:21:16 pi-Weather weewx[5845]: ftpgenerator: ftp'd 0 files in 0.29 seconds

I have tried connecting to my web server using WinSCP and it failed until I alter settings to TLS/SSL Explicit encryption (on port 21) at which point it works.

Any ideas what settings I need to change in the Weewx.conf [[FTP]] section to get it to upload again?

TIA

Neil

Thomas Keffer

unread,

Dec 21, 2019, 6:12:27 PM12/21/19

to weewx-user

In your weewx.conf file, set debug=2. This will give extra information about the FTP connection and, possibly, why, or at least, where, the problem is happening.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8166e9b2-7093-4745-a2e2-c501e695e7a3%40googlegroups.com.

Neil S

unread,

Dec 23, 2019, 4:29:56 AM12/23/19

to weewx-user

Have set Debug=2 but there doesn’t seem to be much more useful info. Presume I have to look at syslog?

Have also checked messages and user.log but doesn’t seem to be anything extra in there either?

I have also replaced the website ftp with the direct IP for the time being.

With secure_ftp = true I get the following in syslog

root@pi-Weather:~# tail -f /var/log/syslog

Dec 23 09:10:17 localhost weewx[9522]: restx: Ambient: url: https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?action=updateraw&ID=ILONDONXXX&PASSWORD=XXX&softwaretype=weewx-3.9.2&dateutc=2019-12-23%2009%3A10%3A00&dailyrainin=0.00&winddir=135&windspeedmph=1.6&windgustmph=1.6&rainin=0.00

Dec 23 09:10:17 localhost weewx[9522]: reportengine: Running reports for latest time in the database.

Dec 23 09:10:17 localhost weewx[9522]: reportengine: Report 'StandardReport' not enabled. Skipping.

Dec 23 09:10:17 localhost weewx[9522]: reportengine: Running report 'SeasonsReport'

Dec 23 09:10:18 localhost weewx[9522]: restx: WOW: Published record 2019-12-23 09:10:00 GMT (1577092200)

Dec 23 09:10:18 localhost weewx[9522]: restx: StationRegistry: Published record 2019-12-23 09:10:00 GMT (1577092200)

Dec 23 09:10:19 localhost weewx[9522]: restx: Wunderground-PWS: Published record 2019-12-23 09:10:00 GMT (1577092200)

Dec 23 09:10:19 localhost weewx[9522]: reportengine: Found configuration file /etc/weewx/skins/Seasons/skin.conf for report 'SeasonsReport'

Dec 23 09:10:25 localhost weewx[9522]: cheetahgenerator: using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']

Dec 23 09:10:25 localhost weewx[9522]: manager: Daily summary version is 2.0

Dec 23 09:10:41 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:11:01 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:11:41 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:12:02 localhost weewx[9522]: cheetahgenerator: Generated 8 files for report SeasonsReport in 98.02 seconds

Dec 23 09:12:02 localhost rsyslogd-2007: action 'action 17' suspended, next retry is Mon Dec 23 09:13:32 2019 [try http://www.rsyslog.com/e/2007 ]

Dec 23 09:12:03 localhost weewx[9522]: manager: Daily summary version is 2.0

Dec 23 09:12:10 localhost weewx[9522]: imagegenerator: Generated 14 images for SeasonsReport in 7.20 seconds

Dec 23 09:12:10 localhost weewx[9522]: copygenerator: copied 5 files to /var/www/weewx

Dec 23 09:12:10 localhost weewx[9522]: reportengine: Report 'SmartphoneReport' not enabled. Skipping.

Dec 23 09:12:10 localhost weewx[9522]: reportengine: Report 'MobileReport' not enabled. Skipping.

Dec 23 09:12:10 localhost weewx[9522]: reportengine: Running report 'FTP'

Dec 23 09:12:10 localhost weewx[9522]: reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report 'FTP'

Dec 23 09:12:10 localhost weewx[9522]: ftpupload: Attempting secure connection to 87.247.245.132

Dec 23 09:12:10 localhost weewx[9522]: ftpupload: Unable to connect or log into server : [Errno 104] Connection reset by peer

Dec 23 09:12:10 localhost weewx[9522]: ftpupload: Attempted 3 times to connect to server 87.247.245.132. Giving up.

Dec 23 09:12:10 localhost weewx[9522]: ftpgenerator: ftp'd 0 files in 0.26 seconds

Dec 23 09:12:10 localhost weewx[9522]: reportengine: Running report 'RSYNC'

Dec 23 09:12:10 localhost weewx[9522]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report 'RSYNC'

Dec 23 09:12:11 localhost weewx[9522]: rsyncgenerator: rsync upload not requested. Skipped.

Dec 23 09:12:22 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:12:27 localhost weewx[9522]: wmr100: Bad USB report received: tuple index out of range

Dec 23 09:12:32 localhost weewx[9522]: wmr100: Bad USB report received: tuple index out of range

Dec 23 09:12:37 localhost weewx[9522]: wmr100: Bad checksum on buffer of length 26

Dec 23 09:12:37 localhost weewx[9522]: wmr100: Bad checksum on buffer of length 7

Dec 23 09:12:52 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:13:03 localhost weewx[9522]: wmr100: Bad checksum on buffer of length 5

Dec 23 09:13:18 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:13:44 localhost weewx[9522]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:13:44 localhost rsyslogd-2007: action 'action 17' suspended, next retry is Mon Dec 23 09:15:14 2019 [try http://www.rsyslog.com/e/2007 ]

with secure_ftp = false I get the following in syslog

Dec 23 09:21:07 localhost weewx[9631]: cheetahgenerator: Generated 8 files for report SeasonsReport in 37.59 seconds

Dec 23 09:21:07 localhost weewx[9631]: manager: Daily summary version is 2.0

Dec 23 09:21:14 localhost weewx[9631]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:21:14 localhost weewx[9631]: imagegenerator: Generated 14 images for SeasonsReport in 6.99 seconds

Dec 23 09:21:14 localhost weewx[9631]: copygenerator: copied 5 files to /var/www/weewx

Dec 23 09:21:14 localhost weewx[9631]: reportengine: Report 'SmartphoneReport' not enabled. Skipping.

Dec 23 09:21:14 localhost weewx[9631]: reportengine: Report 'MobileReport' not enabled. Skipping.

Dec 23 09:21:14 localhost weewx[9631]: reportengine: Running report 'FTP'

Dec 23 09:21:14 localhost weewx[9631]: reportengine: Found configuration file /etc/weewx/skins/Ftp/skin.conf for report 'FTP'

Dec 23 09:21:14 localhost weewx[9631]: ftpupload: Attempting connection to 87.247.245.132

Dec 23 09:21:14 localhost weewx[9631]: ftpupload: Unable to connect or log into server : 421-Sorry, cleartext sessions and weak ciphers are not accepted on this server.#012421 Please reconnect using TLS security mechanisms.

Dec 23 09:21:15 localhost weewx[9631]: ftpupload: Unable to connect or log into server : 421-Sorry, cleartext sessions and weak ciphers are not accepted on this server.#012421 Please reconnect using TLS security mechanisms.

Dec 23 09:21:15 localhost weewx[9631]: ftpupload: Attempted 3 times to connect t o server 87.247.245.132. Giving up.

Dec 23 09:21:15 localhost weewx[9631]: ftpgenerator: ftp'd 0 files in 0.12 seconds

Dec 23 09:21:15 localhost weewx[9631]: reportengine: Running report 'RSYNC'

Dec 23 09:21:15 localhost weewx[9631]: reportengine: Found configuration file /etc/weewx/skins/Rsync/skin.conf for report 'RSYNC'

Dec 23 09:21:15 localhost weewx[9631]: rsyncgenerator: rsync upload not requested. Skipped.

Dec 23 09:21:41 localhost weewx[9631]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

Dec 23 09:21:41 localhost rsyslogd-2007: action 'action 17' suspended, next retry is Mon Dec 23 09:23:11 2019 [try http://www.rsyslog.com/e/2007 ]

Dec 23 09:22:10 localhost weewx[9631]: wmr100: Bad USB report received: could not detach kernel driver from interface 0: No data available

^C

Using nmap against the ISP’s server and port 21 I get

Starting Nmap 7.80 ( https://nmap.org ) at 2019-12-22 17:18 GMT Standard Time

Nmap scan report for honolulu.footholds.net (87.247.245.132)

Host is up (0.020s latency).

PORT STATE SERVICE

21/tcp open ftp

| ssl-enum-ciphers:

| TLSv1.2:

| ciphers:

| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A

| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A

| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A

| TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 2048) - A

| compressors:

| NULL

| cipher preference: server

|_ least strength: A

Does that help?

Neil

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

Thomas Keffer

unread,

Dec 23, 2019, 7:09:54 AM12/23/19

to weewx-user

Not sure what's happening.

If I run nmap from my end, it reveals that your ISP is using an implementation of Pure-FTPd. Looking through the Pure-FTPd documentation, it lists WinSCP as being compatible, but it doesn't say anything one way or another about Python's client implementation. It's possible there's an incompatibility (although I doubt it).

It's also possible it's a firewall issue, although the fact that nmap was able to scan the port suggests not.

You mention you are able to connect with WinSCP. I assume that's on a Windows host, not a Linux host. Are you able to connect using any client on your Linux box, such as FileZilla, or the FTP command line?

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fde5a388-9bc8-4dac-a1f8-fe62fb002a4b%40googlegroups.com.

Neil S

unread,

Dec 23, 2019, 10:15:13 AM12/23/19

to weewx-user

Yes WinSCP is on Win10 machine. Tried Filezilla on Win10 machine and popped up an error about certificates not matching. (Which is reasonable as the core cert is footholds.net

Tried lftp on the raspberry pi itself and came up with this

root@pi-Weather:~# lftp -p 21 -u xxxxx,yyyyyy ftp.mydomain.co.uk

lftp xxxxx...@ftp.mydomain.co.uk:~> ls

ls: Fatal error: Certificate verification: certificate common name doesn't match requested host name 'ftp.mydomain.co.uk'

lftp xxxxx...@ftp.mydomain.co.uk:~>

Note I have replaced username with XXXX, password with YYYYY and my domain name with mydomain

Is it an issues with the remote certificate not being the correct name 'ftpg.mydomain.co.uk' ?

Neil

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fde5a388-9bc8-4dac-a1f8-fe62fb002a4b%40googlegroups.com.

Thomas Keffer

unread,

Dec 23, 2019, 6:52:31 PM12/23/19

to weewx-user

It would appear so, but I'm not much of a SSL expert.

What happens if you try to connect to simply mydomain.co.uk. that is, without the 'ftp' in front?

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ac5063b1-82de-4554-9a1a-164816461472%40googlegroups.com.

Neil S

unread,

Dec 27, 2019, 12:36:44 PM12/27/19

to weewx-user

That doesnt work either but then I didnt expect it to as ftp.mycomain.co.uk and mydomain.co.uk and www.mydomain.co.uk all resolve to the same IP address which is what I tried om Post #3

Anyway thanks for the pointers on lftp. Hammer and possibly nut here but I have now got uploads on the go again.

Installed lftp on the weewxx machine (Raspberry Pi) and in the \etc\lftp.conf put the lines

set ssl:verify-certificate no

set ftp:ssl-protect-data true

(OK I know that ignores the certificate but think that it is OK in this instance as it is my ISP and all I am uploading is public web pages)

Then upload the files into the weather subdirectory on the server...

lftp -p 21 -u user,password ftp.mydomain.co.uk -e "mput /var/www/weewx/* -O /weather/; bye"

Bingo - files uploaded from weewx folder to public web pages.

Now the weewx generates the pages every five minutes (on hour, 5 past 10 past etc) and in general takes about 80 secs to complete its generation and various uploads to Wunderground, WOW Weather etc. So at 2 minutes past, 7 minutes past, 12 minutes past etc I run the lftp comman as a cron job using ..

2-59/5 * * * * lftp -p 21 -u user,password ftp.mydomain.co.uk -e "mput /var/www/weewx/* -O /weather/; bye"

As I said most probably, nut , hammer, crack, nut etc etc but at least things are working again.

Neil

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ac5063b1-82de-4554-9a1a-164816461472%40googlegroups.com.

Thomas Keffer

unread,

Dec 27, 2019, 2:10:26 PM12/27/19

to weewx-user

Whatever works! Sorry I couldn't be of more help.

One other option: try Matthew's SFTP generator: https://github.com/matthewwall/weewx-sftp

Or, of course, rsync. It's much faster, as well as secure.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4a27399a-bdb0-48b2-9618-d96f6c1c271f%40googlegroups.com.

Reply all

Reply to author

Forward