sftp not working properly in WEEWX

[Robert Isbell]

I have Lubuntu 16.10, WEEWX, and a Davis Pro weather station. In the last year 1and1 changed their FTP to SFtp, so I tried installing Matthew Wall's sftp for WEEWX in an attempt to be able to upload to my linux page on 1and1. After installing Matthew's software and the pysftp extensions, I am now getting additional errors as shown in the excerpt of my syslog. Can anyone help?

[mwall]

On Monday, July 9, 2018 at 12:12:11 AM UTC-4, Robert Isbell wrote:

I have Lubuntu 16.10, WEEWX, and a Davis Pro weather station. In the last year 1and1 changed their FTP to SFtp, so I tried installing Matthew Wall's sftp for WEEWX in an attempt to be able to upload to my linux page on 1and1. After installing Matthew's software and the pysftp extensions, I am now getting additional errors as shown in the excerpt of my syslog. Can anyone help?

hi robert,

the exception:

Jul  8 20:30:17 Alder weewx[24552]:         ****  TypeError: from_buffer() cannot return the address of the raw string within a str or unicode or bytearray object

Jul  8 20:30:17 Alder weewx[24552]:         ****  Generator terminated

might be due to mixed up python encryption libraries.

what operating system and operating system version are you running?

could you post the paramiko, cffi, and pysftp versions?  post the output from these commands:

pip freeze | grep paramiko

pip freeze | grep cffi

pip freeze | grep pysftp

(if ubuntu keeps messing up their dependencies like this, we will soon have to spit out an entire python module version listing in the log file)

m 

[Robert Isbell]

Sorry it's taken so long to get back. My CPU fan failed and I needed to get a new one.

I have Ubuntu 16.04 with the XFCE UI. Here is a file with the requested info...

[mwall]

On Friday, July 13, 2018 at 5:44:26 PM UTC-4, Robert Isbell wrote:

Sorry it's taken so long to get back. My CPU fan failed and I needed to get a new one.

I have Ubuntu 16.04 with the XFCE UI. Here is a file with the requested info...

this is one of the stack traces that you posted:

$  pip freeze | grep pysftp

Traceback (most recent call last):

  File "/usr/local/bin/pip", line 7, in <module>

    from pip._internal import main

  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/__init__.py", line 42, in <module>

    from pip._internal import cmdoptions

  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/cmdoptions.py", line 16, in <module>

    from pip._internal.index import (

  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/index.py", line 15, in <module>

    from pip._vendor import html5lib, requests, six

  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py", line 86, in <module>

    from pip._vendor.urllib3.contrib import pyopenssl

  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/urllib3/contrib/pyopenssl.py", line 46, in <module>

    import OpenSSL.SSL

  File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module>

    from OpenSSL import rand, crypto, SSL

  File "/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 118, in <module>

    SSL_ST_INIT = _lib.SSL_ST_INIT

AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

it looks like your python installation is broken, or at least pip is broken.  more specifically, it looks like your python installation is expecting a specific openssl backend implementation, but it is not there.  perhaps something else failed when you were installing other packages and you ignored or missed the failures?

beware that you can easily screw up your operating system if you sometimes use pip to install python packages but other times use apt.  this is more of a problem on redhat and derivatives, but it can also bite you on debian and its derivatives.  in general, you should use one or the other.

unfortunately it looks like the pysftp module is not available as a debian/ubuntu package.  so you must do a pip install or install pysftp directly from its source.

anyway, you need to fix your python installation before you can do anything else.  it is really hard to say how to do that without knowing exactly what commands you executed to get the system into its current, broken state.

m

[Robert Isbell]

I'm afraid I have no record of how I did the install either, and cannot recall. Would I be best off to start with a fresh install of Linux, Weewx, and python? What distro/version of Linux would best accommodate Weewx and python so I can use your sftp driver? I was trying to avoid having to learn about SSL certificates, but 1and1 is making id difficult to upload my files to my web page. I'm not a total neophyte, but not an expert either. I started out with Suse years ago, have had a taste of Redhat, and most recently Ubuntu 

On Sunday, July 8, 2018 at 9:12:11 PM UTC-7, Robert Isbell wrote:

[Robert Isbell]

I just read how to uninstall python and pip. Should I do that then re-install using apt. According to the Weewx documentation I should go with version 2.5, 2.6, or 2.7, but NOT 3.0. Should I re-install version 2.7?

On Sunday, July 8, 2018 at 9:12:11 PM UTC-7, Robert Isbell wrote: