Error with weewx on openwrt: import of driver failed: No backend available (<class 'usb.core.NoBackendError'>)

[Bernhard Frieß]

Hi There, WeeWX Gurus,

I just bought a FineOffset WH1080 weather station and installe dit. For monitoring I purchased a GL.inet GL-MT300N V2.

I extended disk capacity by extroot and an USB stick, installed a full Python 2.7, all necessary modules and then WeeWX 3.8.0 from source.

It seemed to work but it doesn't.

here ist the relevant part of the log file:

Sat Jun 16 17:01:56 2018 user.info weewx[3596]: engine: Initializing weewx version 3.8.0
Sat Jun 16 17:01:56 2018 user.info weewx[3596]: engine: Using Python 2.7.15 (default, Jun 13 2018, 10:54:45)  [GCC 7.3.0]
Sat Jun 16 17:01:56 2018 user.info weewx[3596]: engine: Platform Linux-4.14.48-mips-with-glibc2.0
Sat Jun 16 17:01:56 2018 user.info weewx[3596]: engine: Locale is 'C.UTF-8;C;C;C;C;C'
Sat Jun 16 17:01:56 2018 user.info weewx[3596]: engine: Using configuration file /home/weewx/weewx.conf
Sat Jun 16 17:01:56 2018 user.info weewx[3596]: engine: Loading station type FineOffsetUSB (weewx.drivers.fousb)
Sat Jun 16 17:01:57 2018 user.info weewx[3596]: fousb: driver version is 1.9
Sat Jun 16 17:01:57 2018 user.info weewx[3596]: fousb: polling mode is PERIODIC
Sat Jun 16 17:01:57 2018 user.info weewx[3596]: fousb: polling interval is 60
Sat Jun 16 17:01:58 2018 user.err weewx[3596]: import of driver failed: No backend available (<class 'usb.core.NoBackendError'>)
Sat Jun 16 17:01:58 2018 user.crit weewx[3596]: engine: Unable to load driver: No backend available
Sat Jun 16 17:01:58 2018 user.crit weewx[3596]:     ****  Exiting...

I think the key lines are

Sat Jun 16 17:01:58 2018 user.err weewx[3596]: import of driver failed: No backend available (<class 'usb.core.NoBackendError'>)
Sat Jun 16 17:01:58 2018 user.crit weewx[3596]: engine: Unable to load driver: No backend available
Sat Jun 16 17:01:58 2018 user.crit weewx[3596]:     ****  Exiting...

I attach additionally the configuration file weewx.conf.

Would be great if anyone here has an idea and could help...

Best regards

Bernhard

[Thomas Keffer]

You are missing the underlying usb "C" library.

See this thread.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bernhard Frieß]

Hi Thomas,

Many thanks for your reply. What do you mena with "the underlying usb "C" library? I've installed the package libusb-1.0.

What else is required?

In the weewx documentation it says that for instance python-usb is required but on OpenWRT I can't find that package?! Is this potentially the issue?

The thread your pointing to is this one here?

I'm looking forward for additional help.

Best regards

Bernhard

[Thomas Keffer]

The package python-usb is definitely required. You must have installed it, or something like it, or else the Fine Offset driver would not have gotten far enough to have the "backend" error.

What do you get if you run this:

python -c "import usb; print usb.__version__"

-tk

[Thomas Keffer]

It seems that user Hiljo is having a similar problem.

-tk

[Bernhard Frieß]

root@LEDE:/# python -c "import usb; print usb.__version__"
1.0.0
root@LEDE:/#

but on OpenWRT there is no python-usb to be installed so I installed pyusb 1.0.0 from github. This was mentioned in one of the documentations I found on the web.

[Bernhard Frieß]

Yes, I'm following that topic in parallel

[Thomas Keffer]

So, the 'import usb' command works. That means that pyusb was successfully installed. But, it cannot find a "C" backend.

Yes, I'm following that topic in parallel

​Good. Then I would suggest trying the same command:

find /usr/lib -name "libusb*"​

-tk

[Bernhard Frieß]

nothing:

root@LEDE:/home/weewx# find /usr/lib -name "libusb*"​
root@LEDE:/home/weewx# find /usr/lib -name "libusb*"​ -print
root@LEDE:/home/weewx#

[Bernhard Frieß]

Aber:

root@LEDE:/usr/lib# ls -al libusb*
lrwxrwxrwx    1 root     root            19 Jun 17 10:24 libusb-0.1.so.4 -> libusb-0.1.so.4.4.4
-rwxr-xr-x    1 root     root         12503 Jun 14 06:22 libusb-0.1.so.4.4.4
lrwxrwxrwx    1 root     root            19 Jun 17 05:29 libusb-1.0.so -> libusb-1.0.so.0.1.0
lrwxrwxrwx    1 root     root            19 Jun 17 05:29 libusb-1.0.so.0 -> libusb-1.0.so.0.1.0
-rwxr-xr-x    1 root     root         49795 Jun 14 05:52 libusb-1.0.so.0.1.0
root@LEDE:/usr/lib#

[Thomas Keffer]

OK, so we've established that you have some backend libraries, but for some reason, pyusb is unable to find them. Try this:

PYUSB_DEBUG="debug" python -c "import usb; usb.busses()"

-tk

--

[Bernhard Frieß]

Here we go:

root@LEDE:/usr/lib# PYUSB_DEBUG="debug" python -c "import usb; usb.busses()"
2018-06-17 16:53:02,351 ERROR:usb.libloader:'Libusb 1' could not be found
2018-06-17 16:53:02,356 ERROR:usb.backend.libusb1:Error loading libusb 1.0 backend
2018-06-17 16:53:02,409 ERROR:usb.libloader:'OpenUSB library' could not be found
2018-06-17 16:53:02,413 ERROR:usb.backend.openusb:Error loading OpenUSB backend
2018-06-17 16:53:02,561 ERROR:usb.libloader:'Libusb 0' could not be found
2018-06-17 16:53:02,565 ERROR:usb.backend.libusb0:Error loading libusb 0.1 backend
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/usb/legacy.py", line 353, in busses
    sorted(core.find(find_all=True), key=lambda d: d.bus),
  File "/usr/lib/python2.7/site-packages/usb/core.py", line 1263, in find
    raise NoBackendError('No backend available')
usb.core.NoBackendError: No backend available
root@LEDE:/usr/lib#

[Bernhard Frieß]

Hi There,

it seems I found the issue and were able to solve it.

I overread the the cruical paragraph in the description at https://github.com/weewx/weewx/wiki/weewx-on-openwrt which says:

Then to fix error with backend driver missing you need to help the
/home/weewx/bin/weewx/drivers/wmr200.py driver file find the installed
usb driver:

import usb

import usb.backend.libusb1
backend = usb.backend.libusb1.get_backend(find_library=lambda x: "/usr/lib/libusb-1.0.so")

Which I considered irrelevant for me as it is clearly connected to a different weather station type and a different driver than I use. However the issue is not specific to any specific driver for a weather station but it seems relevant for the drivers of all weather stations which connect via USB to the device where weewx runs on OpenWRT!

After I have changed the file /home/weewx/bin/weewx/drivers/fousb.py in the following way

before:

import usb

import weewx.drivers

after:

import usb

import usb.backend.libusb1
backend = usb.backend.libusb1.get_backend(find_library=lambda x: "/usr/lib/libus

import weewx.drivers

weewx starts and collects data from my weather station.

Many thanks, Thomas, for the good support and the hints which helped me to identify the relevance of a change described in a driver which looked irrelevant at first sight.

Best regards

Bernhard

[Thomas Keffer]

I'm glad that worked out for you, but I'd like to find a more general solution. 

Would you be willing to try the following? For these tests, you need to run the Python interpreter (rather than using the '-c' option):

python

>>> from ctypes.util import find_library

>>> print find_library('c')

>>> print find_library('usb-1.0')

>>> print find_library('usb-0.1')

>>> import usb.core

>>> print usb.core.find()

The first one (find library 'c') just tests the basic functionality of find_library.

The second one ('usb-1.0') tests to see if it can find libusb-1.0

The third is similar, but for libusb-0.1

Let me know what it shows!

-tk

[Bernhard Frieß]

Hi Thomas,

No problem at all! Please find as follows the requested input tgether with the output:

root@LEDE:~# python

Python 2.7.15 (default, Jun 13 2018, 10:54:45)

[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library
>>> print find_library('usb-1.0')
None
>>> print find_library('usb-0.1')
None
>>> import usb.core
>>> print usb.core.find()

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/usr/lib/python2.7/site-packages/usb/core.py", line 1263, in find
    raise NoBackendError('No backend available')
usb.core.NoBackendError: No backend available
>>>

Hope that helps!

Additionally I'd like to propose an update for the weewx installation documentation at http://www.weewx.com/docs/setup.htm

There it reads:

# required packages:
sudo apt-get install python-configobj
sudo apt-get install python-cheetah
sudo apt-get install python-imaging
          
# required if hardware is serial or USB:
sudo apt-get install python-serial
sudo apt-get install python-usb

where my feeling is that python-imaging is outdated. At least it is not available on OpenWRT while pillow should be a fairly well working replacement. At least on my embedded device with OpenWRT on it pillow does the job. Does this make sense for you? Therefore I'd suggest a change to

# required packages:
sudo apt-get install python-configobj
sudo apt-get install python-cheetah
sudo apt-get install Pillow
          
# required if hardware is serial or USB:
sudo apt-get install python-serial
sudo apt-get install python-usb

Best regards

Bernhard

[Thomas Keffer]

Thanks. You didn't run one of the tests, but it appears that the version of find_library() on your version of OpenWRT is not working. Let's confirm by running the missing test:

python

>>> from ctypes.util import find_library

>>> print find_library('c')

Assuming this fails (returns None), then the problem is find_library(). 

Is OpenWRT posix? Confirm by running

python -c "import os; print os.name"

If it is, then find_library() depends on running the system command ldconfig in order to find the library. Confirm that you have ldconfig, and that libusb is in its cache:

which ldconfig

ldconfig -p | grep usb

-tk

[Bernhard Frieß]

See results below inline

Am Montag, 18. Juni 2018 13:46:01 UTC+2 schrieb Thomas Keffer:

Thanks. You didn't run one of the tests, but it appears that the version of find_library() on your version of OpenWRT is not working. Let's confirm by running the missing test:

python

>>> from ctypes.util import find_library

  >>> print find_library('c')

root@LEDE:~# python
Python 2.7.15 (default, Jun 13 2018, 10:54:45)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library

>>> print find_library('c')
None

Assuming this fails (returns None), then the problem is find_library(). 

Yes, your right, the result was "None"

Is OpenWRT posix? Confirm by running

python -c "import os; print os.name"

OpenWRT seems to be posix

root@LEDE:~# python -c "import os; print os.name"
posix

 

If it is, then find_library() depends on running the system command ldconfig in order to find the library. Confirm that you have ldconfig, and that libusb is in its cache:

which ldconfig

ldconfig -p | grep usb

There seems to be no ldconfig on OpenWRT:

root@LEDE:~# which ldconfig
root@LEDE:~# ldconfig -p | grep usb
-ash: ldconfig: not found

The only information I found about ldconfig on OpenWRT.org is an article available in Chinese language only. Here is the translation by google translate: https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=de&ie=UTF-8&u=https%3A%2F%2Fopenwrt.org%2Fstart%3Fid%3Dzh%2Fdocs%2Ftechref%2Fopkg&edit-text=&act=url

[Thomas Keffer]

OK, so we have found the root cause: your version of OpenWRT is missing ldconfig. 

I am not familiar with OpenWRT, but it's quite possible ldconfig can be installed using the package utility opkg. Perhaps it's as simple as 'opkg install ldconfig'??

You will have to consult your OpenWRT documentation, or, perhaps, ask on its support forum.

-tk

[Bernhard Frieß]

Hi Thomas,

Yes and I believe there is no easier fix from your end. I tried already your proposal and unfortunately it didn't work

root@LEDE:~# opkg install ldconfig
Unknown package 'ldconfig'.
Collected errors:
 * opkg_install_cmd: Cannot install package ldconfig.
root@LEDE:~#

But I will check with the OpenWRT eco system and see what I can achieve there. If there is any progress, I will post it here.

Btw did you see my remark about the setup.htm of weewx documentation? It seems that python-imaging could be replaced by Pillow? Does this make sense for you?

Many thnaks for your support!

Best regards

Bernhard

[Thomas Keffer]

Sorry. Forgot to respond about python-imaging.

The documentation for V3.8.1 will reflect this. It will recommend trying python-pil first then, if that doesn't work, python-imaging.

-tk

[Thomas Keffer]

By the way, what version of OpenWRT  are you running?

[Thomas Keffer]

I ran OpenWRT 15.05.1 (on a virtual machine) and was able to install ldconfig using opkg.

However, using OpenWRT 17.04.1, the package was unavailable.

So, it seems that, either on purpose or accidentally, ldconfig was dropped in v17.04

-tk

[Bernhard Frieß]

Hi Thomas,

How can I find out? When I log into the device via ssh it says at the very beginning

OpenWrt SNAPSHOT, r7200-2f3c5fe

I would assume that this means that it is a snapshot of the current development of the 18.x release tree, but I don't know. The reason is that for my device there is no other (more stable) firmware available.

Best regards

Bernhard

[Thomas Keffer]

It is. It was committed just 3 days ago! I'm not surprised that it does not include ldconfig.

I think you need to follow up on the OpenWRT forum.

-tk

[Bernhard Frieß]

will do and keep you posted... Many thanks for the profound support!

If anyone is interested you can find data of my station at the following locations:

http://wow.metoffice.gov.uk/observations/details?site_id=f5a5aecd-7a73-e811-9ccb-0003ff59b2da

https://www.wunderground.com/weather/de/malschenberg/IRAUENBE10

The upload to wetter.com worked a while until yesterday 18:35 and doesn't work since then, I need to check again

I'm running a Fine Offste WH1080 which is connected to a GL.iNet GL-MT300N-V2 running OpenWRT and WeeWX

Additionally I have the GL-MT300N-V2 serving a weather webpage on my local network which I plan to publish on the public web as well but for that I need to familiarize myself with rsync from the GL-MT300N-V2 to one of my Synologies serving a webserver to the public internet.

[bernhard.fr...@gmail.com]

Hi Folks,

I'm having some funny discussions with some OpenWRT/LEDE guys. They believe that the issue is not ldconfig missing on OpenWRT/Lede but weewx requiring ldconfig...

If someone is interested you can read the discussion here.

I will pull myself out of the discussion at the lede forum now because I feel it doesn't bring any tangible result.

Bottom line for me: there is no appetite for ldconfig on OpenWRT/Lede and there will happen nothing in this direction.

Best regards

Bernhard

[gjr80]

Hmmm, if only we could catch some of that forum's vibe here. Maybe we could use the capitalisation police, the recent increased use of weewx/weeWX/weeWx has been disturbing ...
:)

Gary

[Thomas Keffer]

Bernard, I think you would have helped your case if you had pointed out that ldconfig is required by the standard python library, and not directly by WeeWX. 

-tk

Still, 

[RobbH]

At risk of taking a humorous comment too seriously (I did notice the smiley), which capitalization scheme is most correct? I see all three used in official documents, but "weeWX" seems to be the preferred form.

[gjr80]

I am not sure if it was ever documented publicly (unless I missed the 'weeWX Style guide') but I seem to remember an email exchange amongst the developers a good few releases ago (actually looking at the weeWX repo there was wholesale changing of weewx to weeWX in January 2017 in preparation for the 3.7.0 release) that said we would use weeWX as the name for weeWX, but if it was used at the start of a sentence then use WeeWX. Of course weewx.conf is always all lowers.

Tom and Matthew may have a different recollection.

Gary

[Thomas Keffer]

That's pretty much it, but with the introduction of WeeRT, symmetry demanded WeeWX. It also avoids the need for special capitalization at the beginning of a sentence, at the cost of an extra press of the shift key.

-tk

--

[bernhard.fr...@gmail.com]

Hi Thomas,

You're probably right, however I'm not a developer and no I have to live with the result I have achieved or the lack of :)

Thank you and all the other brilliant guys here that you maintain a different climate here!

Best regards

Bernhard

[Liz]

On Mon, 9 Jul 2018 10:21:18 -0700 (PDT)
bernhard.fr...@gmail.com wrote:

> Hi Folks,
>
> I'm having some funny discussions with some OpenWRT/LEDE guys. They
> believe that the issue is not ldconfig missing on OpenWRT/Lede but
> weewx requiring ldconfig...
>
> If someone is interested you can read the discussion here

> <https://forum.lede-project.org/t/ldconfig-missing-on-openwrt-lede/16802/3>.

>
> I will pull myself out of the discussion at the lede forum now
> because I feel it doesn't bring any tangible result.
>
> Bottom line for me: there is no appetite for ldconfig on OpenWRT/Lede
> and there will happen nothing in this direction.
>
> Best regards
>
> Bernhard

"We have Policy A".
"We have Policy C".
Where A and C are mutually incompatible....

I agree, don't waste your time with people who are not interested in
change, another view or even Policy Z.

Liz

[Liz]

On Tue, 10 Jul 2018 18:17:11 -0700
Thomas Keffer <tke...@gmail.com> wrote:

> That's pretty much it, but with the introduction of WeeRT, symmetry
> demanded WeeWX. It also avoids the need for special capitalization at
> the beginning of a sentence, at the cost of an extra press of the
> shift key.
>
> -tk

If we get it wrong, will we be excommunicated?

;)

Liz

[gjr80]

Missed this post, I think this might cause a problem for Matthew though. Fortunately I have a spare keyboard with working shift and caps keys, will get it sent over right away :)

Gary