IOError: open failed

[Mark Whalley]

Having successfully set up a Maplin USB Weather Station (code: N96GY) on my RPi, it had been capturing data and uploading to my web site on the hour every hour for the past few months.

http://www.auvergneitc.com/weather/data/7days.png

This is triggered by a crontab entry:

0 * * * * root /usr/local/bin/weatherupload 1>>/pi01/media/weather/logs/weatherupload.log 2>&1

and weatherupload contains:

#!/bin/bash
cd /pi01/media/weather/
python Hourly.py -vvv data

 A couple of days ago, I noticed it had stopped working.  On checking the log file, I am getting the following errors every hour:

Traceback (most recent call last):
  File "Hourly.py", line 90, in <module>
    sys.exit(main())
  File "Hourly.py", line 87, in main
    return Hourly(args[0])
  File "Hourly.py", line 53, in Hourly
    LogData.LogData(params, raw_data)
  File "/pi01/media/weather/pywws/LogData.py", line 129, in LogData
    ws = WeatherStation.weather_station(ws_type=ws_type)
  File "/pi01/media/weather/pywws/WeatherStation.py", line 353, in __init__
    self.cusb = CUSBDrive(library)
  File "/pi01/media/weather/pywws/WeatherStation.py", line 281, in __init__
    self.dev = USBDevice(0x1941, 0x8021)
  File "/pi01/media/weather/pywws/device_cython_hidapi.py", line 94, in __init__
    self.hid = hid.device(idVendor, idProduct)
  File "hid.pyx", line 42, in hid.device.__cinit__ (hid.c:1064)
IOError: open failed

Going back to basics, I tried a "TestWeatherStation.py", but this also fails:

06:47:09:pywws.WeatherStation.CUSBDrive:using pywws.device_cython_hidapi
Traceback (most recent call last):
  File "TestWeatherStation.py", line 145, in <module>
    sys.exit(main())
  File "TestWeatherStation.py", line 90, in main
    ws = WeatherStation.weather_station()
  File "/pi01/media/weather/pywws/WeatherStation.py", line 353, in __init__
    self.cusb = CUSBDrive(library)
  File "/pi01/media/weather/pywws/WeatherStation.py", line 281, in __init__
    self.dev = USBDevice(0x1941, 0x8021)
  File "/pi01/media/weather/pywws/device_cython_hidapi.py", line 94, in __init__
    self.hid = hid.device(idVendor, idProduct)
  File "hid.pyx", line 42, in hid.device.__cinit__ (hid.c:1064)
IOError: open failed

Although I have some coding experience (shell scripting etc), my Python knowledge is minimal :-(

Any thoughts on why these errors should "suddenly" start appearing?

Thanks in advance
Mark Whalley

[Jim Easterbrook]

On 20/07/13 07:26, Mark Whalley wrote:
> Having successfully set up a Maplin USB Weather Station (code: N96GY) on
> my RPi, it had been capturing data and uploading to my web site on the
> hour every hour for the past few months.
>

> A couple of days ago, I noticed it had stopped working. On checking
> the log file, I am getting the following errors every hour:
>

> self.hid = hid.device(idVendor, idProduct)
> File "hid.pyx", line 42, in hid.device.__cinit__ (hid.c:1064)
> IOError: open failed
>

> Any thoughts on why these errors should "suddenly" start appearing?

It sounds as if it might be the infamous "USB lockup" problem. Does the
station console allow you to see logged records? (Press 'memory' once
and the '+' and '-' buttons should start flashing, allowing you to
examine previous readings.) If you can't see logged data, it's probably
hung up. The only cure is to remove the batteries (and USB lead) to
power down the console, losing its stored data.

You haven't said what version of pywws you're running. I assume it's
from before November 2012 when the software was changed to try and avoid
the USB lockup problem.

--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[Mark Whalley]

On Saturday, 20 July 2013 10:12:25 UTC+1, Jim Easterbrook wrote:

It sounds as if it might be the infamous "USB lockup" problem. Does the
station console allow you to see logged records? (Press 'memory' once
and the '+' and '-' buttons should start flashing, allowing you to
examine previous readings.) If you can't see logged data, it's probably
hung up. The only cure is to remove the batteries (and USB lead) to
power down the console, losing its stored data.

You haven't said what version of pywws you're running. I assume it's
from before November 2012 when the software was changed to try and avoid
the USB lockup problem.

--
Jim Easterbrook                 <http://www.jim-easterbrook.me.uk/>

---
 

Hi Jim

 

Thanks for your response.

 

According to PKG-INFO: Version: 12.05_r521.

 

I can confirm that the station console does show logged records (@ half-hourly intervals).  I have since reset the station console (disconnecting USB cable and removal of batteries).

 

On reconnecting the station, /var/log/messages records the following:

 

Jul 21 06:51:30 pennardpi01 kernel: [86880.206431] usb 1-1.3.4.4: new low-speed USB device number 17 using dwc_otg

Jul 21 06:51:30 pennardpi01 kernel: [86880.666372] usb 1-1.3.4.4: new low-speed USB device number 18 using dwc_otg

Jul 21 06:51:31 pennardpi01 kernel: [86881.126304] usb 1-1.3.4.4: new low-speed USB device number 19 using dwc_otg

Jul 21 06:51:31 pennardpi01 kernel: [86881.626240] usb 1-1.3.4.4: new low-speed USB device number 20 using dwc_otg

 

This morning I renamed the “weather” directory and did a fresh install of pywws (v13.06), but I am still getting the same errors.

 

The following snippet had come from sudo python -vvv -m pywws.TestWeatherStation:

 

# trying /usr/lib/python2.7/lib-dynload/hid.py

# trying /usr/lib/python2.7/lib-dynload/hid.pyc

# trying /usr/local/lib/python2.7/dist-packages/hid.so

dlopen("/usr/local/lib/python2.7/dist-packages/hid.so", 2);

import hid # dynamically loaded from /usr/local/lib/python2.7/dist-packages/hid.so

Traceback (most recent call last):

  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main

    "__main__", fname, loader, pkg_name)

  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code

    exec code in run_globals

  File "/pi01/media/weather/pywws/pywws/TestWeatherStation.py", line 176, in <module>

    sys.exit(main())

  File "/pi01/media/weather/pywws/pywws/TestWeatherStation.py", line 111, in main

    ws = WeatherStation.weather_station()

  File "pywws/WeatherStation.py", line 318, in __init__

    self.cusb = CUSBDrive()

  File "pywws/WeatherStation.py", line 241, in __init__

    self.dev = USBDevice(0x1941, 0x8021)

  File "pywws/device_cython_hidapi.py", line 113, in __init__

    self.hid = hid.device(idVendor, idProduct)

  File "hid.pyx", line 42, in hid.device.__cinit__ (hid.c:1064)

IOError: open failed

# clear __builtin__._

 

I have attached the full verbose output if that is of any help

 

I will continue to investigate, but would welcome any further thoughts you may have.

 

Kindest regards

Mark

 

[Jim Easterbrook]

On 21/07/13 08:24, Mark Whalley wrote:
>
> According to PKG-INFO: Version: 12.05_r521.

That's over a year old (May 2012). I don't know who's responsible for
repackaging pywws, but if they're not going to keep up do date I'd
rather they didn't do it. It's easy enough to install from PyPI.

> I can confirm that the station console does show logged records (@

> half-hourly intervals).I have since reset the station console

> (disconnecting USB cable and removal of batteries).

Doesn't seem to be a lockup then.

> This morning I renamed the �weather� directory and did a fresh install

> of pywws (v13.06), but I am still getting the same errors.
>
> The following snippet had come from sudo python -vvv -m
> pywws.TestWeatherStation:
>
> # trying /usr/lib/python2.7/lib-dynload/hid.py
>
> # trying /usr/lib/python2.7/lib-dynload/hid.pyc
>
> # trying /usr/local/lib/python2.7/dist-packages/hid.so
>
> dlopen("/usr/local/lib/python2.7/dist-packages/hid.so", 2);
>
> import hid # dynamically loaded from
> /usr/local/lib/python2.7/dist-packages/hid.so

Is it usual to use hid on a Pi (USB library option 3 or 4)? I'd have
thought that using PyUSB (library option 1 or 2) would be a better bet.
I wonder if the reason why your working system suddenly stopped was
because something installed hid.

You could try temporarily disabling cython_hidapi. Edit
WeatherStation.py and change the line
from .device_cython_hidapi import USBDevice
to
from .device_cython_hidapi import xxxUSBDevice

[Mark Whalley]

Hi Jim

I've change WeatherStation.py:

if not USBDevice:
    try:
#       from pywws.device_cython_hidapi import USBDevice
        from pywws.device_cython_hidapi import xxxUSBDevice
    except ImportError:
        pass

and now get:

# /usr/local/lib/python2.7/dist-packages/usb/backend/openusb.pyc matches /usr/local/lib/python2.7/dist-packages/usb/backend/openusb.py
import usb.backend.openusb # precompiled from /usr/local/lib/python2.7/dist-packages/usb/backend/openusb.pyc

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/pi01/media/weather/pywws/pywws/TestWeatherStation.py", line 176, in <module>
    sys.exit(main())
  File "/pi01/media/weather/pywws/pywws/TestWeatherStation.py", line 111, in main
    ws = WeatherStation.weather_station()

  File "pywws/WeatherStation.py", line 319, in __init__
    self.cusb = CUSBDrive()
  File "pywws/WeatherStation.py", line 242, in __init__
    self.dev = USBDevice(0x1941, 0x8021)
  File "pywws/device_pyusb1.py", line 96, in __init__
    raise IOError("Weather station device not found")
IOError: Weather station device not found
# clear __builtin__._

Regards
Mark

[Jim Easterbrook]

On 21/07/13 10:08, Mark Whalley wrote:
>
> I've change WeatherStation.py:
>
> if not USBDevice:
> try:
> # from pywws.device_cython_hidapi import USBDevice
> from pywws.device_cython_hidapi import xxxUSBDevice
> except ImportError:
> pass
>
> and now get:
>

> File "pywws/device_pyusb1.py", line 96, in __init__
> raise IOError("Weather station device not found")
> IOError: Weather station device not found

Ah, that's more meaningful. Does it actually show up in the output of
lsusb? (Its ID is 1941:8021.)

jim@brains:~/Documents/projects/pywws$ lsusb
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 1941:8021 Dream Link WH1080 Weather Station / USB
Missile Launcher
Bus 002 Device 007: ID 0451:2046 Texas Instruments, Inc. TUSB2046 Hub
Bus 002 Device 008: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth
Dongle (HCI mode)
Bus 002 Device 009: ID 18e3:9106 Fitipower Integrated Technology Inc
Bus 002 Device 010: ID 413c:2107 Dell Computer Corp.
Bus 002 Device 011: ID 0461:4d65 Primax Electronics, Ltd
Bus 002 Device 012: ID 06f2:0011 Emine Technology Co. KVM Switch Keyboard

[Mark Whalley]

On Sunday, 21 July 2013 10:40:40 UTC+1, Jim Easterbrook wrote:

Ah, that's more meaningful. Does it actually show up in the output of
lsusb? (Its ID is 1941:8021.)

Ah, that's interesting.

My weather station was plugged into a 7-port hub (with its own power supply).  As well as the weather station, I also have a RTL2838 DVB-TV dongle (capturing ADS-B aircraft transmissions - this is working fine).  The hub is connected to 1 of the Pi's USB ports - the other port has a 1TB USB Seagate disk.

I noticed that when I unplugged then plugged in the weather station on the USB hub, rather than showing a constant light on the hub, it would flicker several times before remaining unlit. This corresponded to a number of connect/disconnect messages in /var/log/messages, e.g.:

Jul 21 10:48:06 pennardpi01 kernel: [101075.266264] usb 1-1.3.4.2: USB disconnect, device number 48
Jul 21 10:48:06 pennardpi01 kernel: [101075.505378] usb 1-1.3.4.2: new low-speed USB device number 49 using dwc_otg
Jul 21 10:48:07 pennardpi01 kernel: [101076.105251] usb 1-1.3.4.2: new low-speed USB device number 50 using dwc_otg
Jul 21 10:48:07 pennardpi01 kernel: [101076.234771] usb 1-1.3.4.2: New USB device found, idVendor=1941, idProduct=8021
Jul 21 10:48:07 pennardpi01 kernel: [101076.234801] usb 1-1.3.4.2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Jul 21 10:48:07 pennardpi01 kernel: [101076.258965] hid-generic 0003:1941:8021.0009: hiddev0,hidraw0: USB HID v1.00 Device [HID 1941:8021] on usb-bcm2708_usb-1.3.4.2/input0
Jul 21 10:48:07 pennardpi01 kernel: [101076.289338] usb 1-1.3.4.2: USB disconnect, device number 50
Jul 21 10:48:07 pennardpi01 kernel: [101076.525231] usb 1-1.3.4.2: new low-speed USB device number 51 using dwc_otg
Jul 21 10:48:07 pennardpi01 kernel: [101076.654654] usb 1-1.3.4.2: New USB device found, idVendor=1941, idProduct=8021
Jul 21 10:48:07 pennardpi01 kernel: [101076.654685] usb 1-1.3.4.2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Jul 21 10:48:07 pennardpi01 kernel: [101076.675536] hid-generic 0003:1941:8021.000A: hiddev0,hidraw0: USB HID v1.00 Device [HID 1941:8021] on usb-bcm2708_usb-1.3.4.2/input0

etc

An lsusb was reporting no sign of the weather station :-(

lsusb
Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 001 Device 008: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 0bc2:a013 Seagate RSS LLC
Bus 001 Device 005: ID 0409:0059 NEC Corp. HighSpeed Hub
Bus 001 Device 006: ID 1a40:0101 Terminus Technology Inc. 4-Port HUB
Bus 001 Device 007: ID 0409:0059 NEC Corp. HighSpeed Hub

I then connected the weather station directly to the Pi, and hey presto it worked fine (obviously I had to unplug the hub from the Pi).

Now, I have reconnected the hub to the Pi and the weather station to the hub (as well as the RTL2838), and I now have a constant light on the hub (on the port for the weather station), and it is correctly showing up on an lsusb:

Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 001 Device 075: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 076: ID 1941:8021 Dream Link WH1080 Weather Station / USB Missile Launcher
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 0bc2:a013 Seagate RSS LLC
Bus 001 Device 072: ID 0409:0059 NEC Corp. HighSpeed Hub
Bus 001 Device 073: ID 1a40:0101 Terminus Technology Inc. 4-Port HUB
Bus 001 Device 074: ID 0409:0059 NEC Corp. HighSpeed Hub

So, not sure whether I have a temperamental USB hub / power supply or otherwise, but for the moment, it's working fine.  I'll do some further experimenting (swapping out USB hub / power supply / cables etc), but in the meantime, thanks for all your help :-)

Regards
Mark

[Jim Easterbrook]

On 21/07/13 11:35, Mark Whalley wrote:
>
> So, not sure whether I have a temperamental USB hub / power supply or
> otherwise, but for the moment, it's working fine. I'll do some further
> experimenting (swapping out USB hub / power supply / cables etc), but in
> the meantime, thanks for all your help :-)

Meanwhile, I need to try and get the hid USB module to give some more
informative error messages.

USB hub power supplies are often a bit inadequate. On many switch-mode
"wall wart" types the main smoothing capacitor is a bit under-sized and
has a short life. As it dies, the maximum output current slowly drops,
creating all sorts of strange faults. I've had to replace capacitors in
the PSUs for my Asus router (which runs pywws) and my Topfield PVR.