fousb issue with raspberry pi and stretch.......maybe

[ianpre...@gmail.com]

Hi

Learned people I am in need of some assistance in pointing out what I have done wrong.

I have been using a Aercus 3080 and weewx on a raspberry pi with jessie for over a 2 years now.

I use the port switching ability of weewx to overcome the usb lockup issue and all is good, apart from electrical storms which cause loss of comms between sensors and indoor unit, all outdoor data is lost when this occurs but this is not the reason for this post.

I decided to upgrade to raspberry pi stretch.(cringe)

I copied the config and database files off system.

I shutdown the pi, removed the sd card and inserted a brand new sd with stretch on it.

I then proceeded to install weewx, ddns etc all the bits I did for the previous build.

Copy the saved config and database files back in

So weewx appears to work but stalls sometime later, (see attached)

The python code for usb testing by Matthew Wall appears to have issues. (see attached)

I can shutdown and remove the stretch version sd card and replace it with the jessie sd card and weewx appears to recover (old data while I have been working on the stretch version) and continues working and the python code for usb testing works. I am able to turn the port indicator for the weather station on and off.

I hope I have captured some relevant parts of the log and some terminal work, unfortunately the log fills with a lot of updating to weather sites.

Please let me know if any further info is required.

Regards Ian

[Ian Prescott]

Hi 

following up, weewx doesn't stall as the previous log file shows but I am not sure that the power cycle ability will work.

I have rebuilt my raspberrian stretch SD for the third time and each build has produced the same result.

for the jessie version of weewx start I get

Nov  6 18:44:20 weatherpi weewx[1298]: engine: Debug is 1

Nov  6 18:44:20 weatherpi weewx[1298]: engine: Initializing engine

Nov  6 18:44:20 weatherpi weewx[1298]: engine: Loading station type FineOffsetUSB (weewx.drivers.fousb)

Nov  6 18:44:20 weatherpi weewx[1298]: fousb: driver version is 1.9

Nov  6 18:44:20 weatherpi weewx[1298]: fousb: power cycling enabled for port 4 on hub 001:005

Nov  6 18:44:20 weatherpi weewx[1298]: fousb: polling mode is PERIODIC

Nov  6 18:44:20 weatherpi weewx[1298]: fousb: polling interval is 60

Nov  6 18:44:20 weatherpi weewx[1298]: fousb: found station on USB bus=001 device=006

Nov  6 18:44:20 weatherpi weewx[1298]: engine: Loading service weewx.engine.StdTimeSynch

Nov  6 18:44:20 weatherpi systemd[1]: Started LSB: weewx weather system.

for the stretch version of weewx start I get

Nov  6 18:23:35 weatherpi weewx[2689]: engine: Debug is 1

Nov  6 18:23:35 weatherpi weewx[2674]: Starting weewx weather system: weewx.

Nov  6 18:23:35 weatherpi weewx[2689]: engine: Initializing engine

Nov  6 18:23:35 weatherpi weewx[2689]: engine: Loading station type FineOffsetUSB (weewx.drivers.fousb)

Nov  6 18:23:35 weatherpi systemd[1]: Started LSB: weewx weather system.

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: driver version is 1.9

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: power cycling enabled for port 4 on hub 001:005

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: polling mode is PERIODIC

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: polling interval is 60

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: found station on USB bus= device=

Nov  6 18:23:36 weatherpi weewx[2689]: engine: Loading service weewx.engine.StdTimeSynch

Nov  6 18:23:36 weatherpi weewx[2689]: engine: Finished loading service weewx.engine.StdTimeSynch

So what are the implications of the bolded line......is this why mwall usb testing code has issues on the stretch version and not on jessie version?

will this have consequences for power cycling the usb port when a lockup is detected????

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ian Prescott]

sad news, weewx (stretch version) crashed this morning. It had been running for more than 24hrs.

I am at the end of my abilities and ask for some pointers on how to find a solution.

I have restored the jessie version on the raspberry pi.

Thanks 

Ian

On Sat, 3 Nov 2018 at 10:07, <ianpre...@gmail.com> wrote:

--

[mwall]

On Wednesday, November 7, 2018 at 5:09:18 PM UTC-5, Ian Prescott wrote:

for the jessie version of weewx start I get

Nov  6 18:44:20 weatherpi weewx[1298]: fousb: found station on USB bus=001 device=006

for the stretch version of weewx start I get

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: found station on USB bus= device=

check the pyusb version on each os.  this is due to a difference in the pyusb version, not jessie vs stretch.

the older versions of pyusb reported a bus and device number.  newer versions do not.  this does not affect the functionality of the fousb driver.  it *does* affect the usability of the usb utility i wrote to power cycle in case of failures.  that utility *depends* on the bus and device numbers.

i think they punted the bus and device numbers since bus and device are not reliable ways to identify a usb device - they *may* change when a device is plugged/unplugged.  the preferred way to reference a usb device is using its product/vendor codes, or a serial number (if one is available).  unfortunately, that leaves us with no way to identify devices that have no serial number.

if you are doing something that needs the bus and device numbers, you could try installing pyusb 0.4 instead of pyusb 1.x

m

m

[mwall]

On Thursday, November 8, 2018 at 5:22:01 PM UTC-5, Ian Prescott wrote:

sad news, weewx (stretch version) crashed this morning. It had been running for more than 24hrs.

I am at the end of my abilities and ask for some pointers on how to find a solution.

I have restored the jessie version on the raspberry pi.

Thanks 

Ian

the failure you posted is due to the fact that pyusb 1.x reports None for both bus and device

you should be able to run debian9 (stretch) if you install pyusb 0.4

[ianpre...@gmail.com]

Hi

check the pyusb version on each os.  this is due to a difference in the pyusb version, not jessie vs stretch.

I am using the same usb utility u wrote to test the 4 port usb hub using either stretch or jessie.

It works on jessie but not on stretch as shown below. (the hardware remains the same)

pi@weatherpi:~/Desktop $ sudo python usb_control-0.6.py --hub 001:005 --port 4 --indicator 0

Traceback (most recent call last):

  File "usb_control-0.6.py", line 190, in <module>

    main()

  File "usb_control-0.6.py", line 185, in main

    docmd(CMD_INDICATOR, options.hub, options.port, options.indicator)

  File "usb_control-0.6.py", line 136, in docmd

    devid = "%s:%03d" % (bus.dirname, dev.devnum)

TypeError: %d format: a number is required, not NoneType

pi@weatherpi:~/Desktop $ lsusb

Bus 001 Device 006: ID 1941:8021 Dream Link WH1080 Weather Station / USB Missile Launcher

Bus 001 Device 005: ID 0409:0058 NEC Corp. HighSpeed Hub

Bus 001 Device 004: ID 045e:0745 Microsoft Corp. Nano Transceiver v1.0 for Bluetooth

Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter

Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

So I am confused now, u mentioned pyusb 1.x, ........ so is this your usb utility (later version) or some other code supplied with the os???   and how do I check.

I am only using your usb utility (usb_control-0.6.py) as a test, I rely on weewx fousb to do power cycling in the event of a usb lockup.

The log shows Nov  6 18:23:36 weatherpi weewx[2689]: fousb: found station on USB bus= device=

So is this pyusb code issue or weewx fousb issue??

you should be able to run debian9 (stretch) if you install pyusb 0.4

So how do I get (stretch version) weewx using fineoffset usb driver to use pyusb 0.4 to power cycle a usb port lockup?

So with a limited understanding so far, using the log below, weewx fousb (fineoffset usb driver) is attempting a power cycle of a locked up usb port but cannot do it because of

Nov  6 18:23:36 weatherpi weewx[2689]: fousb: found station on USB bus= device= (for the reason u mentioned above.....no numbers)

this would indicate that weewx fousb (fineoffset usb driver) uses the same principles as your earlier versions of usb utility.

 

Nov  9 07:08:50 weatherpi weewx[3742]: fousb: Attempting to power cycle

Nov  9 07:08:50 weatherpi weewx[3742]: engine: Caught unrecoverable exception in engine:

Nov  9 07:08:50 weatherpi weewx[3742]:     ****  %d format: a number is required, not NoneType

Nov  9 07:08:50 weatherpi weewx[3742]:     ****  Traceback (most recent call last):

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/engine.py", line 871, in main

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      engine = engine_class(config_dict)

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/engine.py", line 78, in __init__

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      self.loadServices(config_dict)

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/engine.py", line 142, in loadServices

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/engine.py", line 467, in __init__

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      if software_interval != self.engine.console.archive_interval:

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/drivers/fousb.py", line 991, in archive_interval

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      return self._archive_interval_minutes() * 60

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/drivers/fousb.py", line 1006, in _archive_interval_minutes

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      power_cycle_station(self.pc_hub, self.pc_port)

Nov  9 07:08:50 weatherpi weewx[3742]:     ****    File "/usr/share/weewx/weewx/drivers/fousb.py", line 715, in power_cycle_station

Nov  9 07:08:50 weatherpi weewx[3742]:     ****      devid = "%s:%03d" % (bus.dirname, dev.devnum)

Nov  9 07:08:50 weatherpi weewx[3742]:     ****  TypeError: %d format: a number is required, not NoneType

Nov  9 07:08:50 weatherpi weewx[3742]:     ****  Exiting.

So how do I get weewx fineoffset usb driver to power cycle a locked up usb port..........or am I missing something??

Thanks

[mwall]

On Friday, November 9, 2018 at 7:24:29 AM UTC-5, ianprescott10 wrote:

Hi

check the pyusb version on each os.  this is due to a difference in the pyusb version, not jessie vs stretch.

I am using the same usb utility u wrote to test the 4 port usb hub using either stretch or jessie.

It works on jessie but not on stretch as shown below. (the hardware remains the same)

the utility only works with pyusb 0.x, not pyusb 1.x.  this is because pyusb 1.x removed the bus and device numbers.

 

So I am confused now, u mentioned pyusb 1.x, ........ so is this your usb utility (later version) or some other code supplied with the os???   and how do I check.

I am only using your usb utility (usb_control-0.6.py) as a test, I rely on weewx fousb to do power cycling in the event of a usb lockup.

The log shows Nov  6 18:23:36 weatherpi weewx[2689]: fousb: found station on USB bus= device=

So is this pyusb code issue or weewx fousb issue??

pyusb is a python library for communicating with usb.  you either installed it as a system package using apt or yum, or you installed it using pip.  or maybe it came with your system.

if you installed using pip, you can get the version using 'pip freeze':

pip freeze | grep usb

if you installed using apt or yum, then use your package manager to figure out the version.  for example:

rpm -qa | grep usb

or

apt list pyusb

 

you should be able to run debian9 (stretch) if you install pyusb 0.4

So how do I get (stretch version) weewx using fineoffset usb driver to use pyusb 0.4 to power cycle a usb port lockup?

you need to install pyusb 0.4 on your system instead of pyusb 1.x.  do this using your system's package manager (apt, yum, etc) if you manage your python using system packages, or pip if you manage your python separately from the system packages.

m

[ianpre...@gmail.com]

aaahhh! (a light bulb moment)

So because I grabbed the latest version of stretch (complete with a later version of pyusb) the later version of pyusb is not compatible with your usb utility, and by assumption and for the same reason the fineoffset usb driver used by weewx also has an issue. 

from the jessie version I get

pi@weatherpi:~ $ apt list pyusb

Listing... Done

pi@weatherpi:~ $ apt -h

apt 1.0.9.8.4 for armhf compiled on Dec 13 2016 22:26:22

Usage: apt [options] command

CLI for apt.

Basic commands: 

 list - list packages based on package names

 search - search in package descriptions

 show - show package details

 update - update list of available packages

 install - install packages

 remove  - remove packages

 upgrade - upgrade the system by installing/upgrading packages

 full-upgrade - upgrade the system by removing/installing/upgrading packages

 edit-sources - edit the source information file

pi@weatherpi:~ $ apt search pyusb

Sorting... Done

Full Text Search... Done

python-usb/oldstable,now 0.4.3-1 armhf [installed,automatic]

  USB interface for Python

pi@weatherpi:~ $

and from the stretch version I get

pi@weatherpi:~ $ apt search pyusb

Sorting... Done

Full Text Search... Done

pypy-libusb1/stable 1.6.3-1 all

  Python wrapper for libusb1 (PyPy)

python-libusb1/stable 1.6.3-1 all

  Python wrapper for libusb1 (Python 2)

python3-libusb1/stable 1.6.3-1 all

  Python wrapper for libusb1 (Python 3)

python3-usb/stable 1.0.0-1 all

  USB interface for Python (Python3)

pi@weatherpi:~ $ apt list pyusb

Listing... Done

pi@weatherpi:~ $

Ok ....so I am in way over my head now. I have no idea on how to change from the current pyusb to a older version.

I did a  "find / -name *us*.py on the stretch version and got back (along with a lot of other stuff)

/usr/lib/python2.7/dist-packages/usb/backend/libusb1.py

/usr/lib/python2.7/dist-packages/usb/backend/openusb.py

/usr/lib/python2.7/dist-packages/usb/backend/libusb0.py

these were the only files that had any usb flavor. 

Would there be any raspberry pi / apt-get gurus willing to give me some guidance on how to find/replace/install an older version python library file over a newer version.

Thanks M for your help and patience....... it took me a while before it clicked on what was happening.

[vince]

On Friday, November 9, 2018 at 3:13:25 PM UTC-8, ianpre...@gmail.com wrote:

Would there be any raspberry pi / apt-get gurus willing to give me some guidance on how to find/replace/install an older version python library file over a newer version.

google "pip install older version of package" if you used pip

google "apt-get install older version of package" if you used a pre-packaged library

 

[Ian Prescott]

So having done a bit of googling around it would appear that installing older versions of packages with later versions of OS could work if one knew what one was doing, but also this could be fraught with danger with regards to dependencies, and so it went on....... into the  

So that I can make considered decision on a way forward with this project to upgrade the raspberry pi OS to the latest version (stretch) could somebody advise me on the following.

(I made some assumptions earlier but never sort clarification )

1) currently raspberry pi stretch OS using weewx and using fineoffset usb driver cannot recover from a usb lockup because the stretch OS uses a later version of python-usb.

2) only a manual disconnect and power down of the weather station can recover a usb lockup with the later version of python-usb

3) currently raspberry pi jessie OS using weewx and using fineoffset usb driver can recover from a usb lockup because the jessie OS uses an earlier version of python-usb.

4) would there any weewx development to regain this functionality to control the usb port using the later version of python-usb

Thanks

--

[Andrew Milner]

Another way of trying to tackle the problem is to reduce as much as possible the need to use a recycling capable hub.  I ran a fine offset station for maybe 5 years and had I think one lock up in that time.  I did however use a powered hub and good uprated power supplies for RPi and hub, and ensured all leads went nowhere near power supplies, computers, screens etc.  I minimised all loads on the rpi as much as possible.  It definitely worked for me and I never felt the need to go investigating the switchable hub option, so always managed to upgrade the OS without any issues.