how to test for null data

[Pila]

I export InTemp to check if my USB connection to the First Offset dropped dead.

InTemp      $current.inTemp.format(add_label=False)
InTemp      $current.inTemp.raw

What will be produced in case of null data? What do I need to test?

In the first case, I expect N/A to mean USB connection is lost. I am guessing N/A will not be in raw data. An empty field?

So, probably the first line is a better choice for such purpose?

[Andrew Milner]

the driver will log an error if communication with the fine offset is lost, and does not continue to store data in the database until communication is restored.

[Andrew Milner]

to test just disconnect the usb and then check the log.  you should not find null data recorded.  when communication is restored weewx will attempt to read data from the fineoffset station which may have been stored whilst there was no communication and will then resume normal loop and rec operations.

[Pila]

I meant: my external program parses output. How will my program know USB broke down? What value in my program should i test to find reset is needed? Will N/A be saved?

[Andrew Milner]

as i said if communication is lost no data is saved in the database - so you will not have n/a, null or anything else in the database.  in fact weewx will ultimately stop running and eventually do a restart.  the symptom will be shown by the time of the generated html file - which will be older than 5 minutes ago (or whatever archive period you have specified).  do my test and see what happens.

On Monday, 7 October 2019 21:15:43 UTC+3, Pila wrote:

I meant: my external program parses output. How will my program know USB broke down? What value in my program should i test to find reset is needed? Will N/A be saved?

[Pila]

Yes, I will. I would just prefer to test with a possibly working solution. This is an excellent explanation of the problem.

Generation is every 5 minutes like you say. My program parses Weewx output 3 minutes later. I test a timestamp of generation.

An alternative would be to separate Weewx log and parse it. The above is less intensive.

Now I can do your test! Many thanks for your kind help and patience.

I thought it will be enough to restart a RPI Zero since it powers the weather station. I tried but it seems I will need a longer cutoff. Have to test if Zero cuts power to the USB while rebooting.

[Pila]

Amazing amount of work and detail went into WeeWX! It is hard to find what can not be adjusted.

I parse data from a small text only report I added. I think:

$current.DateTime.Raw

is adequate. No need for $latest in this context? $current will be the time report was generated. If too old...

[Andrew Milner]

I would be cautious about powering the fineoffset from the pizero, and would prefer to have a powered hub to power the weather station.  The usb interface on the fineoffsets is somewhat finicky at the best of times!!  Inadequate power supplies are behind many issues with rpi and weather stations.

the database should always reflect good data.

[Pila]

I agree. But for starters... Since rebooting Zero does not restart my Fine offset I need an external switch anyway.

But as I live on a an isolated island in a small country, getting particular equipment is not easy. This was just perfectly convenient solution with items I had laying around. Of course nothing can be used if not nearly bulletproof.

I do have an extra powered hub and there can never be shortage of SmartSwitches. Zero will power the hub... Unless I fix is USB cable. Frankenstein style.

[Andrew Milner]

no, no, no - if zero powers the hub it will achieve nothing.  zero should ideally not be supplying power TO anything.  put batteries in the weather station for backup purposes.  use a usb hub powered by its own power supply, not by the pizero. when you say "did not restart fine offset" what exactly do you mean?  what does the log show - answers are always in the log.

On Tuesday, 8 October 2019 23:31:37 UTC+3, Pila wrote:

I agree. But for starters... Since rebooting Zero does not restart my Fine offset I need an external switch anyway.

But as I live on a an isolated island in a small country, getting particular equipment is not easy. This was just perfectly convenient solution with items I had laying around. Of course nothing can be used if not nearly bulletproof.

I do have an extra powered hub and there can never be shortage of SmartSwitches. Zero will power the hub... Unless I fix is USB cable. Frankenstein style.

[Pila]

What you say, I read as: FO can have its batteries, it is enough to kill the power to the USB cable for a short time to fix USB connectivity? Meaning: A nutered (power cut, data only) USB cable between the Zero and USB Hub plus a SmartSWitch on the USB Hub power supply can restore USB when it fails (even with batteries in FO). I understood FO needs to be powered down.

My proffessional deformation is to verify all the facts and measure everythng including things that supposedly do not work or should not be done. And then after having checked all facts, make decisions.

This simple USB connection was both interesting to try and the only immediate thing I could do to connect my Weather Station permanently to RPi Zero without moving the station itself elsewehere. For anything else, I needed to get some more stuf which takes time at my present location. First step is always the same: hook it up, and if it works after a month or so, go on to the next step with it.

I was willing to bet Zero + FO would not work at all! USB cable powering my RPi Zero is 3 meters (10 feet) long! I better not say measurements under load :) I was sure it would not work. Now, I am actually perplexed: it works perfectly fine over a month! On a Zaro where Node-RED and Mosquitto are using power needlesly, plus WeeWx and of course - Zero is powering the Fine Offset itself for few weeks now. But, somehow, against all odds, 5 weeks later, all is well!?! If I did not try it, I would not have believed it.

Previously, I tested connection from my Fine Offset to a PC and USB stopped working. I beleived USB died. Now I decided to do a SmartHome thing which starts with a Weather station. So, when I plugged a Zero into FO, it suddenly worked years after I gave up thinking USB port is dead or corroded (small island, a VERY corrosive surrounding). Them eneloops in FO last forever, over a year.

It was only when I started reading on WeeWX that I learned FO power needs to be fully cycled to restore USB connection. Reading info on Fine Offset, I understand it needs to be completely powered down, screen dead, history emptied. Only then it restores USB connection. So, when I said FO did not react to Zero rebooting, I meant - its screen remained unaffected by Zero rebooting. FO did not loose power long enought to restart. For now, USB connection was never lost, Zero is powering FO for the last 3 weeks (no batteries).

WeeWX log upon manual reboot seems unremarkable:

Oct 10 12:17:56 RPiZero kernel: [    2.673222] usb 1-1: New USB device found, idVendor=1941, idProduct=8021, bcdDevice= 1.00
Oct 10 12:17:56 RPiZero kernel: [    2.688210] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Oct 10 12:17:56 RPiZero kernel: [    2.713617] hid-generic 0003:1941:8021.0001: hiddev96,hidraw0: USB HID v1.00 Device [HID 1941:8021] on usb-20980000.usb-1/input0

Oct 10 12:18:16 RPiZero weewx[345]: engine: Using configuration file /home/weewx/weewx.conf
Oct 10 12:18:16 RPiZero weewx[345]: engine: Loading station type FineOffsetUSB (weewx.drivers.fousb)
Oct 10 12:18:16 RPiZero weewx[345]: fousb: driver version is 1.10
Oct 10 12:18:16 RPiZero weewx[345]: fousb: polling mode is PERIODIC
Oct 10 12:18:16 RPiZero weewx[345]: fousb: polling interval is 60
Oct 10 12:18:17 RPiZero weewx[345]: fousb: found station on USB bus= device=
Oct 10 12:18:17 RPiZero weewx[345]: engine: StdConvert target unit is 0x10
Oct 10 12:18:20 RPiZero weewx[345]: fousb: synchronising to the weather station (quality=1)

[Andrew Milner]

you have misunderstood completely

1. fineoffset usb connection can glitch at any time for no predictable reason - maybe once a month, maybe once a year.  On many occasions it can glitch and weewx is able to recover the connection.

2. sometimes fineoffset glitches in such a way that the only solution is to completely power off the fineoffset, remove any batteries, and restart it - this does not happen very often.  the problem is inside the fineoffset and is not connected to weewx, rpi or anything else.

3. an alternative recover for the problem in 2 above is to have no batteries in the fine offset, power it from a powered usb hub (a usb hub with its own independant power supply) which is able to selectively flip power on individual ports and do the power cut/restore via the usb supply.  Only certain powered usb hubs are able to do this - check weewx threads for more details.

4. the rpi (and pizero) is known for a poor susceptability to power issues.  To avoid any such issues it is suggested to ensure a good beefy power supply for the rpi and to avoid powering other devices off the usb port (eg fineoffset).  however powering the fineoffset via powered usb hub (either without the switcheable ports) is ok

5. no solution has been found to avoid the glitches occurring.  the best one can achieve is to try and recover when weewx is unable to recover.  

6. it can run for months with no issues and then have 3 in a month

7 the problem is a fault in the fineoffset firmware.  recovery demands that the fineoffset hardware has no power (usb or battery) and is restarted.  Just killing the power over usb and running off battery will not stop the problem occurring.

[Pila]

Wow! This is detailed explanation! Thanks. I was not aware there are nuances. Clearly, USB side in the FO is the culprit. We can not change it.

I did not know some glitches are recoverable by Weewx. For the master glitch, we are back to killing USB power with no batteries in FO. I will wire something up.

As for RPi, I have 7 of them running 24/7. RPi 3 is touchy. RPi 2 and 4 not so much. As I said before, my only Zero works 6 weeks perfectly against all odds powering FO. Unfortunately, I can kill USB power only on RPI 2 and 3, not on Zero and 4 would be a very wrong choice for the job anyway.

[Pila]

For people in need of switching USB power to Fine Offset should it be needed, I have a news I did not find is published.

There is another switchable USB hub that is currently available. Plus, it is one of the quite rare hubs not backfeeding power to the RPi (or any other device) controling it!

Transcend USB 3.0 Hub 4 Port Powered Hub3

Take care: there is almost identical HUB2 which does not have external Power Supply. I am talking about the HUB3. Back is labeled: TS-HUB3K. Power supply is declared at 18W.

mine are identified as:
ID 8564:4000 Transcend Information, Inc. RDF8

All 4 ports respond to power commands, but only port 1 actually kills the power. This is the single port on the side - FO should go there.

Power stability is not excellent, we may call it very good:
4,85v @ 0,94A
4.75v @ 1,62A
4.74v @ 1,86A

For RPi running WeeWx and a Fine Offset, I would not expect problems with both RPi and FO powered from it. WeeWx should have no problem controling power to the FO using this hub (I did not verify WeeWX control of the hub!).

Hopefully, they did not "improve" it with new revisions cheaper to produce. Mine were made in 2016.

[Pila]

Fine Offset WS1080 using its factory USB cable and no batteries, draws mostly below 40 mA or tad below 0,2W. Maximal draw is 50 mA (0,26 w) with lights on.

[Pila]

I have cut power to a Fine Offset. After a restart, it is still waiting to set the clock automatically. Wrong time may remain for hours if not days. I can not say at the moment, it was 90 minutes ago.

According to plots, WeeWx seems to read and interpret the data correctly during this 90 minutes.

I would appreciate confirmation if the following statements are true.

Wrong time stamps from the weather station are not spoiling the data as being read by the WeeWx. I can leave time be wrong until it is set automatically. I can not set the FO internal time from a connected RPI Zero.

[Andrew Milner]

I believe you are correct

the hardware guide - http://weewx.com/docs/hardware.htm#fousb_notes says the clock is ignored by weewx as it cannot be trusted.

[Pila]

Thanks for confirmation. FO synced time after some 3 hours.

Btw, I measured Zero W powering FO: powering them both tops at 0,8w. Runs normally at 126 mA 0,63w. That must be perfectly stable.

RPI 3 with LAN powering nothing spends 2,7w booting and 1,5w running.

[Andrew Milner]

i suspect the rpi power issues are more to do with quality of components rather than actual measured values/limits!!

As I no longer use a fineioffset I do not really have an interest nowadays!!

On Sunday, 13 October 2019 18:18:16 UTC+3, Pila wrote:

Thanks for confirmation. FO synced time after some 3 hours.

Btw, I measured Zero W powering FO: powering them both tops at 0,8w. Runs normally at 126 mA 0,63w. That must be perfectly stable.

RPI 3 with LAN powering nothing spends 2,7w booting and 1,5w running.