Bloomsky support

[gjr80]

I have had a Bloomsky Sky2 since October and have finally gotten around to resurrecting some lost work on putting together a driver. As per Matthew's suggestion I have based it on the Netatmo driver and it appears to work well. I only have a Sky2 and not a Storm so have been unable to test it with any rain or wind related obs, though as Matthew says the Bloomsky API is trivial (if anyone has a Storm and would be prepared to let me test with an API key that would be great).

So what. I originally bought the Sky2 due to its fish-eye lens camera that is solar powered and has wifi, the weather observation abilities of the Sky2 played no real part as I would much rather trust my VP2. That being said I would still like to be able to add the Bloomsky obs to weeWX. Given I will not be giving up the VP2, that means I really have 2 options; (1) run 2 instances of weeWX; one each running the vantage and Bloomsky drivers or (2) run a single weeWX instance with the vantage driver and pull in the Bloomsky data via a weeWX service. I don't really want to run 2 instances of weeWX so I am looking at the Bloomsky service approach. The Bloomsky docs state that Bloomsky device phone home to Bloomsky every 5-8 minutes and submit their data. You get Bloomsky data onto your PC via their app or by querying the Bloomsky API, you don't talk directly to the Bloomsky device. When implementing the Bloomsky driver, a separate thread polls the Bloomsky API every so often (lets say 60 seconds) and places the decoded response in a queue. When the driver receives data in the queue it then packages it into a loop packet and emits the packet. WeeWX takes care of the rest. In implementing a service, the Bloomsky service will be bound to either the NEW_LOOP_PACKET or NEW_ARCHIVE_RECORD events and the service will need to augment existing loop packets and/or archive records rather than create loop packets.

How to hit the API and augment a packet/record is simple, already doing that in a driver but the issue I am pondering now is when to hit the API. For example, my VP2 has a 2-odd second loop period, Bloomsky has nothing documented about how often you can query their API but I think every 2 seconds is a bit excessive, especially for such slow changing data. I could stick to polling every minute (user settable) as I do in the Bloomsky driver, but what if this is used with a station/driver whose loop period is somewhat more than 60 seconds (does such a beast exist?), Bloomsky data will back up in a queue, how do I then squeeze more than one Bloomsky obs into a single loop packet. All of the other services I have seen pull data from a register or some other local hardware device where blocking/delays are not an issue. In this case the 'query' is sent via the internet to Bloomsky and it could take some time or even fail after a number of retries/retry wait periods.

Would appreciate any thoughts.

Gary

[Thomas Keffer]

Hi, Gary

A couple of things to keep in mind:

1. Generally, I think you're better off with one instance for each device. Yes, people have written services that can poll additional devices, but this is a bit of a kludge. The one-instance/one-device pattern makes startup simpler, allows the device to participate in a "catch up" (if it has a suitable memory) and makes it easier to keep track of the origin of each bit of data.
   
2. The difference between LOOP packets and archive records is that the former come at irregular intervals and possibly with irregular sets of data, while the latter comes at regular intervals and (usually) with all data types present. If you choose the latter, you should make sure you do the polling at precisely, say, 5 minute intervals. 
3. You definitely do not want to mix observations with different timestamps into a single loop packet.

I'm not sure I answered your questions. If not, give me another try!

-tk

[gjr80]

On Wednesday, 28 June 2017 03:06:54 UTC+10, Tom Keffer wrote:

I'm not sure I answered your questions. If not, give me another try!

Maybe not answered but focussed a rethink.

1. Generally, I think you're better off with one instance for each device. Yes, people have written services that can poll additional devices, but this is a bit of a kludge. The one-instance/one-device pattern makes startup simpler, allows the device to participate in a "catch up" (if it has a suitable memory) and makes it easier to keep track of the origin of each bit of data.

Took me a couple of read throughs before this took (too many devices, APIs etc involved). Whilst there is a bit of 'external' complexity involved in running 2 instances of weeWX I am thinking it is going to simplify a whole pile of 'internal' things. WeeWX has good multi-database support report wise, so separate databases are really not an issue. I agree that in hindsight 2 instances would seem to be best.

2. The difference between LOOP packets and archive records is that the former come at irregular intervals and possibly with irregular sets of data, while the latter comes at regular intervals and (usually) with all data types present. If you choose the latter, you should make sure you do the polling at precisely, say, 5 minute intervals. 

Re-reading some of the Bloomsky docs gave me a better understanding of how the 2 devices (Sky and Storm) work together. The two devices update at Bloomsky central at different rates (one of the order of 5-8 minutes and the other it seems every 30 seconds). Since access to the data is via the Bloomsky API I would expect that you would want to be hitting the API at least every 30 seconds. This would give you loop packets. The API data is not aggregated in any way, it is a snapshot in time so software record generation would be required. Again would seem to point to running separate instances.

3. You definitely do not want to mix observations with different timestamps into a single loop packet.

Yes, would definitely be an issue and having items languish in a queue would tend towards such a situation.

thanks

Gary

[mwall]

gary,

i use the owfs service to augment data on a few installations, but generally speaking i find it better to use a separate weewx instance for each device.  the weewx-multi rc script makes it *really* easy to manage multiple weewx instances (you could do it with a bunch of systemd unit files, i suppose).  i install weewx to /opt/weewx-x.y.z, create a bunch of .conf files in /etc/weewx (one for each device), then use /etc/default/weewx-multi to tell the weewx-multi script how many instances to run.

having a separate process for each device prevents failures on one device from screwing up the data collection from another device.

it also avoids the temporal alignment issues.

i do a simple skin for each device type, then configure the .conf files to emit a separate instance of each skin.

then cron an instance of wee_reports (with its own .conf) to generate an aggregate report that combines data from all the processes.

i also install uploaders such as mqtt, influx, and/or emoncms for centralized aggregate reports and real-time monitoring/diagnostics.

usually i just monitor the central mqtt/influx/emoncms reporting system, but the local, weewx-generated reports are helpful for diagnosing problems and for viewing data when connectivity to the central reporting system goes away (some of the sites i work with have only occasional internet access, or access that is limited to certain periods of time).

(some day maybe i'll implement catch-up for mqtt, influx, and emoncms...)

m