Receive station data via UDP broadcast packets?

[Arthur Emerson]

Since the WeatherFlow API link has been published and is apparently no longer under NDA, I was wondering if anyone is in the process of developing a station driver to capture their UDP broadcast packets on the local subnet and import them into weewx?

https://weatherflow.github.io/SmartWeather/api/udp.html

They also have cloud-based REST and Websocket interface specs published, but I think that the best plan of attack is to pick the broadcasts off of UDP/50222 since I believe that it is broadcast as soon as their hub receives it from each sensor.

Since they are going to [finally!] be shipping at least the Air unit (outdoor temp, humidity, barometer, lightning detector) and Hub to their campaign supporters in the next month, I was thinking that it would be a good time to get support in place for weewx.  Hence, wanting to know if someone else is working on it before trying to cobble together something on my own?????

[mwall]

On Sunday, August 6, 2017 at 1:35:40 PM UTC-4, Arthur Emerson wrote:

> WeatherFlow is finally about to ship their crowd-funded PWS. Their basic architecture is a
> wireless network hub device, with Bluetooth BLE links to one or more "Air" and "Sky" outdoor
> devices that each contain groups of sensors. They have already published all kinds of cloud
> API's (yawn), and are promising to publish their BLE API in the near future.
>
> TL;DR - They are also doing something that is interesting for easily capturing station data,
> namely broadcasting UDP packets (apparently as they arrive at their wireless network hub)
> onto the local network segment for any device to see. No fake DNS and web servers, no
> pass-through network ports, just listen for the UDP packets and decode the JSON goodness!
> I have simplified rtl_433 down to this:

> #!/usr/bin/python
>
> from socket import *
> s=socket(AF_INET, SOCK_DGRAM)
> s.bind(('255.255.255.255',50222))
> while True:
> m=s.recvfrom(1024)
> print m[0]

> This is what an output packet from an "Air" device should look like:

> {"serial_number":"AR-00009876","type":"obs_air","obs":[[1502287861,1009.30,21.90,85,0,0,3.46,1]],"firmware_revision":12}
>
> (All of their other status packets are pure JSON, but for some reason they chose to send out
> observations as a list in the "obs" field. The format is completely documented in their API docs.)
>
> Long story short, this shares so much in common with the SDR broadcast decoder that I'm
> wondering if this SDR station code should be renamed to a medium-agnostic broadcast
> decoder, and encourage development of external programs other than rtl_433 as input?
>
> I'm not overly familiar with all of the available weewx station input types, but seeing the
> simplicity of grabbing broadcast packets off of the wire I think that I may actually start to
> send similar packets from my growing collection of DIY IoT sensors and use this same
> basic format. Hence, thinking that the sensor-mapping feature in the SDR station code
> is way more flexible than my half-finished station driver with everything hard-coded. If
> there's a more appropriate way of doing this, I'm all ears.....

[mwall]

i'm leaning toward a generic weewx-wu driver that captures (sniff or listen) tcp/ip packets that are wu protocol (http GET). that driver would work with acurite bridge, fine offset observer, and any other hardware that sends directly to weather underground.

there is already a working implementation in the weewx-interceptor driver. we would push the generic tcp/ip components into the weewx 'driver' module, then distribute a simple weewx-wu driver.

the weewx-sdr driver would be much simpler if it only had to recognize the json output. i'm happy to make that change sooner than later, and push the pressure for consistency and versioning down onto the rtl_433 development. that could also facilitate the use of sdr dongle for capturing davis signals, not just 433Mhz signals.

but i still don't have a clear vision of the "one json-speaking driver to rule them all"

is it a weewx-json driver that accepts any json input? then that driver has plugins/modules/api to feed it via tcp/ip (e.g., for the weatherflow broadcasts) or pipe (e.g., for rtl_433 or rtldavis)?

or do we just push the json-handling stuff into a simple class or two in the weewx drivers module. then implement weewx-weatherflow driver that listens for udp broadcasts (using tcp/ip components in weewx.drivers) and handles them (using json components in weewx.drivers). and we implement weewx-sdr that is just a simplification of the current weewx-sdr.

or something else?

[Arthur Emerson]

I do not claim to be a programmer or have more than a limited grasp of Python.  My thought was to go with code re-use, and write a simple program as a replacement for rtl_433 to capture the WeatherFlow (WF) UDP packets and pipe them into the existing weewx-sdr stuff.  WF's JSON isn't much different than rtl_433's JSON packets, and the sensor-mapping code is already there in the weewx-sdr setup that would also apply to the WF packets.  Hence, floating the idea of making the current weewx-sdr stuff into a weewx-broadcast station type, and letting the user set their own external program to feed it the broadcast packets regardless if they came in from an SDR, over the ethernet cable, or were picked off off of the air via Bluetooth LE.

The holy grail (a single weewx-json driver) seems like it would quickly morph back into another weewx-sdr, since one thing that is certain in the software universe is that vendors will find ways to implement non-standard standards.  WF chose to send their weather observations as a list in their JSON "obs" field, so getting the temperature requires parsing the list and not just saying something like temperature=pkt['temperature'] or however the Python syntax goes.  All of their station status packets are "proper" JSON, though.  I can see other stations deciding to do similar weird things in the future.

Doing a weewx-wu seems like a good idea, since more than one station type sends that packet format.

Since everyone else here has a gazillion times more familiarity with the weewx layout, Python, and what the pain points are with other vendors' stations, I'll pass on making a recommendation for how best to proceed.  I'm also not opposed to finishing my WF-specific station code that's about 50% complete, but would be hesitant to encourage others to use that spaghetti knowing how little I know about Python and how ugly the code will be.....

[Arthur Emerson]

FYI, I just cobbled together a working weewx driver interface for the WeatherFlow UDP broadcast protocol.  It ignores the sensor addresses for now, so it will get confused if there are more than one sensor unit of a given type attached to the Hub.

I still plan to implement the tuple sensor names and mappings like the SDR driver does to allow multiple sensor units, but will have to dig around that driver to see how it works.  As I said, Python might as well be Klingon to me.

I'll throw it up on Github once I have seen it run for a few days without problems.  If anyone wants to live dangerous, reply here and I'll upload what I'm testing.....

[Arthur Emerson]

FYI, I have just created a master branch of the WeatherFlow UDP station driver and committed v1.00 to GitHub:

https://github.com/captain-coredump/weatherflow-udp

The sensor_map logic works, and has been tested with multiple devices reporting to the same WeatherFlow Hub.....