dealing with events vs binary sensors

[Greg Troxel]

I'll assume that the larger world is rtl_433, mqtt, and Home Assistant
(HA). I think for systems that aren't that, the needed semantics are
close, but if you see it otherwise please describe.

rtl_433 has doctrine to just represent a transmission straightforwardly
in json. That has tremendous merit and I'm not trying to change that,
but it doesn't make sense to carry this approach all the way to the end
of the larger system.

Some things are "sensors" (HA term), which have new values over time.
The canonical example is temperature. Temperature fundamentally *is* a
sensor, and most devices transmit it periodically.

Some things are "binary sensors", which have values limited to on/off,
over time. Examples are doors and moisture sensors. Some devices might
send a value once a minute, and some might send a value when it changes.
Given lack of acks, and the desire for prompt notification, it seems
obvious that best practice (for a transmitter designer!) is to send the
value when it changes and also at some background rate. (We recently
have been talking about Govee which doesn't do it this way.)

Some things are "events", which is like "button pushed". This can be a
motion detection, and it can be a "user pushed button". The latter can
be used to "transition house to night mode", for example.

(Motion detection is conceptually messy. It can make sense to think of
it as a binary sensor, and it can make sense to think of it as an event.
An event firing corresponds to the off/on transition. There is no
"motion no longer detected".)

HA's notion of event is that it fires once and then can fire again. The
state is only the last time pushed. There is no notion of on and off.
https://www.home-assistant.io/integrations/event

There is support for events via mqtt:

https://www.home-assistant.io/integrations/event.mqtt/


I think it makes sense to sort things into two categories:

things intended to be binary sensors

things intended to be events

and to adjust the json keys in decoders to align appropriately, perhaps
using event_ as a prefix for events.

Then, in the mqtt scripts, we can produce mqtt event config for events.

A nuance is that if a device sends a button press, and sends 4
transmissions for redundancy, we really should be sending 1 event. This
means the json/mqtt bridge has to have state and keep track of the
last-fired time and suppress new events at least in the next 100ms or
whatever. (Really, devices should send a packet number or something to
allow merging the duplicate transmissions, but they are how they are.)


There was recently an query about the retained flag. It seems obvious
that sensors should have the retained flag set if they post only
occasionally, and not have it set if they post regularly. One would
like a sensor that is not reporting to show as unavailable. It also
seems obvious that events should never have the retained flag.


My own usage is using rtl_433_mqtt_relay.py with hand-configured
sensors, and I have not tried to set retain, nor have I noticed it.
That is working well, but I don't have any rtl_433 devices that only
report on state change.


I wonder if people think this makes sense, or if others see it
differently.

[Benjamin Larsson]

Hi.

On 24/09/2023 15:28, Greg Troxel wrote:
>
> Some things are "sensors" (HA term), which have new values over time.
> The canonical example is temperature. Temperature fundamentally *is* a
> sensor, and most devices transmit it periodically.
>
> Some things are "binary sensors", which have values limited to on/off,
> over time. Examples are doors and moisture sensors. Some devices might
> send a value once a minute, and some might send a value when it changes.
> Given lack of acks, and the desire for prompt notification, it seems
> obvious that best practice (for a transmitter designer!) is to send the
> value when it changes and also at some background rate. (We recently
> have been talking about Govee which doesn't do it this way.)
>
> Some things are "events", which is like "button pushed". This can be a
> motion detection, and it can be a "user pushed button". The latter can
> be used to "transition house to night mode", for example.
>

What would this imply? To me it just sounds like a more formalized key
naming convention (which would be good).

[...]

>
> A nuance is that if a device sends a button press, and sends 4
> transmissions for redundancy, we really should be sending 1 event. This
> means the json/mqtt bridge has to have state and keep track of the
> last-fired time and suppress new events at least in the next 100ms or
> whatever. (Really, devices should send a packet number or something to
> allow merging the duplicate transmissions, but they are how they are.)
>

There has been some suggestions for a duplicate check. That would filter
out duplicate messages in a specific time frame.

>
> There was recently an query about the retained flag. It seems obvious
> that sensors should have the retained flag set if they post only
> occasionally, and not have it set if they post regularly. One would
> like a sensor that is not reporting to show as unavailable. It also
> seems obvious that events should never have the retained flag.
>

I'm not sure this belongs in rtl_433 unless it can be handled
automatically by the naming of keys. But what would happen if a message
contains several types?

IMO this is something a consumer needs to handle if it can't be added in
a structured way to rtl_433. Otherwise rtl_433 would need to split 1
message into 2 parts, one with the retained flag and one without.

MvH
Benjamin Larsson

[Greg Troxel]

"'Benjamin Larsson' via rtl_433" <rtl...@googlegroups.com> writes:

> On 24/09/2023 15:28, Greg Troxel wrote:
>> Some things are "sensors" (HA term), which have new values over
>> time.
>> The canonical example is temperature. Temperature fundamentally *is* a
>> sensor, and most devices transmit it periodically.
>> Some things are "binary sensors", which have values limited to
>> on/off,
>> over time. Examples are doors and moisture sensors. Some devices might
>> send a value once a minute, and some might send a value when it changes.
>> Given lack of acks, and the desire for prompt notification, it seems
>> obvious that best practice (for a transmitter designer!) is to send the
>> value when it changes and also at some background rate. (We recently
>> have been talking about Govee which doesn't do it this way.)
>> Some things are "events", which is like "button pushed". This can
>> be a
>> motion detection, and it can be a "user pushed button". The latter can
>> be used to "transition house to night mode", for example.
>>
>
> What would this imply? To me it just sounds like a more formalized key
> naming convention (which would be good).

In rtl_433, I think it is mostly a key naming convention. HA has
different semantics for sensors vs events, and while I don't want to
recast rtl_433 as an HA input scheme, I think that sort of semantics is
likely pretty universal. The idea would be to be able to send mqtt
autoconf json that takes keys with event_foo or button_foo (after we
figure out the details) and configures them as events/buttons, vs things
that are configured as sensors (numeric values) or binary sensors
(on/off).

> There has been some suggestions for a duplicate check. That would
> filter out duplicate messages in a specific time frame.

Now formally on the table as

https://github.com/merbanan/rtl_433/issues/2640

>> There was recently an query about the retained flag. It seems
>> obvious that sensors should have the retained flag set if they post
>> only occasionally, and not have it set if they post regularly. One
>> would like a sensor that is not reporting to show as unavailable. It
>> also seems obvious that events should never have the retained flag.
>
> I'm not sure this belongs in rtl_433 unless it can be handled
> automatically by the naming of keys. But what would happen if a
> message contains several types?

Two really good points.

First, I think the builtin mqtt, influxdb, etc. support should by
doctrine have no semantic processing. mqtt receivers should simply
expect to get all decodes as they happen. That more or less means no
retain flags. For "take reports of decodes and send them via mqtt to
some consumer that is not a log", I think we need something that does
semantic impedance matching.

As for a message with both events and sensors, probably the middleware
needs to split it and send on two topics, with only things that really
deserve retain separated. The only situations I see that fully make
sense are:

A binary sensor that is transmitted when it changes, and at some very
low rate (1h? 4h? daily?) otherwise. Here, you want HA (or equiv)
starting up to see a retained message, mostly. But you don't want it
to think the device has reported recently, when that isn't true. So
I'm not sure it's a good idea even then.

A sensor that reports at low rate, like once per hour. You might want
a data consumer that starts up to see the last value. But, that also
has the "is that value fresh and is it reporting". HA now will save
the value of sensors (including binary) on shutdown and re-read them.

> IMO this is something a consumer needs to handle if it can't be added
> in a structured way to rtl_433. Otherwise rtl_433 would need to split
> 1 message into 2 parts, one with the retained flag and one without.

I agree it doesn't belong in rtl_433 proper. I see it as fitting maybe
in rtl_433_mqtt_relay and rtl_433_mqtt_hass. (I also think some of
those "examples" should get elevated to real programs that are
first-class.)



Can anyone articulate why it makes sense to use retained at all for
these reports?