battery status reporting: docs vs reality

[Greg Troxel]

We've had a lot of cleanup and I want to check in about where we are.
This is prompted by a proposal to add non-standard battery fields to
WH52 (which I've objected to).
[>
In docs/DATA_FORMAT.md we have:

* **battery_ok** (double) (Optional)
* Battery status indication as a level between 0 (empty) and 1 (full). If the sensor can only report a binary status the value shall be 1 for "OK" and 0 for "LOW".

* **battery_V** (**battery_mV**) (double) (Optional)
* Battery level in Volts. Should be supplemented by *battery_ok* status indication if possible.


I interpret this as:

- battery voltage

+ If the device reports battery voltage, then the driver should
output "battery_V" for the voltage io volts. This must be as
DATA_DOUBLE.

+ We acknowledge that some devices output battery voltage as mV to
battery_mV. It is left implicit that this might be DATA_INT, even
though it's said to be properly in DATA_DOUBLE.

+ The use of battery_mV is a historical situation that should be
resolved.

- battery level

+ The "battery_ok" data item represents the battery ranging between
0.0 (at device failure) and 1.0 (brand new). Ideally this will
linearly decline between putting in a brand-new battery and the
moment the device goes offline or first behaves erratically due to
low battery. (In practice there is guesswork, but the linear
decline with failure at 0 is the target we are guessing at.)

+ As a way to deal with devices that have only a "battery is ok" vs
"battery is not ok" indication, we map ok to 1.0 and not ok to
0.0.

- other comments

+ Remember that the first prime directive is to represent faithfully
what was transmitted, but that faithfully includes scale and unit
transformation. The second prime directive is to bring data into
a common representation so that it's usable without being an
expert about the particular device.

+ Absent discussion and modifying the design document, we should not
add other kinds of battery entities.

+ If a device reports voltage and a % remaining lifetime, they
should bothe be reported.

+ If a device reports voltage only, a driver must report that and
should create a synthetic battery_ok.

+ voltage should never be synthesized.

+ If a device reports voltage that can usefully be mapped to
battery_ok and also a binary ok/not-ok, we should discuss how to
proceed.


(This note intentionally does not talk about back compat and getting from
here to there. I'm trying to establish the current design, and
deferring migration.)


Agreement? Disagreement?

[Greg Troxel]

And

When reporting battery_ok for a device with coarse quantization, the
chosen % should be more or less the lowest remaining lifetime that is
consistent with the voltage, at least in giving the right idea about
the low end. E.g. a device that reports 1300 mV at remaining
lifetimes of 40% to 1% should be mapped to a value that indicates the
battery is critically low. The exact details of this mapping are not
specified by this document.

[Christian Z.]

> + As a way to deal with devices that have only a "battery is ok" vs
"battery is not ok" indication, we map ok to 1.0 and not ok to

0.0.

The current scheme is to map a battery low flag to integer 0 (and integer 1 otherwise).

This helps consumers detect if a range of values is available but comes at the cost of having a variable type.

Since the type can always be parsed as a float it should not be a problem.

> + If a device reports voltage that can usefully be mapped to
battery_ok and also a binary ok/not-ok, we should discuss how to
proceed.

I think that we won't need a binary flag if a range of steps is available. The binary battery low info is included in something like "battery_ok <= 0.2".

We would need to document where we want that last level to be. Imagine a device with four steps, is the last one 25 %? Or maybe a device with three steps (full/ good/ low) -- is that 100 / 67 / 33 %? Or perhaps better 100 / 50 / 20 %? I.e. demand the last level to be at or below 0.2?

[Greg Troxel]

[Greg Troxel]

Thanks for answering on list.

"Christian Z." <chri...@zuckschwerdt.org> writes:

>> + As a way to deal with devices that have only a "battery is ok" vs
>> "battery is not ok" indication, we map ok to 1.0 and not ok to
>> 0.0.
>
> The current scheme is to map a battery low flag to integer 0 (and integer 1
> otherwise).
> This helps consumers detect if a range of values is available but comes at
> the cost of having a variable type.
> Since the type can always be parsed as a float it should not be a problem.

It is not really a problem, except that it is irregular for no good
reason. If battery_ok is defined to be a float, is there any downside
to changing the type to DATA_FLOAT everywhere it's used this way?

Or are you saying that it's a feature that value is polymorphic and that
the type denotes the semantics? That in my view is way too subtle. I
feel that if we want metadata about what's expressed then we should do
that somehow.

Are there any known users that key off 1.0 vs 1 in output? This feels theoretical.

>> + If a device reports voltage that can usefully be mapped to
>> battery_ok and also a binary ok/not-ok, we should discuss how to
>> proceed.
>
> I think that we won't need a binary flag if a range of steps is available.
> The binary battery low info is included in something like "battery_ok <=
> 0.2".

I'm fine with that. My goal here is first to get us on "written plan,
follow it".

> We would need to document where we want that last level to be. Imagine a
> device with four steps, is the last one 25 %? Or maybe a device with three
> steps (full/ good/ low) -- is that 100 / 67 / 33 %? Or perhaps better 100 /
> 50 / 20 %? I.e. demand the last level to be at or below 0.2?

(The value is 0-1 and we as people are discussing %. That's totally
fine but not an agreement that we should change the definition!)

It depends. I would say the rules are:

First, determine how the reported values correspond to % lifetime.
There is absolutely no way to have a rule that takes n steps from
random devices and gets that right.

Second, realize that the point of battery reporting is to convey
information that the battery is low and might need attention.
Therefore, if one has a report of step value X, the right % is the
largest remaining % such that it's a reasonably safe bet that with
that report, there is that much left.

Third, relax the second rule to be some blend of literal and close
enough that there is the right impression. Accept that 10% means "the
device is working but the battery is so low that you're on thin ice
and if you value reliable operation you should replace it right now.".


So imagine a device with 4 levels, and that reports them for a number of
days, after a fresh battery, with assumed constant drain

A: 30
B: 100
C: 120
D: 50

with the implication being that it stops transmitting after those 50 in
D. I'm totally making this example up, and to figure out how to map a
real device, one would need to put in a fresh battery and data log the
values for the batteries entire lifetime.

This sums to 300 days. I would map this

A: 100%
B: 50%
C: 20%
D: 10%

commentary:

A is "the battery is obviously new and things are great". I would be
fine with mapping A to 90% as well.

B is the battery is on the first half of usage. Might be 90%, might
be 50%. call it 50%

C is the battery is getting low, but it's not critical. 20% is close
to the 50/300, a bit higher but gives the right impression

D means you're on thin ice. Use the 10% signal, vs 0% when we are
sure it's dead.

[Greg Troxel]

With a 1-bit input, you are dealing with:

battery is not low, so it might be from X% ish to 100%
battery is low, so it might be from 0% to X%

where X will vary according to the device, and our job is to figure out
what it is, maybe - we certainly can't choose it. If designing a
device, I'd say it ought to be 15% or 20%, roughly.


As for the user perception when mapped, I think 0% for 'low' is totally
fair. The device has said "the battery is not ok" and it is not capable
of anything further.

We could play games where

if there's a real % or battery V with mV resolutio, just use it

if there are 10ish levels, map them, but avoid 100% and 0% in favor of
95/5 as the extremes

if it's binary, use 100/0

and that could be a clue.

My experience in home assistant (not because HA matters to rtl_433; it's
just a situation where I monitor too many batteries) is that there are
all sorts of devices. A very few count down 100->0 pretty linearly.
Some (ecolink flood sensors, zwave) are 100/99/0. Some go 100->85% but
85% is basically dead so that's wrong. Some are 100/80/60/40/20/0, but
they fail a bit after 40 and never report 20. And some are just 'help
my battery is low' vs not. So I don't think anybody can really do much
useful other than "when device reports below 15%, replace it now".


Getting the mapping right is hard. I am totally ok with guesses
intended to be as close as we can be, to be refined with experience.
That's a bug fix, not an API change, so there's no compat issue.