Need solid line in graph plot

[bgra...@umw.edu]

I'm moving from weewx 4.10.2 (setup.py) on Ubuntu to an RPI5 with (pip install) Bookworm.

Thought I could figure this one out...

The data is coming every 20 minutes via MQTT and my VP2+ has an archive period of 15 minutes. Thus, there are three updates per hour which don't always correspond with the archive period.

weewx.sdb shows, for example:

extraTemp3
81.79
82.2
82.36
null
83.14
83.86
84.07
null
84.06
83.25
82.4
null

My skin.conf uses:

[[[dayextraTemp3]]]
             yscale = None, None, 0.5
            [[[[extraTemp3]]]]
                label = Newell Temperature

I've looked at the wiki but can't figure out how to fix it.

Thanks for any pointers.

Cheers,

Bob

[Jeff A. D.]

https://weewx.com/docs/5.1/custom/image-generator/?h=gap#line-gaps

So you should have a line that says line_gap_fraction=0.01 you can try removing or commenting out.  I think changing the fraction to 0.02 or more would work also.

[bgra...@umw.edu]

Jeff,

Thanks for the reply. I was showing line_gap_fraction=0.05 so I commented it out. I then tried 0.02 but neither worked. I removed the .png file and restarted weewx + ran the Standard report each time. No luck with either setting. In 4.10.2 my setting was  line_gap_fraction=0.01 and it was working. Trying 0.01 here didn't work. Is there something else I need to run to rebuild the plot using the changes in skin.conf?

Thanks for the help.

Bob

[bell...@gmail.com]

I don’t fully understand the `line_gap_fraction` option. But I seem to remember that `None` values will always cause a break in the plot…

Bob, I think we had worked out how to use MQTTSubscribe caching for this observation. I think with the updates you made to get MQTTSubscribe V3 to work, your configuration is no longer using the caching function….

- rich

[bgra...@umw.edu]

Rich,

You are correct, of course, since I changed the source of the data to another MQTT broker I didn’t carry over all of the original code you setup for me on 4.10.2. Let me play with that and see what I can do. Thanks.

[gjr80]

On Saturday 20 July 2024 at 07:42:39 UTC+10 bell...@gmail.com wrote:

I don’t fully understand the `line_gap_fraction` option. But I seem to remember that `None` values will always cause a break in the plot…

line_gap_fraction does seem a little mystical in operation but once you understand how it works it is fairly easy to use. Think of line_gap_fraction as the proportion of the overall x-axis timeframe that the plot engine considers to be a gap in data. So for a typical day plot (27 hours or 97200 seconds) a line_gap_fraction of 0.05 means that any gap in time between successive plot points of 4860 seconds (81 minutes) or longer will be considered a gap and result in a gap in the plotted line. If line_gap_fraction = 0.01 the gap drops to 16.2 minutes. So for data with a 15 minute archive period a line_gap_fraction = 0.01 will plot a continuous line provided there are no missing points. Any missing points result in a gap of at least 30 minutes so line_gap_fraction = 0.01 will result in gaps. In this case a line_gap_fraction = 0.02 or greater is required. The default line_gap_fraction = 0.05 should work fine. If it is not then something else is amiss (first thing that springs to mind is the possibility that line_gap_fraction is being overridden somewhere in the respective skin.conf or weewx.conf).

The same applies for week, month and year plots, but you now apply line_gap_fraction to a much longer overall x-axis timeframe meaning the gap in seconds for a given line_gap_fraction value is greater . Also, week, month and year plots by default plot an aggregate so the data points are now spaced every 'aggregate interval' seconds and not necessarily every 'archive period' seconds.

Also, when line_gap_fraction is omitted (or set to a value <0 or >1) any missing data points will be considered a gap and plotted as such. In this case you definitely do not want to omit line_gap_fraction or you will always have gaps.

Gary

[bell...@gmail.com]

Gary,

Thanks for the very clear explanation.  I guess I was a bit lazy. After reading your post I found the documentation for it here, http://www.weewx.com/docs/5.1/reference/skin-options/imagegenerator/#line_gap_fraction

Also, digging into the code seems to confirm that null/None values will always cause a break. See, https://github.com/weewx/weewx/blob/95210c4e6008b67c14220157e4a7934c1e03bd42/src/weeplot/utilities.py#L472

- rich

[bgra...@umw.edu]

Rich,

As you suggested, I added the additional MQTTSubscribe code to my weewx.conf file and it seems to be filling in the gaps. Thanks for clearing this up for me.

[[[lht65b]]]    #105a inside temp
ignore = false    # opt in for this field
                name = extraTemp2
expires_after = None    # cache never expires
                contains_total = false
                conversion_type = float

           [[[[message]]]]
                type = individual

Gary, I "sort of" understand the line_gap_fraction--thanks for the details. Since mine was set to 0.05 (weewx.conf 5.1), I'm not sure what might be overiding this value. I guess it's possible that some other format setting is affecting it. I've looked at skin.conf and weewx.conf but nothing at least directly mentions the line_gap_fraction.

Everyone's help here has been greatly appreciated.

Cheers,

Bob

[bell...@gmail.com]

Bob,

Thanks for letting us know that you are up and running. I greatly appreciate knowing when a suggestion has worked (as I bet future google searchers do too). A couple of comments.

The key option you add is `expires_after = None`. This enables MQTTSubscribeService’s cache. The `contains_total` and `conversion_type` should have no impact. The values you set them to are the default. But you have something working, so I wouldn’t change anything.

I did a bit more research on `line_gap_fraction`. From reading the forums, it further confirms my understanding that it doesn’t work with null values. Stating differently, it works when a complete archive record is not in the DB but not for missing values in an archive record. So in your case, no setting of `line_gap_fraction` will fix your plots. You need to get the extraTemp2 value into the DB, which is what the MQTTSubscribeService cache does.

Hopefully this makes some sense and also correct.

rich

[Graham Eddy]

maybe report interval aggregation, to (mostly) eliminate NULLs

⊣GE⊢

[bgra...@umw.edu]

My problem is definitely caused by the archive period (15 minutes) and the sensor reporting time (20 minutes). The reporting time can be adjusted up and down, however the battery is supposed to last 10 years at the default setting. The Dragino lht65 is a nice temp/hum unit which costs around $35. It transmits LoRa data from a neighbor’s Stevenson shelter several blocks away to my weewx station. The range is supposed to be ok for several kilometers. Just thought I’d pass this along.

Again, much appreciated to all and especially Rich and MQTTScribe.

Cheers,

Bob

Http://grattans.org/wx (soon to be running weewx 5.x)

[Graham Eddy]

i have same situation - archive interval 5m, lorawan sensors 20m - so my 5 min reports aggregate the lorawan sensors over last 20m to pick up the last value each time e.g.

[[[plots/dayrivercomb]]]

    aggregate_type = last

    aggregate_interval = 1200  # 20 mins

    line_gap_fraction = 0.05

    [[[[river_level]]]]

    [[[[river_level2]]]]

    [[[[river_level4]]]]

        color = black

cheers

⊣GE⊢

[bell...@gmail.com]

Cool. Thanks for sharing.

rich