Belchertown, graphics are faulty

[geni08...@gmail.com]

On 3 days we had rain of 9mm, 8.6mm and 0.6mm. The graphic now shows 0mm, 15mm and 3.2mm on the corresponding days. The sum is correct but wrongly divided. Where is the mistake?

[month]
    # Chart Timespan Defaults
    title = "Letzte 30 Tage"
    show_button = true
    button_text = "Letzte 30 Tage"
    type = spline
    time_length = 2592000 # Last 30 days
    tooltip_date_format = "dddd LL"
    aggregate_type = max
    aggregate_interval = 86400 # 1 day
    gapsize = 86400000 # 1 day in milliseconds
   
    [[chart3]]
        title = Regenx
        type = line
        aggregate_type=sum
        [[[rainTotal]]]
            name = Regen Total
        [[[rain]]]   
            yAxis = 1
            type = column
   

Live: https://affolter.familyds.net/wetter/graphs/?graph=month

[geni08...@gmail.com]

@Pat: The error is not only with me, also with @Andre. I see the error when the graphic "last 30 days" has been selected and the rain is selected as the daily total bar. The graphics are one day late. No daily value is correct, the sum of 3-4 days is correct again. In my opinion the graphic has problems to calculate the correct local start and end of the day.

[Pat]

Which one is the error? rainTotal does not use the aggregate_type since it is already aggregating these totals. 

[Geni 0815]

I would like to have a graph of the last 30 days with a column with the amount of rain per day and a curve that accumulates these daily amounts.
What should the definition in fer Graphic.conf look like?
Do these daily values have to match the NOAA reports?

[Pat]

The rainTotal is the accumulation over the time span, so that will be your line graph. 

The rain will show you rain over the aggregate_interval - so for this chart it's 1 day. 

So your graph looks right, but I'm seeing the data a little wrong too. For example on June 28 I had 2.41 inches rain, but the graph shows it on June 29. 

I'll look at why it's showing the wrong day. I think it's a setting with the way the json generator is running

[Pat]

Edit your bin/user/belchertown.py file. 

Look for this line:

        if time_length == "today" or time_length == "timespan_specific" or isinstance(time_length, int):

Change it to

        if time_length == "today" or time_length == "timespan_specific":

Update your graphs.conf with this

[month]
    # Chart Timespan Defaults
    title = "Letzte 30 Tage"
    show_button = true
    button_text = "Letzte 30 Tage"
    type = spline
    time_length = 2592000 # Last 30 days
    tooltip_date_format = "dddd LL"
    aggregate_type = max
    aggregate_interval = 86400 # 1 day
    gapsize = 86400000 # 1 day in milliseconds
   
    [[chart3]]
        title = Regenx

        [[[rainTotal]]]
            type = line
            name = Rain Total
        [[[rain]]]  
            aggregate_type = sum
            yAxis = 1
            type = column

Restart weewx and let me know if this works for you.

Highcharts looks to be doing a little bit of rounding. I'm seeing my 2.42

[Geni 0815]

Hi Pat, for me the mistake is more than just a rounding.

Day     in the report      in graphics

26            20.0                23.4
27             3.4                  0.0

28             8.6                33.2

29           24.8                  0.2

01            13.4                18.6
02             5.4                 0.8

03             0.6                 0.0

10             9.0                15.0
11             8.6                 3.2
12             1.0                 0.4

[Geni 0815]

Hi Pat, I just noticed the following:
As an example in the graphic "last 30 days", "rain per day". These values sometimes change every hour!
For my understanding, the amount of rain per day is to be understood on a calendar day within the last 30 days.
In the graphics, however, the day begins fluently.
I want a day to be a calendar day and not flowing 24h.
This is how it is handled in the Templete Season.
How do I have to make the graphic settings so that a day is a calendar day?
I hope I have clearly described my wish.

[Pat]

Sounds like you may have some problems with your database. I suggest you take a backup and follow the database drop-daily and rebuild-daily and see if that helps

The graphs uses your archive table

The reports uses your daily tables. 

Seems like those aren't in sync for you.

[Geni 0815]

Why is the value correct here?

https://affolter.familyds.net/wetter/graphs/?graph=monthnow

[Andre]

I can reproduce the display issues.

I have dropped all daily reports and rebuild daily.

Last 7 days

Last 30 days...

Am Mittwoch, 15. Juli 2020 14:19:19 UTC+2 schrieb Pat:

[Pat]

I'm at a loss because I can't recreate this. 

Please send me a copy of your database as well as your graphs.conf, and public_html/belchertown/json/weewx_data.json

My time to dedicate to this is slim right now, so if anyone else finds the problem that'd be helpful and I can merge it to development branch. 

[Manfred Maier]

Couldn't this just be an issue with how the timespan for the chart is defined?

The chart displays the past 2592000 seconds and splits it into pieces of 86400 seconds (24 hours). But those 24 hours might go from 11am - 11am.

Just a thought and probably wrong ...

[Pat]

Great point Manfred. If you guys change your time_length to month, do the values match?

[Andre]

With "time_length = month" the values are correct of course.

Maybe we need a fix to generate real calendar days for "time_length = 2592000".

[Pat]

What do you mean real calendar days? The 2592000 is the previous 2592000 seconds. This number can be anything you want. 

[Andre]

It was just a thought, because with "time_length = 2592000" we do not get the desired values.

[Pat]

So if you don't want the last 2592000 seconds, maybe you want "the last 30 days" as a rolling window?

In that case, use time_length = days_ago and time_ago = 30

Please do read the charts wiki, everything is covered there. 

[geni08...@gmail.com]

Hi Pat, the Wicki doesn't get me any further because I don't understand some things and I can't get any graphics as I wish. I want three examples
each the amount of rain per unit of time in mm as a column and the cumulative total as a line.

Example 25h: The variant with mm / 10min works, but the variant with mm / h only results in nonsense.

[25h]
    # Chart Timespan Defaults
    title = "Letzte 25h"
    show_button = true
    button_text = "Letzte 25h"
    time_length = 90000
    tooltip_date_format = "LLL"
    gapsize = 600000 # This should be your archive_interval from weewx.conf multiplied by 1000 to get milliseconds. Standard is 5 minutes

    [[chart1]]
        title = Regen

        [[[rain]]]
          yAxis = 1
          type = column

          name = Regen mm/10min

        [[[rainTotal]]]
          name = Regen Total
   

    [[chart2]]
        title = Regen
        aggregate_interval = 86400
        [[[rain]]]
          aggregate_type = sum

          yAxis = 1
          type = column

          name = Regen mm/h

        [[[rainTotal]]]
          name = Regen Total

The two variants "Last 7 days" and "Last 30 days", because nothing is plotted during my attempt.
Can I get the corrected samples?
Thank you for your help

[Pat]

So on 1 page, you want 3 charts?

1. Rain for 25 hours

2. Rain for 7 days

3. Rain for 30 days

Yes, we need to use seconds for this as that is the rolling time period. The time_ago does not do a rolling period, only the calendar day for that time ago. 

Is this what you want? https://belchertownweather.com/graphs/?graph=testrainpage

[Pat]

This conversation has me playing with new time spans too. A rolling calendar day, like you mentioned. 

Options could be hour, day, week, month, year. For example:

1. time_length = day_rolling with time_ago = 90 would show 90 days ago to "now"

[2020-04-17 00:00:00 EDT (1587096000) -> 2020-07-16 12:05:00 EDT (1594915500)]

2. time_length = week_rolling with time_ago = 1,  gives you from midnight 1 week ago to "now". 

[2020-07-09 00:00:00 EDT (1594267200) -> 2020-07-16 12:00:00 EDT (1594915200)]

3. time_length = month_rolling with time_ago = 1, would be midnight 1 month ago, to "now". 

[2020-06-16 00:00:00 EDT (1592280000) -> 2020-07-16 12:00:00 EDT (1594915200)]

4. time_length = year_rolling with time_ago = 4 would show 4 years ago to "now"

[2016-07-16 00:00:00 EDT (1468641600) -> 2020-07-16 12:05:00 EDT (1594915500)]

I think we'll also need a "timestamp ago" to "now". So I could specify a chart from Jan 1, 2018 to now for example. 

time_length = timestamp_rolling with time_ago = an epoch timestamp like 1514782800 (this is Jan 1, 2018 midnight Eastern time)

[2018-01-01 00:00:00 EST (1514782800) -> 2020-07-16 12:10:00 EDT (1594915800)]

I will look to add these new time lengths to the 1.2 development branch for testing

[geni08...@gmail.com]

I would like to:
1. Rain 25h -> mm per h, not every 5 min
2 Rain Last 7 Day -> mm per day, not per hour
3 Rain Last 30 Day -> is what I want

[geni08...@gmail.com]

How does this season do it?

Pat schrieb am Donnerstag, 16. Juli 2020 um 18:30:19 UTC+2:

1. Don't think this is possible. The graphs do not do any additional math. They show what is in the database and aggregate those values if needed (such as max, min, sum, etc.). 

2. Don't think this is possible. The graphs do not do any additional math. They show what is in the database and aggregate those values if needed (such as max, min, sum, etc.). 

3. Rain last 30 days is:

[testrainpage]
    title = "Rain Test"
    show_button = true
    button_text = "Rain Test"
    tooltip_date_format = "LLL"


    [[rain30d]]
        title = "Rain Last 30 Days"

        time_length = 2592000 # Last 30 days

        aggregate_type = sum
        aggregate_interval = 86400 # 1 day
        gapsize = 86400000
        [[[rain]]]
            name = Rain
            type = column
        [[[rainTotal]]]
            name = Rain Total

[geni08...@gmail.com]

Season is the standard at weewx.

The reports report rain per day :

Pat schrieb am Donnerstag, 16. Juli 2020 um 18:38:34 UTC+2:

I don't know, I don't use seasons. Can you provide the chart code for those graphs? 

[Pat]

Looking at the source code of Seasons, try this in your graphs.conf. I think this is what you're looking for with the aggregations you want. 

[testrainpage]
    title = "Rain Test"
    show_button = true
    button_text = "Rain Test"
    tooltip_date_format = "LLL"

   
    [[rain25h]]
        title = "Rain 25h"
        time_length = 90000 # 25 hours
        [[[rain]]]
            name = Rain (hourly total)
            type = column
            aggregate_type = sum
            aggregate_interval = 3600

        [[[rainTotal]]]
            name = Rain Total

    [[rain7d]]
        title = "Rain Last 7 Days"
        time_length = 604800 # Last 7 days
        gapsize = 3600000
        [[[rain]]]
            name = Rain (daily total)
            type = column
            aggregate_type = sum
            aggregate_interval = 86400

        [[[rainTotal]]]
            name = Rain Total

    [[rain30d]]

        title = "Rain Last 30 Days"
        time_length = 2592000 # Last 30 days

        gapsize = 86400000
        [[[rain]]]
            name = Rain (daily total)
            type = column
            aggregate_type = sum
            aggregate_interval = 86400

        [[[rainTotal]]]
            name = Rain Total

On Thursday, July 16, 2020 at 12:36:01 PM UTC-4, geni08...@gmail.com wrote:

[geni08...@gmail.com]

that's not what i want, your 30 day version is correct

https://www.affolter.familyds.net/wetter/graphs/?graph=25h

[Pat]

I'm at a loss now. I've copied the aggregation types and intervals charts you showed me from Seasons.

[geni08...@gmail.com]

A sum for a certain time, in the example 1 day must start with 0. In the example, the graphic starts the day with 0.2mm, which results in a difference of 0.2mm between the real daily amount and the daily amount of the graphic.

How can I move the graphic not to start at 0h but for example 1 second later then the precipitation from 23:50 to 24:00 would probably not be counted.

These time slots appear to be a general problem for the graphics engine, which also creates the previous problems.
When it rains, I am particularly interested in how many liters of water I get in a certain time, how quickly it falls down may be interesting due to soil erosion or water drainage problems.

The Current Month graphic is very interesting because the current daily total is correct again

[geni08...@gmail.com]

Hi PAT, I have version 1.2rc2 since yesterday. So now "start_at_midnight = true" also works. Very important, this is the only way that the daily rain sums are correct.