Full day highcharts question

[Didier Decoodt]

Hi

For the current day graph, I would like to have the xAxis from the beginning of the day to the end of the day (midnight).

When I use time_length = today, I have from the beginning of the dya (ok) to the current hour

My question is: which parameter I have to write in my graphs.conf?

Many thanks

[Pat]

Right now the time_length = today will only show you the values that have happened since midnight. 

[Didier Decoodt]

Yes, but How to have 24 hours duration for xAxis?

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0bd9ab2a-3b47-433b-847c-8705840614a4o%40googlegroups.com.

[Pat]

Not possible at this time. Curious; why do you want to show blank space?

On Sunday, June 14, 2020 at 10:38:08 AM UTC-4, Didier Decoodt wrote:

Yes, but How to have 24 hours duration for xAxis?

Le dim. 14 juin 2020 à 16:31, Pat <p...@obrienphoto.net> a écrit :

Right now the time_length = today will only show you the values that have happened since midnight. 

On Sunday, June 14, 2020 at 4:22:04 AM UTC-4, Didier Decoodt wrote:

Hi

For the current day graph, I would like to have the xAxis from the beginning of the day to the end of the day (midnight).

When I use time_length = today, I have from the beginning of the dya (ok) to the current hour

My question is: which parameter I have to write in my graphs.conf?

Many thanks

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Didier Decoodt]

I see this example on a site

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d280aebe-b614-4139-ac11-07c5541d7466o%40googlegroups.com.

[Pat]

I will say that I noticed in weewx 4, that the "year" timespan shows the full year now. I'm not sure what changed and haven't had the time to dive into it. I'm not sure if it's a change with archiveYearSpan() or with highcharts. But the archiveDaySpan() still shows just midnight to "now" with the belchertown charts.  

[Didier Decoodt]

Exactly, it's work for a year...

I will contact the site's owner for more info

Thank's Pat for your answer

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5e27f508-41dd-4620-add7-58f667c925cbo%40googlegroups.com.

[Manfred Maier]

This new behavior (ie. showing the entire year / month on the x-axis) has been the main motivation for me for switching to weewx 4.0.

In my opinion, this makes the charts much more readable. 

Personally I would love to use this 'feature' also for the daily charts.

[Pat O'Brien]

I'll have to look into it for the day charts. Like I said I'm not sure why the year charts are doing it but I haven't had time to look at it yet.

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/UWSnqt7tY3Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/95b20ca2-ed9a-430d-8df7-e8327d8ccbaeo%40googlegroups.com.

[Manfred Maier]

My post didn't want to say that you have to look into it NOW. 

I just wanted to give a '+1' for this feature request ;)

... perhaps it's even something that is not possible at all with weewx 4.0. 

[Pat]

haha i know what you meant! hope to look into it soon though

[Pat]

Coming up with a loss on this one. Both the "today" and the "year" charts are the full time spans generated by weewx for their time period, and both go through the same SQL function. Yet the year query results is populated 366 times (once a day), but the day query results is only populated for midnight to "now", not midnight to midnight like the timespan suggests. 

Anyone else have any insight on why:

The day span below does not return 288 results? (based on a 5 minute archive interval)

archive.getSqlVectors(TimeSpan(1592107200, 1592193600), outTemp, None, None)

However, this one return 366 results (based on the full year span)

archive.getSqlVectors(TimeSpan(1577854800, 1609477200), outTemp, max, 86400)

Thoughts? Tom? Matthew? Gary?

[gjr80]

What you are seeing is some subtly different behaviour brought about by the introduction of xtypes in WeeWX v4. One of the things that xtypes did was replace the manager.getSqlVectors() method; it now uses xtypes.get_series() (getSqlVectors() still exists and continues to be supported in v4 for backwards compatibility, it just calls xtypes.get_series()).

If the old getSqlVectors() was called with no aggregate, SqlVectors() simply queried the archive for data in the timespan concerrned and this query only returned records that currently exist in the archive (ie nothing in the future unless for some reason you had future dated data in your database). The number of data points returned by a non-aggregate call for current day, week, month, year spans was equal to the number of records from the start of the timespan to the current time. If an aggregate was included then the aggregate was calculated for all aggregate periods in the timespan (including future dated aggregate periods). Of course the aggregate for future dated aggregate periods is None/nothing and getSqlVectors() ignored these None/nothing rows. The number of data points returned by an aggregate call for current day, week, month, year spans was equal to the number of aggregate periods from the start of the timespan to the current time.

Now under xtypes.get_series(), the vectors returned when there is no aggregate follow the old getSqlVecrtors() approach; the archive is queried and only the data that is returned from the query is returned in the vectors (ie nothing future dated because no future date records exist) and again the number of data points returned by a non-aggregate call for current day, week, month, year spans was equal to the number of records from the start of the timespan to the current time. However, when an aggregate is specified xtypes.get_series() uses the same query as was used by getSqlVectors() but future dated aggregate results are now included. The number of data points returned by an aggregate call for current day, week, month, year spans was equal to the number of aggregate periods in the timespan.

So considering your examples, under the old getSqlVectors() I would expect your 'day span' call would return somewhere from 0 to 288 data points depending on time of day it was called. Your 'year span' call would return somewhere from 0 to 366 data points depending on the day it was called. Under the new v4 xtypes.get_series() I would expect your 'day span' call would return somewhere from 0 to 288 data points depending on time of day it was called. Your 'year span' call will always return 366 data points but of those 366 data points the last x data points will be None where x = 366 - day number of the year the call was made.

I am not sure whether the changed behaviour was a deliberate choice on Tom's behalf or not, I suspect not, best left for Tom when he is back in contact in a week or so. I will raise an issue on this so it is not lost.

Gary

[Manfred Maier]

Thanks, Gary, for this comprehensive explanation.

Based on what you wrote I was now able to plot an entire day.

The trick is quite simple: I just had to define an aggregation for the daily charts. I.e. adding the following two lines to the graphs.conf:

    aggregate_type = max

    aggregate_interval = 300

[didier....@gmail.com]

Thanks it's work !!!!

[Pat]

Awesome! Glad that it worked for you

[didier....@gmail.com]

Oh, just a small problem:

For Rain graph, the curve continue after "now"! (screenshot attached)

All others curves stop at datetime=now, it's correct)

The problem concern "rainTotal" and is also available for day, week, month and year time_length

Have you an idea?

hereafter the part of grah.conf

[week]

    # Chart Timespan Defaults

    title = "Cette semaine"

    show_button = true

    button_text = "SEMAINE"

    time_length = week

    tooltip_date_format = "LLLL"

    type = spline

    aggregate_type = max

    aggregate_interval = 3600 # 1 hour

    gapsize = 3600000 # 1 hour in milliseconds

    [[chart1]]

        title = Température

        [[[outTemp]]]

            color = "#b2df8a"

        [[[dewpoint]]]

            yAxis_label = ( °C )

            yAxis_tickInterval = 2

    yAxis_softMin = 0

    

    [[chart2]]

        title = Pluie

        [[[rainTotal]]]                          <============ Problem

            color = "#f7a35c"

            name = Cumul

[[[rain]]]

color = "#268bd2"

aggregate_type = sum

name = pluie

type = column

                yAxis_tickInterval = 1

                yAxis_label = ( mm )

yAxis_min = 0

[Pat]

No, not sure. I assume it's because the data points aren't reset to "None" using this new way of getting data for the rain observation?

Gary - did you create an issue for this? I'd hate to patch something in the skin to only undo it if/when Tom properly patches this.

[Pat]

I raised this as an issue and Tom mentioned for the day charts you should probably be using last as the aggregate, not max. 

I'll look into rainTotal since that's a Belchertown observation. 

[Didier Decoodt]

Pat, I put aggregate_type = none instead of max, and it's work!!!

I don't really understand aggregate_type effect....

differences between none, sum, min, max and avg?

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/35aa4d91-2dd6-4fc5-882d-78ddfd99c18co%40googlegroups.com.

[Pat]

rain total works? That's good though I don't know how! :)

For the aggregates, Belchertown borrows that from weewx's Seasons and Standard skins. Here's more information from their documentation. 

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Didier Decoodt]

Thanks for this documentation

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d738ffbd-0729-42a0-8b0e-15aa37381d95o%40googlegroups.com.

[Didier Decoodt]

Oupss

It seems to be correct but the aggregate_interval is not correct, every 5 mn instead of 1 day for monthly graphic...

aggregate_type=none is not appropriate...

(with  aggregate_type=last the curve continue after "now")

I think the problem is linked with rainTotal variable, 

[Pat]

Change your aggregate_interval to match your gapSize and see if that helps. 

On Monday, June 22, 2020 at 7:05:00 PM UTC-4, Didier Decoodt wrote:

Oupss

It seems to be correct but the aggregate_interval is not correct, every 5 mn instead of 1 day for monthly graphic...

aggregate_type=none is not appropriate...

(with  aggregate_type=last the curve continue after "now")

I think the problem is linked with rainTotal variable, 

Thanks for this documentation

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d738ffbd-0729-42a0-8b0e-15aa37381d95o%40googlegroups.com.

[Didier Decoodt]

I have:

    aggregate_interval = 86400 # 1 day
    gapsize = 86400000 # 1 day in milliseconds

but if aggregate_type=none, then I have a dot each 5 mn (may be it's normal)

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4a5a30f5-2eef-40ba-976d-f80e8ff11893o%40googlegroups.com.

[Pat]

I think I have a fix I will add to the development branch. Are you using the development branch? Can you try testing it too?

We haven't had rain for a long time, so I am trying this on my year chart. You can see attached you can see rainTotal stops today and doesn't carry to the end of the year anymore. 

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4a5a30f5-2eef-40ba-976d-f80e8ff11893o%40googlegroups.com.

[Didier Decoodt]

Yes I use the development branch. What is the procedure for upgrading?

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/e1f5cc02-3585-4aa7-af87-fc37df0e5ba5o%40googlegroups.com.

[Pat]

You can 

a) download the development zip file and run wee_extension --install to install it again just like you did previously

or 

b) download the development zip file and manually replace all files

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/e1f5cc02-3585-4aa7-af87-fc37df0e5ba5o%40googlegroups.com.

[Didier Decoodt]

Thanks 

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fc7ec2c8-8d1c-415c-be60-62dda8f956aeo%40googlegroups.com.

[Manfred Maier]

Thanks so much, Pat!

Works perfectly well.

[Didier Decoodt]

Hi Pat

After update, the graphic rainTotal does not start at the right date... the curve should start the 13 jun, not the 1st of june (I have no data before 13 june)

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/690bccbc-85af-4d11-bcac-f4c2d23cb975n%40googlegroups.com.

[Manfred Maier]

Hmm ... it's working on my end.

[Didier Decoodt]

Yes’ it’s working when you have data from the beginning but in my case, this is a new station, data start in the middle of time_length month and year 

For week it’s ok

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/dbd12f3f-6fa0-426d-8e5a-2facf9152a3dn%40googlegroups.com.

[Pat]

Please paste your full graph that's broken so I can take a look at it

[Didier Decoodt]

Please find enclosed:

week.jpg graphic

month.jpg graphic

year.jpg graphic

graphs.conf file

Le mer. 24 juin 2020 à 13:50, Pat <p...@obrienphoto.net> a écrit :

Please paste your full graph that's broken so I can take a look at it

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2669f288-4099-45af-88e7-5382397bf5b9o%40googlegroups.com.

[Manfred Maier]

Good catch, Didier!

I just found that I get the same wrong behavior for my 2019 chart. My station started in July only. And the RainTotal chart shows values for Jan-June.

It's the second one from the bottom: https://www.wetter-zorneding.de/graphs/?graph=2019

[Pat]

Interesting. Would one of you guys be able to share you database with me so I have test data to use? We haven't had rain in a long time so this would be easier to figure out where my math is going wrong. 

[Manfred Maier]

Sent by email. The file has been too big for an upload here (or the database content isn't allowed?).

Thanks for looking into it!

[Pat]

Got it - thanks!

[Pat]

Can you try the latest belchertown.py file from the devleopment branch? I think I got it working. See screenshot.

[Manfred Maier]

Solved!

Thanks so much! 

[Didier Decoodt]

Perfect !

Thanks

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/295f9456-430d-44fb-9c4e-9b61640794e0n%40googlegroups.com.

[Manfred Maier]

Hi Pat,

apologies, it's me again.

Yesterday (and today) I've just copied the new belchertown.py in my user folder.

Now I've seen that i get a two error messages in my syslog.

The first one I've been able to resolve. It was caused by the "check for update" functionality that you have removed recently. After setting the update setting to zero the error was gone.

The second one I'm not able to locate. This is the error I get:

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'NameMapper.NotFound'>'

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/Belchertown/js/belchertown.js.tmpl

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: **** Reason: cannot find 'forecast_provider'

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in generate

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_Belchertown_js_belchertown_js_tmpl.py", line 1154, in respond

Jun 24 22:20:20 raspberrypi weewx[6177] ERROR weewx.cheetahgenerator: ****  NameMapper.NotFound: cannot find 'forecast_provider'

[Pat O'Brien]

Update your skins.conf to latest version too

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/UWSnqt7tY3Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4e605379-67f9-4db4-b1a9-07b752dcbcebn%40googlegroups.com.

[Manfred Maier]

Didn't work :(

I don't understand this error. The 'forecast_provider' is in my skin.conf and the weewx.conf:

            # Forecast defaults

            forecast_enabled = 1

            forecast_provider = "darksky"

            forecast_api_id = "b6b2add3..."

            forecast_api_secret = "b6b2add3..."

            forecast_units = "ca"

            forecast_lang = "de"

            forecast_stale = 3540

            forecast_alert_enabled = 1

            forecast_alert_limit = 1

            forecast_show_daily_forecast_link = 0

            forecast_daily_forecast_link = ""

[Pat O'Brien]

Can you try to reinstall the whole development skin and restart weewx?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/aa56d448-ba0a-4d31-a737-ba2eb27c191fn%40googlegroups.com.

[Manfred Maier]

I'll have to try that tomorrow. 

My skin installation includes a number of individual adaptions (e.g. pi page, individual website widget, horizontal lines in some graphs, etc) and it always takes me 1-2 hours to update the skin. 

[Pat O'Brien]

Wow! I'm not sure why you're getting that error either. Something is missing somewhere

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/712411b5-3388-4713-b710-adf4523f36f6n%40googlegroups.com.

[Manfred Maier]

A new installation of the entire skin fixed it.

I have no idea what went wrong with the old one. My suspicion would be that there was some encoding issue with a file. 

Something I noticed after the new installation:

* In the old version of the weewx.conf, I had included the DarkSky secret key in quotation marks

* After the re-install of the development skin, those quotation marks were gone

--> This caused a weird behavior: When triggering a report with wee_reports everything worked fine. But in the next regular report loop, the DarkSky forecast disappeared . With a wee_reports it came back. And disappeared again in the next regular reporting cycle.
* Putting back the quotation marks fixed this issue

[Christian Gruber]

should this still work with weewx 4.5.1 ?

i updated and my full day charts are gone

[James Taylor]

Not sure.   Under weewx 4.4.0 my Temperature chart were as follows under Belchertown

Under weewx 4.5.1 it was displaying as the following

I made no changes other than upgrade to 4.5.1.

# Global Chart Defaults
# These are fallback options that charts will use if an option is not defined.
aggregate_type = None
time_length = 90000 # Last 25 hours
type = line
colors = "#7cb5ec, #b2df8a, #f7a35c, #8c6bb1, #dd3497, #e4d354, #268bd2, #f45b5b, #6a3d9a, #33a02c"
tooltip_date_format = "LLL"

[day]
    # Chart Timespan Defaults
    title = "Today since Midnight"
    show_button = true
    button_text = "Day"
    time_length = today
    tooltip_date_format = "LLL"

    aggregate_type = max
    aggregate_interval = 300

    gapsize = 300000 # This should be your archive_interval from weewx.conf multiplied by 1000 to get milliseconds. Standard is 5 minutes

    [[chart1]]
        title = Temperature
        start_at_midnight = true
        [[[outTemp]]]
            zIndex = 1
            name = Temperature
        [[[windchill]]]
        [[[heatindex]]]
            color = "#f7a35c"
        [[[dewpoint]]]
            color = purple

Obviously 4.4.0 came out this year so I'm wondering if something has been changed in Tom's code.

James

[Arend]

This new behaviour is caused by this WeeWX 4.5.1 commit:

https://github.com/weewx/weewx/commit/a6098878562e4a35345051c45ee6aadf31edbdce

            for stamp in weeutil.weeutil.intervalgen(startstamp, stopstamp, aggregate_interval):

                # Get the aggregate as a ValueTuple

                agg_vt = get_aggregate(obs_type, stamp, do_aggregate, db_manager)

                if agg_vt[0] is None:

                    continue

                if unit:

                    # It's OK if the unit is unknown (=None).

                    if agg_vt[1] is not None and (unit != agg_vt[1] or unit_group != agg_vt[2]):

Comment out the text marked as bold in xtypes.py to restore previous behaviour or wait for Pat (or someone else) to create a fix for Belchertown skin.

Op maandag 3 mei 2021 om 19:48:20 UTC+2 schreef James Taylor:

[James Taylor]

Thanks.

What I'm trying to work out is if this is expected / correct behavior of weewx now.   Reading Gary's comment back in June 2020 sort of makes sense, but I'm still not 100% sure.

James