Loop through all years in db

[Praveen Chandrasekaran]

Hi,

I get how to loop through all months of a particular year from example. I want to add one more loop outside to loop through all years.

I am trying to build all years monthly highs, lows etc in one page 

How do I loop through all years available in the db? I assume I have to use .years but (unknown).years. What should the unknown be. 

Regards,

Praveen 

[Tom Keffer]

I haven't tried it, but I don't see why $alltime.years wouldn't do it. Requires WeeWX v4.6 or later.

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/CA%2BW%3DTmXZNV4uE_96DQDVABBM9Js5t9Fh1VLk%2BYQPRQQ6Ezwumw%40mail.gmail.com.

[Praveen Chandrasekaran]

Realized that i did reply instead of reply all. 

On Sat, 13 Aug, 2022, 09:05 Praveen Chandrasekaran, <pravee...@gmail.com> wrote:

Follow up - if i want to get max/min/average etc across all years for a particular month - do i need to loop around all years and do or is there an easier way? Like june.$alltime.avg etc? Is it possible?

On Fri, 12 Aug, 2022, 21:53 Praveen Chandrasekaran, <pravee...@gmail.com> wrote:

Thanks. This was exactly what I was looking for. 

[Tom Keffer]

So you did! Here's my response:

There's no explicit tag to do that. You could loop over all months, then test the date to see if it is in a particular month. Something like (NOT TESTED):

#for month in $alltime.months

#  if month.start.format("%b") == 'Jun'

#    <p>Max for year $month.start.format("%Y") is $month.outTemp.max</p>

#  end if

#end for

Alternatively, you could write an xtypes extension and create a new aggregation type. Similar to the "historical_min" and "historical_max" aggregations in the weewx-xaggs extension, except it would be the historical min and max for the month, instead of the day.

[Praveen Chandrasekaran]

Finally managed to get a page like what I wanted up for one month:

http://bedi.co.in/~pcws/tabular.html?report=Climatalogy.txt

Attached template file used for same.

Some inputs on below would be helpful:

1) The way it is done now, I need to have 12 templates - one for each file. With changes in just 2 lines. Looking for a way to do it smarter. I see that SummaryByMonth etc creates timespans and loops. But in this scenario, the timespans I am interested in are not continuous (Jan of each year, Feb of each year and so on). So I dont think I can create timespans for these

2) xaggs may not work I think since it seems to query daily summary from db. There is no monthly summary in db however as I understand

3) How do I make different month templates run at different times. No point in running for month off Jan daily in August for example.

PS : I am a noob on cheetah etc. Just reading through stuff now to get hang of things.

[Tom Keffer]

It's true that there is no "monthly summary" table, but that shouldn't stop you from doing an xtypes extension. I think all that would be necessary is to modify the SQL queries slightly in weewx-xaggs. For example, to return the all time historical high for a month the query (line 45)

"SELECT MAX(`max`) FROM {table}_day_{obs_type} "
                  "WHERE STRFTIME('%m-%d', dateTime,'unixepoch','localtime') = '{month:02d}-{day:02d}';",

becomes

"SELECT MAX(`max`) FROM {table}_day_{obs_type} "
                  "WHERE STRFTIME('%m', dateTime,'unixepoch','localtime') = '{month:02d}';",

and so on. Call it something like "historical_month_max". That leaves the problem of where does "month" come from. In weewx-xaggs, it comes from the month of the date whose page is being generated. However, you want it for a specific month, not necessarily "today." You can use the optional parameter "option_dict" to set it (requires WeeWX V4.8 or greater). Then your template looks like

#for month in range(1,13)

# <p>The historical high for month $month is $alltime.outTemp.historical_month_max($month)</p>

#end for

Unfortunately, "option_dict" is not documented in the xtypes wiki. I'll try to get around to that today. Or, perhaps Gary can do it.

I realize this is a rather "stream-of-consciousness" post. Hope I'm making myself clear.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/e6b5d587-912f-41ae-b3af-0e6dbd0f5d7bn%40googlegroups.com.

[Praveen Chandrasekaran]

Thanks. On further digging i also see that sarogata skin has function for monthly stats across years. Looks like something i can steak my code from!

[Praveen Chandrasekaran]

Though i agree that xtypes will achieve this much easier instead of having to add new search lists etc. 

[Praveen Chandrasekaran]

I assume if go the Saratoga way of SLE for monthly stats all monthly stats get computed every archive interval irrespective of whether reports are generated every archive interval. That isn't good if so.

Again xtypes seems better as the db fetches are done only on demand there as i understand 

[Praveen Chandrasekaran]

Also the xtypes for monthly stats needs some more tweaks to ensure partial months aren't used for average computation and such.

Gary's SLE in saratoga extensions already takes care of them :)

Adding all that to xtypes seems beyond my noob skill level.

Gary,

How many seconds does the SLE for monthly stats take every record? Is it a big drag? If so any way to run it more infrequently?

Also do you plan to replace it with xtypes at some point?

[Tom Keffer]

Nothing wrong with a search-list extension. The chief advantage of an xtypes extension is that a new aggregation type becomes a first-class citizen of the tag system. Indeed, all the existing aggregations are implemented using xtypes. 

$month.outTemp.historical_month_high

vs something like

$historical_high(outTemp, 'month')

[Praveen Chandrasekaran]

After struggling a bit through SLE today I hav decided to go down the route of xtypes. Is there documentation for option_dict that you mentioned other day?

[Tom Keffer]

Last night, I did put a very brief mention of option_dict in the wiki article. https://github.com/weewx/weewx/wiki/WeeWX-V4-user-defined-types

[Tom Keffer]

Praveen,

On thinking about your problem a little more, I now understand the issue with needing 12 different templates. However, there may be a solution using the #set global command, and then using #include.

Briefly, it would look something like this:

#for i in range(1,13

#  set global month=$i

#  include "Climatology.txt"

#end for

Then the file "Climatology.txt" would be very similar to what you have, except it would be parameterized by the variable $month.

[Praveen Chandrasekaran]

This would be part of template itself?

[Praveen Chandrasekaran]

I have the xtypes for monthly historical running locally. The one for rain needs a bit more work as there is a sum+average involved to find monthly total average. Will generate a pull request on git hub this weekend with it if that's fine with you.