Cydia pomonella
157 views
Skip to first unread message
Chuck Rhode
Jun 22, 2020, 1:37:14 PM6/22/20
to weewx...@googlegroups.com
A couple of years ago, I developed growing-degree-day graphs to display
codling moth development using *weeWX*.
o http://lacusveris.com/cydia/index.shtml
I've been lurking on the weewx...@googlegroups.com mailing list
since then. Now, I'm finally ready to upgrade to *weeWX 4*, but I
have some embroidery on the *python 2* code that I need to migrate to
*python 3*. The magnitude of this task causes me to rethink my
two-year-old approach.
I cloned weeWX's *ImageGenerator* to access its own database, do its
own growing-degree-day calculations, and generate its own Python
Imaging Lib (PIL) images. This was not easy, but it was good enough
... then.
Today, I'm bedeviled by notions that perhaps a more general approach
would be nearly as easy (or hard) to achieve and might find utility
for other *weeWX* applications. I guess you'd say I've fallen in love
with *cheetah* and have taken to looking at all problems as templating
applications. (Full disclosure: I seem to recall that, in a previous
life, I was a SPERRY (Univac/UNiSYS) *SKEL* programmer.)
... so I ask this somewhat rhetorical question:
Has there been any discussion about templating image generation
through *cheetah*?
I don't recall seeing any such discussion although I've seen some talk
about others' frustrations with the limitations of *ImageGenerator*
that mirror my own. Apparently the limitation of not being able to
draw y-axis horizons has been lifted in *weeWX 4*, but limitations
such as not being able to apply a second y-axis have not. To be sure,
these limitations do not wreck the whole notion of *weeWX* or prevent
users' workarounds like mine. On the other hand, they are examples
only and surely don't exhaust the kinds of drawings that
*ImageGenerator* in particular and PIL in general are NOT suited to.
In my mind templating is for text only, so I'm mystified by what I'm
asking. I can't feature what an imaging template would look like.
On the other hand, I'm attracted to the notion of providing the
growing-degree-day calculations through *get_extension_list* for use
by *cheetah* in templates for ... generating images.
o Perhaps images could be conjured in real-time using something like
Google Chart's Visualization JavaScript APIs. Embedding the
JavaScript (and data) in *cheetah* templates should be
straightforward. This relies on (an indeterminate amount of) client
horsepower being available to conjure the images. Also, the host
site will wish to serve the (huge burden of) JavaScript on every
access to avoid the appearance of cross-site scripting if possible.
And it makes any such application dependent on a big, fickle outside
software-development organization, which will be forever totally
oblivious of the needs of the *weeWX* community among others.
o Perhaps images could be conjured ahead of time as they are now using
a *python* script, but the script itself would be a *cheetah*
template. This would allow specially calculated data to be
embedded. Images could then be uploaded and served to all comers
without further modification until the next generation cycle with
great savings of client-side and server-side resources.
I'm not talking here about a generalized image-scripting language,
but a *python* module that imports some handy special-purpose
classes to make *weeWX* image generation fairly terse. Of course,
images are deceptively simple things that hide excruciatingly
detailed specs. Templating images will not be easy. It will not
replace *ImageGenerator*, nor will it ever be for the
faint-of-heart.
I think generating a *python* image-generation script from *cheetah*
has the advantage, though, of abstracting image generation from the
database structure, reporting cycles, and *weeWX* implementation
details. It would promote *weeWX* extensibility. Also, it opens
the possibility of using plotting packages other than PIL. My
inclination is to use *matplotlib* instead. Would those who have
used both recommend something else?
> If you can looke into the Seedes of Time,
> And say, which Graine will grow, and which will not,
> Speake then to me, who neyther begge, nor feare
> Your fauors, nor your hate.
I'd like my development not to conflict with previous or future
*weeWX* development. Are there things that have been tried and
abandoned that would impact image templating? Are there things in
the works that would conflict with using *matplotlib*? Are there
better options for image templating?
codling moth development using *weeWX*.
o http://lacusveris.com/cydia/index.shtml
I've been lurking on the weewx...@googlegroups.com mailing list
since then. Now, I'm finally ready to upgrade to *weeWX 4*, but I
have some embroidery on the *python 2* code that I need to migrate to
*python 3*. The magnitude of this task causes me to rethink my
two-year-old approach.
I cloned weeWX's *ImageGenerator* to access its own database, do its
own growing-degree-day calculations, and generate its own Python
Imaging Lib (PIL) images. This was not easy, but it was good enough
... then.
Today, I'm bedeviled by notions that perhaps a more general approach
would be nearly as easy (or hard) to achieve and might find utility
for other *weeWX* applications. I guess you'd say I've fallen in love
with *cheetah* and have taken to looking at all problems as templating
applications. (Full disclosure: I seem to recall that, in a previous
life, I was a SPERRY (Univac/UNiSYS) *SKEL* programmer.)
... so I ask this somewhat rhetorical question:
Has there been any discussion about templating image generation
through *cheetah*?
I don't recall seeing any such discussion although I've seen some talk
about others' frustrations with the limitations of *ImageGenerator*
that mirror my own. Apparently the limitation of not being able to
draw y-axis horizons has been lifted in *weeWX 4*, but limitations
such as not being able to apply a second y-axis have not. To be sure,
these limitations do not wreck the whole notion of *weeWX* or prevent
users' workarounds like mine. On the other hand, they are examples
only and surely don't exhaust the kinds of drawings that
*ImageGenerator* in particular and PIL in general are NOT suited to.
In my mind templating is for text only, so I'm mystified by what I'm
asking. I can't feature what an imaging template would look like.
On the other hand, I'm attracted to the notion of providing the
growing-degree-day calculations through *get_extension_list* for use
by *cheetah* in templates for ... generating images.
o Perhaps images could be conjured in real-time using something like
Google Chart's Visualization JavaScript APIs. Embedding the
JavaScript (and data) in *cheetah* templates should be
straightforward. This relies on (an indeterminate amount of) client
horsepower being available to conjure the images. Also, the host
site will wish to serve the (huge burden of) JavaScript on every
access to avoid the appearance of cross-site scripting if possible.
And it makes any such application dependent on a big, fickle outside
software-development organization, which will be forever totally
oblivious of the needs of the *weeWX* community among others.
o Perhaps images could be conjured ahead of time as they are now using
a *python* script, but the script itself would be a *cheetah*
template. This would allow specially calculated data to be
embedded. Images could then be uploaded and served to all comers
without further modification until the next generation cycle with
great savings of client-side and server-side resources.
I'm not talking here about a generalized image-scripting language,
but a *python* module that imports some handy special-purpose
classes to make *weeWX* image generation fairly terse. Of course,
images are deceptively simple things that hide excruciatingly
detailed specs. Templating images will not be easy. It will not
replace *ImageGenerator*, nor will it ever be for the
faint-of-heart.
I think generating a *python* image-generation script from *cheetah*
has the advantage, though, of abstracting image generation from the
database structure, reporting cycles, and *weeWX* implementation
details. It would promote *weeWX* extensibility. Also, it opens
the possibility of using plotting packages other than PIL. My
inclination is to use *matplotlib* instead. Would those who have
used both recommend something else?
> If you can looke into the Seedes of Time,
> And say, which Graine will grow, and which will not,
> Speake then to me, who neyther begge, nor feare
> Your fauors, nor your hate.
I'd like my development not to conflict with previous or future
*weeWX* development. Are there things that have been tried and
abandoned that would impact image templating? Are there things in
the works that would conflict with using *matplotlib*? Are there
better options for image templating?
Tom Keffer
Jun 22, 2020, 9:40:02 PM6/22/20
to weewx-user
Thank you for your thoughtful letter!
Two things come to mind:
First, "growing degree days" is now a first-class type, analogous to heating- and cooling-degree days.
Second, see issue #341, which discusses support for generating JSON series using the Cheetah generator. The intention was to use these series to drive things like Highchart plots. Nothing much has happened to the issue recently (it's nearly 2 years old), but perhaps this is what you had in mind?
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/20200622123641.08a30e2c%40quixote.LacusVeris.com.
Chuck Rhode
Jun 23, 2020, 1:40:57 PM6/23/20
to weewx...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 22 Jun 2020 18:39:28 -0700
Tom Keffer <tke...@gmail.com> wrote:
> First, "growing degree days
> <http://www.weewx.com/docs/customizing.htm#[[DegreeDays]]>" is now a
identical code to cooling-degree days. This is the max-min
calculation used in crop science.
What I refer to as growing-degree-days is a wide family of
insect-development models. These depend on temperature but, like
max-min, are non-linear at the threshold. Whether they become
independent of temperature above the cutoff or actually decline is up
to the modeler's interpretation of the behavior of his favorite insect
(sub)species, and he has various mathematical techniques with which to
work his will.
I hope to be able to provide *python* functions for
growing-degree-days on daily max/min temps, threshold, and cutoff that
will exhaust the research literature.
Here is why I regard growing-degree-days models superior to max-min.
For me in Sheboygan:
The growing degree-days (GDD) calculated by Baskerville and Emin's
single sine horizontal cutoff method outruns the conventional
degree-days (DD) calculated by the max-min method by about a week.
o http://lacusveris.com/cydia/models.shtml#x83
> Second, see issue #341 <https://github.com/weewx/weewx/issues/341>,
widths, of course) would be nifty. This would make *html*-embedded
*javascript* code very terse.
I'm under the impression the absence of the patch is not a stopper
because I've been generating *.csv files from *cheetah* all along ...
with *for* loops, of course.
I'm also under the impression that *cheetah* can generate *python*
code from a template. I'd hope to keep all the specs for an image or
a suite of images together in one *python* module including the data,
cutoff dates, curve names, colors, axis labels, titles, etc, etc, etc.
It's not something you'd do with a compiled language, but interpreted
languages almost beg to be abused in this way.
I haven't tried any of this, yet. I reserve the right to change my
mind.
- --
.. Be Seeing You,
.. Chuck Rhode, Sheboygan, WI, USA
.. Weather: http://LacusVeris.com/WX
..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAl7yPqAACgkQYNv8YqSjllLjDgCdHq+dKB0k8gLDjZU5dMJHaJlF
dKQAni3fbSe33utKBToZ6ojJtFA6xcmd
=iQZf
-----END PGP SIGNATURE-----
Hash: SHA1
On Mon, 22 Jun 2020 18:39:28 -0700
Tom Keffer <tke...@gmail.com> wrote:
> First, "growing degree days
> <http://www.weewx.com/docs/customizing.htm#[[DegreeDays]]>" is now a
> first-class type, analogous to heating- and cooling-degree days.
Thanks for your reply. Yes, but it's more than analogous; it's the
identical code to cooling-degree days. This is the max-min
calculation used in crop science.
What I refer to as growing-degree-days is a wide family of
insect-development models. These depend on temperature but, like
max-min, are non-linear at the threshold. Whether they become
independent of temperature above the cutoff or actually decline is up
to the modeler's interpretation of the behavior of his favorite insect
(sub)species, and he has various mathematical techniques with which to
work his will.
I hope to be able to provide *python* functions for
growing-degree-days on daily max/min temps, threshold, and cutoff that
will exhaust the research literature.
Here is why I regard growing-degree-days models superior to max-min.
For me in Sheboygan:
The growing degree-days (GDD) calculated by Baskerville and Emin's
single sine horizontal cutoff method outruns the conventional
degree-days (DD) calculated by the max-min method by about a week.
o http://lacusveris.com/cydia/models.shtml#x83
> Second, see issue #341 <https://github.com/weewx/weewx/issues/341>,
> which discusses support for generating JSON series using the Cheetah
> generator. The intention was to use these series to drive things like
> Highchart plots. Nothing much has happened to the issue recently
> (it's nearly 2 years old), but perhaps this is what you had in mind?
Yes, *json* or *csv*-formatted strings (wrapped to sensible margin
> generator. The intention was to use these series to drive things like
> Highchart plots. Nothing much has happened to the issue recently
> (it's nearly 2 years old), but perhaps this is what you had in mind?
widths, of course) would be nifty. This would make *html*-embedded
*javascript* code very terse.
I'm under the impression the absence of the patch is not a stopper
because I've been generating *.csv files from *cheetah* all along ...
with *for* loops, of course.
I'm also under the impression that *cheetah* can generate *python*
code from a template. I'd hope to keep all the specs for an image or
a suite of images together in one *python* module including the data,
cutoff dates, curve names, colors, axis labels, titles, etc, etc, etc.
It's not something you'd do with a compiled language, but interpreted
languages almost beg to be abused in this way.
I haven't tried any of this, yet. I reserve the right to change my
mind.
- --
.. Be Seeing You,
.. Chuck Rhode, Sheboygan, WI, USA
.. Weather: http://LacusVeris.com/WX
..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAl7yPqAACgkQYNv8YqSjllLjDgCdHq+dKB0k8gLDjZU5dMJHaJlF
dKQAni3fbSe33utKBToZ6ojJtFA6xcmd
=iQZf
-----END PGP SIGNATURE-----
Tom Keffer
Jun 23, 2020, 1:47:28 PM6/23/20
to weewx-user
Cool.
I don't see why the new xtypes facility can't be used to calculate your version of growing degree days. It allows you to define new types. Take a look at it and see if it helps.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/20200623124048.5f5ff7e9%40quixote.LacusVeris.com.
Chuck Rhode
Jun 25, 2020, 7:34:59 AM6/25/20
to weewx...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hash: SHA1
On Tue, 23 Jun 2020 10:46:56 -0700
Tom Keffer <tke...@gmail.com> wrote:
> I don't see why the new xtypes facility
> <https://github.com/weewx/weewx/wiki/WeeWX-V4-user-defined-types>
Tom Keffer <tke...@gmail.com> wrote:
> I don't see why the new xtypes facility
> can't be used to calculate your version of growing degree days. It
> allows you to define new types.
I'll pursue this. Thanks.
> allows you to define new types.
.. 60° — Wind W 3 mph
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
ab4AnizLC2ju1cahhgnZ+7qVy4Oomzoe
=X3b9
-----END PGP SIGNATURE-----
Chuck Rhode
Dec 17, 2020, 12:14:00 PM12/17/20
to weewx...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hash: SHA1
On Tue, 23 Jun 2020 10:46:56 -0700
Tom Keffer <tke...@gmail.com> wrote:
> I don't see why the new xtypes facility can't be used to calculate
> your version of growing degree days. It allows you to define new
> types. Take a look at it and see if it helps.
It has taken me awhile to catch up with your suggestion. I see where
Tom Keffer <tke...@gmail.com> wrote:
> I don't see why the new xtypes facility can't be used to calculate
> your version of growing degree days. It allows you to define new
> types. Take a look at it and see if it helps.
XType.get_scalar is called in tags.CurrentObj, and I see where
XType.get_aggregate is called in tags.AggTypeBinder, but I don't see
where XType.get_series is ever called. Do you have examples of
configs or templates that mention "observation types" evaluated with
*get_series*?
.. 30° — Wind N 5 mph — Sky overcast. Light snow.
-----BEGIN PGP SIGNATURE-----
iF0EARECAB0WIQT+MY/5I/LMPSswTbVg2/xipKOWUgUCX9uRzgAKCRBg2/xipKOW
UpySAJ9NXSY4Q6DcUH3geT32Pl70R7ED9QCfQTi9wZVnfApucReVJMOdJDnB0Z4=
=MKSI
-----END PGP SIGNATURE-----
Tom Keffer
Dec 17, 2020, 3:27:28 PM12/17/20
to weewx-user
get_series() is used by the image generator, not the Cheetah generator. It's used to create plot lines. See line 176 in imagegenerator.py
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/20201217111350.7db804b9%40wealthy.
Chuck Rhode
Dec 17, 2020, 4:22:15 PM12/17/20
to weewx...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hash: SHA1
On Thu, 17 Dec 2020 12:26:56 -0800
Tom Keffer <tke...@gmail.com> wrote:
> get_series() is used by the image generator, not the Cheetah
> generator. It's used to create plot lines. See line 176 in
> imagegenerator.py
> <https://github.com/weewx/weewx/blob/master/bin/weewx/imagegenerator.py#L176>
Tom Keffer <tke...@gmail.com> wrote:
> get_series() is used by the image generator, not the Cheetah
> generator. It's used to create plot lines. See line 176 in
> imagegenerator.py
I see. Thanks.
.. 31° — Wind Calm — Sky overcast.
-----BEGIN PGP SIGNATURE-----
iF0EARECAB0WIQT+MY/5I/LMPSswTbVg2/xipKOWUgUCX9vL+wAKCRBg2/xipKOW
UmjzAJ9mhx1M/86VP2KfIK7rfmeK5DyQ8wCeL8CNtj9vg0UvG+oDpcYqmE0SjJ4=
=5Rub
-----END PGP SIGNATURE-----
Andy
Dec 23, 2020, 10:34:40 AM12/23/20
to weewx-user
I will be glad to test this when you get it going.
Andy
Chuck Rhode
Dec 23, 2020, 7:41:28 PM12/23/20
to weewx...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hash: SHA1
On Wed, 23 Dec 2020 07:34:40 -0800 (PST)
Andy <rockl...@gmail.com> wrote:
> I will be glad to test this when you get it going.
Actually, as of yesterday, it *is* going. I have a Cheetah template
Andy <rockl...@gmail.com> wrote:
> I will be glad to test this when you get it going.
that prints reports for selected insect-development models. I just
haven't packaged it as a WeeWX extension, yet. I'm working on
cumulative growing degree day graphs. I expect this part of the
effort to extend through Ground Hog Day before I can turn to package
details. The Good Lord willin' and the creek don't rise, the
extension should be ready for the next growing season.
.. 44° — Wind SW 26 mph — Sky mostly clear.
-----BEGIN PGP SIGNATURE-----
iF0EARECAB0WIQT+MY/5I/LMPSswTbVg2/xipKOWUgUCX+PjrgAKCRBg2/xipKOW
UmS0AJ4veo/htou5GfY3tjNywOtng91caACdGzkX+K2de8+EXVEh3xD0YCEOagM=
=E2Eq
-----END PGP SIGNATURE-----
Reply all
Reply to author
Forward
0 new messages