--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/807eae4f-404c-48bc-9e27-43de1ac73c07n%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEAxQaySc-b%3D2aH0dwDadK3A%2BkuKGvovidVsi7%3DYK8FdAw%40mail.gmail.com.
Re: editing the Belchertown skin, nope haven’t touched it, other than interfacing with it via the include files (as generated by my BelchertownWxFeeds extension). When all the endpoints (for testing) are enabled index.html is about 1.8MB, so perhaps that’s causing the problem. I can easily reduce / eliminate endpoints and prefer to do that before eliminating the Belchertown skin.
On Dec 31, 2020, at 11:44, vince <vince...@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/57a5bfc7-d0b4-4419-b178-6342564642edn%40googlegroups.com.
On Jan 7, 2021, at 11:07, vince <vince...@gmail.com> wrote:
Are you closing the file every time after you write it ?
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/228a5a48-1aa1-490e-826d-e01e6bfb2da1n%40googlegroups.com.
Garry,
I took your WxFeedsMemory.py and made some modifications.
First, I changed the logging to simple print calls. this was just a convenience for me to easily see what was going on.
Second, I stole some code from Vince’s mem extension, https://github.com/vinceskahan/vds-weewx-v3-mem-extension. I did this because I am used to the data it returns. (Note, I highly recommend this when debugging memory.)
Third, I added code so I could call your extension repeatedly in a loop.
When I called new_archive_record 100 times there was a slight increase in memory, but nothing alarming. But it was slow, approximately 2 minutes a call.
So, I commented out the flush and fsync calls and called new_archive_record 10,000 times. The slight increase seemed to stop around 6,000 calls.
I’m wondering if the amount of time it takes to run is causing problems when run as a service. I plan to uncomment the code and run it as an extension. I’ll let you know what I see.
rich
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d5e41e91-5dab-41bc-bc34-682826ba4753n%40googlegroups.com.
On Jan 9, 2021, at 19:11, vince <vince...@gmail.com> wrote:
If your memory usage is stable, is there a problem ?
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/26d2e298-1f5a-4d54-8cc8-89acaf65dab3n%40googlegroups.com.
On Jan 16, 2021, at 11:09, bell...@gmail.com <bell...@gmail.com> wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0f3d3ae7-ad42-4ebb-a395-81ddc2887496n%40googlegroups.com.
My experience is that when I recreate an include file for Belchertown on each archive cycle, as an extension to Belchertown or as a service, memory usage increases with each cycle and eventually errors start.
RPi 4 - 8 GBUbuntu Raspberry Pi OS 20.10Python3 - 3.8.6WeeWX 4.3Belchertown 1.2- Problem was first noticed on RPi Zero W with Raspberry Pi OS Lite.- Problem was reproduced on RPi 4 - 4 & 8 GB with Raspberry Pi OS Desktop.- My extension was reduced to minimum code needed to reproduce the problem - that code is attached.
On Jan 17, 2021, at 13:11, vince <vince...@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/44242da6-de69-4f68-a0b2-963689c6f46an%40googlegroups.com.
On Jan 17, 2021, at 16:06, vince <vince...@gmail.com> wrote:
garry - I installed your thing and did not see a memory increase 'without' belchertown running, and but it 'is' leaking' it seems 'with' belchertown 1.2 installed. I'm running the simulator and ubuntu-2004 in a Vagrant VM running python3 weewx 4.3.0 via setup.py - Here's some raw numbers with a little commentary for when I did what.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/14a90da6-c84c-40ab-9a37-a85ae5aeddcan%40googlegroups.com.
Syntax:
The #include directive is used to include text from outside the template definition. The text can come from an external file or from a $placeholder variable. When working with external files, Cheetah will monitor for changes to the included file and update as necessary.
This example demonstrates its use with external files:
And this example demonstrates use with $placeholder variables:
By default, included text will be parsed for Cheetah tags. The argument ``raw'' can be used to suppress the parsing.
Cheetah wraps each chunk of #include text inside a nested Template object. Each nested template has a copy of the main template's searchList. However, #set variables are visible across includes only if the defined using the #set global keyword.
All directives must be balanced in the include file. That is, if you start a #for or #if block inside the include, you must end it in the same include. (This is unlike PHP, which allows unbalanced constructs in include files.)"
I decided to insert the 'raw' argument into the 4 '#include' directives to see if anything changed.
After about 1 hour. memory usage did not grow above about 65 MB!
I just removed the 'raw' argument and restarted WeeWX. After just a few minutes, memory usage is almost 300 MB and growing!
As before, the station website is at: https://lockyer.ca/weather/OsoyoosLakeNorthEast and the mem graphs are at https://lockyer.ca/weather/OsoyoosLakeNorthEast/mem.
So, I think I have conclusively shown that the problem is within Cheetah and is related to it parsing an include file.
I'm going to re-insert the 'raw' arguments and carry on.
Should I report this issue to the Cheetah folks or is there someone closer to the Cheetah developers that can better do that?
Regards,
Garry
On Jan 19, 2021, at 13:15, vince <vince...@gmail.com> wrote:
I'm thinking it's actually working as designed if you assume Pat intended that cheetah directives should be parsed within a .inc(luded) file. You just might be the edge case where you have a rather massive include that does 'not' want cheetah directive parsing.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/75c3e93e-6e99-4e60-a73d-39ff3b269f14n%40googlegroups.com.
I prefer restarting WeeWX daily over hand editing index.html.tmpl but that frequency may not work for all cases so I’ll probably live with editing.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/bfd80498-e70b-4a26-9f76-6a922439a1ben%40googlegroups.com.
On Jan 19, 2021, at 16:16, Tom Keffer <tke...@gmail.com> wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEBLSWhzeEFOyksrx64Pro2M0F2skgOCePZ0DUWSEoH7rw%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7F58A737-4F41-4C62-8C7F-A5CA7678C672%40gmail.com.
On Jan 19, 2021, at 19:38, vince <vince...@gmail.com> wrote:
Just a followup - I took belchertown out of the picture and used my minimal demo-skin and added 'one line' that #include(d) a file generated by Garry's service, and it 'is' leaking memory albeit less quickly than the belchertown example.....
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4916786b-3bb2-49a6-93b1-87a60a822dbdn%40googlegroups.com.
<Screen Shot 2021-01-19 at 7.33.46 PM.png>
On Jan 20, 2021, at 06:24, bell...@gmail.com <bell...@gmail.com> wrote:
Note, it is growing - just much slower.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/71e3dde7-0174-42d2-908e-1d6ea27b42ban%40googlegroups.com.
On Jan 20, 2021, at 06:24, bell...@gmail.com <bell...@gmail.com> wrote:
Note, it is growing - just much slower.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/71e3dde7-0174-42d2-908e-1d6ea27b42ban%40googlegroups.com.
On Jan 22, 2021, at 12:00, Pat <pobri...@gmail.com> wrote:
I'm tracking this thread a bit slowly... Is the raw portion within the #include custom to your setup - or is this something that should be added to the Belchertown skin?
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9d1ce1c1-3017-43f2-b66f-13c5637748e6n%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/772028EA-5967-47A8-A130-5864076C10DD%40gmail.com.
On Jan 22, 2021, at 17:14, Timothy L <lecoqacr...@gmail.com> wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAGW8XMQMwDDpKmfMgan0jkGtG1uOxBXg3LzW8cRFH%2BpYMd3XwA%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/04E005A6-BF4B-40A8-AE3F-0B3E80292402%40gmail.com.
In my experimental skin I was running into this issue, so I decided to delve into Cheetah’s internals and learned the following. As part of the compilation process the module is added to sys.modules and the generated code is cached. If one is dynamically generating template source and it is different, then it will not be found in the cache and an additional entry will be added to sys.modules and the cache. This will result in ever increasing memory usage. Since the OP was dynamically generating html and not Cheetah source, it could be included as ‘raw’. This skips the compilation step and the addition of it to sys.modules and the cache.
- rich
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c33494f0-1ff1-464e-b37c-df48f02dc1d3n%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f6a401d0-614d-4b7d-b44e-9f58f0870f8en%40googlegroups.com.
Nicely done, Rich. Fits the facts perfectly.
import Cheetah.Template.TemplaceCheetah.Template.Template._CHEETAH_useCompilationCache = False
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0813965d-814f-4297-ac1b-b4c40a364396n%40googlegroups.com.
That won't fix anything. Possible solutions:
import Cheetah.Template.TemplaceCheetah.Template.Template._CHEETAH_useCompilationCache = False