Out of memory

[GilRoss]

My system has 30k dedicated to the Python heap. I've written a module (along with the imported modules) that uses all this memory. Is it possible to break this one module into two modules: Mod1, Mod2, then have a 3rd module (ModMaster) import these subordinate modules one at a time such that only one is using Python heap memory at any one time? That is, import Mod1 into the heap, use it, remove Mod1 from the heap, import Mod2 into the heap, use it, remove Mod2 from the heap.

Gil

[Dean Hall]

Hmmm... if you are using r0X series, having more modules will take up more RAM.

If you are using v10 branch, then more modules will take up more Flash. It might be possible to do what you say. But there is module caching that you'll have to comment out. See src/lib/__bi.py line 944 (variable __md) and src/vm/interp.c line 1469 where the IMPORT_NAME bytecode stores a reference to the module in the __md dict.

You have to get rid of those extra reference so that after "del modulename" there are no reference to the module and the garbage collector can whisk it away.

!!Dean

On Apr 24, 2013, at 7:58 PM, GilRoss wrote:

> My system has 30k dedicated to the Python heap. I've written a module (along with the imported modules) that uses all this memory. Is it possible to break this one module into two modules: Mod1, Mod2, then have a 3rd module (ModMaster) import these subordinate modules one at a time such that only one is using Python heap memory at any one time? That is, import Mod1 into the heap, use it, remove Mod1 from the heap, import Mod2 into the heap, use it, remove Mod2 from the heap.
>
> Gil
>

> --
> --
> You are subscribed to the "python-on-a-chip" (or p14p for short) Google Group.
> Site: http://groups.google.com/group/python-on-a-chip
>
> ---
> You received this message because you are subscribed to the Google Groups "python-on-a-chip" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to python-on-a-ch...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[pir...@gmail.com]

It would be cool that there's a config flag for this things... :-)

[Dean Hall]

Could do a flag in C, but it's hard to make a flag in Python because Python doesn't have a way to conditionally compile code. (e.g. no #ifdef). If you see a way, let me know.

!!Dean

[pir...@gmail.com]

I was thinking about pmConfigure.py (It's called this way, isn't it?)

2013/4/28 Dean Hall <dwha...@gmail.com>:

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

[Dean Hall]

Yes, very close. pmfeatures.py

You just made me think of this::

import pmfeatures.py

if PM_FEATURES['HAVE_MODULE_CACHE']:
global __md
__md = {}

Which would get the job done, but it would add the PM_FEATURES dict to the builtins namespace... unless I deleted it at the end of __bi.py

Hmmm......

!!Dean

[pir...@gmail.com]

> Yes, very close. pmfeatures.py
>
Oh, yes, pmfeatures.py and pmConfigure.py are very similar... <ironic/>

:-P

> You just made me think of this::
>
> import pmfeatures.py
>
> if PM_FEATURES['HAVE_MODULE_CACHE']:
> global __md
> __md = {}
>

Why Python? I though this was a C problem, that's the reason I talked
about pmfeatures.py and compile time...

> Which would get the job done, but it would add the PM_FEATURES dict to the builtins namespace... unless I deleted it at the end of __bi.py
>
> Hmmm......
>

Sweets nightmares tonight... ;-)

[Dean Hall]

In the VM there are usually two parts to every problem/solution.
For the case of module caching, there's the actions in the C part of the VM:
when the IMPORT_NAME bytecode is executed, the module is put in the cache.
However, the cache itself is declared in the builtins namespace as variable __md (short for module dict).

!!Dean