Optimizing memory

[Sensei]

Hi!

I wrote a parser for PDB files that calculates atom and residue
inertia matrices, and I am sure I have written the worst code ever :)
I'm new to Mathematica, so I don't think I know how to optimize my
code. One particular concern is this.

Using Module[] and defining many local symbols, is the memory
associated to them deallocated when the call exits? Or is it possible
to force memory deallocation?

myParser[filename_]:=Module[
{ (* many symbols *) },
symbol1 = ...;
symbol2 = ...;
(* very BIG memory allocation *)
...
]

Thanks for any information!

--
Sensei <sens...@mac.com>

The optimist thinks this is the best of all possible worlds.
The pessimist fears it is true. [J. Robert Oppenheimer]

[Peter Pein]

Sensei schrieb:

Consider the following example:
In[1]:=
HilbertMemoryEater[foo_] := Module[{bar = Array[1/(#1 + #2 - 1), {foo, foo}]},
Print[MemoryInUse[]]]
In[2]:=
MemoryInUse[]
HilbertMemoryEater[3333]
MemoryInUse[]
Share[]
MemoryInUse[]
Out[2]= 2408424
From In[2]:= 525622432
Out[4]= 2425616
Out[5]= 123296
Out[6]= 2303064

Out[2] shows the memory used by the freshly started system (I load some packages via init.m).
"From In[2]" shows the memory used by the system plus the huge amount used by the local variable bar.
Out[4] shows that the system uses slightly more memory after the execution of HilbertMemoryEater[] than before.
Out[5] shows the amount of memory, which has been freed by the forced garbage collection (gc)
and finally Out[5] shows that the gc led to less memory consumption than the freshly started system used.

This indicates that memory used by local variables is freed as soon as the variable isn't used any more.

Hope this helps,
Peter

[Christopher Arthur]

There is a function called "Share" for which the intention seems to be
garbage collection, but I have had mixed results using it. Otherwise, you
can call Names["symbol*"] to get a list of all the defined symbols that
start with "symbol". This will tell you what you want since local variables
are identified uniquely from one instance to another by a suffixed module
number; e.g. if your variable is called "myVariable", then when the module
runs it will get a name like "myVariable$2341" where 2341 identifies the
unique instance. If you call Names after your modules execute and you still
see a lot of dollar signs and numbers, then you may have a problem.

[Maxim]

The basic idea is that local Module variables are discarded if no
references to them exist when Module quits:

In[1]:= Module[{x = 1}, x++]; Names["x$*"]

Out[1]= {}

In[2]:= Module[{x = 1}, f[] := x++]; Names["x$*"]

Out[2]= {"x$39"}

In[3]:= Array[f[]&, 5]

Out[3]= {1, 2, 3, 4, 5}

There is one complication, though, which is related to assignments to
parts of expressions:

In[4]:= $Version

Out[4]= "5.2 for Microsoft Windows (June 20, 2005)"

In[5]:= Table[
Module[{L = Array[0&, 32764]},
L[[#]]++& /@ Range[32764]];
Names["L$*"],
{2}]

Out[5]= {{"L$40"}, {"L$51"}}

In[6]:= Table[
Module[{L = Array[0&, 32765]},
L[[#]]++& /@ Range[32765]];
Names["L$*"],
{2}]

Out[6]= {{"L$61"}, {"L$61", "L$71"}}

We can see that in both cases the local variable still exists after the
Module terminates. In the first case we can say that it gets overwritten
on the subsequent calls to Module. In the second example, however, the
memory taken by the list L is never freed. This is also confirmed by
MaxMemoryUsed: the amount of allocated memory will remain almost constant
after each subsequent iteration in the first case and increase linearly in
the second case. This definitely seems to be a problem with part
assignments.

Maxim Rytin
m...@inbox.ru