Pymples ahead

[Ludwig Hähne]

Hi guys,

as we already discussed offline, progress has somewhat stagnated
lately due to our various other commitments. In order to keep Pympler
alive as a project, I'd like to discuss in which direction we want to
go in the (near) future. So here is my opinion:

Where are we?

With asizeof, muppy and tracker (aka heapmonitor) under one banner,
Pympler has (IMHO) sufficient features to be usable and it is already
a good tool in many different memory profiling-related scenarios. I
believe the weak point of Pympler is the lack of accessibility. In
practice, memory-profiling boils down to "My program/web-application
takes too much memory. What can I do about it?". As it is, I'm not
sure Pympler is immediately helpful. In order to exploit Pympler
mechanisms (beyond asizeof), one would have to read the whole
documentation just to get a vague understanding of what module one
might need.

To make things worse, memory profiling is not well understood. One can
ignore memory bloat/leaks/waste a very long time before those issues
start to create actual problems. By that time a Python application
takes multiple gigabytes of memory and is almost impossible to profile
as-is.

Where to go from here?

A web interface might help here. Something simple, like what "import
pdb; pdb.set_trace()" is for debugging; put "pympler.gui.web.show()"
on a spot of interest and the system gives immediate feedback. My idea
is to provide some basic information, e.g. process memory footprint,
garbage overview and pointers to the docs of associated facilities in
Pympler. Please have a look at the webgui-branch of how this looks at
the moment. Dynamic content could easily be added, e.g. per-class
statistics. There is a certain risk of creating feature bloat aka the
unmaintainable mess - so I'd like to keep the webgui simple and
decoupled from the rest of the project.

Another topic ... releases:

Pympler 0.2 is long overdue. As far as the issue tracker is concerned,
the most critical features are already implemented (py3 support,
better test coverage, heapmonitor refactored). The documentation
probably needs some care, but I'd be ok to go with what we have. What
do you think?

Cheers, Ludwig

BTW, Pympler will soon have its (her,his?) first anniversary :)

[Robert Schuppenies]

Hi Ludwig.

Many thanks for the good summary of the status quo. I agree with you in many
points. So in short I say, go ahead and take Pympler to the next level, both
with the web gui and the 0.2 release. You currently have the drive to keep
Pympler going and I don't want you to be stopped by lack of time on our side.

I played around with the gui.web module a bit and I really like it. I always
wanted to do something like this and now you actually did it. I think it will
be very helpful to users as it might achieve what the muppy couldn't: a usable
interface to explore the depths of memory usage.

Also, I just had a look and the first check-in into the Google code repository
was August 26th, so here a late "Happy Birthday" to you guys :)

cheers,
robert