memory leak?

[Rick Ree]

Hi,

If I create a new app and put a single function in default.py:

def f():

    return dict(a=list(range(100000)))

and then repeatedly call this using wget, e.g. wget http://localhost:8000/default/f.html, then memory usage of web2py will creep higher with each call. Is this a memory leak, or will the memory eventually be reclaimed? Is there something I can do in f() to avoid this?

thanks,

-Rick

[Derek]

question 1 - sure, the memory should be reclaimed eventually. You could tell the gc to collect immediately, but it operates on it's own time. Even the .NET VM will hold on to memory for longer than it should if there's nothing that needs that memory. You just call gc.collect() to have it collect immediately.

question 2 - yea, don't do that :) try 'xrange(100000)' which is more memory efficient. 

[Rick Ree]

On Tuesday, April 1, 2014 3:55:43 PM UTC-5, Derek wrote:

question 1 - sure, the memory should be reclaimed eventually. You could tell the gc to collect immediately, but it operates on it's own time. Even the .NET VM will hold on to memory for longer than it should if there's nothing that needs that memory. You just call gc.collect() to have it collect immediately.

The problem is that in the real app, gc never occurs before memory usage gets so huge that the os (Ubuntu 12.04 server) kills web2py.

 

question 2 - yea, don't do that :) try 'xrange(100000)' which is more memory efficient. 

I don't see how that will help if the full list needs to be rendered in a view.

Basically my problem is that I have a controller function that constructs a dictionary from some db queries that gets serialized in a json view. Repeated calls to this function leak memory, and I'm trying to figure out if this is a bug in web2py. The example I gave is the simplest case that shows the problem.

[Massimo Di Pierro]

This should not be happening. What OS? What wev2py version?

[Rick Ree]

Ubuntu 12.04 and 13.10. I'm running web2py from the git repo, but observed this in 2.8.2 as well.

[Paolo Valleri]

I run nearly the same example:

def f():

    a=list(range(100000))

    return 'ok'

After about 200 downloads the web2py process grew up to 100M. Tested on a system with 8Gb ram, ubuntu 12.04

paolo

[Richard Ree]

Yes, the leak seems to be associated with rendering data in a view.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/_cB4Ibm5ilc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rick Ree]

So, I'm pretty sure I was mistaken about this being a memory leak. My app had a bug in which a SELECT form field (options widget) was being populated by hundreds of thousands of items from the db. This caused memory use to exceed server limits when the view was rendered multiple times in rapid succession. In the absence of this, garbage collection does seem to proceed normally. Sorry for the confusion.

To unsubscribe from this group and all its topics, send an email to web2py+unsubscribe@googlegroups.com.

[Massimo Di Pierro]

Better a false positive than an un-reported bug. :-)