How to catch Out of Memory errors?

[Waleed Abdulla]

I'm getting a lot of "Out of Memory" errors, and when that happens GAE just kills the instance right away and all I see in the logs is something like this:

Exceeded soft private memory limit with 277.09 MB after servicing 1 requests total

    To find where the problem is, I put logging.info() statements in the code, but the problem is that nothing is logged when the instance is killed this way. I also tried using logservice.flush(), but that doesn't seem to help. I cannot get any data out of that instance to help me debug the problem. 

    Any tips?

Waleed

[Brandon Wirtz]

Add your own events to put in the log, and they will be…

 

Instance started,

Defined some variables

Started a loop

Looped

Looped again count 1

Looped again count 2-infinity

Finished looping

 

Wrote something to datastore

Output the HTML Life is happy

 

That Kind of stuff.

Along the way you will likely find you have  a loop until the end of time statement somewhere

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Waleed Abdulla]

That's the problem, nothing I log gets recorded. It's like when the instance is killed, everything is deleted, including any events that were logged before that point. 

[Brandon Wirtz]

Hmm… I don’t think that happens for me… Are you dying at spin up?

[André Pankraz]

Sometimes I wonder if you really use GAE too ;) Or are Java and Python this different?


All ServletResponses and Log-Events are (for me) cached (and flushed only at the end of the request).

E.g.: If you have a 1 hour backend job no log entry appears in the Admin console till the request is complete...HttpResponse.out.write/flush will return nothing either till the response is complete. You can flush all you want...nothing happens.


If you get an OutOfMem instance kill or a (for me) famous "Connection to client lost." no log entries will appear in the console, no output will be written to the servlet response stream.
This is especially funny because im sure that the Blob-creation API in Java has severe Memory leaks.


How do we debug this stuff? I miss jvisialvm or hsperf-dumps or or or...



I know Brandon...we all should only write 50 ms Proxy-Caches or we are wrong here. ;)

[Brandon Wirtz]

I don’t actually use GAE.  I troll the forums with mis-information so that everyone gets it in their head how great the service is, then when they have invested months in code that doesn’t do what they expect I buy their forclosed home.  It’s great, Google gets more business, I get cheap property. We all live happily ever after.

 

I’ll take a look at if Java does the same thing.

 

From: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] On Behalf Of André Pankraz
Sent: Tuesday, December 06, 2011 12:03 PM
To: google-a...@googlegroups.com
Subject: Re: [google-appengine] How to catch Out of Memory errors?

 

Sometimes I wonder if you really use GAE too ;) Or are Java and Python this different?

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/1ZeWxwnrKEQJ.