Memory leak with ndb.put_multi

51 views
Skip to first unread message

Attila-Mihaly Balazs

unread,
Jan 6, 2017, 2:00:48 AM1/6/17
to Google App Engine
I posted a bug with this already (including a sample application to reproduce it) here: https://code.google.com/p/googleappengine/issues/detail?id=13485

I'm just re-posting it here so that people are aware:

It seems that ndb.put_multi has a serious memory leak when using large (1000) batches of heterogeneous entries which makes it unsuitable for batch-load tasks (ie. I want to load 1m entries in the datastore). The workarounds I found are:

- only use it with homogeneous entities
- use smaller batch size (100)

Hope that this gets resolved soon.

Jordan (Cloud Platform Support)

unread,
Jan 6, 2017, 10:08:25 AM1/6/17
to Google App Engine
Thank you for reporting this issue. All further progress will occur within the PIT
Reply all
Reply to author
Forward
0 new messages