Development Server Performance Issue

Richard Arrano

unread,

Jul 29, 2011, 6:13:01 AM7/29/11

to Google App Engine

Hello,
I've noticed that ever since I installed 1.5.2, the performance of my
development server has degraded terribly. It used to use ~250 MB of
memory and now, without any major changes to my application, it
consistently uses ~600-800 MB. Writing to the local datastore has now
become incredibly slow; writing ~100 entries now takes ~3-5 minutes.
Any idea why this might be? Some flag I need to set? Any help is much
appreciated it, it's majorly hampering my development.

Thanks,
Richard

Ikai Lan (Google)

unread,

Jul 29, 2011, 4:40:23 PM7/29/11

to google-a...@googlegroups.com

Is this the Python or Java dev server? Has anyone else experienced similar issues?

--

Ikai Lan
Developer Programs Engineer, Google App Engine

plus.ikailan.com | twitter.com/ikai

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Richard Arrano

unread,

Jul 30, 2011, 2:37:11 AM7/30/11

to Google App Engine

It was on the Python dev server. I should refine my statement and say
that previously, it would sometimes hit 400 MB of memory used but now
it starts at 600 and usually balloons to 800+. The write times are
horrendous for db.put() and db.delete().

On Jul 29, 1:40 pm, "Ikai Lan (Google)" <ika...@google.com> wrote:
> Is this the Python or Java dev server? Has anyone else experienced similar
> issues?
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
> plus.ikailan.com | twitter.com/ikai
>

Tim Hoffman

unread,

Jul 30, 2011, 5:21:27 AM7/30/11

to google-a...@googlegroups.com

Are you using sqlite backend. Maybe you aren't and your datastore size is growing and the default datastore performs terribly when it gets big.

Rgds

T

Richard Arrano

unread,

Jul 31, 2011, 7:30:47 PM7/31/11

to Google App Engine

I haven't changed anything in ages, so if it used to be default then
yes. How can I change this?

-Richard

Tim Hoffman

unread,

Jul 31, 2011, 7:52:32 PM7/31/11

to google-a...@googlegroups.com

--usesqlite

If you datastore gradually grows and you keep adding to it then it will get really slow.
The 1.5.2 unless you use --default_partition argument you end up with a different namespace so
the datastore may appear empty (you can't find any data) so you chuck some more data in and
boom your datastore is twice as big as it was before.

Just a guess mind you.

Rgds

Tim

Richard Arrano

unread,

Jul 31, 2011, 8:17:37 PM7/31/11

to Google App Engine

Thanks Tim, you nailed it. I attributed it to 1.5.2 but I realized I
created a moderate amount of data around the same time, hence the
slowdown. Using SQLite fixed it completely. Thanks!

-Richard

Ikai Lan (Google)

unread,

Aug 1, 2011, 8:44:38 PM8/1/11

to google-a...@googlegroups.com

The thing to remember here is that the datastore stub is that: a stub. The whole thing gets loaded into memory. That's how we're able to determine what indexes are required, do full datastore scans, etc.

--

Ikai Lan
Developer Programs Engineer, Google App Engine

plus.ikailan.com | twitter.com/ikai

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

Reply all

Reply to author

Forward