Index building taking over 12h

[Eric Rannaud]

I have an index that has been building for over 12h, on less than
200MB of data, with 500,000 entities. (App: 911pagers)

Is there something I should do?

Thanks.

[Nick Johnson (Google)]

Hi Eric,

Indexes are built in a queue across all apps, so the index build time has more to do with what's in the queue ahead of you than how large your data is. 12 hour build times are not uncommon.

-Nick Johnson

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

[jpmorganuk]

Hi Nick,

What can be done to speed this up?

Regards,
jpmorganuk

[Nick Johnson (Google)]

Hi,

We're working on speeding up index building times. There's nothing you can do as a user to speed up index building.

-Nick Johnson

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Brade]

This is one of those "ugly truths" about app engine, as glorious as it
is otherwise.
Perhaps there should be more emphasis on the necessity of deploying a
new index definition before adding new features to one's app?
My app www.klection.com luckily is not "mission critical" but the
aggregated comments section "What You Say" is about 12 hours along on
its "building" status, as all I added was a __key__ desc index for
comments (which there are less than 50 of right now). BTW my local dev
environment failed to add this index automagically for some odd
reason, so it took me a while even to troubleshoot the fact I needed
to add it manually (because the functionality was working fine in my
local environment).

I will be starting fairly soon on my next GAE app, and these sorts of
issues make me wary. As fantastic as it is in many respects, the
indexing problems really affect the reliability of the system. It's
nice to see the recent emphasis in the Articles section about the
Datastore, but hopefully in the near future there will be better
information reporting when deploying your app, because things that
work great on your local dev environment should not completely break
once deployed. That seems borderline unacceptable at present.

--brad g.

On Dec 3, 4:12 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> Hi,
>
> We're working on speeding up index building times. There's nothing you can
> do as a user to speed up index building.
>
> -Nick Johnson
>
> On Wed, Dec 2, 2009 at 6:35 PM, jpmorganuk

> <market...@farrellheyworth.co.uk>wrote:

>
>
>
> > Hi Nick,
>
> > What can be done to speed this up?
>
> > Regards,
> > jpmorganuk
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-appengi...@googlegroups.com<google-appengine%2Bunsu...@googlegroups.com>

[Joshua Smith]

The technique I use is to deploy it as a new version number, and wait for the indexes to finish, and then proceed to make it the default.

The indexes are shared between versions, so this "just works."

It does mean that you have to plan for a 24 hour delay in deploying a new version, but for most developers, that forced QA time is probably good discipline anyway!

-Joshua

> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[Ikai L (Google)]

Joshua,

Great tip! I'm wondering if there's a way we can better surface App Engine's versioning ability as a great way to stage a new release of an application.

For those not familiar with what Joshua is talking about: when you change the version number in your appengine-web.xml or app.yaml file and deploy your application, you can access all versions of your application at VERSION.latest.APPID.appspot.com. All versions of your application will have access to the data store, and you can cut over from a production version to a new release with the click of a button in the Admin Console.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

[Brade]

GOOD LORD MY INDEX IS STILL "BUILDING"...

On Dec 18, 10:53 am, Brade <bradez...@gmail.com> wrote:
> This is one of those "ugly truths" about app engine, as glorious as it
> is otherwise.
> Perhaps there should be more emphasis on the necessity of deploying a
> new index definition before adding new features to one's app?

> My appwww.klection.comluckily is not "mission critical" but the

[Ikai L (Google)]

Brade, your index could be stuck. What is your app ID?

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--

[Brade]

filmologue

It's still "building" today >_<
So hopefully you can help me out.

Thanks a lot!
--brad g.

On Dec 21, 4:02 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> Brade, your index could be stuck. What is your app ID?
>
>
>
> On Sun, Dec 20, 2009 at 3:46 PM, Brade <bradez...@gmail.com> wrote:
> > GOOD LORD MY INDEX IS STILL "BUILDING"...
>
> > On Dec 18, 10:53 am, Brade <bradez...@gmail.com> wrote:
> > > This is one of those "ugly truths" about app engine, as glorious as it
> > > is otherwise.
> > > Perhaps there should be more emphasis on the necessity of deploying a
> > > new index definition before adding new features to one's app?

> > > My appwww.klection.comluckilyis not "mission critical" but the

> > <google-appengine%2Bunsu...@googlegroups.com<google-appengine%252Buns...@googlegroups.com>

>
> > > > > .
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > > > --
> > > > Nick Johnson, Developer Programs Engineer, App Engine
> > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
> > Number:
> > > > 368047
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengi...@googlegroups.com<google-appengine%2Bunsu...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --