Building a new index is taking way too long
6 views
Skip to first unread message
Alex Popescu
Apr 27, 2009, 7:59:05 AM4/27/09
to Google App Engine
Hi,
I have defined a new index for an entity based on a datetime and 1char
string property. Currently there are only 58 such entities in the
datastore and after 1h15min it is still not ready. I do not see any
reasons why building the index for such a small amount of entities
would take such a long time, except a problem with the backend.
The application id: dailycloud.
./alex
I have defined a new index for an entity based on a datetime and 1char
string property. Currently there are only 58 such entities in the
datastore and after 1h15min it is still not ready. I do not see any
reasons why building the index for such a small amount of entities
would take such a long time, except a problem with the backend.
The application id: dailycloud.
./alex
Alex Popescu
Apr 28, 2009, 2:30:25 AM4/28/09
to Google App Engine
Is there any status update on this issue? I have noticed other posts
on the list speaking about the same problem.
./alex
On Apr 27, 2:59 pm, Alex Popescu <the.mindstorm.mailingl...@gmail.com>
wrote:
on the list speaking about the same problem.
./alex
On Apr 27, 2:59 pm, Alex Popescu <the.mindstorm.mailingl...@gmail.com>
wrote:
Jason (Google)
Apr 30, 2009, 8:02:11 PM4/30/09
to google-a...@googlegroups.com
Hi Alex. We definitely need to (and will) document this better, but index build time is not directly proportional to the number of entities in the datastore but to the number of other applications who are building indexes at the same time. Since index building is managed by a fixed number of "workers," your application's indexes will take longer to build if there are a higher number of applications building indexes. It appears that this particular issue affected a number other developers earlier in the week. We'll take a closer look at this problem and how it can be better mitigated going forward, but it should be fairly uncommon for the most part.
- Jason
- Jason
Ubaldo Huerta
May 6, 2009, 11:02:43 AM5/6/09
to Google App Engine
I'm having same issue. Index building, for a alpha app at this moment
(just 100 entities), and it's stuck building index. Sounds like one
has to factor in this lag time in the development cycle, that is,
first update index definitions, wait wait and wait, then upload the
app (well, if one uses versions, can conceivably promote the version
to be the default one when index building has finished )
An indication of how many people are ahead in the queue or better yet
(expected duration although I realize that it can be hard to
estimate), email notification when it's done, adding more "workers",
would be nice. In any case, the message that version is ready to start
serving upon upload is misleading. version might be capable of serving
http requests, but it all indexes in index.yaml aren't built you
should get a warning at least
On a separate note, vacuum_indexes more often than not, hangs. Well,
maybe it isn't hung, but it appears so, because output just says
Fetching index definitions diff, and stale indexes still show in the
console.
On May 1, 2:02 am, "Jason (Google)" <apija...@google.com> wrote:
> Hi Alex. We definitely need to (and will) document this better, but index
> build time is not directly proportional to the number of entities in the
> datastore but to the number of other applications who are building indexes
> at the same time. Since index building is managed by a fixed number of
> "workers," your application's indexes will take longer to build if there are
> a higher number of applications building indexes. It appears that this
> particular issue affected a number other developers earlier in the week.
> We'll take a closer look at this problem and how it can be better mitigated
> going forward, but it should be fairly uncommon for the most part.
>
> - Jason
>
> On Mon, Apr 27, 2009 at 11:30 PM, Alex Popescu <
>
(just 100 entities), and it's stuck building index. Sounds like one
has to factor in this lag time in the development cycle, that is,
first update index definitions, wait wait and wait, then upload the
app (well, if one uses versions, can conceivably promote the version
to be the default one when index building has finished )
An indication of how many people are ahead in the queue or better yet
(expected duration although I realize that it can be hard to
estimate), email notification when it's done, adding more "workers",
would be nice. In any case, the message that version is ready to start
serving upon upload is misleading. version might be capable of serving
http requests, but it all indexes in index.yaml aren't built you
should get a warning at least
On a separate note, vacuum_indexes more often than not, hangs. Well,
maybe it isn't hung, but it appears so, because output just says
Fetching index definitions diff, and stale indexes still show in the
console.
On May 1, 2:02 am, "Jason (Google)" <apija...@google.com> wrote:
> Hi Alex. We definitely need to (and will) document this better, but index
> build time is not directly proportional to the number of entities in the
> datastore but to the number of other applications who are building indexes
> at the same time. Since index building is managed by a fixed number of
> "workers," your application's indexes will take longer to build if there are
> a higher number of applications building indexes. It appears that this
> particular issue affected a number other developers earlier in the week.
> We'll take a closer look at this problem and how it can be better mitigated
> going forward, but it should be fairly uncommon for the most part.
>
> - Jason
>
> On Mon, Apr 27, 2009 at 11:30 PM, Alex Popescu <
>
Dejan Juric
May 22, 2009, 9:00:37 AM5/22/09
to Google App Engine
Hi,
I'm also having the issue of stuck index building. the app is
quickbookmarkcom.
The indexes have been in the "Building" phase for a couple of days
already, and i cannot remove them (vacuum_indexes gives: "indexes were
not deleted. Most likely this is because they no longer exist.")
I have since changed my app and the indexes are no longer needed.
I would highly appreciate if somebody from Google could have a look at
it and remove the indexes that are in the Building phase.
Thank you very much,
Dejan
On May 6, 5:02 pm, Ubaldo Huerta <uba...@gmail.com> wrote:
> I'm having same issue.Indexbuilding, for a alpha app at this moment
> (just 100 entities), and it's stuckbuildingindex. Sounds like one
I'm also having the issue of stuck index building. the app is
quickbookmarkcom.
The indexes have been in the "Building" phase for a couple of days
already, and i cannot remove them (vacuum_indexes gives: "indexes were
not deleted. Most likely this is because they no longer exist.")
I have since changed my app and the indexes are no longer needed.
I would highly appreciate if somebody from Google could have a look at
it and remove the indexes that are in the Building phase.
Thank you very much,
Dejan
On May 6, 5:02 pm, Ubaldo Huerta <uba...@gmail.com> wrote:
> I'm having same issue.Indexbuilding, for a alpha app at this moment
> (just 100 entities), and it's stuckbuildingindex. Sounds like one
> has to factor in this lag time in the development cycle, that is,
> first updateindexdefinitions, wait wait and wait, then upload the
> app (well, if one uses versions, can conceivably promote the version
> to be the default one whenindexbuildinghas finished )
>
> An indication of how many people are ahead in the queue or better yet
> (expected duration although I realize that it can be hard to
> estimate), email notification when it's done, adding more "workers",
> would be nice. In any case, the message that version is ready to start
> serving upon upload is misleading. version might be capable of serving
> http requests, but it all indexes inindex.yaml aren't built you
> An indication of how many people are ahead in the queue or better yet
> (expected duration although I realize that it can be hard to
> estimate), email notification when it's done, adding more "workers",
> would be nice. In any case, the message that version is ready to start
> serving upon upload is misleading. version might be capable of serving
> should get a warning at least
>
> On a separate note, vacuum_indexes more often than not, hangs. Well,
> maybe it isn't hung, but it appears so, because output just says
> Fetchingindexdefinitions diff, and stale indexes still show in the
>
> On a separate note, vacuum_indexes more often than not, hangs. Well,
> maybe it isn't hung, but it appears so, because output just says
> console.
>
> On May 1, 2:02 am, "Jason (Google)" <apija...@google.com> wrote:
>
> > Hi Alex. We definitely need to (and will) document this better, butindex
> > build time is not directly proportional to the number of entities in the
> > datastore but to the number of other applications who arebuildingindexes
> > at the same time. Sinceindexbuildingis managed by a fixed number of
>
> On May 1, 2:02 am, "Jason (Google)" <apija...@google.com> wrote:
>
> > Hi Alex. We definitely need to (and will) document this better, butindex
> > build time is not directly proportional to the number of entities in the
> > datastore but to the number of other applications who arebuildingindexes
> > "workers," your application's indexes will take longer to build if there are
> > a higher number of applicationsbuildingindexes. It appears that this
> > particular issue affected a number other developers earlier in the week.
> > We'll take a closer look at this problem and how it can be better mitigated
> > going forward, but it should be fairly uncommon for the most part.
>
> > - Jason
>
> > On Mon, Apr 27, 2009 at 11:30 PM, Alex Popescu <
>
> > the.mindstorm.mailingl...@gmail.com> wrote:
>
> > > Is there any status update on this issue? I have noticed other posts
> > > on the list speaking about the same problem.
>
> > > ./alex
>
> > > On Apr 27, 2:59 pm, Alex Popescu <the.mindstorm.mailingl...@gmail.com>
> > > wrote:
> > > > Hi,
>
> > > > I have defined a newindexfor an entity based on a datetime and 1char
> > We'll take a closer look at this problem and how it can be better mitigated
> > going forward, but it should be fairly uncommon for the most part.
>
> > - Jason
>
> > On Mon, Apr 27, 2009 at 11:30 PM, Alex Popescu <
>
> > the.mindstorm.mailingl...@gmail.com> wrote:
>
> > > Is there any status update on this issue? I have noticed other posts
> > > on the list speaking about the same problem.
>
> > > ./alex
>
> > > On Apr 27, 2:59 pm, Alex Popescu <the.mindstorm.mailingl...@gmail.com>
> > > wrote:
> > > > Hi,
>
> > > > string property. Currently there are only 58 such entities in the
> > > > datastore and after 1h15min it is still not ready. I do not see any
> > > > reasons whybuildingtheindexfor such a small amount of entities
> > > > datastore and after 1h15min it is still not ready. I do not see any
Reply all
Reply to author
Forward
0 new messages