How can i increase indexes of my application

[Prakhil Samar]

Hi

I have a application and my datastore indexes are full i have created
200 indexes.

is there any way that indexes can be increased ?

Can i increase indexes by enabling billing on my application ?

[Nicanor Babula]

I also want to know if that would be possible. Actually my app is using 186 out of 200 and I am afraid that soon more than 200 will be needed.

Is there something to buy?

[Kenneth]

I would suggest optimising the number of indexes, 200 is a lot of indexes!

Read this and see if you can reduce the number of indexes given some of the improvements last year:

http://code.google.com/appengine/articles/indexselection.html

[Nicanor Babula]

I already did that.. In fact my app doesn't have exploding indexes, because I limited the searches and the sort-able columns in the list views in order to contain the number of indexes. I am sure there is still space for optimization, but 200 is a finite number and we are adding new features on a daily basis and looking at the trend, in maximum one year, we will have reached the limit. What is to be done then? (Our app uses about 50 kinds)

Are there any other techniques on how to reduce the number of indexes?

Thanks,
Cristian.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/IUlbvakF0OwJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nicanor Cristian Babula

[Prakhil Samar]

Thanks for the link.

But i have a very vast application, and i found that day before
yesterday my indexes were full 200 of 200

i vacuumed some of them, but still the quota value shows 200 of 200,
instead of the full it should show number of indexes created

Also, after every 24 hrs. database is reset, but from two days its
still showing the quota details indexes 200 of 200 and not allowing me
to add any new indexes

can anyone provide some solution ??

[Nicanor Babula]

Prakhil,

The indexes quota doesn't reset every 24 hours.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nicanor Cristian Babula

[Andreas]

did you create those indexes manually or are these indexes created automatically?

what kind of queries do you have to make that you need 200 indexes?

[Mike Wesner]

The 200 limit made sense before the new billing scheme, but now it doesn't as much.  We pay (a lot!) for storage and the additional datastore operations that result from having more composite indexes.  This limit should probably be removed entirely.

Mike

[Nicanor Babula]

Indeed!

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/RSyf_nV1G4MJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nicanor Cristian Babula

[de Witte]

Try http://support.google.com/code/bin/request.py?contact_type=AppEngineQuotaRequest 

200 indexes for 50 entities is a lot and makes your application expensive. 

There are tricks to sort a ListView without using an index for each column.

Op woensdag 21 maart 2012 08:47:06 UTC+1 schreef prakhil samar het volgende:

[Nicanor Babula]

For example? I am sure there is space for optimizing the list views..

The only trick I know is to reduce the number of order-able columns in the list view...

For example, just yesterday I developed a statistics module which required me to add another 2 indexes for 2 queries like:

select * from Customers where accountId = 'currentAccountId' and creationDate > [from] and creationDate < [toDate] and status = 'available'

Is there a trick to avoid using indexes on queries like that?

[Prakhil Samar]

Hi GAE team,

I have billing enabled on my application id “xxxxxxx”

From last week my “Quota details” for datastore indexes are not
getting reset its showing 200 of 200.

Is there something I need to do with billing stuff OR will you please
reset the datastore indexes count.

I have 147 indexes serving in the “Datastore Indexes”, but Quota
details shows 200

On Mar 23, 4:40 pm, Nicanor Babula <nicanor.bab...@gmail.com> wrote:
> For example? I am sure there is space for optimizing the list views..
> The only trick I know is to reduce the number of order-able columns in the
> list view...
>
> For example, just yesterday I developed a statistics module which required
> me to add another 2 indexes for 2 queries like:
> select * from Customers where accountId = 'currentAccountId' and
> creationDate > [from] and creationDate < [toDate] and status = 'available'
>
> Is there a trick to avoid using indexes on queries like that?
>
> Il giorno venerdì 23 marzo 2012 12:28:47 UTC+1, de Witte ha scritto:
>
>
>
>
>
> > Try

> >http://support.google.com/​code/bin/request.py?contact_​type=AppEngineQuotaRequest<http://support.google.com/code/bin/request.py?contact_type=AppEngineQ...>

[Nicanor Babula]

Hi there,

Have you considered executing appcfg vacuum_indexes?

http://code.google.com/appengine/docs/java/tools/uploadinganapp.html#Deleting_Unused_Indexes

Cristian.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nicanor Cristian Babula

[Wolfram Gürlich]

In many places you can just use an indexed property instead of another custom index. It will cost you double however, since it actually creates two index entries - ASC and DESC.

Combine the properties you want to go in the index into one string and put it in a new property that just serves as your index.

Wolfram

[Jeff Schnitzer]

2012/3/26 Wolfram Gürlich <w.gue...@gmail.com>:

> In many places you can just use an indexed property instead of another
> custom index. It will cost you double however, since it actually creates two
> index entries - ASC and DESC.

It should be noted that compound indexes require single-property
indexes on all fields mentioned in the compound index. If you put()
an entity without a single-property index on any of the specified
properties, GAE will not put the entity in the multi-property index.

So it's not either/or; it's single-property indexes or both.

Jeff

[Prakhil Samar]

Yes i have tried vaccum_indexes

the problem is with the indexes count not getting reset.

I just have 147 indexes serving on my application and my number of indexes under the Quota Details shows 200

What should i do now?

GAE experts please suggest OR reset my indexes quota

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nicanor Cristian Babula

[Alfred Fuller]

It seems like there are two separate issues here:

We have a bug where the index quota isn't being updated correctly (we are looking into this).

When you have a lot of kinds you may actually need a lot of indexes. For example 200 indexes / 50 kinds = 4 indexes per kind. This is reasonable. I am curious, for all those that actually have 200 indexes, what is the maximum number of indexes / kind for all your kinds?

[Nicanor Babula]

I ran into the the same issue as Prakhil. I took a look at the new Query planner and optimized the indexes for some of my kinds, the number of indexes going to 200. I vacuumed them, but the number of indexes is still at 200, when the would have to be at 168. What can I do?

appid: domodentweb2.

datastore: HRD.

Thank you.

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/R4t0qUZC5QMJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nicanor Cristian Babula

[Nicanor Babula]

And here is the log message:

[log]

Unable to update indexes:

com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=s%7Edomodentweb2&version=8l&

400 Bad Request

Over index creation quota: The API call datastore_v3.CreateIndex() required more quota than is available.

at com.google.appengine.tools.admin.AbstractServerConnection.send1(AbstractServerConnection.java:285)

at com.google.appengine.tools.admin.AbstractServerConnection.send(AbstractServerConnection.java:245)

at com.google.appengine.tools.admin.AbstractServerConnection.post(AbstractServerConnection.java:224)

at com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.java:619)

at com.google.appengine.tools.admin.AppVersionUpload.updateIndexes(AppVersionUpload.java:303)

at com.google.appengine.tools.admin.AppAdminImpl.updateIndexes(AppAdminImpl.java:179)

at com.google.appengine.tools.admin.AppCfg$UpdateIndexesAction.execute(AppCfg.java:736)

at com.google.appengine.tools.admin.AppCfg.<init>(AppCfg.java:172)

at com.google.appengine.tools.admin.AppCfg.<init>(AppCfg.java:69)

at com.google.appengine.tools.admin.AppCfg.main(AppCfg.java:65)

com.google.appengine.tools.admin.AdminException: Unable to update indexes for app: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=s%7Edomodentweb2&version=8l&

400 Bad Request

Over index creation quota: The API call datastore_v3.CreateIndex() required more quota than is available.

at com.google.appengine.tools.admin.AppAdminImpl.updateIndexes(AppAdminImpl.java:183)

at com.google.appengine.tools.admin.AppCfg$UpdateIndexesAction.execute(AppCfg.java:736)

at com.google.appengine.tools.admin.AppCfg.<init>(AppCfg.java:172)

at com.google.appengine.tools.admin.AppCfg.<init>(AppCfg.java:69)

at com.google.appengine.tools.admin.AppCfg.main(AppCfg.java:65)

Caused by: com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=s%7Edomodentweb2&version=8l&

400 Bad Request

Over index creation quota: The API call datastore_v3.CreateIndex() required more quota than is available.

at com.google.appengine.tools.admin.AbstractServerConnection.send1(AbstractServerConnection.java:285)

at com.google.appengine.tools.admin.AbstractServerConnection.send(AbstractServerConnection.java:245)

at com.google.appengine.tools.admin.AbstractServerConnection.post(AbstractServerConnection.java:224)

at com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.java:619)

at com.google.appengine.tools.admin.AppVersionUpload.updateIndexes(AppVersionUpload.java:303)

at com.google.appengine.tools.admin.AppAdminImpl.updateIndexes(AppAdminImpl.java:179)

... 4 more

[/log]

--
Nicanor Cristian Babula

[Nicanor Babula]

SOLVED!!!

For anybody who might run into the same problem, try to change the default version back and forth and the quota will be reset.

Cheers. Appengine rules.

--
Nicanor Cristian Babula