Search API questions for app engine team

[Mahron]

Hi,

the doc says there is a 250 GB limit on search data storage.

Is that only for the preview ? Or will this limit be also for production ?

Also, would it be possible to give us details about how search indexes are stored, like it is done for the datastore, in order to understand the storage overheads.

Thanks. 

[Mahron]

Sorry for upping this, need an answer.

[Chris Ramsdale]

We're looking into per-index limits with a more relaxed aggregate index limit. This is still in the research phase and I expect the total 250 GB limit to persist through GA. Feel free to reach out directly if this limit is blocking your use of the Search API.

-- Chris

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Mahron]

This limit will be a problem down the road if it can not be lifted, so I am not really sure if I should implement it or not.

I also do not understand if those 250 GB are for the text only, or does it include whatever storage if used for indexing (which would make things even worse).

[David Hardwick]

Can this limit be segmented by namespaces?  So you can have one search index for namespaceA and then another for namespaceB?  (each maxing out at 250GB during the GA period)

[Kaan Soral]

250gb is really funny, the reason I use appengine (and probably other people's main reason too) is that it allows an app to grow out of boundaries

you can buy dedicated machine(s) that will beat the 250gb indexed search limitation pretty easily, many people were using simple lucene etc based search solutions, so the current state of "Search" is no better than their placeholder solutions

If the index limit is 250gb's, it seems that the actual document size limitation should be a lot lower

I also invested a lot lately on my search api implementation, I think the feature will make appengine much much more valuable than it already is

Assuming it would become as scalable as the db, it might even replace db queries for many use cases (probably cost of search api would be > db, that's why it won't replace all use cases)

[Mahron]

How do I reach out directly ? is there a place to ask question without special support (because I don't have that)?

I also added a feature request for non indexed fields : https://code.google.com/p/googleappengine/issues/detail?id=9619