Searching a datastore?

[and...@gmail.com]

One feature that doesn't appear to be present in the Datastore API is
search. Given Google's primary product, this seems like a significant
omission. Are there plans to add this sort of functionality in a
future release, or should we start looking into ways to implement
search features using what we've been given?

[ryan]

We've included a short-term full text search library in the
google.appengine.ext.search module. It's limited, so we don't discuss
it much in the documentation. We expect to provide a more robust, full
featured solution eventually.

Until then, here's the google.appengine.ext.search docstring:

"""Full text indexing and search, implemented in pure python.

Defines a SearchableModel subclass of db.Model that supports full text
indexing and search, based on the datastore's existing indexes.

Don't expect too much. First, there's no ranking, which is a killer
drawback.
There's also no exact phrase match, substring match, boolean
operators,
stemming, or other common full text search features. Finally, support
for stop
words (common words that are not indexed) is currently limited to
English.

To be indexed, entities must be created and saved as SearchableModel
instances, e.g.:

from google.appengine.ext import search

class Article(search.SearchableModel):
text = db.TextProperty()
...

article = Article(text=...)
article.save()

To search the full text index, use the SearchableModel.all() method to
get an
instance of SearchableModel.Query, which subclasses db.Query. Use its
search()
method to provide a search query, in addition to any other filters or
sort
orders, e.g.:

query = article.all().search('a search
query').filter(...).order(...)
for result in query:
...

The full text index is stored in a property named
__searchable_text_index. If
you want to use search() in a query with an ancestor, filters, or sort
orders,
you'll need to create an index in index.yaml with the
__searchable_text_index
property. For example:

- kind: Article
properties:
- name: __searchable_text_index
- name: date
direction: desc
...

Note that using SearchableModel will noticeable increase the latency
of save()
operations, since it writes an index row for each indexable word. This
also
means that the latency of save() will increase roughly with the size
of the
properties in a given entity. Caveat hacker!
"""