[ANN]: full-text search for app-engine-patch/Django

[Waldemar Kornewald]

Hi,
we'd like to announce the immediate availablility of our full-text
search package.
It's called gae-search:
http://gae-full-text-search.appspot.com/

See it in action by searching our documentation (which is indexed with
gae-search). We also have a few demos.

Note that gae-search requires app-engine-patch (Django).

Features:
* index only specific properties (instead of all string/text
properties like in SearchableModel)
* Porter stemmers (increase search quality)
* sort your results (at least a little bit) via chain-sorting
* make "DISTINCT" queries using a so-called "values index"
* auto-completion via a jQuery plugin
* key-based pagination (fully unit-tested implementation of Ryan
Barrett's algorithm)
* easy to use views and templates (add search support in just a few lines)

Since it took a lot of effort to implement all features and make them
easy to use we can't give this away for free, though. We initially
implemented it for our own projects, but after so many people
complained about the lack of full-text search we though we could
provide it to others - for a little compensation.

Bye,
Waldemar Kornewald & Thomas Wanschik (the creators of app-engine-patch)

[Nick Johnson (Google)]

Hi Waldemar,

Impressive!

I'm curious - what's the dependency on Django / App Engine patch? This seems like it should be doable as a purely datastore-oriented library.

-Nick Johnson

--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

[Waldemar Kornewald]

Hi Nick.

On Jun 24, 1:28 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi Waldemar,
>
> Impressive!

Thanks a lot! It's nice to hear that from a Google developer. :) We
hope it's enough to bridge the gap until Google releases super-
powerful full-text search support (with ranking and spellchecking and
phonetic matching and ...).

> I'm curious - what's the dependency on Django / App Engine patch? This seems
> like it should be doable as a purely datastore-oriented library.

Some low-level parts *should* work directly with any framework (as we
just derive from db.StringListProperty). We haven't tested it, but I'm
pretty sure if you rip out the values index the Base package would run
with webapp, too, though a few app-engine-patch/ragendja utility
functions might still be needed.

The values index (or "DISTINCT" index) requires special app-engine-
patch extensions because it has to be compatible with transactions. If
you modify an entity within a transaction gae-search has to wait for
the transaction to finish and only then it's possible to update the
values index (which would be in different entity groups). Also, we
have to know whether the transaction finished successfully before we
can actually update the values index. Here, app-engine-patch provides
a few extra signals (post_save_committed, post_delete_committed) which
we handle in order to update the index. We plan to port the updater to
the new background processing API and maybe that can obviate the need
for those signals. So, a future release of the Base package might even
support webapp (if there is enough interest).

The higher-level parts in the Premium package (e.g., the views and
templates) require more complete Django support (more than what django-
helper can provide). Though, we already removed the dependency on
generic views, so it might be possible to support django-helper in a
future release, too.

Finally, the "auto-complete" jQuery plugin is independent of any
framework, of course.

Bye,
Waldemar Kornewald

[GenghisOne]

Hi Waldemar

Kudos for trying to address a serious limitation with google app
engine.

What could we do to convince you to open-source your search
technology? By we, I mean the AppEngine dev community.

If it is a monetary issue, maybe Google could cut your company a
cheque to cover your investment and we could finally put this crippled
search issue to rest.

[kugutsumen]

One happy customer here (probably one of the first :) )

Impressive work from Waldemar and Thomas! I'm going to play
with gae-search over the weekend. I might use it to let people
search version banners my nkill gae project.