field-specific search + gae-search

[Dan Frankowski]

I am seeking advice.

I know gae-search is deprecated, but these same issues would come up in nonrel-search, and I fear it is hours of time to convert.

gae-search is set up to throw many different fields into one searchable property in an entity separate from the one being searched for.

For our application, we wish to have search by many fields (e.g., name, sex, date of birth, residence, caregiver name) but field-specific. That is, searching the name field for "dan" would not come up with "dan" from the caregiver name. However, I still would like it to be case-insensitive and with prefix matching.

Possibility #1 is to create shadow fields, e.g.

name = db.StringProperty()
name_for_search = db.StringProperty()
date_of_birth = db.DateProperty()
date_of_birth_for_search = db.StringProperty()

and stick lowercased data in *_for_search, and build complex queries with prefix searches. Seems like a pain, especially since the number of indexes goes up rapidly.

Possibility #2 is to create an array with field name prefixes, e.g.

search_array = ["name:dan" , "date_of_birth:1980-01-01", ..]

and create searches on search_array.

If I put search_array on the original entity and hook it up with a Django post-save trigger, I wonder if I get a recursive mess. If I put search_array in its own search-entity (with a parent of the original entity), I don't see how to integrate it with gae-search.

Advice?

Dan

[Waldemar Kornewald]

You can't do multiple prefix searches in a single query. The simplest
solution is to disable the relation index and create multiple search
indexes (one for every searchable property: "name", "date_of_birth",
etc.). Then, you can filter with "=" on those properties as if they
were StringListProperties. Since you only want prefix search that's
probably good enough. You should also split() the search strings, so
you can match multiple words via "startswith" ("d frank" would match
"Dan Frankowski").

Bye,
Waldemar Kornewald

--
Django on App Engine, MongoDB, ...? Browser-side Python? It's open-source:
http://www.allbuttonspressed.com/blog/django

[Dan Frankowski]

When you say "create multiple search indexes" you mean multiple SearchIndexProperty properties on the same object? Will a single call to show_search_results() search multiple such properties?

--
You received this message because you are subscribed to the Google Groups "gae-search" group.
To post to this group, send email to gae-s...@googlegroups.com.
To unsubscribe from this group, send email to gae-search+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gae-search?hl=en.

[Waldemar Kornewald]

On Tue, Sep 7, 2010 at 5:59 PM, Dan Frankowski <dfra...@gmail.com> wrote:
> When you say "create multiple search indexes" you mean multiple
> SearchIndexProperty properties on the same object?

Yes.

> Will a single call to
> show_search_results() search multiple such properties?

No it won't (there's no way to specify which GET key is associated
with which index). You'll have to define your own view (or extend the
existing one). There's also no way to call search() on an existing
query. That would have to be added to SearchIndexProperty.search() as
an optional keyword argument (so that is used instead of
self.model.all()).

[Dan Frankowski]

When you say "disable the relation index" you don't mean set relation_index=False, do you? You mean modify the code substantially? The code in SearchIndexProperty.search() says:

        if self.values_index:
            return self._values_index_model.index_0.search(query, filters,
                chain_sort=chain_sort, language=language, keys_only=keys_only)
        elif self.relation_index:
            items = getattr(self._relation_index_model, self.name).search(query,
                filters, chain_sort=chain_sort, language=language,
                keys_only=True)

However, this new beast is neither a relation_index (because that can't be merge-joined), nor a values_index (that has all distinct values according to your docs). Or could I use a values_index?

I am beginning to think it might be easier to rewrite than understand.

Dan

On Tue, Sep 7, 2010 at 6:23 AM, Waldemar Kornewald <wkorn...@gmail.com> wrote:

[Dan Frankowski]

Ah, never mind. It can fall through that if/elif and still return a search.  However, there is a separate query object, so that query object would have to be modified to take something other than model.all().

Dan

[Dan Frankowski]

Never mind again, "query" looks like just text. However, SearchableListProperty and SearchIndexProperty would both have to be modified. I think I'm starting to see.

Dan

[Dan Frankowski]

Not even that is true. SearchIndexProperty.{paginated_query, _get_paginated_query} don't used SearchableListProperty.search? That is unexpected.

Dan

[Dan Frankowski]

Ah, they are for when there is no query. I think I'm figuring this out. Sorry for all the emails. I'll stop until I have a solid question.

Dan

[Dan Frankowski]

Waldemar,

Based on your advice, I modified gae-search to do what I want (refine a query by multiple parameters). I attach the patch file in case it is of interest.

Thanks again. Our project now has data for over 1000 children and growing.

Dan

P.S. My commit comment:

Edit gae-search to allow query refinement through "previous_query" parameters

Add previous_query to:
- SearchableListProperty.filter()
- SearchableListProperty.search()
- SearchIndexProperty.search()

Add to views:

- query_param_search
- show_search_results_from_results

[Waldemar Kornewald]

Hi Dan,
thanks a lot for the patch. BTW, the same functionality is already
part of nonrel-search (in case you ever find the time to port it
over). Good luck with your project.

Bye,
Waldemar