field-specific search + gae-search - gae-search

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion field-specific search + gae-search
Received-SPF: pass (google.com: domain of dfran...@gmail.com designates 209.85.213.169 as permitted sender) client-ip=209.85.213.169;
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=N7lv119oOYqhQnVemi/fdUdXQ3PA/Qc6qaYRXQux8xcDDCxIuJ3Jaoc9CyRmw082JK
         v4tBokXeL7ySF3B+V+QXmJa4Gyfqx9j84EaiLy40ELIHp+hi6Rk3tX1dETKFoK3eLRGp
         DVfegd1xBVOT6UFKrStXGlriaD5HAIQRzPIoQ=
MIME-Version: 1.0
In-Reply-To: <AANLkTikMD3ZsN-xX=FzVYD0hS9Bzc==ErAB9H2ogX...@mail.gmail.com>
References: <AANLkTi=qCSrYLV4UdfbGjRbuN0De8t3xyOYQ=GwKt...@mail.gmail.com>
	<AANLkTinxfQ+Z9dJtF6eb12kBuRYiiSHXPHHfCj8Mt...@mail.gmail.com>
	<AANLkTinZRXfLU0N7igUc7zQHc4g1H9HT88KLbkYN4...@mail.gmail.com>
	<AANLkTi=e5rfP3M6R2SjNSp6o5tUw1wg=GG4J151Ho...@mail.gmail.com>
	<AANLkTikMD3ZsN-xX=FzVYD0hS9Bzc==ErAB9H2ogX...@mail.gmail.com>
Date: Sun, 26 Sep 2010 19:05:52 -0500
Message-ID: <AANLkTin3Np1k-Mgz8dFQ1KCzkOVLzWbMFJf9o1gJg...@mail.gmail.com>
Subject: Re: [gae-search] field-specific search + gae-search
From: Dan Frankowski <dfran...@gmail.com>
To: gae-search@googlegroups.com
Content-Type: multipart/alternative; boundary=0016e68db6e6d97ae80491327d86

--0016e68db6e6d97ae80491327d86
Content-Type: text/plain; charset=ISO-8859-1

Not even that is true. SearchIndexProperty.{paginated_query,
_get_paginated_query} don't used SearchableListProperty.search? That is
unexpected.

Dan

On Sun, Sep 26, 2010 at 6:29 PM, Dan Frankowski <dfran...@gmail.com> wrote:

> Never mind again, "query" looks like just text. However,
> SearchableListProperty and SearchIndexProperty would both have to be
> modified. I think I'm starting to see.
>
> Dan
>
>
> On Sun, Sep 26, 2010 at 6:21 PM, Dan Frankowski <dfran...@gmail.com>wrote:
>
>> Ah, never mind. It can fall through that if/elif and still return a
>> search.  However, there is a separate query object, so that query object
>> would have to be modified to take something other than model.all().
>>
>> Dan
>>
>>
>> On Sun, Sep 26, 2010 at 6:20 PM, Dan Frankowski <dfran...@gmail.com>wrote:
>>
>>> When you say "disable the relation index" you don't mean set
>>> relation_index=False, do you? You mean modify the code substantially? The
>>> code in SearchIndexProperty.search() says:
>>>
>>>         if self.values_index:
>>>             return self._values_index_model.index_0.search(query,
>>> filters,
>>>                 chain_sort=chain_sort, language=language,
>>> keys_only=keys_only)
>>>         elif self.relation_index:
>>>             items = getattr(self._relation_index_model, self.name
>>> ).search(query,
>>>                 filters, chain_sort=chain_sort, language=language,
>>>                 keys_only=True)
>>>
>>> However, this new beast is neither a relation_index (because that can't
>>> be merge-joined), nor a values_index (that has all distinct values according
>>> to your docs). Or could I use a values_index?
>>>
>>> I am beginning to think it might be easier to rewrite than understand.
>>>
>>> Dan
>>>
>>> On Tue, Sep 7, 2010 at 6:23 AM, Waldemar Kornewald <wkornew...@gmail.com
>>> > wrote:
>>>
>>>> On Tue, Sep 7, 2010 at 2:04 AM, Dan Frankowski <dfran...@gmail.com>
>>>> wrote:
>>>> > I am seeking advice.
>>>> >
>>>> > I know gae-search is deprecated, but these same issues would come up
>>>> in
>>>> > nonrel-search, and I fear it is hours of time to convert.
>>>> >
>>>> > gae-search is set up to throw many different fields into one
>>>> searchable
>>>> > property in an entity separate from the one being searched for.
>>>> >
>>>> > For our application, we wish to have search by many fields (e.g.,
>>>> name, sex,
>>>> > date of birth, residence, caregiver name) but field-specific. That is,
>>>> > searching the name field for "dan" would not come up with "dan" from
>>>> the
>>>> > caregiver name. However, I still would like it to be case-insensitive
>>>> and
>>>> > with prefix matching.
>>>> >
>>>> > Possibility #1 is to create shadow fields, e.g.
>>>> >
>>>> > name = db.StringProperty()
>>>> > name_for_search = db.StringProperty()
>>>> > date_of_birth = db.DateProperty()
>>>> > date_of_birth_for_search = db.StringProperty()
>>>> >
>>>> > and stick lowercased data in *_for_search, and build complex queries
>>>> with
>>>> > prefix searches. Seems like a pain, especially since the number of
>>>> indexes
>>>> > goes up rapidly.
>>>> >
>>>> > Possibility #2 is to create an array with field name prefixes, e.g.
>>>> >
>>>> > search_array = ["name:dan" , "date_of_birth:1980-01-01", ..]
>>>> >
>>>> > and create searches on search_array.
>>>> >
>>>> > If I put search_array on the original entity and hook it up with a
>>>> Django
>>>> > post-save trigger, I wonder if I get a recursive mess. If I put
>>>> search_array
>>>> > in its own search-entity (with a parent of the original entity), I
>>>> don't see
>>>> > how to integrate it with gae-search.
>>>> >
>>>> > Advice?
>>>>
>>>> You can't do multiple prefix searches in a single query. The simplest
>>>> solution is to disable the relation index and create multiple search
>>>> indexes (one for every searchable property: "name", "date_of_birth",
>>>> etc.). Then, you can filter with "=" on those properties as if they
>>>> were StringListProperties. Since you only want prefix search that's
>>>> probably good enough. You should also split() the search strings, so
>>>> you can match multiple words via "startswith" ("d frank" would match
>>>> "Dan Frankowski").
>>>>
>>>> Bye,
>>>> Waldemar Kornewald
>>>>
>>>> --
>>>> Django on App Engine, MongoDB, ...? Browser-side Python? It's
>>>> open-source:
>>>> http://www.allbuttonspressed.com/blog/django
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "gae-search" group.
>>>> To post to this group, send email to gae-search@googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> gae-search+unsubscribe@googlegroups.com<gae-search%2Bunsubscribe@googlegroups.com>
>>>> .
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/gae-search?hl=en.
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>

--0016e68db6e6d97ae80491327d86
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Not even that is true. SearchIndexProperty.{paginated_query, _get_paginated=
_query} don&#39;t used SearchableListProperty.search? That is unexpected.<b=
r><br>Dan<br><br><div class=3D"gmail_quote">On Sun, Sep 26, 2010 at 6:29 PM=
, Dan Frankowski <span dir=3D"ltr">&lt;<a href=3D"mailto:dfran...@gmail.com=
">dfran...@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; borde=
r-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Never mind again,=
 &quot;query&quot; looks like just text. However, SearchableListProperty an=
d SearchIndexProperty would both have to be modified. I think I&#39;m start=
ing to see.<br>
<font color=3D"#888888"><br>Dan</font><div><div></div><div class=3D"h5"><br=
><br><div class=3D"gmail_quote">
On Sun, Sep 26, 2010 at 6:21 PM, Dan Frankowski <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:dfran...@gmail.com" target=3D"_blank">dfran...@gmail.com</a>&g=
t;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin: 0pt =
0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex=
;">

Ah, never mind. It can fall through that if/elif and still return a search.=
=A0 However, there is a separate query object, so that query object would h=
ave to be modified to take something other than model.all().<br><font color=
=3D"#888888"><br>

Dan</font><div><div></div><div><br>
<br><div class=3D"gmail_quote">On Sun, Sep 26, 2010 at 6:20 PM, Dan Frankow=
ski <span dir=3D"ltr">&lt;<a href=3D"mailto:dfran...@gmail.com" target=3D"_=
blank">dfran...@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gma=
il_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(20=
4, 204, 204); padding-left: 1ex;">


When you say &quot;disable the relation index&quot; you don&#39;t mean set =
relation_index=3DFalse, do you? You mean modify the code substantially? The=
 code in SearchIndexProperty.search() says:<br><br>=A0=A0=A0=A0=A0=A0=A0 if=
 self.values_index:<br>



=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 return self._values_index_model.index_0.s=
earch(query, filters,<br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 chai=
n_sort=3Dchain_sort, language=3Dlanguage, keys_only=3Dkeys_only)<br>=A0=A0=
=A0=A0=A0=A0=A0 elif self.relation_index:<br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0 items =3D getattr(self._relation_index_model, <a href=3D"http://self.na=
me" target=3D"_blank">self.name</a>).search(query,<br>



=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 filters, chain_sort=3Dchain_s=
ort, language=3Dlanguage,<br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =
keys_only=3DTrue)<br><br>However, this new beast is neither a relation_inde=
x (because that can&#39;t be merge-joined), nor a values_index (that has al=
l distinct values according to your docs). Or could I use a values_index?<b=
r>



<br>I am beginning to think it might be easier to rewrite than understand.<=
br><font color=3D"#888888"><br>Dan<br><br></font><div class=3D"gmail_quote"=
><div>On Tue, Sep 7, 2010 at 6:23 AM, Waldemar Kornewald <span dir=3D"ltr">=
&lt;<a href=3D"mailto:wkornew...@gmail.com" target=3D"_blank">wkornewald@gm=
ail.com</a>&gt;</span> wrote:<br>



</div><div><div></div><div><blockquote class=3D"gmail_quote" style=3D"margi=
n: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-le=
ft: 1ex;"><div><div></div><div>On Tue, Sep 7, 2010 at 2:04 AM, Dan Frankows=
ki &lt;<a href=3D"mailto:dfran...@gmail.com" target=3D"_blank">dfrankow@gma=
il.com</a>&gt; wrote:<br>




&gt; I am seeking advice.<br>
&gt;<br>
&gt; I know gae-search is deprecated, but these same issues would come up i=
n<br>
&gt; nonrel-search, and I fear it is hours of time to convert.<br>
&gt;<br>
&gt; gae-search is set up to throw many different fields into one searchabl=
e<br>
&gt; property in an entity separate from the one being searched for.<br>
&gt;<br>
&gt; For our application, we wish to have search by many fields (e.g., name=
, sex,<br>
&gt; date of birth, residence, caregiver name) but field-specific. That is,=
<br>
&gt; searching the name field for &quot;dan&quot; would not come up with &q=
uot;dan&quot; from the<br>
&gt; caregiver name. However, I still would like it to be case-insensitive =
and<br>
&gt; with prefix matching.<br>
&gt;<br>
&gt; Possibility #1 is to create shadow fields, e.g.<br>
&gt;<br>
&gt; name =3D db.StringProperty()<br>
&gt; name_for_search =3D db.StringProperty()<br>
&gt; date_of_birth =3D db.DateProperty()<br>
&gt; date_of_birth_for_search =3D db.StringProperty()<br>
&gt;<br>
&gt; and stick lowercased data in *_for_search, and build complex queries w=
ith<br>
&gt; prefix searches. Seems like a pain, especially since the number of ind=
exes<br>
&gt; goes up rapidly.<br>
&gt;<br>
&gt; Possibility #2 is to create an array with field name prefixes, e.g.<br=
>
&gt;<br>
&gt; search_array =3D [&quot;name:dan&quot; , &quot;date_of_birth:1980-01-0=
1&quot;, ..]<br>
&gt;<br>
&gt; and create searches on search_array.<br>
&gt;<br>
&gt; If I put search_array on the original entity and hook it up with a Dja=
ngo<br>
&gt; post-save trigger, I wonder if I get a recursive mess. If I put search=
_array<br>
&gt; in its own search-entity (with a parent of the original entity), I don=
&#39;t see<br>
&gt; how to integrate it with gae-search.<br>
&gt;<br>
&gt; Advice?<br>
<br>
</div></div>You can&#39;t do multiple prefix searches in a single query. Th=
e simplest<br>
solution is to disable the relation index and create multiple search<br>
indexes (one for every searchable property: &quot;name&quot;, &quot;date_of=
_birth&quot;,<br>
etc.). Then, you can filter with &quot;=3D&quot; on those properties as if =
they<br>
were StringListProperties. Since you only want prefix search that&#39;s<br>
probably good enough. You should also split() the search strings, so<br>
you can match multiple words via &quot;startswith&quot; (&quot;d frank&quot=
; would match<br>
&quot;Dan Frankowski&quot;).<br>
<br>
Bye,<br>
Waldemar Kornewald<br>
<br>
--<br>
Django on App Engine, MongoDB, ...? Browser-side Python? It&#39;s open-sour=
ce:<br>
<a href=3D"http://www.allbuttonspressed.com/blog/django" target=3D"_blank">=
http://www.allbuttonspressed.com/blog/django</a><br>
<font color=3D"#888888"><br>
--<br>
You received this message because you are subscribed to the Google Groups &=
quot;gae-search&quot; group.<br>
To post to this group, send email to <a href=3D"mailto:gae-search@googlegro=
ups.com" target=3D"_blank">gae-search@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:gae-search%=
2Bunsubscribe@googlegroups.com" target=3D"_blank">gae-search+unsubscribe@go=
oglegroups.com</a>.<br>
For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/gae-search?hl=3Den" target=3D"_blank">http://groups.google.com/group/g=
ae-search?hl=3Den</a>.<br>
<br>
</font></blockquote></div></div></div><br><br clear=3D"all"><br>
</blockquote></div><br><br clear=3D"all"><br>
</div></div></blockquote></div><br><br clear=3D"all"><br>
</div></div></blockquote></div><br><br clear=3D"all"><br>

--0016e68db6e6d97ae80491327d86--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
