searchable Model - error when item contains too much text - suggestions/fixes?

[sofia]

Hi

I'm consistently getting the same error when trying to insert or
update an item from a searchableModel that has too much text. By too
much text i mean something like the example below:

@project:nottee @type:note @status:todo Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aliquam et lectus eget diam dapibus
placerat. Nam posuere, purus tristique ultrices porta, sapien lectus
interdum nisi, eget tempus odio felis nec lacus! Nunc sit amet mi sit
amet odio molestie placerat.

- it's not that much.. If i remove some of the text everything goes
smoothly.

Here's my model definition

class Note(search.SearchableModel):
user = db.UserProperty(required=True)
description = db.TextProperty(required=True)
created = db.DateTimeProperty(auto_now_add=True)
updated = db.DateTimeProperty(auto_now=True)
tags = db.TextProperty(required=False)

Am i doing somethong wrong? Is there at least any way to limit the
searchable index to the tags property?

Thank you for any help. the way it is it leaves my little app little
room to breathe ;)

[Bill]

Hi Sofia,

I ran into this problem as well. I pulled the SearchableModel code
and
made some modifications so you can specify which properties to
ignore from full indexing. You can also increase the minimum length
of the word before it's indexed.

I plan on fully releasing my hacked version hopefully late tonight or
tomorrow. That said, I don't understand why so small a text size
is causing an error. I wasn't hitting the error until it was a very
lengthy
article and only on the AppEngine server. What's the exact error
you're getting?

-Bill

[sofia]

Hi

The error message is below [shortened]

Traceback (most recent call last):
[..]
BadRequestError: Too many indexed properties for entity: app:
"notee",path <Element {type: "Note",id: 168}>
This index put it over the limit: entity_type: "Note",ancestor:
false,Property { name: "__searchable_text_index", direction:
ASCENDING},Property { name: "__searchable_text_index", direction:
ASCENDING},Property { name: "__searchable_text_index", direction:
ASCENDING},Property { name: "user", direction: ASCENDING},Property
{ name: "updated", direction: DESCENDING}

This is happening on the AppEngine Server - right now i don't remember
if it also happens on my local server but i will check later.

Sorry for my ignorance here but if you release the hacked version
could i use it live? If so, how - i mean we only have code for the app
itself, so where do i put code for the appengine itself or is it just
a question of inheriting and so overwriting?

[sofia]

Does anyone have any idea why i'm hitting the search index limit at
such low bounds?

When i save i'm just doing this

note = Note(description=description, user=user, tags=tag_strings)
note.put()

where tag_strings is something like this 'project:nottee type:note
status:todo'.

Am i doing something wrong? Does anyone have any suggestion?

[sofia]

I changed the Note model making the tags property a String instead of
a Text property - what it should have been anyway but although it
works locally, deployed it still gives errors..

class Note(search.SearchableModel):
user = db.UserProperty(required=True)
description = db.TextProperty(required=True)
created = db.DateTimeProperty(auto_now_add=True)
updated = db.DateTimeProperty(auto_now=True)

tags = db.StringProperty(required=False)

[thedude]

i'm having the same problem with a very very small text size.

what's frustrating, is it was working for a while, now it isn't, so
besides having to try to delete everything on the server to retry...

crossing my fingers it's an app engine problem today, but seriously,
adding a small class with 3 fields and about 100 characters of text
is "too many indexed properties"??? that's a big wtf..

[Bill]

If you are still talking about indexing errors, look at this thread:
http://groups.google.com/group/google-appengine/browse_thread/thread/d5f4dcb7d00ed4c6/bac9b1d0d50e5a5d#bac9b1d0d50e5a5d

You can also pull my slightly hacked version of SearchableModel out of
Bloog. Here's the link to the code:
http://github.com/DocSavage/bloog/tree/master/models/search.py
With your own version of SearchableModel, you can increase
_FULL_TEXT_MIN_LENGTH and remove any text-based properties you don't
want indexed. You can also try hacking on the index size to remove
the errors.
To use in your own code, just change the superclass of SearchableModel
back to db.Model.
-Bill

On Aug 14, 6:59 pm, sofia <sofiacard...@gmail.com> wrote:

[Bill]

I should also mention that tags as String would probably increase the
indexing since Text and Blob values are not indexing, according to
this:
http://code.google.com/appengine/docs/datastore/queriesandindexes.html

On Aug 14, 6:59 pm, sofia <sofiacard...@gmail.com> wrote:

[thedude]

bill,

thanks for this link to the search.py.

can you provide a little more specificity into how to integrate this
into an app?

if we were using search.SearchableModel previously to extend my
Article class. do i change it back to db.Model and include
master_models_search.py in my app update directory up on the GAE
server?

tnks,

On Aug 24, 5:51 pm, Bill <billk...@gmail.com> wrote:
> If you are still talking about indexing errors, look at this thread:http://groups.google.com/group/google-appengine/browse_thread/thread/...

[Bill]

I just posted a blog entry on the subject:
http://billkatz.com/2008/8/A-SearchableModel-for-App-Engine

Just browse the Bloog source code and you'll see how to use it. Put
the search.py into your app folder, import it using something like
"import search" (instead of doing "import
google.appengine.ext.search"), and it should work.

I think Marzia said SearchableModel was updated. Need to make sure
any changes get put into my version.
-Bill