Ignore Entitys

[Flips]

Hi,

i am using Google App Engine Entitys which implement the
"SearchableModel". For each Entity there will be another one generated
by the App Engine (Search Index) and I do not want to replicate this
Entity.
Is there any possibility to ignore it?

Cheers

[Kurt Daal]

If I'm not mistaken SearchableModel simply adds another property to
entity called "__searchable_text_index".

You should be able to exclude it from replication by using
RECEIVE_FIELDS_EXCLUDE config flag like this:

SERVICES = {
"ReplicateYourEntity": {
TYPE: REPLICATION_SERVICE,
KIND: "YourEntity",
RECEIVE_FIELDS_EXCLUDE: ["__searchable_text_index"],
},

[Flips]

Hi Kurt,

thanks for the fast reply but unfortunately this solution is not
working. :(

I have added the following code in rocket.py, line 75 as a "Bugfix":

if field == "__searchable_text_index":
continue

Now I have the next problem with the "__searchable_text_index", If I
create a new Entity and use the put() function the index will be
filled automatically. If I create a new Entity via MySQL and approcket
replicates it to the App Engine the "__searchable_text_index" seems to
be empty.

Cheers

[Kaspars]

Are you using the most recent version of AppRocket?
RECEIVE_FIELDS_EXCLUDE option was only added to 2.0.0 which has been
just released.

About the other problem - it's because AppRocket operates at lower
level than App Engine Model classes. It uses low level datastore
interfaces that are not aware about model classes directly. However I
think there is one potential solution to this. There is a way to
trigger a custom function after an entity has been uploaded to App
Engine. You need to specify additional config flag AFTER_SEND and set
it to full name of the function that you want to call, like this:

SERVICES = {
"ReplicateYourEntity": {
TYPE: REPLICATION_SERVICE,
KIND: "YourEntity",
RECEIVE_FIELDS_EXCLUDE: ["__searchable_text_index"],

AFTER_SEND: "some_package.some_module.after_send"
},
}

Then in the specified module, create a function like this:

def after_send(entity):
YourEntity.get(entity.key).put()

This should trigger the generation of index by SearchableModel. Note -
AFTER_SEND is supported in AppRocket 2.0.0

[Flips]

Hi Kaspars,

I use version 2.0.0 and I have tested RECEIVE_FIELDS_EXCLUDE with a
regular field and it worked but it does not work with
"__searchable_text_index". I think you should know that I am working
with the development server and did not test it on the App Engine.

Your solution to my second problem is almost perfect but get
(entity.key) does not work you have to use get(entity) :-)

[Waleed Abdulla]

Thanks guys for this thread, it was timely. I got the SearchableModel working using Kaspars' method. I'm surprised that the AFTER_SEND option is not documented!  It's such a great feature. This is my after_send method for reference:

def rocket_after_send(entity):

    MyEntity.get(entity.key()).put()

   Since the entity is being passed to the function, can't we skip the get() part and avoid the extra hit to the datastore; and simply cast the entity to the right type instead? It sounds doable, but I don't know much about the insides of app rocket to tell if it would break something. Any thoughts?

Or, maybe app rocket can skip putting the entity in the data store when the after_send option is present and let that function handle it. This will prevent double putting. 

On an unrelated note, I'm considering switching to Bill Katz's SearchableModel[1] instead because it stores the index list in a separate entity to keep the main entities small, and it has an explicit index() method to only update the index which I can call rather than loading and saving the entity. 

Regards,

Waleed

[1] http://www.billkatz.com/2008/8/A-SearchableModel-for-App-Engine

--

You received this message because you are subscribed to the Google Groups "AppRocket" group.
To post to this group, send email to appr...@googlegroups.com.
To unsubscribe from this group, send email to approcket+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/approcket?hl=en.

[Kurt Daal]

Somehow AFTER_SEND option slipped past documentation. Certainly will
be added.

About your comment of skipping get part, I don't think you can cast
the entity because AppRocket operates with low level datastore classes
and not the Model objects. Maybe there is some way of constructing
Model object from datastore entity without doing get, but I need to
check App Engine internals a bit to have a more specific answer.

The idea of skiping put entirely and delegating it to a custom
function might work though. We will consider supporting it in the next
update.

> > approcket+...@googlegroups.com<approcket%2Bunsu...@googlegroups.com>
> > .

[Waleed Abdulla]

Thanks Kurt. I played with it a little bit and implemented a simple change to let the after_send method do the update for me because I need to do some data customization before saving. For anyone who might want to see the changes, here is a minor patch to rocket.py:

@@ -153,8 +153,6 @@

                         entity[field_name] = rocket_to_ae(field_type, value)

                         

     

-        datastore.Put(entity)

-        

         after_send = self.request.get("after_send")

         if after_send:

             try:

@@ -166,6 +164,8 @@

             except Exception, e:

                 logging.exception("Error invoking AFTER_SEND event handler (%s):" % after_send)

                 self.response.out.write(u'<error>Error invoking AFTER_SEND event handler (%s): %s</error>\n' % (after_send,e))                            

+        else:

+            datastore.Put(entity)

         

         self.response.out.write(u'OK')

    And I changed my after_send method as follows:

def rocket_after_send(entity):

        # ... do custom updates before entity converts to a model .. #

    mymodel = MyModel.from_entity(entity)

        # ... do custom updates after entity becomes a model .. #

    mymodel.put()

Regards,

Waleed

To unsubscribe from this group, send email to approcket+...@googlegroups.com.