Saving an entity from a Reducer invoking my DAO (DatastoreService.put)

[Antonio Fornié Casarrubios]

Hi all. I didn't find any answer or info for this:

Context: Java Google App Engine project, using Datastore. For a certain kind, Sale, I save the entities in two ways:

1. From a typical SaleDAO#save(saleEntity)
2. During MapReduce, from my SaleReducer#emit(saleEntity)
   

So I was investigating how to reuse some behavior as part of both cases, so I made something like:

    class SaleDAO {
        DatastoreService datastore
        public save(Entity entity) {
            // My common behavior that I want to reuse
            datastore.put(entity)
            // Some more common behavior that I want to reuse
        }
    }

And then for the MapReduce part, I extended OutputWriter to call my DAO, instead of the DatastoreMutationPool#put(entity) method provided for that purpose.

    

    public class SaleOutputWriter extends OutputWriter<Entity> {
        @Override
        public void write(Entity entity) {
            //pool.put(entity); - try to invoke the DAO instead
            saleDAO.save(sale)
        }
    }

1. My surprise is that it works! Well, the MR jobs seem to go to a strange state, but the entities are correctly saved. Still I'm 99% sure that this must be a bad practice or bring some problems. Can somebody tell me
2. If that's not the case, why are we given a certain API for MapReduce and another one for normal persistence? Will be the same to invoke DatastoreMutationPool.put than DatastoreService.put (any implementation)?
   

Honestly, this idea sounded crazy in my head and now that it seems to work I want to know more about it.

[Tom Kaitchuck]

The point of DatastoreMutationPool is to provide batching of updates to increase throughput. It is fine to use something else.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/61276548-616e-4f94-9263-00bcfcb5e33d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Antonio Fornié Casarrubios]

Thanks Tom. So it's that all? The only difference/disadvantage of this approach would be to have less performance?

[Tom Kaitchuck]

Correct. You still have to deal with slice / shard retries. But it should work fine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1cc85d13-ff75-4284-8e50-06b8a4eb17a5%40googlegroups.com.