Intermittent Issue Writing to Google Appengine Datastore

[user242153 via StackOverflow]

I have a functioning app and recently have had intermittent problems writing to the datastore.

I did not make any relevant code changes, however in the last few days my attempts to write to the datastore sometimes work and sometimes don't.

I am trying to save an object that is in a many to one relationship with an existing persisted parent. So, the logic works like this:

1) Parent pulled from the datastore

2) Child created / instantiated using constructor

3) Parent.addSingleChild(child); // the "addSingleChild" method just adds the object argument to the collection of children

4) child.setParent(Parent); // sets the Parent object to the parent field

I am using transactions as explained in the documentation ending with "finally {if (tx.isActive()) {tx.rollback(); } }"

When the servlet is called, the parent is called from the datastore and the child object is created and added to the many to one mapping to the pre-existing parent.

The child should automatically be persisted, since the parent is already persistent, and the child is added to the collection of children that map to the parent. And it worked this way in the past. However, to be sure, I did add a pm.makePersistent(child). Doesn't seem to help, still have the intermittent problem.

Any suggestions would be appreciated, and if you need to see the actual code I can post. Thanks

*** UPDATE *****

Here is the code for reference. For background, there is a class called OMRUser which is a user class, and there is Location3 class. There is a one to many owned relationship between a user and locations

snippet of USER class:

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class OmrUser {
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    @Persistent
    private User gaeuser;

    @Persistent(mappedBy = "omruser")
    @Element(dependent = "true")
    private List userlocs;    

    @Persistent(mappedBy = "omruser")
    @Element(dependent = "true")
    private List userdrivers;        

    @Persistent
    private String accounttype;    

snippet of Location class:

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Location3 implements Comparable {

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    @Persistent
    private OmrUser omruser;        
    

snippet of servlet to save locations

PersistenceManager pm = PMF.get().getPersistenceManager();
Transaction tx = pm.currentTransaction();
try {
    tx.begin();

    String select_query = "select from " + OmrUser.class.getName(); 
    Query query = pm.newQuery(select_query);
    query.setFilter("gaeuser == paramAuthor"); 
    query.declareParameters("java.lang.String paramAuthor"); 

    List<OmrUser> omruserquery = (List<OmrUser>) pm.newQuery(query).execute(user);

    if(omruserquery.iterator().hasNext()) {
        OmrUser thisomruser = omruserquery.iterator().next();
        if(thisomruser.getNumlocs() < MaxSaveLocs) {
            thisomruser.addSingleLoc(location);
            location.setOmruser(thisomruser);
            pm.makePersistent(location);    
            log.info("Adding Location to existing OMR User");
        } 
        else {
            msg = "You have exceeded the maximum number of locations you can save, please delete a location first.";
            log.info("FAIL - Exceed max num locs");
        }
    } 
    else {
        OmrUser newuser = new OmrUser(user, "Basic");
        newuser.addSingleLoc(location);
        pm.makePersistent(newuser);
        location.setOmruser(newuser);
        log.info("Created new user " + newuser.getGaeuser().getEmail());
    }            

    tx.commit();
} 
finally {
     if (tx.isActive()) {
         tx.rollback();
     }
}

ANOTHER UPDATE: Was able to replicate the error in local dev environment. still intermittent, however I was able to get this stack trace:

Jun 17, 2010 6:02:55 PM com.google.apphosting.utils.jetty.JettyLogger warn WARNING: /SaveLoc javax.jdo.JDODataStoreException: Datastore Failure at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:304) at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:132) at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56) at omr.SaveLoc.doPost(SaveLoc.java:74) at omr.SaveLoc.doGet(SaveLoc.java:24) at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:51) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:122) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:349) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) NestedThrowablesStackTrace: com.google.appengine.api.datastore.DatastoreFailureException: handle 1 not found at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:51) at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:67) at com.google.appengine.api.datastore.DatastoreServiceImpl$2.run(DatastoreServiceImpl.java:188) at com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java:30) at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:176) at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:156) at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:152) at org.datanucleus.store.appengine.RuntimeExceptionWrappingDatastoreService.put(RuntimeExceptionWrappingDatastoreService.java:105) at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:187) at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:139) at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:134) at org.datanucleus.store.appengine.DatastorePersistenceHandler.updateObject(DatastorePersistenceHandler.java:536) at org.datanucleus.state.JDOStateManagerImpl.flush(JDOStateManagerImpl.java:4576) at org.datanucleus.ObjectManagerImpl.flushInternal(ObjectManagerImpl.java:2814) at org.datanucleus.ObjectManagerImpl.flush(ObjectManagerImpl.java:2754) at org.datanucleus.ObjectManagerImpl.preCommit(ObjectManagerImpl.java:2893) at org.datanucleus.TransactionImpl.internalPreCommit(TransactionImpl.java:369) at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:256) at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83) at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56) at omr.SaveLoc.doPost(SaveLoc.java:74) at omr.SaveLoc.doGet(SaveLoc.java:24) at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:51) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:122) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:349) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/3050773/intermittent-issue-writing-to-google-appengine-datastore