Why does Datastore produce Timeout while fetching URL?

[Joshua Fox via StackOverflow]

For a short time today, my code running in Flexible Environment using Google Cloud Datastore API experienced java.net.SocketTimeoutException: Timeout while fetching URL on inserting an Item to Datastore in another GAE project.

In addition, Dataflow failed to insert Items at that time; almost certainly the same problem.

This occured from 13:37 to 13:57 UTC. Before and after this, a lot of other data did get inserted correctly; including a rerun on this same data.

Googling the error suggests it can be caused by downtime in the Google Cloud, but the Google Cloud Status Dashboard shows green.

What caused this? How can we avoid it in the future?

com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities run: ERROR in CopyEntities(commerceDocs/RFQ) java.lang.RuntimeException: com.google.cloud.datastore.DatastoreException: I/O error at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.retryIfAllowed(GCloudApiDSBackup.java:963) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntities(GCloudApiDSBackup.java:857) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.retryIfAllowed(GCloudApiDSBackup.java:959) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntities(GCloudApiDSBackup.java:857) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntitiesByParts(GCloudApiDSBackup.java:991) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.run(GCloudApiDSBackup.java:801) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at 
java.lang.Thread.run(Thread.java:745) Caused by: com.google.cloud.datastore.DatastoreException: I/O error at 
com.google.cloud.datastore.spi.DefaultDatastoreRpc.translate(DefaultDatastoreRpc.java:105) at 
com.google.cloud.datastore.spi.DefaultDatastoreRpc.commit(DefaultDatastoreRpc.java:133) at 
com.google.cloud.datastore.DatastoreImpl$4.call(DatastoreImpl.java:390) at 
com.google.cloud.datastore.DatastoreImpl$4.call(DatastoreImpl.java:387) at 
com.google.cloud.RetryHelper.doRetry(RetryHelper.java:179) at 
com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:244) at 
com.google.cloud.datastore.DatastoreImpl.commit(DatastoreImpl.java:386) at 
com.google.cloud.datastore.DatastoreImpl.commitMutation(DatastoreImpl.java:380) at 
com.google.cloud.datastore.DatastoreImpl.put(DatastoreImpl.java:340) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntities(GCloudApiDSBackup.java:836) ... 9 more Caused by: com.google.datastore.v1.client.DatastoreException: I/O error, code=UNAVAILABLE at 
com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:126) at 
com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:95) at 
com.google.datastore.v1.client.Datastore.commit(Datastore.java:84) at 
com.google.cloud.datastore.spi.DefaultDatastoreRpc.commit(DefaultDatastoreRpc.java:131) ... 17 more Caused by: java.net.SocketTimeoutException: Timeout while fetching URL: https://datastore.googleapis.com/v1/projects/freightos-prod-backup2:commit at 
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:173) at 
com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:45) at 
com.google.api.client.extensions.appengine.http.UrlFetchRequest.execute(UrlFetchRequest.java:74) at 
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981) at 
com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:87) ... 19 more

Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/40830727/why-does-datastore-produce-timeout-while-fetching-url