Store data to Cloud storage In GAE Managed VMs

74 views
Skip to first unread message

Wang Steven

unread,
Jul 27, 2015, 12:14:12 PM7/27/15
to Google App Engine
Running app in GAE Managed VMs.
appengine version : 1.9.24
appengine-gcs-client version : 0.4.4

code : 
 private final GcsService gcsService = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
      @SuppressWarnings("resource")
      GcsOutputChannel outputChannel = gcsService.createOrReplace(fileName, GcsFileOptions.getDefaultInstance());
      outputChannel.write(ByteBuffer.wrap(bytes));
      outputChannel.close();
com.google.appengine.tools.cloudstorage.RetriesExhaustedException: RetryHelper(5.619 s, 10 attempts, com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$1@721b2ad5): Too many failures, giving up
    at com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:127)
    at com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:166)
    at com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:156)
    at com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.close(GcsOutputChannelImpl.java:198)
    at java.nio.channels.Channels$1.close(Channels.java:178)
    at java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1827)
    at java.io.ObjectOutputStream.close(ObjectOutputStream.java:741)
    at com.deepnet.idigisign.services.google.impl.GoogleCloudStorageServiceImpl.writeObjectToFile(GoogleCloudStorageServiceImpl.java:55)
    at com.deepnet.idigisign.services.google.impl.GoogleCloudStorageServiceImpl.write(GoogleCloudStorageServiceImpl.java:123)
    at com.deepnet.idigisign.services.document.impl.DocumentServiceImpl.setTemplateDocuments(DocumentServiceImpl.java:148)
    at com.deepnet.idigisign.services.template.impl.TemplateServiceImpl.postTemplateDocuments(TemplateServiceImpl.java:114)
    at com.deepnet.idigisign.rest.template.impl.TemplateProviderImpl.postTemplateDocuments(TemplateProviderImpl.java:101)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
    at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
    at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
    at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
    at com.deepnet.idigisign.filters.ApiOriginFilter.doFilter(ApiOriginFilter.java:25)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.deepnet.idigisign.filters.OauthOriginFilter.doFilter(OauthOriginFilter.java:49)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.deepnet.idigisign.filters.UserIdentificationServiceFilter.doFilter(UserIdentificationServiceFilter.java:42)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at com.google.apphosting.runtime.jetty9.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
    at com.google.apphosting.vmruntime.jetty9.VmRuntimeWebAppContext.doScope(VmRuntimeWebAppContext.java:461)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.google.appengine.api.files.FinalizationException
    at com.google.appengine.api.files.FileServiceImpl.translateException(FileServiceImpl.java:617)
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:594)
    at com.google.appengine.api.files.FileServiceImpl.open(FileServiceImpl.java:527)
    at com.google.appengine.api.files.FileServiceImpl.openForAppend(FileServiceImpl.java:471)
    at com.google.appengine.api.files.FileServiceImpl.openForAppend(FileServiceImpl.java:190)
    at com.google.appengine.api.files.FileServiceImpl.openWriteChannel(FileServiceImpl.java:171)
    at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.append(LocalRawGcsService.java:181)
    at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.finishObjectCreation(LocalRawGcsService.java:245)
    at com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$1.call(GcsOutputChannelImpl.java:201)
    at com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$1.call(GcsOutputChannelImpl.java:198)
    at com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:108)
    ... 76 more
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 101: File is already finalized
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.convertRemoteError(VmApiProxyDelegate.java:276)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.runSyncCall(VmApiProxyDelegate.java:174)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeApiCall(VmApiProxyDelegate.java:141)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeSyncCallWithTimeout(VmApiProxyDelegate.java:115)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeSyncCall(VmApiProxyDelegate.java:104)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeSyncCall(VmApiProxyDelegate.java:47)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:112)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:64)
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:590)
    ... 85 more

Can someone tell me where it all went wrong?

kcc...@gmail.com

unread,
Jul 27, 2015, 6:42:35 PM7/27/15
to Google App Engine, wzl1...@gmail.com
Have you create your bucket in GCS?

Nick (Cloud Platform Support)

unread,
Jul 28, 2015, 11:08:28 AM7/28/15
to Google App Engine, wzl1...@gmail.com, wzl1...@gmail.com
Hi Steven,

Google Groups isn't the place to post specific technical issues, as this forum is meant more for general discussion of the platform and services. 

If you would like help with a technical issue, you should post to stackoverflow [1] or serverfault [2]. 

If you believe you've identified and can reproduce an issue with the platform itself (behaviour is different from documentation or error occurs during normal use), then you should proceed to open a public issue tracker [3] issue with enough detail to reproduce the issue on our side, or if possible, an attached app that can be used to directly observe the behaviour. 

Your issue report contains not enough information to determine what is happening, even in the event that you did post it to one of those other locations, so I suggest adding more detail before doing so. Some of that information might be:

* What kind of data are you sending, how large is it
* The full code context
* Details about what steps you've taken to set up
* Details about what you have tried to fix, what you haven't tried, what you think might be relevant, etc.

If you would like to open a thread in this forum discussing the platform or services in more broad terms, starting a discussion that would be useful for other users to join in to, feel free to do so.

Have a great day!

Reply all
Reply to author
Forward
0 new messages