Stability of Java 8 on Standard Environment

1,106 views
Skip to first unread message

Patrick Jackson

unread,
Jul 6, 2017, 3:45:38 PM7/6/17
to Google App Engine
Super excited to see Java 8 support.  The removal of the white-list is also huge!  Curious about other's experiences with it so far.  Any instability, things to note or consider.
I know this is beta and no SLA until GA and all that, just would be interested to hear from any devs that have been using it for real apps and traffic.

-Patrick Jackson

Jeff Schnitzer

unread,
Jul 7, 2017, 11:12:43 AM7/7/17
to Google App Engine
I’m using it for a real (reasonably complicated, including guice with a lot of AOP) app, although very light traffic. AFAICT it works as advertised.

Jeff

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/bcbfcc51-120b-4dc4-baf3-fdd616e310ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Emanuele Ziglioli

unread,
Jul 8, 2017, 5:00:02 AM7/8/17
to Google App Engine, je...@infohazard.org
I've been amazed: I've deployed it three days ago and utilization has dropped to the floor.



Instances, no more than one at the time. I'm now thinking of reducing the size of the instance


I'm only experiencing one small problem on the development local server when loading files from GCS whose name contains a colon character.
Other than that, I'm running it using the java command line, not using the Eclipse plugin, I've got more control on where to run it from.

Better late, than never!!

E.

On Saturday, 8 July 2017 03:12:43 UTC+12, Jeff Schnitzer wrote:
I’m using it for a real (reasonably complicated, including guice with a lot of AOP) app, although very light traffic. AFAICT it works as advertised.

Jeff
On Thu, Jul 6, 2017 at 12:45 PM, Patrick Jackson <patjac...@gmail.com> wrote:
Super excited to see Java 8 support.  The removal of the white-list is also huge!  Curious about other's experiences with it so far.  Any instability, things to note or consider.
I know this is beta and no SLA until GA and all that, just would be interested to hear from any devs that have been using it for real apps and traffic.

-Patrick Jackson

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Patrick Jackson

unread,
Jul 11, 2017, 9:18:03 AM7/11/17
to Google App Engine, je...@infohazard.org
That's sounds wonderful. Thanks for the reply.  Do you know exactly what caused the drop in utilization?  Did you change other things than upgrading to Java 8?

-Patrick

You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/2UTg4Eqskyk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

Emanuele Ziglioli

unread,
Jul 11, 2017, 5:17:52 PM7/11/17
to Google App Engine, je...@infohazard.org
No other changes, I swear! Our traffic is pretty steady, we've been paying pretty much the same bill for years.
Yesterday I reduced the instance size from F4 to F2 and so far I've seen no errors.
Typically one of my tasks would hit the 10m limit.
Not sure what caused the performance improvement, we should ask Ludo.
Surely some generic speedup due to simply upgrading the runtime. Perhaps the security sandbox in the Java 7 runtime was slowing everything down.

E.

Thomas Wiradikusuma

unread,
Jul 11, 2017, 9:00:04 PM7/11/17
to Google App Engine
I noticed that my app is 0.5mb smaller but takes longer to load and uses more memory. This is simply by comparing data from GCP console (open new tab of same console page, refresh, see what's different).

Ludovic Champenois

unread,
Jul 12, 2017, 9:24:06 AM7/12/17
to google-a...@googlegroups.com, Thomas Wiradikusuma
Hi,

Thanks for the feedback.

A newer push (soon) will address the load time issue that is related to
a wrong timeout value,

Cheers,

Ludo

Robert Dyas

unread,
Jul 13, 2017, 3:22:27 PM7/13/17
to Google App Engine
Glitches: Main glitch so far is that we needed to set  <url-stream-handler>urlfetch</url-stream-handler> in our appengine-web.xml file for some reason. Still trying to figure out why we can make the request on java7 and 8 via urlfetch, but it doesn't work on native on java8. We could be doing something stupid.... but can't figure it out yet...

Performance: for our use case (hitting databases on GCE) it is much, much faster than java7. Huge improvement not only in speed, but in consistency of speed. We suspected that the security manager was causing a lot of socket overhead under java7... so maybe the lack of a security manager in java8 is why our particular use case runs so much better.

Torbjørn Smørgrav

unread,
Jul 17, 2017, 3:03:53 AM7/17/17
to Google App Engine
One thing I have experienced with Java8 is that the mail service seems to only handle attachments smaller than 3.5MB. 

Robert Dyas

unread,
Jul 17, 2017, 12:05:58 PM7/17/17
to google-a...@googlegroups.com
Also using the native java8 https request, it throws an exception if you call conn.getInputStream() and the response code is 400 series. That code worked fine in java7 and java8 with urlfetch, but in java8 native you have to call conn.getErrorStream() after testing the response code to know what method to call to get the body.


On Thursday, July 6, 2017 at 3:45:38 PM UTC-4, Patrick Jackson wrote:

Christian Maan

unread,
Jul 20, 2017, 12:37:56 PM7/20/17
to Google App Engine
We switched to the java8 runtime recently, and for the most part didn't experience any major issues (yet). Performance did definitely improve for us, compared to java7. On standard environment, we use a bunch of Google cloud services, including Datastore, Vision API, Storage, Compute Engine, Memcache, Pubsub, UserService, SearchService and BigQuery. All of them seem to work without issues after upgrading to java8.

Currently, we try to make use if some of the new grpc feature java8 is advertised with. Unfortunately, things seem to be considerably more unstable there.

Based on the example here [1], we tried to get Spanner to work on java8, using the ideomatic Java client libraries from google-cloud-java. The example itself executes without problems, but any slight variation in code structure will throw off the grpc initialization code and break the example. We tracked the problem down to GrpcUtils.TIMER_SERVICE and GrpcUtils.SHARED_CHANNEL_EXECUTOR. In the official example, classloading side effects seem to determine the type of ThreadFactory that backs these final static fields. If the effective ThreadFactory turns out to be a ThreadManager.currentRequestThreadFactory(), because the initialization happened in a context where MoreExecutors.isAppEngine() returns true, everything falls apart and we are left with a NullPointerException in ApiProxyImpl:1267 later on, when the Spanner session pool tries to create a session [2].

If we - somewhat brutally - force GrpcUtils.TIMER_SERVICE and GrpcUtils.SHARED_CHANNEL_EXECUTOR to use a Executors.defaultThreadFactory() like this [3], we get a working grpc Spanner setup on Appengine Standard!

So java8 looks promising, but from this experience we think it still needs some work.

- c


Message has been deleted

Kenworth (Google Cloud Platform)

unread,
Jul 21, 2017, 8:25:09 PM7/21/17
to Google App Engine

Thank you all for the very interesting discussion points. Please note that if your post is either a suspected bug and/or feature request, it is highly recommended to file this on Public Issue Tracker for better tracking purposes.


For technical questions you can post on StackOverflow or Serverfault. Please visit our community support page for the list of tags we monitor. For general discussions about products and release notes, you can always post here on Google Groups.


James Baldassari

unread,
Jul 26, 2017, 11:02:30 AM7/26/17
to Google App Engine
Christian, thanks for that interesting (and kind of scary) work-around (https://gist.github.com/cmaan/7752e3c4fd0b1ba90a745cb6db232206).  I was having the same problem but with the PubSub client rather than the Spanner client.  By switching the thread factory I was able to avoid the NPE at ApiProxyImpl.java:1267, but now I'm getting a slightly different problem:

Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED at io.grpc.Status.asRuntimeException(Status.java:543) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:442) ~[grpc-stub-1.4.0.jar:1.4.0] at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:76) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:512) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:429) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:544) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52) ~[grpc-core-1.4.0.jar:1.4.0] at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:117) ~[grpc-core-1.4.0.jar:1.4.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_112-google-v7] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_112-google-v7] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_112-google-v7] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:295) ~[na:1.8.0_112-google-v7] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_112-google-v7] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_112-google-v7] ... 1 common frames omitted Caused by: java.io.IOException: Could not get the access token. at com.google.auth.oauth2.AppEngineCredentials.refreshAccessToken(AppEngineCredentials.java:136) ~[google-auth-library-oauth2-http-0.7.0.jar:na] at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:149) ~[google-auth-library-oauth2-http-0.7.0.jar:na] at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:135) ~[google-auth-library-oauth2-http-0.7.0.jar:na] at io.grpc.auth.GoogleAuthLibraryCallCredentials$1.run(GoogleAuthLibraryCallCredentials.java:110) ~[grpc-auth-1.4.0.jar:1.4.0] ... 7 common frames omitted Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112-google-v7] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112-google-v7] at com.google.auth.oauth2.AppEngineCredentials.refreshAccessToken(AppEngineCredentials.java:131) ~[google-auth-library-oauth2-http-0.7.0.jar:na] ... 10 common frames omitted Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call memcache.Get in a thread that is neither the original request thread nor a thread created by ThreadManager at com.google.apphosting.api.ApiProxy$CallNotFoundException.foreignThread(ApiProxy.java:844) ~[runtime-shared.jar:na] at com.google.apphosting.api.ApiProxy$1.get(ApiProxy.java:183) ~[runtime-shared.jar:na] at com.google.apphosting.api.ApiProxy$1.get(ApiProxy.java:180) ~[runtime-shared.jar:na] at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:93) ~[appengine-api.jar:na] at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28) ~[appengine-api.jar:na] at com.google.appengine.api.memcache.MemcacheServiceImpl.get(MemcacheServiceImpl.java:51) ~[appengine-api.jar:na] at com.google.appengine.api.appidentity.AppIdentityServiceImpl.getAccessToken(AppIdentityServiceImpl.java:300) ~[appengine-api.jar:na] ... 14 common frames omitted

I'll keep investigating.  Maybe there is another thread factory that needs to be updated somewhere...

James Baldassari

unread,
Jul 26, 2017, 11:13:00 AM7/26/17
to Google App Engine

Christian Maan

unread,
Jul 26, 2017, 11:22:13 AM7/26/17
to Google App Engine
Ah, I just found your issue on github and was writing up a reply with the same workaround.

Automagic authentication does not work in a plain java thread pool, to avoid `UNAUTHENTICATED` errors you would need to use a PubSub client with explicit credentials, e.g.:
```
Publisher.defaultBuilder(topicName)
.setCredentialsProvider(/* ... */)
.build()
```

For reference, I will also add a comment to your issue https://github.com/GoogleCloudPlatform/google-cloud-java/issues/2275

Kenworth (Google Cloud Platform)

unread,
Jul 26, 2017, 7:25:12 PM7/26/17
to Google App Engine
@James  I can see that you are already getting responses from my colleague(s) on GitHub, you can continue troubleshooting on your GitHub thread or any one of the relevant links you mentioned (link1, link2, link3). If no resolution is viable, please feel free to update your Public Issue Tracker case providing reproducible steps and/or providing a sandbox project.

James Baldassari

unread,
Jul 27, 2017, 1:11:26 PM7/27/17
to Google App Engine

I was able to find a work-around to be able to publish to a PubSub topic in the Java 8 standard environment until a permanent fix is released: https://github.com/GoogleCloudPlatform/google-cloud-java/issues/2275


It's necessary to first obtain credentials from within an App Engine request thread. Then, pass those credentials to a new Publisher that is created inside a non-request thread, such as one created by a Java ExecutorService. This combination works but is not very efficient.

Reply all
Reply to author
Forward
0 new messages