play-ws NoSuchMethodError: io.netty.util.internal.PlatformDependent.newAtomicIntegerFieldUpdater (Netty dependency issue)

[ja...@roobricks.com]

I'm using a play-ws client to post requests to a Spray server endpoint that fronts a Spark driver program.  When I post using the injected ws, I get this error:

Caused by: java.lang.NoSuchMethodError: io.netty.util.internal.PlatformDependent.newAtomicIntegerFieldUpdater(Ljava/lang/Class;Ljava/lang/String;)Ljava/util/concurrent/atomic/AtomicIntegerFieldUpdater;

at org.asynchttpclient.netty.NettyResponseFuture.<clinit>(NettyResponseFuture.java:52)

at org.asynchttpclient.netty.request.NettyRequestSender.newNettyResponseFuture(NettyRequestSender.java:311)

at org.asynchttpclient.netty.request.NettyRequestSender.newNettyRequestAndResponseFuture(NettyRequestSender.java:193)

at org.asynchttpclient.netty.request.NettyRequestSender.sendRequestWithCertainForceConnect(NettyRequestSender.java:129)

at org.asynchttpclient.netty.request.NettyRequestSender.sendRequest(NettyRequestSender.java:107)

at org.asynchttpclient.DefaultAsyncHttpClient.execute(DefaultAsyncHttpClient.java:216)

at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:184)

at play.api.libs.ws.ahc.AhcWSClient.executeRequest(AhcWS.scala:45)

at play.api.libs.ws.ahc.AhcWSRequest$.execute(AhcWS.scala:90)

at play.api.libs.ws.ahc.AhcWSRequest$$anon$2.execute(AhcWS.scala:166)

at play.api.libs.ws.ahc.AhcWSRequest.execute(AhcWS.scala:168)

at play.api.libs.ws.WSRequest$class.post(WS.scala:510)

at play.api.libs.ws.ahc.AhcWSRequest.post(AhcWS.scala:107)

at webservices.DataFrameService.serializeDataset(DataFrameService.scala:36)

It appears that play-ws is picking up an older version of Netty?  A potentially salient point is that this error occurs when I compile the application against Spark-2.2-SNAPSHOT but does not occur when I compile the project against Spark-2.1.  The Spark-specific code is in a different project, however.

I'm including spark-ws in my project like so:

"com.typesafe.play" %% "play-ws" % "2.5.12"

Here is what i've tried in sbt to rectify:

1. Added an explicit ("io.netty" % "netty-all" % "4.0.43.Final") to my dependencies

2. Added exclude statements to the spark imports like so:

"org.apache.spark" %% "spark-sql" % sparkV exclude("org.jboss.netty", "netty") exclude ("io.netty", "netty")

3. Changed the order in which the play-ws module is added to the project dependencies (moved it to the end, moved it to the beginning)

[Will Sargent]

This is a common conflict when you use Spark, because Spark doesn't shade the version of Netty that it uses and so it conflicts with Play.

What we've done in Play 2.6.x is shade AHC's use of Netty, so if you're using Play-WS standalone it doesn't interfere:

https://github.com/playframework/play-ws#shading

However, if you are using Play with the Netty engine and include Spark, you'll still have the same problem.

--

Will Sargent

Engineer, Lightbend, Inc.

will.s...@lightbend.com

--
You received this message because you are subscribed to the Google Groups "Play Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/play-framework/264bbbd4-6d0c-45fe-8e87-6be9b4575e27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thibault Kruse]

For anyone having to manually manage the netty version (to resolve conflicts): The version is determined by this chain of dependencies:

com.typesafe.play:play-ws_2.11:2.5.14

|    +--- org.asynchttpclient:async-http-client:2.0.24

|    |    +--- org.asynchttpclient:async-http-client-netty-utils:2.0.24

|    |    |    +--- io.netty:netty-buffer:4.0.42.Final

To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.