Lagom usage with Coursier

Andrew Moreland

unread,

May 31, 2017, 6:39:01 PM5/31/17

to Lagom Framework Users

I have a lagom project with many sub-services. I've experienced two build pain points:

(1) when I run `sbt compile` I spend a lot of time waiting for dependency resolution -- each service

independently resolves all of its dependencies

(2) when I am working with intellij, I get a lot of contention for the ivy lock between intellij and my interactive sbt session

I've experimented with sbt's experimental subgraph resolution cache and this seems to help (1) a bit.

However, it's still slow and (2) is still a problem.

I was told by someone with more experience in the scala ecosystem to give Coursier (https://github.com/coursier/coursier)

a try since it removes the global ivy lock and it seems to resolve dependencies much more quickly.

I dropped it in to my sbt build and everything compiles fine. When I try to _run_ my project's integration tests I see:

[error] o.a.c.s.CassandraDaemon - Exception encountered during startup
java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.newDirectExecutorService()Lcom/google/common/util/concurrent/ListeningExecutorService;
at org.apache.cassandra.index.SecondaryIndexManager.<clinit>(SecondaryIndexManager.java:118)
at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:383)
at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:361)
at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:573)
at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:550)
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:305)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
at org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:887)
java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.newDirectExecutorService()Lcom/google/common/util/concurrent/ListeningExecutorService;
at org.apache.cassandra.index.SecondaryIndexManager.<clinit>(SecondaryIndexManager.java:118)
at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:383)
at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:361)
at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:573)
at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:550)
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:305)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
at org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:887)
at org.apache.cassandra.service.StartupChecks$8.execute(StartupChecks.java:304)
at org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:107)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:175)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:568)
at akka.persistence.cassandra.testkit.CassandraLauncher$.start(CassandraLauncher.scala:125)
at com.lightbend.lagom.scaladsl.testkit.ServiceTest$.startServer(ServiceTest.scala:192)
at money.haven.plaid.impl.PlaidItemServiceSpec.server$lzycompute(PlaidItemServiceSpec.scala:16)
at money.haven.plaid.impl.PlaidItemServiceSpec.server(PlaidItemServiceSpec.scala:16)
at money.haven.plaid.impl.PlaidItemServiceSpec.<init>(PlaidItemServiceSpec.scala:29)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:435)
at sbt.ForkMain$Run$2.call(ForkMain.java:296)
at sbt.ForkMain$Run$2.call(ForkMain.java:286)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

which, of course, fails the test.

The root cause seems to be that Guava 16 is ending up first on the classpath, while embedded Cassandra requires >= Guava 18.

I'm not sure how to actually address this in sbt / lagom, though. I've tried to do something like:

lazy val lagomSettings = Seq(
libraryDependencies ++= Seq(
lagomScaladslPersistenceCassandra,
lagomScaladslTestKit,
macwire
),
dependencyOverrides ++= Set(
"com.google.guava" % "guava" % "18.0"
)
)

so that everything that depends on embedded cassandra should see Guava >= 18, but this has had no effect.

Does anyone here have enough knowledge to recommend a path towards fixing this?

I know that this is not totally in-scope for Lagom, but it seems like there is community sentiment that Coursier is

significantly better than the status-quo SBT dependency resolver, and I expect that other people using lagom / coursier

will run into this problem.

Best,

Andy

Andrew Moreland

unread,

May 31, 2017, 8:35:24 PM5/31/17

to Lagom Framework Users

Ah. I realized that the problem had nothing to do with Lagom (aside from that it needs guava >= 18). I had a com.google.guava:guava-jdk5 transitive dependency which I didn't know about and this was bringing in the outdated classes. When I excluded this dependency everything started working.

Best,

Andy

rajesh krishnan

unread,

Apr 3, 2018, 8:24:04 AM4/3/18

to Lagom Framework Users [deprecated]

Hi Andy,

I am trying to achieve the similar effect of increasing the download speed by adding coursier. I have added coursier to plugins.sbt within the project but still no luck. Interested to know the steps you have taken to accomplish this.