Problems with sbt-assembly

[sapi...@gmail.com]

Hey guys,

Kamon.io looks really promising, but it's not usable in current form for me :)

I build artifacts with sbt-assembly. There are 2 problems:

1) Multiple aop.xml files. sbt-assebmly uses deduplicate strategy:

[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /home/ubuntu/.ivy2/cache/kamon/kamon-core/jars/kamon-core-0.0.14.jar:META-INF/aop.xml
[error] /home/ubuntu/.ivy2/cache/kamon/kamon-spray/jars/kamon-spray-0.0.14.jar:META-INF/aop.xml
[error] /home/ubuntu/.ivy2/cache/kamon/kamon-testkit/jars/kamon-testkit-0.0.14.jar:META-INF/aop.xml

I'm not AspectJ expert, so I'm not sure what is the solution here. Ideally I want sbt-assembly to be smart, so it can merge these files and get perfect XML as a result. But now it can only merge it in a very simple way, one by one, so we have incorrect XML.

2) reference.conf. For *.conf files sbt-assembly uses concat strategy. As a result (part of compiled reference conf with Akka, Spray and Kamon.io):

trace {
ask-pattern-tracing = off
}
}kamon {
spray {
include-trace-token-header = true
trace-token-header-name = "X-Trace-Token"
}
}######################################
# Akka Testkit Reference Config File #
######################################

And error:

Caused by: com.typesafe.config.ConfigException$Parse: reference.conf: 1456: Expecting end of input or a comma, got 'kamon' (if you intended 'kamon' to be part of the value for 'kamon', try enclosing the value in double quotes)

Solution: just add a comment in the beginning or empty line in the end of the every reference.conf file.

Anyway, thanks for cool library ;-)

[Ivan Topolnjak]

Hello sapiensy,

Thanks for using Kamon! We are sure that you will have fun while using it :).

1) Multiple aop.xml files. sbt-assebmly uses deduplicate strategy:

[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /home/ubuntu/.ivy2/cache/kamon/kamon-core/jars/kamon-core-0.0.14.jar:META-INF/aop.xml
[error] /home/ubuntu/.ivy2/cache/kamon/kamon-spray/jars/kamon-spray-0.0.14.jar:META-INF/aop.xml
[error] /home/ubuntu/.ivy2/cache/kamon/kamon-testkit/jars/kamon-testkit-0.0.14.jar:META-INF/aop.xml

I'm not AspectJ expert, so I'm not sure what is the solution here. Ideally I want sbt-assembly to be smart, so it can merge these files and get perfect XML as a result. But now it can only merge it in a very simple way, one by one, so we have incorrect XML.

We all wish sbt-assembly to be smart enough to merge XML files cleanly, but until that happens (if it ever happens) I can suggest you to create your own aop.xml file and ignore all other files. I manually merged the aop.xml files from kamon-core, kamon-spray and kamon-testkit for v0.0.14 and here is the resulting file contents:

<!DOCTYPE aspectj PUBLIC "-//AspectJ//DTD//EN" "http://www.eclipse.org/aspectj/dtd/aspectj.dtd">

<aspectj>
  <aspects>
    <!-- Actors -->
    <aspect name="akka.instrumentation.RepointableActorRefTraceContextMixin"/>
    <aspect name="akka.instrumentation.SystemMessageTraceContextMixin"/>
    <aspect name="akka.instrumentation.ActorSystemMessagePassingTracing"/>
    <aspect name="akka.instrumentation.EnvelopeTraceContextMixin"/>
    <aspect name="akka.instrumentation.BehaviourInvokeTracing"/>
    <aspect name="kamon.instrumentation.ActorLoggingTracing"/>

    <!-- Futures -->
    <aspect name="kamon.instrumentation.FutureTracing"/>

    <!-- Patterns -->
    <aspect name="akka.instrumentation.AskPatternTracing"/>

    <!-- Spray Server -->
    <aspect name="spray.can.server.ServerRequestInstrumentation"/>

    <!-- Spray Client -->
    <aspect name="spray.can.client.ClientRequestInstrumentation"/>

    <!-- TestKit -->
    <aspect name="akka.testkit.TestProbeInstrumentation"/>   

  </aspects>

  <weaver options="-verbose -showWeaveInfo">
    <include within="scala.concurrent..*"/>
    <include within="akka..*"/>
    <include within="spray..*"/>
    <include within="kamon..*"/>
  </weaver>

</aspectj>

That should be enough to bypass this issue.

 

2) reference.conf. For *.conf files sbt-assembly uses concat strategy. As a result (part of compiled reference conf with Akka, Spray and Kamon.io):

  trace {
    ask-pattern-tracing = off
  }
}kamon {
  spray {
    include-trace-token-header = true
    trace-token-header-name = "X-Trace-Token"
  }
}######################################
# Akka Testkit Reference Config File #
######################################

And error:

Caused by: com.typesafe.config.ConfigException$Parse: reference.conf: 1456: Expecting end of input or a comma, got 'kamon' (if you intended 'kamon' to be part of the value for 'kamon', try enclosing the value in double quotes)

Solution: just add a comment in the beginning or empty line in the end of the every reference.conf file.

Thanks for the suggestion, adding a short comment to each reference.conf file will do the trick, we are about to release Kamon 0.0.15, expect this little fix to be there :)

 

Anyway, thanks for cool library ;-)

Thanks! We appreciate that you take your time not only to try out Kamon, but also to suggest solutions to the problems you find, we hope to see you around often, best regards!

[lorenzo...@gmail.com]

You could also use this on your assembly.sbt:

// Create a new MergeStrategy for aop.xml files
val aopMerge: MergeStrategy = new MergeStrategy {
val name = "aopMerge"
import scala.xml._
import scala.xml.dtd._

def apply(tempDir: File, path: String, files: Seq[File]): Either[String, Seq[(File, String)]] = {
val dt = DocType("aspectj", PublicID("-//AspectJ//DTD//EN", "http://www.eclipse.org/aspectj/dtd/aspectj.dtd"), Nil)
val file = MergeStrategy.createMergeTarget(tempDir, path)
val xmls: Seq[Elem] = files.map(XML.loadFile)
val aspectsChildren: Seq[Node] = xmls.flatMap(_ \\ "aspectj" \ "aspects" \ "_")
val weaverChildren: Seq[Node] = xmls.flatMap(_ \\ "aspectj" \ "weaver" \ "_")
val options: String = xmls.map(x => (x \\ "aspectj" \ "weaver" \ "@options").text).mkString(" ").trim
val weaverAttr = if (options.isEmpty) Null else new UnprefixedAttribute("options", options, Null)
val aspects = new Elem(null, "aspects", Null, TopScope, false, aspectsChildren: _*)
val weaver = new Elem(null, "weaver", weaverAttr, TopScope, false, weaverChildren: _*)
val aspectj = new Elem(null, "aspectj", Null, TopScope, false, aspects, weaver)
XML.save(file.toString, aspectj, "UTF-8", xmlDecl = false, dt)
IO.append(file, IO.Newline.getBytes(IO.defaultCharset))
Right(Seq(file -> path))
}
}

// Use defaultMergeStrategy with a case for aop.xml
// I like this better than the inline version mentioned in assembly's README
val customMergeStrategy: String => MergeStrategy = {
case PathList("META-INF", "aop.xml") =>
aopMerge
case s =>
defaultMergeStrategy(s)
}

// Use the customMergeStrategy in your settings
mergeStrategy in assembly := customMergeStrategy

make sure you erase the newline between import scala.xml.dtd._ and def apply.

This piece of code is a courtesy from @colestanfield in github. The original link for the gist is:

https://gist.github.com/colestanfield/fac042d3108b0c06e952

Cheers.

[danilo...@gmail.com]

I tried to add that to assembly.sbt but got a bunch of sbt errors, mainly related to missing imports.
Can you help?


error: not found: type MergeStrategy
error: not found: value MergeStrategy
error: not found: value PathList
error: not found: value defaultMergeStrategy

[Lorenzo Fundaró]

Do you have the assembly plugin installed properly ? 

defaultMergeStrategy comes with assembly sbt. You should be able to find that value. 

Maybe recheck that you install assembly properly[1], and also add import AssemblyKeys._ on top of your assembly.sbt file.

[1] https://github.com/sbt/sbt-assembly

Lorenzo Fundaró García

[Danilo Tuler]

sbt-assembly is installed as described in the docs. I'm able to use it.

Tried to add import AssemblyKeys._, didn't work.

error: not found: object AssemblyKeys

Should I add the code snipped to assembly.sbt or build.sbt?

​Is there any example project?

Thanks

Danilo

[Lorenzo Fundaró]

You normally need to add the code snippet to assembly.sbt. But the fact that you're getting errors when importing AssemblyKeys makes me think that you need to check your classpath. Even though assembly plugin is installed, somehow sbt cannot find it when calling the import of AssemblyKeys. 

Lorenzo Fundaró García

[Lorenzo Fundaró]

Do you have assembly.sbt at the top of your project ? 

Lorenzo Fundaró García

[Danilo Tuler]

Yes.

I managed to make it work.

I put part of the code in a new 'project/Build.scala'.

And the strategy addition in build.sbt.

Thanks!

[Ivan Topolnjak]

@Danilo, thanks for sharing!

@Lorenzo, thanks for helping out!

Have a great day guys, and keep using Kamon :D, best regards!