"Failed to load any of the given libraries" using netty-tcnative-boringssl-static

[rgon...@sevone.com]

I'm trying to make a grpc client in Scala and am using the statically linked openssl provided by netty-tcnative-boringssl-static dependency. I checked the dependency tree and believe I have all compatible versions of grpc-netty (1.6.1), netty-codec-http2 (4.1.14.Final), and netty-tcnative-boringssl-static (2.0.5.Final). I'm testing on osx, with apr 1.6.3.

My main() function first checks whether OpenSSL is available as follows:

def main(args: Array[String]): Unit = {
  val hasOpenSsl = io.netty.handler.ssl.OpenSsl.isAvailable()
  println("has openssl? " + hasOpenSsl)
  println("stack trace: " + io.netty.handler.ssl.OpenSsl.unavailabilityCause().printStackTrace())

Running the app produces:

has openssl? false
java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty-tcnative-osx-x86_64, netty-tcnative, netty_tcnative]
    at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:180)
    at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:423)
    at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:91)
    at com.sevone.analytics.drivers.FoundationConsumerDriver$.main(FoundationConsumerDriver.scala:9)
    at com.sevone.analytics.drivers.FoundationConsumerDriver.main(FoundationConsumerDriver.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at scala.reflect.internal.util.ScalaClassLoader.$anonfun$run$2(ScalaClassLoader.scala:99)
    at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:34)
    at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:30)
    at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:125)
    at scala.reflect.internal.util.ScalaClassLoader.run(ScalaClassLoader.scala:99)
    at scala.reflect.internal.util.ScalaClassLoader.run$(ScalaClassLoader.scala:91)
    at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:125)
    at scala.tools.nsc.CommonRunner.run(ObjectRunner.scala:22)
    at scala.tools.nsc.CommonRunner.run$(ObjectRunner.scala:21)
    at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
    at scala.tools.nsc.CommonRunner.runAndCatch(ObjectRunner.scala:29)
    at scala.tools.nsc.CommonRunner.runAndCatch$(ObjectRunner.scala:28)
    at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
    at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:66)
    at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:85)
    at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:96)
    at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:101)
    at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
stack trace: ()

Subsequent to this, NettyChannelBuilder produces the "Jetty ALPN/NPN has not been properly configured" exception.

My main question is: do I need to do some setup before checking for calling OpenSsl.isAvailable(), or am I missing something else?

Thanks for any help!!

---

SevOne, Inc. reserves the right to monitor the transmission of this message and to take corrective action against any misuse or abuse of its e-mail system or other components of its network.

The information contained in this e-mail may be confidential and/or legally privileged. It is intended solely for the addressee.  If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited and may be unlawful. If you have received this communication in error, please reply to the sender and destroy all copies of the message.  To contact us directly, send to postm...@sevone.com. 

[Norman Maurer]

Is this the whole stack trace ? I would expect there is more.

--
You received this message because you are subscribed to the Google Groups "Netty discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netty+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/19cc71e8-64fc-4324-982c-e08412b886c2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rgon...@sevone.com]

I posted this after spending 2 days tracking the problem down, and of course found the issue right after posting it...

We are using the sbt-assembly plugin to create a fat jar file for Spark deployment. The assemblyMergeStrategy was discarding the META-INF directory from the netty-tcnative-boringssl-static-2.0.5.Final.jar file, which happens to contain the native ssl libraries. The solution was to update the merge strategy in build.sbt as follows. Hope this helps someone else.

val uberJarMerge = new sbtassembly.MergeStrategy {
  override def name: String = "uberJarMerge"
  override def apply(tempDir: File, path: String, files: Seq[File]): Either[String, Seq[(File, String)]] = {
    Right(
      for {
        // collects not jar-origin files plus jar-origin "native" directory
        f <- files if (!AssemblyUtils.sourceOfFileForMerge(tempDir, f)._4) || (f.getPath contains "/native/")
      } yield {
        f -> path
      }
    )
  }
}

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs @ _*) => uberJarMerge
  case x => MergeStrategy.first
}

[Norman Maurer]

Ah this makes sense…

Thanks for closing the loop.

Norman

---

SevOne, Inc. reserves the right to monitor the transmission of this message and to take corrective action against any misuse or abuse of its e-mail system or other components of its network.

The information contained in this e-mail may be confidential and/or legally privileged. It is intended solely for the addressee.  If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited and may be unlawful. If you have received this communication in error, please reply to the sender and destroy all copies of the message.  To contact us directly, send to postm...@sevone.com. 

--
You received this message because you are subscribed to the Google Groups "Netty discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/e84d8d38-f1d4-4a85-864a-be95c873f1f4%40googlegroups.com.