Building grpc-java on Power8

[nishi...@gmail.com]

Hi,

I'm trying to build grpc-java on Power8 in order to build bazel 0.3.0 (need protoc-gen-grpc-java built on power). However, I'm getting below errors while doing so -

A problem occurred configuring project ':grpc-compiler'.

> Exception thrown while executing model rule: NativeComponentRules#createBinaries

> Invalid NativePlatform: ppcle_64

Could anyone please guide me on how to build grpc-java on Power? I've also gone through a similar post for "arm" but it didn't help. Looks like some other places also need to be modified in order to add support for Power. Not sure if skipcodegen=true is expected in my case, but I've also tried that and it also fails for missing netty-tcnative-boring-ssl jar for Power.

Thanks in advance,

Nishidha

[Eric Anderson]

This is probably similar to a previous discussion.

We've not done any work to build the native grpc-java components on Power or ARM (and Android doesn't count, since that stack doesn't need any native components to be built).

For building grpc-compiler, all that may be necessary is to add a new platform to the list we have. This is very gradle-specific, and you might need to reference the docs. But I'd really expect very few changes necessary (like up to 5 lines). If you get it working, we'd be happy to accept a PR. We've talked with Bazel devs in the past; we may want to talk to them some more to make sure that using Bazel on those platforms is smooth.

netty-tcnative-boring-ssl doesn't have pre-built binaries either for ARM/Power. However, I question if they are necessary. I don't think bazel is using TLS (since it is local-only communication). I wouldn't be surprised if they needed netty-epoll or something like that though, since that allows using Unix Domain Sockets. There are docs on how to build tcnative, if it turns out you need it.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/1b0d3426-61fc-4089-84dd-fef0a5b7c495%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nishi...@gmail.com]

Hi Eric,

Thanks a ton. I've made changes for ppc referring all the docs and links you shared. And I'm able to build grpc-compiler successfully. Although further I get error for netty-tcnative-boring-ssl but the executable I need gets already generated before this error. Still, I tried building netty-tcnative too but it still had problems even after fixing many. So, I left it there.

For PR, let me check with IBM legal team if we've CLA.

I'll get back to you.

Thanks again,

Nishidha

[dw5...@gmail.com]

Nishidha - 

ran into this post and am looking to get ARM working - specifically aarch64. Can you share what process/changes you took to get things working? Am more than happy to do PR and include ppcle_64 as well if you can give me directional guidance on what to do.

Thanks!

[nishi...@gmail.com]

Hello,

I'd actually built only grpc-compiler project from grpc-java which I needed for building Bazel 0.3. Entire grpc-java is yet to be built completely. I'm still seeing one test failure for netty-tcnative not being able to be loaded, although I built it from source locally. So, currently working on this issue. I shall get back to you with the working changes as soon as possible.

Thanks,

Nishidha

[nishi...@gmail.com]

One more thing, for ARM platform, I see two github issues which Alex Ryan has reported and also successfully built grpc-java. 

https://github.com/grpc/grpc-java/issues/2198

https://github.com/grpc/grpc-java/issues/2202

Hope these are helpful or you may want to talk to Alex.

Regards,

Nishidha

On Thursday, 14 July 2016 17:15:24 UTC+5:30, nishi...@gmail.com wrote:

[dwight]

Awesome. I eventually got grpc-java/compiler to build for 'arm64' with the below changes. This said, it still crashes on ../gradlew test

Diff:

```

diff --git a/compiler/build.gradle b/compiler/build.gradle

index 7e116d4..4183a59 100644

--- a/compiler/build.gradle

+++ b/compiler/build.gradle

@@ -47,6 +47,9 @@ model {

       }

     }

     gcc(Gcc) {

+       target("linux_aarch64"){

+           cppCompiler.executable = "/usr/bin/gcc"

+       }

     }

     clang(Clang) {

     }

@@ -59,11 +62,15 @@ model {

     x86_64 {

       architecture "x86_64"

     }

+    linux_aarch64 {

+      architecture "arm64"

+      operatingSystem "linux"

+    }

   }

 

   components {

     java_plugin(NativeExecutableSpec) {

-      if (arch in ['x86_32', 'x86_64']) {

+      if (arch in ['x86_32', 'x86_64', 'arm64']) {

         // If arch is not within the defined platforms, we do not specify the

         // targetPlatform so that Gradle will choose what is appropriate.

         targetPlatform arch

```

result of ..gradlew test

```

FAILURE: Build failed with an exception.

* What went wrong:

Could not resolve all dependencies for configuration ':grpc-compiler:protobufToolsLocator_javalite'.

> Could not find protoc-gen-javalite-linux-aarch_64.exe (com.google.protobuf:protoc-gen-javalite:3.0.0).

  Searched in the following locations:

      https://repo1.maven.org/maven2/com/google/protobuf/protoc-gen-javalite/3.0.0/protoc-gen-javalite-3.0.0-linux-aarch_64.exe

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

```

Any idea what that means? the result output to build bazel 0.3.0 also fails, so think something isn't configured right

--
You received this message because you are subscribed to a topic in the Google Groups "grpc.io" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/grpc-io/DpsdI6kSxJI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/353a482b-83c1-42b5-be15-fd78bd1fae65%40googlegroups.com.

[nishi...@gmail.com]

This error means, it needs protoc-gen-javalite binary built on arm as you've one for protoc. Build google/protobuf's tag "v3.0.0-javalite" that will generate the required binary and install it. And then run ./gradlew build can be run with -Pprotoc-gen-javalite=<pathto/protoc-gen-javalite>.

Other changes look good for adding support on arm.

Regards,

Nishidha

On Thursday, 14 July 2016 17:15:24 UTC+5:30, nishi...@gmail.com wrote:

[nishi...@gmail.com]

Hello Eric,

I'm still stuck with error while running command 

./gradlew build -Pprotoc=/usr/local/bin/protoc -Pprotoc-gen-javalite=/usr/local/bin/protoc-gen-javalite

I'm seeing below error despite building netty-tcnative on Power -

:grpc-interop-testing:test[jetty-alpn-agent] Using: alpn-boot-8.1.7.v20160121.jar

io.grpc.testing.integration.Http2NettyTest > classMethod FAILED

    java.lang.UnsatisfiedLinkError: failed to load the required native library

        at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:241)

        at io.netty.handler.ssl.OpenSslContext.<init>(OpenSslContext.java:169)

        at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:347)

        at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:337)

        at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:413)

        at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:391)

        at io.grpc.testing.integration.Http2NettyTest.startServer(Http2NettyTest.java:70)

        Caused by:

        java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty-tcnative-linux-ppcle_64, netty-tcnative-linux-ppcle_64-fedora, netty-tcnative]

            at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:167)

            at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:325)

            at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:90)

            at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:137)

            at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:119)

            at io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:100)

            at io.grpc.testing.integration.Http2NettyTest.startServer(Http2NettyTest.java:65)

212 tests completed, 1 failed, 4 skipped

:grpc-interop-testing:test FAILED

I tried copying the generated jar of netty-tcnative at $HOME/.gradle/cache/modules/file*/io.netty/netty-tcnative. Could you please help me with this setting?

Thanks,

Nishidha

On Thursday, 14 July 2016 17:15:24 UTC+5:30, nishi...@gmail.com wrote:

[nishi...@gmail.com]

Hi,

I got it working finally. Please see my comments in this issue on github for the solution. I've also generated a PR for Power specific changes.

Thanks,

Nishidha 

On Thursday, 14 July 2016 17:15:24 UTC+5:30, nishi...@gmail.com wrote: