grpc-java on Raspberry Pi ARM
885 views
Skip to first unread message
warr...@embodied.me
Sep 27, 2017, 3:10:50 PM9/27/17
to grpc.io
Hi gRPC team,
I wanted to share my experience building gRPC on a Raspberry Pi model 3, linux ARM 32-bit. Reading previous posts, it simply wasn't such a popular HW :) I personally found gRPC to be great to use. It is a huge enabler for the robotic projects we're working on.
Summary of my open questions:
(1) does protoc-gen-javalite need to correspond with protoc version releases? The latest protoc-gen-javalite is 3.0.1 while protoc is 3.4? Does it have significant implications down the line?
(2) building netty-tcnative is an important pre-req. Should I raise a separate issue with netty-tcnative project about building on arm32? It seems like they only support 64bit platforms, but arm32 built successfully. Do you have any insights here?
(3) building protoc-gen-grpc-java on arm32 requires patching to grpc-java/compiler/build.gradle https://github.com/neo-titans/odroid/blob/master/build_tensorflow/grpc-java.v0.15.0.patch
(4) would the protobuf-gradle-plugin need a platform target flag for arm32/64? Below I ran into a problem where the locally built version resolved my compile errors, whereas no corresponding version was found in mavenCentral ?
A potential blocker is the dependency on building netty-tcnative on arm32 first (see pre-reqs below). But if there's a way, maybe we can work towards a (or PR) to add linux_arm-32 support?
---------------------------------------------------
My goal was to build and run the grpc-java/examples on Raspberry pi 3
pi@raspberrypi:~/gitcode/grpc-java/examples $ ./gradlew installDist
pi@raspberrypi:~/gitcode/grpc-java/examples $ ./gradlew test
---------------------------------------------------
Pre-reqs:
(1) Oracle JDK 8
(2) Expand swap, use a 1G file on /var/myswap for example
(3) Build netty-tcnative first
---------------------------------------------------
I ran into this error
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8:run (native-jar) on project netty-tcnative-boringssl-static: An Ant BuildException has occured: The directory /home/bolinux/Desktop/netty-tcnative/boringssl-static/target/native-lib-only/META-INF/native/linux64 does not exist
[ERROR] around Ant part ...<exec resolveexecutable="true" failonerror="true" dir="/home/bolinux/Desktop/netty-tcnative/boringssl-static/target/native-lib-only/META-INF/native/linux64/" executable="strip">... @ 11:184 in /home/bolinux/Desktop/netty-tcnative/boringssl-static/target/antrun/build-main.xml
---------------------------------------------------
I had to edit netty-tcnative/pom.xml
---------------------------------------------------
<archBits>64</archBits>
---> to
<archBits>32</archBits>
---------------------------------------------------
build protoc and protoc-gen-javalite for arm32
---------------------------------------------------
pi@raspberrypi:~/gitcode/protobuf $ git checkout v3.0.1-javalite
---------------------------------------------------
had to edit grpc/examples/build.gradle to point to local protoc=/usr/local/bin/protoc
---------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java/examples $ nano build.gradle
---------------------------------------------------
and retry installDist
---------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java/examples $ ./gradlew installDist
...........
Could not resolve all files for configuration ':protobufToolsLocator_grpc'.
> Could not find protoc-gen-grpc-java-linux-arm_32.exe (io.grpc:protoc-gen-grpc-java:1.6.1).
......
---------------------------------------------------
so build protoc-gen-grpc-java using patch from https://github.com/neo-titans/odroid/blob/master/build_tensorflow/grpc-java.v0.15.0.patch
---------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java/compiler $ patch -p0 < ~/0001-grpc-java-compiler-build.gradle-to-compile-on-armv7.patch
pi@raspberrypi:~/gitcode/grpc-java/compiler $ ../gradlew java_pluginExecutable
pi@raspberrypi:~/gitcode/grpc-java/compiler $ ../gradlew install
---------------------------------------------------
hitting compiler errors now
---------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java/examples $ ./gradlew installDist
.......
:compileJava/home/pi/gitcode/grpc-java/examples/build/generated/source/proto/main/java/io/grpc/examples/helloworld/HelloRequest.java:48: error: cannot find symbol
if (!parseUnknownFieldProto3(
^
symbol: method parseUnknownFieldProto3(CodedInputStream,Builder,ExtensionRegistryLite,int)
location: class HelloRequest
....................................................................
---------------------------------------------------
google search on the errors leads to the notion, other parts of grpc-java needs compiling onto arm7 ?
whatever is pulled in via mavenCentral isn't working?
so does this mean a full build of grpc-java? or just an assemble?
---------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java $ export CXXFLAGS="$(pkg-config --cflags protobuf)" LIBS="$(pkg-config --libs protobuf)"
pi@raspberrypi:~/gitcode/grpc-java $ ./gradlew build -PskipCodegen=true -Pprotoc=/usr/local/bin/protoc
---------------------------------------------------
get hit with the same "none of the libraries are loaded [netty-tcnative]"
---------------------------------------------------
:grpc-interop-testing:test[jetty-alpn-agent] Using: alpn-boot-8.1.6.v20151105.jar
io.grpc.testing.integration.Http2NettyTest > classMethod FAILED
java.lang.UnsatisfiedLinkError: failed to load the required native library
at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:329)
.............
at io.grpc.testing.integration.Http2NettyTest.startServer(Http2NettyTest.java:53)
Caused by:
java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty-tcnative-linux-arm_32, netty-tcnative-linux-arm_32-fedora, netty-tcnative, netty_tcnative]
---------------------------------------------------
Recall from above we've already built netty-tcnative for arm32. it's a problem of locating it in the local .m2 repo
so I'm going to add the arm32 classifier at the end
---------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java $ git diff build.gradle
diff --git a/build.gradle b/build.gradle
index 5a5ace91..8fec12bd 100644
--- a/build.gradle
+++ b/build.gradle
@@ -1,7 +1,7 @@
buildscript {
repositories {
- mavenCentral()
mavenLocal()
+ mavenCentral()
maven {
}
@@ -53,8 +53,8 @@ subprojects {
targetCompatibility = 1.6
repositories {
- mavenCentral()
mavenLocal()
+ mavenCentral()
}
[compileJava, compileTestJava].each() {
@@ -199,7 +199,7 @@ subprojects {
netty: "io.netty:netty-codec-http2:[${nettyVersion}]",
netty_epoll: "io.netty:netty-transport-native-epoll:${nettyVersion}" + epoll_suffix,
netty_proxy_handler: "io.netty:netty-handler-proxy:${nettyVersion}",
- netty_tcnative: 'io.netty:netty-tcnative-boringssl-static:2.0.5.Final',
+ netty_tcnative: 'io.netty:netty-tcnative-boringssl-static:2.0.6.Final-' + osdetector.classifier,
// Test dependencies.
junit: 'junit:junit:4.11',
----------------------------------------------------------
running again gives more hints on naming it properly:
----------------------------------------------------------
* What went wrong:
Could not resolve all files for configuration ':grpc-benchmarks:runtime'.
> Could not find io.netty:netty-tcnative-boringssl-static:2.0.6.Final-linux-arm_32.
Searched in the following locations:
file:/home/pi/.m2/repository/io/netty/netty-tcnative-boringssl-static/2.0.6.Final-linux-arm_32/netty-tcnative-boringssl-static-2.0.6.Final-linux-arm_32.pom
file:/home/pi/.m2/repository/io/netty/netty-tcnative-boringssl-static/2.0.6.Final-linux-arm_32/netty-tcnative-boringssl-static-2.0.6.Final-linux-arm_32.jar
Required by:
project :grpc-benchmarks
pi@raspberrypi:~/gitcode/grpc-java $ ls ~/.m2/repository/io/netty/netty-tcnative-boringssl-static/2.0.6.Final/
netty-tcnative-boringssl-static-2.0.6.Final.jar
netty-tcnative-boringssl-static-2.0.6.Final-linux-arm_32.jar
netty-tcnative-boringssl-static-2.0.6.Final.pom
netty-tcnative-boringssl-static-2.0.6.Final-sources.jar
_remote.repositories
----------------------------------------------------------
but I know the 2.0.6.Final.jar does not have the .so files, while the *Final-linux-arm_32.jar does
so the workaround I make is to copy and rename properly
----------------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java $ cp -R ~/.m2/repository/io/netty/netty-tcnative-boringssl-static/2.0.6.Final ~/.m2/repository/io/netty/netty-tcnative-boringssl-static/2.0.6.Final-linux-arm_32
-----------------------------------------------------------
-----------------------------------------------------------
$ cd protobuf/java
$ mvn test
$ mvn install
pi@raspberrypi:~/gitcode/protobuf/java $ ls ~/.m2/repository/com/google/protobuf/protobuf-
protobuf-java/ protobuf-java-util/ protobuf-lite/ protobuf-parent/
-----------------------------------------------------------
now back to grpc-java
-----------------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java $ ./gradlew build -PskipCodegen=true -Pprotoc=/usr/local/bin/protoc
:compileJava/home/pi/gitcode/grpc-java/examples/build/generated/source/proto/main/java/io/grpc/examples/helloworld/HelloRequest.java:48: error: cannot find symbol
if (!parseUnknownFieldProto3(
^
symbol: method parseUnknownFieldProto3(CodedInputStream,Builder,ExtensionRegistryLite,int)
location: class HelloRequest
/home/pi/gitcode/grpc-java/examples/build/generated/source/proto/main/java/io/grpc/examples/helloworld/HelloRequest.java:475: error: cannot find symbol
return super.setUnknownFieldsProto3(unknownFields);
^
-----------------------------------------------------------
Is it the gradle plugin not built locally on arm32, so these errors still come up?
-----------------------------------------------------------
567 git clone https://github.com/google/protobuf-gradle-plugin.git
568 cd protobuf-gradle-plugin/
573 ./gradlew install -Pprotoc=/usr/local/bin/protoc
pi@raspberrypi:~/gitcode/protobuf-gradle-plugin $ ls ~/.m2/repository/com/google/protobuf/protobuf-gradle-plugin/
0.8.4-SNAPSHOT/ maven-metadata-local.xml
-----------------------------------------------------------
So edit build.gradle under examples to point to the locally built plugin
-----------------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java/examples $ git diff build.gradle
diff --git a/examples/build.gradle b/examples/build.gradle
index ac676b02..e726c01e 100644
--- a/examples/build.gradle
+++ b/examples/build.gradle
@@ -3,18 +3,19 @@ apply plugin: 'com.google.protobuf'
buildscript {
repositories {
+ mavenLocal()
mavenCentral()
}
dependencies {
// ASSUMES GRADLE 2.12 OR HIGHER. Use plugin version 0.7.5 with earlier
// gradle versions
- classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.1'
+ classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.4-SNAPSHOT'
}
}
repositories {
- mavenCentral()
mavenLocal()
+ mavenCentral()
}
// IMPORTANT: You probably want the non-SNAPSHOT version of gRPC. Make sure you
@@ -37,7 +38,7 @@ dependencies {
protobuf {
protoc {
- artifact = 'com.google.protobuf:protoc:3.3.0'
+ path='/usr/local/bin/protoc'
}
plugins {
grpc {
-----------------------------------------------------------
Finally, success:
-----------------------------------------------------------
pi@raspberrypi:~/gitcode/grpc-java/examples $ ./gradlew installDist
:compressingHelloWorldClient UP-TO-DATE
:helloWorldClient UP-TO-DATE
:helloWorldServer UP-TO-DATE
:extractIncludeProto
:extractProto
:generateProto
:compileJava
:processResources
:classes
:jar
:routeGuideClient
:routeGuideServer
:startScripts SKIPPED
:installDist
BUILD SUCCESSFUL in 23s
12 actionable tasks: 9 executed, 3 up-to-date
pi@raspberrypi:~/gitcode/grpc-java/examples $ ./gradlew test
:extractIncludeProto UP-TO-DATE
:extractProto UP-TO-DATE
:generateProto UP-TO-DATE
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:extractIncludeTestProto
:extractTestProto
:generateTestProto NO-SOURCE
:compileTestJavaNote: /home/pi/gitcode/grpc-java/examples/src/test/java/io/grpc/examples/header/HeaderServerInterceptorTest.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processTestResources NO-SOURCE
:testClasses
:test*** shutting down gRPC server since JVM is shutting down
*** shutting down gRPC server since JVM is shutting down
*** shutting down gRPC server since JVM is shutting down
*** server shut down
*** server shut down
*** shutting down gRPC server since JVM is shutting down
*** server shut down
*** server shut down
BUILD SUCCESSFUL in 26s
9 actionable tasks: 4 executed, 5 up-to-date
Thanks,
Warren
keywords: raspberry pi, arm32, arm64, armv7l
Carl Mastrangelo
Oct 3, 2017, 12:27:46 AM10/3/17
to grpc.io
Responses inline
On Wednesday, September 27, 2017 at 12:10:50 PM UTC-7, warr...@embodied.me wrote:
Hi gRPC team,I wanted to share my experience building gRPC on a Raspberry Pi model 3, linux ARM 32-bit. Reading previous posts, it simply wasn't such a popular HW :) I personally found gRPC to be great to use. It is a huge enabler for the robotic projects we're working on.Summary of my open questions:(1) does protoc-gen-javalite need to correspond with protoc version releases? The latest protoc-gen-javalite is 3.0.1 while protoc is 3.4? Does it have significant implications down the line?
No, but int he future protolite will align with the proto versions. I think it is being improved internally before catching up with the main proto version.
(2) building netty-tcnative is an important pre-req. Should I raise a separate issue with netty-tcnative project about building on arm32? It seems like they only support 64bit platforms, but arm32 built successfully. Do you have any insights here?
You can, but I think the better solution is using Conscrypt, which was made for android devices. (typically arm). I don't know if its ready yet, but it will be the eventual way for using SSL with grpc java
(3) building protoc-gen-grpc-java on arm32 requires patching to grpc-java/compiler/build.gradle https://github.com/neo-titans/odroid/blob/master/build_tensorflow/grpc-java.v0.15.0.patch
We (developers) don't have a lot of devices, so it's hard to test and maintain this. We are kind of stuck on this front. Perhaps Bazel will be the prefered build system and solve this.
(4) would the protobuf-gradle-plugin need a platform target flag for arm32/64? Below I ran into a problem where the locally built version resolved my compile errors, whereas no corresponding version was found in mavenCentral ?
I think it picks one of the prebuilt binaries (for x86 or amd64) based on your platform, which is why it doesn't work. Again, we don't really have the devices to test this, which is why it doesn't exist.
Reply all
Reply to author
Forward
0 new messages