grpc make can't find protoc 3 (Mac OS Mavericks)

[Thad Humphries]

I posted this last night on Stackoverflow . Since then I've worked a bit more with it (and caught one error in my post) but I still cannot complete an install of grpc for C and C ++.

To reiterate, I installed grpc-java according to the instructions on my Mac (running Mavericks). This included installing protobuf v3.0.0-alpha-2, which I did:

    $ which protoc

    /usr/local/bin/protoc

    $ /usr/local/bin/protoc --version

    libprotoc 3.0.0

The Java examples work fine. Now I'd like to try the C ++ examples and, ideally, have a C ++ client talk to a Java server and visa-versa.

I cloned grpc and attempted to install it. `sudo make install` and, subsequently, `make verify-install` fail, telling me that protobuf 3 is not installed:

    $ sudo make install

    ...

    [INSTALL] Installing grpc protoc plugins

    [INSTALL] Installing root certificates

    We could not find protoc 3.0.0+ installed on your system. While this

    will not prevent grpc from working, you will not be able to compile

    and run any meaningful code with it.

    Please download and install protobuf 3.0.0+ from:

       https://github.com/google/protobuf/releases

    Once you've done so, or if you think this message is in error,

    you can re-run this check by doing:

       make verify-install

But as you see, protoc is installed. Why the error?

This morning I started over, going through all of the gprc installation instructions, including those especially called out for the Mac. `make test` exited with an error (see below) and `sudo make install` failed as before.

    $ make test

    ...

    I0311 13: 50: 35.953012000 140735270560528 transport_end2end_tests.c: 676] waiting for expectations to complete

    D0311 13: 50: 36.151828000 4507860992 transport_end2end_tests.c: 282] recv_batch: 8192 ops on server final_state = GRPC_STREAM_SEND_CLOSED

    D0311 13: 50: 36.151848000 4507860992 transport_end2end_tests.c: 332] [00] GRPC_OP_SLICE len = 1

    D0311 13: 50: 36.151875000 4507860992 transport_end2end_tests.c: 342] + 8191 more

    I0311 13: 50: 36.151994000 140735270560528 transport_end2end_tests.c: 676] waiting for expectations to complete

    E0311 13: 50: 36.252335000 140735270560528 transport_end2end_tests.c: 678] Timeout waiting for expectation completion

    I0311 13: 50: 36.252382000 140735270560528 transport_end2end_tests.c: 659] EXPECTED: Waiting for request to close: client = 0x7fff531f5200, server = 0x7fab6b004c00

    I0311 13: 50: 36.252392000 140735270560528 transport_end2end_tests.c: 659] EXPECTED: Waiting for message to finish: length = 140734587949568 read_pos = 409343 begun = 1792017040

    / Bin / sh: line 1: 60304 Abort trap: 6 ./bins/opt/chttp2_transport_end2end_test

    test chttp2_transport_end2end_test failed

    make: *** [test_c] Error 1

Any ideas how I can fix this?

[Nicolas Noble]

I guess I should create a Stackoverflow account sometime and pay more attention to these questions :-)

I probably should amend the message a bit more on make install. The message you get on sudo make install isn't an actual failure - grpc's installation succeeded. It's just a warning you that it couldn't find the protoc compiler on your system. There might be several reasons for this, the most common one I've seen so far is that the "sudo" environment is culling /usr/local from the path. Basically, I bet that if you do sudo protoc --version, you'll get an error, which is why sudo make install can't find protoc, as it inherits root's environment that doesn't cover /usr/local. However, make verify-install, ran without sudo, should properly find it, and tell you your system is okay. If it still doesn't, that's something to investigate, but still not an actual blocking problem.

The test failure you get is unrelated, and seems like an actual problem somewhere in grpc, which would be worth investigating.

[Thad Humphries]

Thanks for the answer. No, protoc is found with sudo

$ sudo which protoc

/usr/local/bin/protoc

$ sudo protoc --version

libprotoc 3.0.0

and `make verify-install` gives me the same error message as I got on 'sudo make install`.

You say grpc should still work. By that I assume that there is nothing in the Makefile that needs to be done after the check for protoc. Is that correct?

However, I cannot get the grpc-common/cpp/helloworld example to run:

$ cd grpc-common/cpp/helloworld

$ make

protoc -I ../../protos --cpp_out=. --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` ../../protos/helloworld.proto

g++ -std=c++11 -I/usr/local/include -pthread  -c -o helloworld.pb.o helloworld.pb.cc

g++ -std=c++11 -I/usr/local/include -pthread  -c -o greeter_client.o greeter_client.cc

g++ helloworld.pb.o greeter_client.o -L/usr/local/lib -lgrpc++_unsecure -lgrpc -lgpr -lprotobuf -lpthread -ldl -o greeter_client

g++ -std=c++11 -I/usr/local/include -pthread  -c -o greeter_server.o greeter_server.cc

g++ helloworld.pb.o greeter_server.o -L/usr/local/lib -lgrpc++_unsecure -lgrpc -lgpr -lprotobuf -lpthread -ldl -o greeter_server

rm helloworld.pb.cc

$ ./greeter_server

E0311 15:03:52.096212000 140735270560528 server.c:617] assertion failed: grpc_is_initialized() && "call grpc_init()"

Abort trap: 6

(And just for completeness...)

$ ./greeter_client

E0311 15:04:05.440178000 140735270560528 channel.c:67] assertion failed: grpc_is_initialized() && "call grpc_init()"

Abort trap: 6

I'd really like to see this stuff work. We've got a large, multi-platform, multi-language client-server app who's RPC architecture could do with replacement. I'm hoping we can find something that moves seamlessly between C to Java and Objective C (and more) vs having to rewrite the protocol and API for each language.

[Nicolas Noble]

Correct: nothing more is needed after installing grpc, really.

I've grabbed my Macbook to run some tests, and yes, I can see something's broken. It's not tied to running the protoc command however, it's tied to try to compile a check. There's two ways the protobuf 3.0.0 message can appear on make verify-install: either the protoc command doesn't return something that looks like 3.0.0, or it failed to compile a test software. You can see that by doing make run_dep_checks. I'll spend some time troubleshooting that one, but it looks like it's tied to my usage of /dev/null in the tests. Now that I think about it, I've seen that before, but I didn't think it was causing the whole test to fail however. The solution might be as simple as replacing /dev/null by `mktemp` or something. It's currently preventing grpc from using the system openssl and protobuf, spending useless CPU cycles to compile them from the third_party directory. It's not critical, but it's probably annoying enough.

I've also run the tests there, and I can confirm that there's currently a problem with the mac port of grpc. Some of the tests are failing, and I don't link the output I am seeing; it's looking like there's some kind of overflow. But that's just me glancing at the error output. But this is definitely something to investigate. I'll file a proper issue in our github project.

As for the hello world example, after some poking around, I was able to reproduce what you're experiencing. I need to investigate more, but a quick workaround I've come up with is to delete the dynamic versions of the libraries. Basically, delete the *.dylib versions of grpc, grpc++ and gpr from your /usr/local/lib directory. What I'm saying is that I managed to run the helloworld example under mac while linking with grpc statically instead of dynamically. Which is going to be a fun one to debug.

Thanks for your reports and efforts! We definitely need to make the MacOS experience better.

[Nicolas Noble]

Oh and, forgot to say: if you're starting from scratch, you can avoid installing the dynamic libraries by doing `sudo make install-static` instead of `sudo make install`

[Nicolas Noble]

Issue filed for the test failures:

https://github.com/grpc/grpc/issues/1007

Pull request to fix the various "make" issues:

https://github.com/grpc/grpc/pull/1009

[jsmr...@gmail.com]

Hello, 

I am finding similar issues while installing gRPC on with my native toolchain (modified the flags in Makefile). Even though I have protoc3 while doing a make, it still fails to recognize it and tries to install the protobuf. 

While it doesn't do the installation successfully where it fails with crnt.o not found. 

Please let me know if there are any system requirements specifically needed for the gRPC (CPP)

This is a serious problem we are facing and any help in this regards is greatly appreciated. 

Cheers,

JS