Contributing to gRPC Ruby plugin, questions running tests locally (MacOS, Ruby)

[Matt Todd]

Hi there,

I'm looking for guidance on setting up my local development environment to run the Ruby generation tests on MacOS; I'm trying to reproduce a bug but encountering issues running the tests no my machine. I've attempted to setup my local environment according to the documentation but encounter problems when running the tests; I do not see the same issues running the tests in Docker.

CONTEXT:

The bug I'm trying to reproduce involves defining request/response message types under one package (with its own `ruby_package` option) and the gRPC service in another package that includes and uses the request/response types from the first package. When generating the service file, the request/response message types ignore the `ruby_package` option which results in incorrect constant references.

I've generated these Ruby files on my MacOS machine using `protoc` installed with `homebrew` and the gRPC Ruby plugin installed via homebrew (via https://formulae.brew.sh/formula/grpc).

SETTING UP LOCAL DEVELOPMENT:

My first goal was to see if I could write a quick test case to reproduce the issue within the `github.com/grpc/grpc` repository; I created a new protobuf definition file in `src/proto/grpc/testing/` that imports `grpc/testing/package_options.proto`, then I added a corresponding test to `src/ruby/spec/pb/codegen/package_option_spec.rb`.

Next I pulled up the contributing guide (https://github.com/grpc/grpc/blob/master/CONTRIBUTING.md) which directed me to first clone the repository following the BUILDING instructions (https://github.com/grpc/grpc/blob/master/BUILDING.md#clone-the-repository-including-submodules).

I then built the prerequisites for running the Ruby tests with:

python tools/run_tests/run_tests.py -l ruby --build_only 

Next I tried running the Ruby language tests with:

* python tools/run_tests/run_tests.py -l ruby

This failed with the following error message:

```

  1) Code Generation Options should generate and respect package options

     Failure/Error: expect(gen_file).to be_truthy

       expected: truthy value

            got: nil

```

When I debugged a bit, I noticed that the exit code was 127, indicating that the `protoc` binary was not found; it was looking for the binary at `bins/opt/protobuf/protoc` but the binary was not there. (Likewise, the expected plugin binary was missing from the expected path as well.) I reran the tests with `-c opt` to make sure that was being set correctly and that the build step would force the build into the correct directory but this had no discernible affect.

At this point I hardcoded the path to the `protoc` binary I had installed on my system in the test file and was able to get the tests to generate the files as expected, except that the tests now failed (without any other changes) with the following error:

```

  1) Code Generation Options should generate and respect package options

     Failure/Error: expect(require('grpc/testing/package_options_services_pb')).to be_truthy

     NameError:

       uninitialized constant Grpc::Testing::TestService::Service::TestRequest

```

The constant that is expected is defined at `Grpc::Testing::Package::Options` but when the service file is being required, the generated service file has the service generated under `Grpc::Testing::TestService::Service` and references the `TestRequest` and `TestResponse` message classes without a package prefix (which would work if the service file also generated under `Grpc::Testing::Package::Options`).

At this point, it appeared to me that the tests might be failing to build on my platform (MacOS).

DOCKER:

Next I decided to try running the tests in a Docker container (with the --use_docker option) to eliminate inconsistencies between platforms:

python tools/run_tests/run_tests.py -l ruby --use_docker -c opt

The test that was failing before passes.

CONCLUSION:

There are a few points of confusion I have regarding my development environment and my platform.

Either I've missed critical steps in my local development environment setup (specifically getting `bin/opt/protobuf/protoc` and the accompoanying plugin binary setup correctly) or the tests are not intended to be run on my platform/outside of Docker/whatever build environment most contributors are using.

Further, when I override the tests to run using my local install of `protoc` and `grpc_ruby_plugin`, both of which are the latest versions available, the existing test I'm interested in fails in the same way that my bug report intended to reproduce; and yet running the tests in Docker does not reproduce this failure. Does this indicate that the binary I'm using built on MacOS (installed with homebrew) behaves distinctly from the Linux build?

NEXT STEPS:

I will open up a Pull Request with my proposed reproduction (sometime this week).

I will regenerate the Ruby definitions with my local MacOS install of `protoc` and `grpc_ruby_plugin` and compare that to files generated inside of a Linux container (using Docker).

I'm hoping that someone here with familiarity with building and running the Ruby tests on MacOS can chime in with some insight I'm missing.

Thanks for your time and consideration,

Matt

[apo...@google.com]

Re: context

I'm not clear on what the bug is at this point, or if there is one. If I understand, the issues encountered here were while attempting to reproduce a suspected bug?

Re: SETTING UP LOCAL DEVELOPMENT

I'm actually surprised that your invocation of "python tools/run_tests/run_tests.py -l ruby" didn't work (that *should* be building all of the pre-reqs, and it's basically the same command being ran on CI)

Re: CONCLUSION

The tests are meant to be ran outside of docker. You should be able to run them on mac.

Please open the pull request and put an example, with ideally steps to reproduce, as you've attempted to do here :)

[Matt Todd]

Thanks for following up! My apologies for not getting back to this sooner.

I've opened https://github.com/grpc/grpc/pull/16728 to attempt to reproduce the specific failure I'm seeing, and created https://gist.github.com/mtodd/cb6c189b2d3dec494c7f81f5f5e288f6 to provide a focused reproduction case. I'll follow up to include specific reproduction steps but ran short on time today.

I believe the PR and the gist clearly demonstrates that the generated service file ignores the `ruby_package` option of the imported message types.

[apo...@google.com]

Commented on the PR, thanks for bringing this to attention. There are a couple of different issues here, looking in to it