What's the general way to patch an external dependency package in Bazel?

Jianjun Liu

unread,

Aug 28, 2019, 8:07:00 AM8/28/19

to bazel-discuss

Dear All:

I am new to bazel.

When I test to build tensorflow/serving, because of glibc 2.30 upgrading, there is a need to patch grpc (with patch as https://github.com/grpc/grpc/pull/18950). grpc is an external dependency of tensorflow, instead of tensorflow/serving.

What's the general way to patch this? Any guide and link on this?

Would appreciate for your sharing!

Jianjun

Brian Silverman

unread,

Aug 28, 2019, 2:49:35 PM8/28/19

to Jianjun Liu, bazel-discuss

Hi,

In general, it depends how the external dependency is brought in. Many of the repository rules (http_archive for example) have a `patches` attr which will let you specify patch strings.

I looked for the place where tensorflow/serving pulls in grpc, but couldn't find it after a few minutes. Is it an http_archive hiding somewhere, or something else?

Brian Silverman

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/a5383427-86b9-468d-8287-26a01d675fd3%40googlegroups.com.

Jianjun Liu

unread,

Aug 28, 2019, 9:10:29 PM8/28/19

to bazel-discuss

Hi,

grpc is the external dependency of tensorflow (included via tf_http_archive in tensorflow/workspace.bzl).

Because tensorflow is the external dependency of tensorflow/serving (included via tensorflow_http_archive in WORKSPACE), grpc becomes the external dependency of tensorflow/serving.

Is there a step we could patch any external dependency, which have been fetched to local?

Would appreciate for your sharing!

Jianjun

On Thursday, August 29, 2019 at 2:49:35 AM UTC+8, Brian Silverman wrote:

Hi,

In general, it depends how the external dependency is brought in. Many of the repository rules (http_archive for example) have a `patches` attr which will let you specify patch strings.

I looked for the place where tensorflow/serving pulls in grpc, but couldn't find it after a few minutes. Is it an http_archive hiding somewhere, or something else?

Brian Silverman

On Wed, Aug 28, 2019 at 5:07 AM Jianjun Liu <jianj...@intel.com> wrote:

Dear All:

I am new to bazel.
When I test to build tensorflow/serving, because of glibc 2.30 upgrading, there is a need to patch grpc (with patch as https://github.com/grpc/grpc/pull/18950). grpc is an external dependency of tensorflow, instead of tensorflow/serving.
What's the general way to patch this? Any guide and link on this?

Would appreciate for your sharing!
Jianjun

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-...@googlegroups.com.

Brian Silverman

unread,

Sep 3, 2019, 5:03:43 PM9/3/19

to Jianjun Liu, bazel-discuss

Hi,

It looks like `tf_http_archive` doesn't have support for overriding a dependency. A common pattern in other projects is to skip creating individual repository rules if another with the same name already exists, which allows the top level project to simply declare a modified version (with different patches etc) before calling the main rule (ie `tf_repositories`). Protobuf's protobuf_deps.bzl does this, for example. It looks like tensorflow would need to wrap `tf_http_archive` in a macro which could check `native.existing_rule` before actually creating the repository via the custom rule. It's a straightforwards change (rename the rule to `_tf_http_archive` and write the macro named `tf_http_archive`, and then none of the other .bzl files need to change), but I don't know whether the tensorflow maintainers will agree.

Then your WORKSPACE would look something like this:

```

http_archive(

name = "grpc",

patches = [ "your_thing.patch" ],

urls = [ "where ever you want to fetch from, maybe the same as tensorflow does" ],

sha256 = "important to fill this out",

)

load("however you're doing it now", "tf_serving_workspace")

tf_serving_workspace()

```

For the replacement rule, you could either use the same `tf_http_archive` rule after adding a `patches` attr, or just use plain `http_archive`. I believe implementing `patches` in `tf_http_archive` would be simple using patch from @bazel_tools//build_defs/repo:utils.bzl the same way http_archive does it. Alternatively, if you're not using any parts of `tf_http_archive` which make it different than `http_archive`, you could just ignore them and use `http_archive` instead.

Brian

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/dfa2edac-0f12-4c0b-bba2-d3a20edbdae6%40googlegroups.com.

Jianjun Liu

unread,

Sep 3, 2019, 8:55:40 PM9/3/19

to bazel-discuss

Hi, Brian:

As refer to the latest workspace.bzl at https://github.com/tensorflow/serving/blob/master/tensorflow_serving/workspace.bzl, there is a way to override the dependency package of tensorflow.

Following the same way, I could override grpc package with the patch now.

Very appreciate for your help!

diff --git a/tensorflow_serving/workspace.bzl b/tensorflow_serving/workspace.bzl

index 7c6ab33..1746c53 100644

--- a/tensorflow_serving/workspace.bzl

+++ b/tensorflow_serving/workspace.bzl

@@ -1,6 +1,7 @@

# TensorFlow Serving external dependencies that can be loaded in WORKSPACE

# files.

+load("@org_tensorflow//third_party:repo.bzl", "tf_http_archive")

load("@org_tensorflow//tensorflow:workspace.bzl", "tf_workspace")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

@@ -48,3 +49,16 @@ def tf_serving_workspace():

strip_prefix = "libevent-release-2.1.8-stable",

build_file = "@//third_party/libevent:BUILD",

)

+

+ # ===== Override TF defined `grpc` to fix gettid conflict because of glibc 2.30.

+ tf_http_archive(

+ name = "grpc",

+ patch_file = str(Label("//third_party:Rename-gettid-functions.patch")),

+ sha256 = "67a6c26db56f345f7cee846e681db2c23f919eba46dd639b09462d1b6203d28c",

+ strip_prefix = "grpc-4566c2a29ebec0835643b972eb99f4306c4234a3",

+ system_build_file = str(Label("@org_tensorflow//third_party/systemlibs:grpc.BUILD")),

+ urls = [

+ "http://mirror.tensorflow.org/github.com/grpc/grpc/archive/4566c2a29ebec0835643b972eb99f4306c4234a3.tar.gz",

+ "https://github.com/grpc/grpc/archive/4566c2a29ebec0835643b972eb99f4306c4234a3.tar.gz",

+ ],

+ )

Br, Jianjun

Brian

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/dfa2edac-0f12-4c0b-bba2-d3a20edbdae6%40googlegroups.com.

Reply all

Reply to author

Forward