Using rules_foreign_cc to run cmake but some external dependency files are missing

[Suet-fei Li]

I am working on a project that uses Bazel, but some of the external dependencies only support cmake. In particular, I am trying to get https://github.com/google/ukey2 to work. However, I keep running into the missing file problem. Basically my projects depends on ukey2, which depends on protobuf (https://github.com/google/ukey2/tree/master/third_party), but somehow the protobuf directory is not getting populated. I tried passing in " -Dprotobuf_BUILD_TESTS=OFF", but somehow the tests are still being run.

Any idea what is going on and how to fix it?

--------------- error message -----------

CMake Error at tests.cmake:2 (message):

  Cannot find third_party/googletest directory that's needed to build tests.

  If you use git, make sure you have cloned submodules:

    git submodule update --init --recursive

  If instead you want to skip tests, run cmake with:

    cmake -Dprotobuf_BUILD_TESTS=OFF

-------------

--- I use  new_git_repository in WORKSPACE to clone all the submodules  recursively ---

new_git_repository(

    name = "com_google_ukey2",

    remote = "https://github.com/google/ukey2",

    branch = "master",

    build_file_content = _ALL_CONTENT,

    init_submodules = True,

    recursive_init_submodules = True,

)

---------------

---- Bazel BUILD file --------

load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")

cmake(

    name = "ukey2",

    cache_entries = {

        "CMAKE_CXX_FLAGS": "-std=c++11 -Dprotobuf_BUILD_TESTS=OFF",

    },

    generate_args = [

        "-Dukey2_USE_LOCAL_PROTOBUF=ON",

        "-Dukey2_USE_LOCAL_ABSL=ON",

        "-Dprotobuf_BUILD_TESTS=OFF",

    ],

    lib_source = "@com_google_ukey2//:all_srcs",

    linkopts = ["-pthread -Dprotobuf_BUILD_TESTS=OFF"],

    out_static_libs = ["libukey2.a"],

)

----------

[Suet-fei Li]

CLARIFICATION: looks like Bazel is not pulling the 3rd level dependencies. My project depends on ukey2, ukey2 depends on protobuf, protobuf depends on googletest. But the following directory is empty: 

~/.cache/bazel/_bazel_suetfei/fc109dcda9742f37d2114134b7994456/execroot/__main__/external/com_google_ukey2/third_party/protobuf/third_party/googletest/

Which I think is why I am getting the "CMake Error at tests.cmake:2 (message): Cannot find third_party/googletest directory that's needed to build tests." error.

Is this expected? Or a bug? Any work around?

Thanks.

[Konstantin]

It seems ukey2 depends on Protobuf, but Bazel does not know about it. You only specified cmake rule for ukey2. What it does is opaque for Bazel. I would try to add Protobuf to the "deps" attribute of ukey2 cmake rule hoping it would hint Bazel to build it first.

Konstantin

[Suet-fei Li]

I used "git clone" to download and then in Bazel, define new_local_repository instead. Seems to me there is a bug in "new_git_repository" with " init_submodules = True", that it is not pulling the whole tree. If it requires additional dependencies, then the documentation should say so.

[Konstantin]

Are you aware that "init_submodules" attribute only works one level deep and you supposed to use "recursive_init_submodules" attribute to pull the whole tree?

[Suet-fei Li]

Referring back to my original post, both attributes were set to true. That is why I suspect this is a "bug" on Bazel's side.

--- I use  new_git_repository in WORKSPACE to clone all the submodules  recursively ---

new_git_repository(

    name = "com_google_ukey2",

    remote = "https://github.com/google/ukey2",

    branch = "master",

    build_file_content = _ALL_CONTENT,

    init_submodules = True,

    recursive_init_submodules = True,

)

---------------

[Keith Smiley]

Looks like the current logic is `init_submodules` has to be false to pickup `recurisve_init_submodules`

https://github.com/bazelbuild/bazel/blob/2e723f228efee008bcfd62ceb74a176a357c4c32/tools/build_defs/repo/git_worker.bzl#L107-L112

Here's a PR to flip that https://github.com/bazelbuild/bazel/pull/13903

--
Keith Smiley

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/a0826688-091e-428f-8e2b-8d8fcd61ef91n%40googlegroups.com.

[Suet-fei Li]

Thanks! I will wait for https://github.com/bazelbuild/bazel/pull/13903 to be in and then try it out again.

[Keith Smiley]

You should be able to do it now with `init_submodules = False, recursive_init_submodules = True`. Worth a try in case there's another bug since this won't make it into a LTS release for a long time.

--
Keith Smiley

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/c9f8ca73-63f2-4f49-8c5b-c707c683ee63n%40googlegroups.com.