Using rules_foreign_cc to run cmake but some external dependency files are missing

1,188 views
Skip to first unread message

Suet-fei Li

unread,
Aug 14, 2021, 12:09:13 AM8/14/21
to bazel-discuss
I am working on a project that uses Bazel, but some of the external dependencies only support cmake. In particular, I am trying to get https://github.com/google/ukey2 to work. However, I keep running into the missing file problem. Basically my projects depends on ukey2, which depends on protobuf (https://github.com/google/ukey2/tree/master/third_party), but somehow the protobuf directory is not getting populated. I tried passing in " -Dprotobuf_BUILD_TESTS=OFF", but somehow the tests are still being run.

Any idea what is going on and how to fix it?

--------------- error message -----------
CMake Error at tests.cmake:2 (message):
  Cannot find third_party/googletest directory that's needed to build tests.
  If you use git, make sure you have cloned submodules:
    git submodule update --init --recursive
  If instead you want to skip tests, run cmake with:
    cmake -Dprotobuf_BUILD_TESTS=OFF
-------------

--- I use  new_git_repository in WORKSPACE to clone all the submodules  recursively ---

new_git_repository(
    name = "com_google_ukey2",
    branch = "master",
    build_file_content = _ALL_CONTENT,
    init_submodules = True,
    recursive_init_submodules = True,
)
---------------

---- Bazel BUILD file --------
load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")
cmake(
    name = "ukey2",
    cache_entries = {
        "CMAKE_CXX_FLAGS": "-std=c++11 -Dprotobuf_BUILD_TESTS=OFF",
    },
    generate_args = [
        "-Dukey2_USE_LOCAL_PROTOBUF=ON",
        "-Dukey2_USE_LOCAL_ABSL=ON",
        "-Dprotobuf_BUILD_TESTS=OFF",
    ],
    lib_source = "@com_google_ukey2//:all_srcs",
    linkopts = ["-pthread -Dprotobuf_BUILD_TESTS=OFF"],
    out_static_libs = ["libukey2.a"],
)
----------

Suet-fei Li

unread,
Aug 14, 2021, 12:09:16 AM8/14/21
to bazel-discuss
CLARIFICATION: looks like Bazel is not pulling the 3rd level dependencies. My project depends on ukey2, ukey2 depends on protobuf, protobuf depends on googletest. But the following directory is empty: 

~/.cache/bazel/_bazel_suetfei/fc109dcda9742f37d2114134b7994456/execroot/__main__/external/com_google_ukey2/third_party/protobuf/third_party/googletest/

Which I think is why I am getting the "CMake Error at tests.cmake:2 (message): Cannot find third_party/googletest directory that's needed to build tests." error.

Is this expected? Or a bug? Any work around?

Thanks.

Konstantin

unread,
Aug 15, 2021, 3:53:15 PM8/15/21
to bazel-discuss
It seems ukey2 depends on Protobuf, but Bazel does not know about it. You only specified cmake rule for ukey2. What it does is opaque for Bazel. I would try to add Protobuf to the "deps" attribute of ukey2 cmake rule hoping it would hint Bazel to build it first.

Konstantin

Suet-fei Li

unread,
Aug 17, 2021, 6:37:07 PM8/17/21
to bazel-discuss
I used "git clone" to download and then in Bazel, define new_local_repository instead. Seems to me there is a bug in "new_git_repository" with " init_submodules = True", that it is not pulling the whole tree. If it requires additional dependencies, then the documentation should say so.

Konstantin

unread,
Aug 18, 2021, 12:18:34 AM8/18/21
to bazel-discuss
Are you aware that "init_submodules" attribute only works one level deep and you supposed to use "recursive_init_submodules" attribute to pull the whole tree?

Suet-fei Li

unread,
Aug 25, 2021, 2:55:55 AM8/25/21
to bazel-discuss
Referring back to my original post, both attributes were set to true. That is why I suspect this is a "bug" on Bazel's side.

--- I use  new_git_repository in WORKSPACE to clone all the submodules  recursively ---

new_git_repository(
    name = "com_google_ukey2",
    branch = "master",
    build_file_content = _ALL_CONTENT,
    init_submodules = True,
    recursive_init_submodules = True,
)
---------------

Keith Smiley

unread,
Aug 25, 2021, 12:41:47 PM8/25/21
to Suet-fei Li, bazel-discuss
Looks like the current logic is `init_submodules` has to be false to pickup `recurisve_init_submodules`


--
Keith Smiley


--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/a0826688-091e-428f-8e2b-8d8fcd61ef91n%40googlegroups.com.

Suet-fei Li

unread,
Aug 25, 2021, 6:25:35 PM8/25/21
to bazel-discuss
Thanks! I will wait for https://github.com/bazelbuild/bazel/pull/13903 to be in and then try it out again.

Keith Smiley

unread,
Aug 25, 2021, 6:27:29 PM8/25/21
to Suet-fei Li, bazel-discuss
You should be able to do it now with `init_submodules = False, recursive_init_submodules = True`. Worth a try in case there's another bug since this won't make it into a LTS release for a long time.
--
Keith Smiley


Reply all
Reply to author
Forward
0 new messages