What's the canonical alternative to CMake's `install` in bazel?

[imm...@gmail.com]

Hi,

Is there any canonical way to export and package all the libraries together with header files?

I'm trying to migrate several projects to bazel. Those projects produce static/dynamic libraries for other projects.

however, not all projects in my working environment can be migrated into a monorepo.

So bazel projects must be able to build and export those libraries and headers, preferably in a packaged file format, which would be later imported in other non-bazel projects.

Previously we were using cmake's `install` to copy headers into output directory, and other commands to pack them up.

In bazel, since `cc_library` already has information about `hdrs`, I guess the export/package procedure should be easier than cmake, but I can find any rules for now.

Thanks,

[Brian Silverman]

Hi,

rules_pkg has various packaging rules. pkg_tar is the basic one to build tarballs. There are also pkg_deb and pkg_rpm for building those up into installable packages.

Those rules don't directly know anything about how to package C++ code. There are a lot of decisions to make about what to package where which I think you'll need to answer for your use case. I see three options which might make sense, depending on what you're looking for:

The easiest approach is manually specifying all the files (headers, output libraries, etc) you want to package in pkg_tar rules. This is very straightforward and should work fine for small/simple projects.

The next-more-complicated approach is writing a macro to automatically creates a pkg_tar and corresponding cc_library, and then using it instead of cc_library directly. This doesn't require digging into any complicated APIs from the C++ rules, while still letting you customize the paths everything ends up at. To me, the main limitation is having to manually track dependencies between libraries, which will get complicated if you have deeply nested dependency trees. You could write a pkg_tar which collects up the pkg_tars from each library to combine them at the end. Something like this:

def cc_install_library(name, srcs, hdrs):

  native.cc_library(name = name, srcs = srcs, hdrs = hdrs):

  pkg_tar(name = name + '_install', srcs = hdrs + [':%s.so' % name], package_dir = 'usr/lib')

The most complicated approach would be a custom rule that looks at the CcInfo provider from the cc_library rules and packages the files based on that. This provides access to all of the information about the libraries, their source files, their outputs, and how those all relate. The downside is having to work with the APIs that provide all of that information. You can re-use some of the pieces from rules_pkg to build up tarballs of the files you want (I'd start by looking at how pkg_tar works).

Brian

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/5a7ed510-9508-44fb-8f5c-7206bd19a7f8%40googlegroups.com.

[imm...@gmail.com]

Brian,

Thanks for the reply, it's a good idea to wrap cc_library and pkg_tar into a macro!

I also tried to implement a custom rule, however I encountered a problem with CcInfo: I cannot distinguish header files in `hdrs` from those in `srcs`.

here's my rule definition:

cc_package = rule(
    implementation = _package_impl,
    attrs = {
        "library" : attr.label(),
        "strip_include": attr.string(default = ""),
        "prepend_include": attr.string(default = "")
    }
)

and the usage:

cc_library(
    name = "hello-time",
    srcs = ["hello-time.cc", "hello-internal.h"],
    hdrs = ["hello-time.h"],
    visibility = ["//visibility:public"],
)


cc_package(
    name = "package_hello",
    library = ":hello-time"
)

`package_hello` depends on cc_library: `hello-time`, with public header file `hello-time.h`, and internal header file `hello-internal.h`

and I can access ctx.attr.library[CcInfo] in `_package_impl`. however, I cannot find information equivalent to `hdrs`.

The closest field I can find is ctx.attr.library[CcInfo].compilation_context.headers, but it has both `hello-time.h` and `hello-internal.h`.

Is there any clue?

Thanks,

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-...@googlegroups.com.

[Brian Silverman]

Hi,

CcInfo only has information to build dependent libraries. Looks like that doesn't include where header files come from.

An easy workaround is moving the public ones (hdrs) to a separate cc_library which you feed to the cc_package, and also put in the deps of a (separate) cc_library with the source files etc. That avoids duplicating anything and isn't many more lines.

You could also solve it with an aspect. That basically lets you build up information through the dependency tree by looking at all the attributes for each rule. That does mean even more moving parts to keep track of though.

Brian

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/604c2f3a-bca9-4940-8bcf-248f534c4f0d%40googlegroups.com.

[imm...@gmail.com]

Brian,

Thanks, I've just learned about aspect, seems a little overkill here.

Now I still have one problem implementing rules:

I customized pkg_zip from rules_pkg to accept cc_library as dependency, However the rule complains about library missing:

FileNotFoundError: [Errno 2] No such file or directory: 'bazel-out/darwin-fastbuild/bin/stage3/lib/libhello-time2.so'

the dependency `cc_library` seems not being executed before my custom rule.

however the dependency graph seems correct:

...

  "//stage3/lib:hello-time2_zip"

  "//stage3/lib:hello-time2_zip" -> "//stage3/lib:hello-time2"

  "//stage3/lib:hello-time2_zip" -> "//stage3/lib:hello-time.h"

...

  "//stage3/lib:hello-time2"

  "//stage3/lib:hello-time2" -> "//stage3/lib:hello-internal.h\n//stage3/lib:hello-time.cc"

  "//stage3/lib:hello-time2" -> "//stage3/lib:hello-time.h"

...

what might be the problem? 

Thanks again for your advice,

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/604c2f3a-bca9-4940-8bcf-248f534c4f0d%40googlegroups.com.

[imm...@gmail.com]

I figured it out, I need to declare these files as `input` of the action.

Thanks!