Per file "copt" available in cc_library?

[Sarah Maven]

I'm trying to compile festvox/speech_tools with bazel instead of make and I ran into an issue where I need to specify different command line options (copt) for each file.

I tried separating each set of files into its own cc_library with the differing copts separated by library. The only issue is that when linking, the libraries cannot find the implementations of methods defined in the separate library. If I remove the -Wl,--end-lib -Wl,--start-lib that separates the libraries' .o files in the generated .params file, I can compile manually, but i cannot compile automatically this way.

To give a example of what i'm trying to say, I have 6 files, A.cc B.cc C.cc 1.cc 2.cc 3.cc; A, B, and C need -DINSTANTIATE_TEMPLATES, and 1, 2, and 3 must not be compiled with this option. For the sake of argument lets say all files reference functions defined in every other file in this set. I want to successfully compile these into a library that can be used elsewhere.

I will probably try making cc_binary rules to generate specialized .o files and combine those into a library, but i wanted to see if there are any other suggestions on tackling this.

Thanks!

[Sarah Maven]

After more searching, doing individual cc_binary rules with .o is annoying because you need to use "-r" linkopt as "-c" does not produce a file. This required for me turning off certain "feature" flag groups. That led me to `features = ["-libraries_to_link"],` which should, in theory, turn off the start and end library flag that was messing me up in the first place. I will report back when I've tested this on if it works.

[Sarah Maven]

Ah, nevermind, '@local_config_cc//:toolchain' unconditionally implies feature 'libraries_to_link'. So I have to manually do things with `-r`.

[Alex Garcia Mayans]

Have you tried “always_link=1” on those libraries?

Regards,

Alexandre Garcia Mayans

On 22 Feb 2021, at 03:37, Sarah Maven <zarra...@gmail.com> wrote:

Ah, nevermind, '@local_config_cc//:toolchain' unconditionally implies feature 'libraries_to_link'. So I have to manually do things with `-r`.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/85fb419d-07b9-4ccb-88de-62a936e73e6cn%40googlegroups.com.

[Sarah Maven]

I did, that gives me a bunch of undefined reference errors. I have been also trying to do more with "-r" but I'm managing to create duplicate function definitions that way. I suppose I may have to create a genrule to manually invoke the gcc reference provided by bazel to create the ".o" files in a way i can reference in other build rules. That should be fairly immune to errors as I can reproduce exactly the calls I'm looking for

[Sarah Maven]

So fun addendum, trying to 'just get access to gcc' is actually surprisingly difficult to do, because you have to grab it off of a toolchain inside a build rule instead of a gen rule. This would be fine but I'm running into the issue of CcToolchianInfo being created as just a plain ToolchainInfo, which has none of the fields I need. So I need to figure out how to fix that. https://github.com/bazelbuild/intellij/issues/1344 runs into a similar issue but I will have to investigate further

def _cc_object_impl(ctx):

    if type(ctx.attr._compiler[cc_common.CcToolchainInfo]) != "CcToolchainInfo":

        fail("CcToolchainInfo not of correct type")

    pass

cc_object = rule(

    implementation = _cc_object_impl,

    attrs = {

        "srcs": attr.label_list(allow_files = [".cc", ".cpp"]),

        "deps": attr.label_list(),

        "_compiler": attr.label(

            default = "@local_config_cc//:cc-compiler-k8",

            providers = [cc_common.CcToolchainInfo],

        ),

    },

)

[Austin Schuh]

This smells like something that the C++ sandwich is supposed to make easier.  https://docs.bazel.build/versions/master/skylark/lib/cc_common.html#compile

Austin

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/241160b2-4abc-4ed0-af13-835620d96afbn%40googlegroups.com.

[Sarah Maven]

Thank you, I think you have located exactly what I'm trying to find, will update if successful, but also I did figure out how to get the correct toolchain by ripgrep-ing rules_cc. The code is

def _cc_object_impl(ctx):

    cc_toolchain = ctx.attr._cc_toolchain[cc_common.CcToolchainInfo]

    if type(cc_toolchain) != "CcToolchainInfo":

        fail("_compiler not of type CcToolchainInfo. Found: " + type(cc_toolchain))

    print(cc_toolchain.compiler_executable)

cc_object = rule(

    implementation = _cc_object_impl,

    attrs = {

        "srcs": attr.label_list(allow_files = [".cc", ".cpp"]),

        "deps": attr.label_list(),

        "_cc_toolchain": attr.label(

            default = Label("@bazel_tools//tools/cpp:current_cc_toolchain"),

            providers = [cc_common.CcToolchainInfo],

        ),

    },

)

[Sarah Maven]

So I may be reading this wrong, but it appears cc_common.compile does not actually compile, but creates information needed to create a CcInfo. rules_cc's examples don't make use of it but are unfortunately outdated. https://github.com/bazelbuild/rules_cc/blob/master/examples/my_c_compile/my_c_compile.bzl That being said I should be able to use the CcInfo to tell other rules like cc_library about the .o files I'm creating and merge multiple cc_infos together if I have one set of object files depending on the relevant headers of another. So I will have to invoke the compiler myself still, but it is possible CcInfo will lessen the pain

[Sarah Maven]

Ah so actually turns out compile does compile, but only if you return info about it (like DefaultInfo) from your rule, otherwise it does nothing

[Sarah Maven]

Since my code now works, at least in my test project here's what I did in the end

# [new] rules_cc_object.bzl

def _cc_object_impl(ctx):

    cc_toolchain = ctx.attr._cc_toolchain[cc_common.CcToolchainInfo]

    if type(cc_toolchain) != "CcToolchainInfo":

        fail("_compiler not of type CcToolchainInfo. Found: " + type(cc_toolchain))

    

    feature_configuration = cc_common.configure_features(

        ctx = ctx,

        cc_toolchain = cc_toolchain,

        requested_features = ctx.features,

        unsupported_features = ctx.disabled_features,

    )

    deps_compilation_contexts = [dep[CcInfo].compilation_context for dep in ctx.attr.deps]

    # does it respect the toolchains built_in_include_directories?

    (compilation_context, compilation_outputs) = cc_common.compile(

        name = ctx.label.name,

        actions = ctx.actions,

        feature_configuration = feature_configuration,

        cc_toolchain = cc_toolchain,

        srcs = ctx.files.srcs,

        public_hdrs = ctx.files.hdrs,

        private_hdrs = ctx.files.private_hdrs,

        quote_includes = ctx.attr.includes,

        local_defines = ctx.attr.defines,

        user_compile_flags = ctx.attr.copts,

        compilation_contexts = deps_compilation_contexts,

    )

    cc_info = CcInfo(

        compilation_context = compilation_context,

    )

    transitive = [dep[DefaultInfo].files for dep in ctx.attr.deps]

    output_files = depset(

        compilation_outputs.pic_objects + compilation_outputs.objects,

        transitive = transitive,

    )

    file_set_produced = DefaultInfo(files = output_files)

    return file_set_produced, cc_info

cc_object = rule(

    implementation = _cc_object_impl,

    attrs = {

        "srcs": attr.label_list(allow_files = [".cc", ".cpp"]),

        "hdrs": attr.label_list(allow_files = [".h", ".hh"]),

        "private_hdrs": attr.label_list(allow_files = [".h", ".hh"]),

        "includes": attr.string_list(),

        "defines": attr.string_list(),

        "copts": attr.string_list(),

        "deps": attr.label_list(),

        "_cc_toolchain": attr.label(

            default = Label("@bazel_tools//tools/cpp:current_cc_toolchain"),

            providers = [cc_common.CcToolchainInfo],

        ),

    },

    # required to access certain cc_common methods

    fragments = ["cpp"],

)

# [new] BUILD

load("rules_cc_object.bzl", "cc_object")

cc_object(

    name = "dependency",

    srcs = [

        "dependency.cc"

    ],

    hdrs = [

        "dependency.hh",

    ],

)

cc_object(

    name = "test",

    srcs = [

        "main.cc"

    ],

    deps = [

        ":dependency", # we extract headers from dependencies

    ],

)

cc_binary(

    name = "main",

    deps = [

        ":test", # cc_binary only expects headers in deps

    ],

    srcs = [

        ":test", # but we also have .o files to act as sources

    ],

)

[Konstantin]

I have solved similar problem by introducing my own cc_compile build rule which takes all the same attributes as cc_library, but does not produce the library. Instead it provides the produced object files as its default output, so other cc rules, such as cc_library can take my cc_compile targets in the list of their srcs. I will attach a couple files to give you the idea.

Konstantin