genrule running ./configure almost works

[Gregg Reynolds]

Hi,

I'm using new_http_archive to import libcoap

new_http_archive(

    name = "libcoap",

    urls = ["https://github.com/obgm/libcoap/archive/v4.1.2.zip"],

    sha256 = "4caf73446a4994a0571afaf5e60a62a749bda92a29e9407174614733e984c238",

    strip_prefix = "libcoap-4.1.2",

    build_file = "config/libcoap.BUILD",

    # urls = ["https://github.com/dthaler/libcoap/archive/IoTivity-1.2.1d.zip"],

    # sha256 = "aac5b9101fad4cf722f0166f2739ebcefe55ad0ca7aa475550b11c8eb4740511",

    # strip_prefix = "libcoap-IoTivity-1.2.1d",

)

The lib must be configured, which involves running autogen.sh (which runs autoreconf etc.) and then configure.

The following genrule seems to be doing this. Note that my output is just for while I'm developing; once I get configure running successfully I'll deal with the outputs.

genrule(

    name = "config",

    srcs = ["autogen.sh",

            "configure.ac",

            "Makefile.am",

            "NEWS",

            "README",

            "AUTHORS",

            "ChangeLog",

            "doc/Doxyfile.in",

            "doc/Makefile.am",

            "examples/Makefile.am",

            "examples/coap-client.txt.in",

            "examples/coap-rd.txt.in",

            "examples/coap-server.txt.in",

            "include/coap/coap.h.in",

            "tests/Makefile.am",

    ],

    outs = ["autotest"],

    cmd  = "\n".join([

        "echo 'foo' > $@",

        "cd external/libcoap",

        "./autogen.sh 2>&1",

        "sleep 1",

        "./configure --disable-examples"

    ])

)

However, it chokes on a config macro:

checking whether to build static libraries... yes

./configure: line 12764: syntax error near unexpected token `-Wlogical-op,'

./configure: line 12764: `AX_CHECK_COMPILE_FLAG(-Wlogical-op, WARNING_CFLAGS="$WARNING_CFLAGS -Wlogical-op",,-Werror)'

Target @libcoap//:config failed to build

But if I manually download the library, run autogen.sh and then configure, it does not throw this error.

So my inference is that Bazel's environment is doing something that configure cannot abide.  Any suggestions?

Thanks,

Gregg

[Gregg Reynolds]

Never mind.  It turned out to be a question of listing all the files needed for input. This works:

genrule(

    name = "config",

    srcs = ["autogen.sh",

            "configure.ac",

            "Makefile.am",

            "NEWS",

            "README",

            "AUTHORS",

            "ChangeLog",

            "doc/Doxyfile.in",

            "doc/Makefile.am",

            "examples/Makefile.am",

            "examples/coap-client.txt.in",

            "examples/coap-rd.txt.in",

            "examples/coap-server.txt.in",

            "include/coap/coap.h.in",

            "tests/Makefile.am",

            "m4/ax_check_compile_flag.m4",

            "m4/ax_check_link_flag.m4",

            "libcoap-1.pc.in"

    ],

    outs = ["AUTOTEST"],

    cmd  = "\n".join([

        "echo 'foo' > $@",

        "cd external/libcoap",

        "bash -c ./autogen.sh 2>&1",

        "./configure --disable-examples --disable-tests --disable-documentation"

   ])

)

The two missing m4 files were causing the error.  NB: the 2>&1 bit keeps the stdout/stderr msgs of the two cmds from intermingling.

Sorry about the noise

g

[Dmitry Lomov]

Nice! Also take a look at https://github.com/bazelbuild/rules_foreign_cc.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAO40Mi%3DQZVV7%3DyVJHr9Kfn%3DVB7T0rX61-s746D-8V64w%2Bc9iaA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Google Germany GmbH

Erika-Mann-Straße 33, 80636 München, Germany

[Gregg Reynolds]

On Thu, Jun 7, 2018, 12:27 PM Dmitry Lomov <dsl...@google.com> wrote:

Nice! Also take a look at https://github.com/bazelbuild/rules_foreign_cc.

Thanks!  I'm loving Bazel - to the point where I fear I'm beginning to annoy people, heh.

This use case replaces a scons-based version. IMHO the Bazel version is simpler and cleaner by at least an order of magnitude.

Regarding rules_foreign_cc: looks interesting, but how would that accommodate cross-platform builds? I'm experimenting now with my genrule; you have to set the path to your toolchain and invoke configure with --host, --build, etc. It's a little clumsy, but doable with select. Based on the readme for rules_foreign_cc it isn't clear to me how that would be handled.

G

[Gregg Reynolds]

I've got a cross-build configure running, but there's one problem: the toolchain path is hard-coded. I have:

CTNG_PATH="/Volumes/CrossToolNG"

RPI_CONFIG  = "".join([

    "echo 'foo'  > $@\n",

    "(cd external/libcoap; ",

    "bash -c ./autogen.sh 2>&1; ",

    "export PATH=\"", CTNG_PATH, "/armv8-rpi3-linux-gnueabihf/bin:$${PATH}\"; ",

    " ./configure",

    " --build=x86_64-apple-darwin",

    " --host=armv8-rpi3-linux-gnueabihf",

    " --disable-examples",

    " --prefix=/usr",

    " --mandir=\"/usr/share/man\"",

    " --libdir=\"/usr/lib/arm-linux-gnueabihf\"",

    " CPPFLAGS=\"-P\";)"]

)

And then RPI_CONFIG is used as the cmd in the genrule.

I have the toolchain in my WORKSPACE:

new_local_repository(

  name = "ctng",

  path = "/Volumes/CrossToolNG",

  build_file = 'tools/ctng/ctng.BUILD'

)

Is there any way I can pull the path from this new_local_repository rule?

Thanks,

Gregg

[Dmitry Lomov]

Hi Gregg,

Thanks for your kind words about Bazel!

> Regarding rules_foreign_cc: looks interesting, but how would that accommodate cross-platform builds? I'm experimenting now with my genrule; you have to set the path to your toolchain and invoke configure with --host, --build, etc. It's a little clumsy, but doable with select. Based on the readme for rules_foreign_cc it isn't clear to me how that would be handled.

Build rules, as opposed to genrule and repository rules, have access to toolchain information. rules_foreign_cc utilize that information (see https://github.com/bazelbuild/rules_foreign_cc/blob/master/cc_configure_make/def.bzl#L20). Bear in mind that APi to access C++ toolchain information is currently undergoes many changes (I am cc'ing Marcel and Pedro for this, here is the roadmap), but in general a special build rule (as opposed to genrule or macro or repository rules) is the answer if you need access to configuration.

So to answer your second question, you will need to define a  C++ toolchain for your crosscompiler and reference that in your build: here is the current tutorial for that https://github.com/bazelbuild/bazel/wiki/Yet-Another-CROSSTOOL-Writing-Tutorial, but be aware of the disclaimerson that page.

Incidentally, currently Irina (cc'd) is picking up work to make Bazel interface more easily with other build systems, starting with C++ build systems. What is your interest in this work? We would be happy if you take early prototypes for a test drive, or even better if you have time to contribute to this effort.

Cheers,

Dmitry

[Gregg Reynolds]

On Mon, Jun 11, 2018, 10:15 AM Dmitry Lomov <dsl...@google.com> wrote:

Hi Gregg,

Thanks for your kind words about Bazel!

...

 What is your interest in this work?

https://github.com/OpenOCF/iochibity

See also https://github.com/OpenOCF/iotivity/tree/bazel?files=1, which bazelizes the legacy Iotivity repo (which uses Scons). About 90% done; it turned out to be much easier than I had feared.

I think these are good demo projects for Bazel. They require multiple build platforms (Linux, Mac, windows), multiple build options, and cross-build support (embedded systems). And they depend on remote third-party libs that must be patched and/or configured. And there's a Java API binding, and Android support. A relatively small project that stretches the build system. The legacy project based on scons is weak in all four areas (imho) but with Bazel everything just works! Well, maybe "just" is a little strong; the learning curve was fairly steep. But it was worth the effort; now I can target raspberry pi e.g. with a simple --config=rpi-arm8! (Uses a crosstool-NG CROSSTOOLS file I  managed to come up with.) Except I have to run a config script first, heh. 

Before I discovered Bazel I had the usual collection of environment vars, shell scripts, etc. to support cross-builds. It worked but it was cumbersome, error-prone, and hideous. Bazel is better by several orders of magnitude, imho.

We would be happy if you take early prototypes for a test drive, or even better if you have time to contribute to this effort.

I'm happy to help, time permitting.

Gregg

[Marcel Hlopko]

Hi all,

From my perspective the optimal solution is what Irina is working on - a custom rule that uses Skylark API to the C++ toolchain (tracking issue) to get the right flags to pass to make/other build system, plus that uses C++ sandwich (tracking issue) to expose C++ providers so other C++ and other rules can depend on it properly. Macro/genrule based approaches have their limitations (the most serious that access to the C++ toolchain information is not customizable enough). Both projects are seeing good progress (I'm starting to migrate internal users to the new toolchain API today, and we already have some internal rules migrated to the C++ sandwich, so I'm defensively guessing 2 months + bazel release to get stable enough implementation so you can depend on it in your work.

--

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891