Response to stack overflow question about compile tests:

[Austin Schuh]

I don't have enough reputation to reply to the stack overflow question, so I'm taking the response back here...  Sigh.

https://stackoverflow.com/questions/44532306/how-to-do-verify-that-some-code-doesnt-compile-with-bazel/44537817#44537817

In the case that Brian is asking about, the test file depends on a custom cross-tool file, hundreds of C++ files, and autogenerated protobuf code.  It isn't practical to re-run bazel in a test.  How would you test that something fails to compile there?

We've got some cases with one of our libraries which we want to verify get caught at compile time.  Currently that has to be done manually, which isn't good.

Austin

[Steren Giannini]

On Mon, Jun 19, 2017 at 8:44 PM Austin Schuh <austin...@gmail.com> wrote:

I don't have enough reputation to reply to the stack overflow question, so I'm taking the response back here...  Sigh.

According to this page: "The most basic privilege of all -- the right to ask a question, and the right to contribute an answer. This is generally available to everyone, regardless of reputation level."

Can you explain what is the problem? Maybe you are trying to comment? In which case, yes, I think out mistake is that we should not have posted the question on your behalf.

Steren

 

https://stackoverflow.com/questions/44532306/how-to-do-verify-that-some-code-doesnt-compile-with-bazel/44537817#44537817

In the case that Brian is asking about, the test file depends on a custom cross-tool file, hundreds of C++ files, and autogenerated protobuf code.  It isn't practical to re-run bazel in a test.  How would you test that something fails to compile there?

We've got some cases with one of our libraries which we want to verify get caught at compile time.  Currently that has to be done manually, which isn't good.

Austin

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CABsbf%3DFwMM0E4BbV%2BmPTyQeJYDmxXhL2RMj3YkOXruDSW-3N9g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Austin Schuh]

Thanks Steren.

I'm trying to comment on Lazlo's response to Brian to continue the discussion.  (Brian and I work together).  In theory, I could ask a follow-on question, but I'm pretty certain that isn't going to link everything together and leave a useful result.

Austin

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CANHKq5EU_qyMGNFupMbUxy_0bm03NDpixMQqQ2fAnqJMy1TNnA%40mail.gmail.com.

[Marcel Hlopko]

Could you elaborate why it's not practical to depend on the crosstool, cc_libraries, and proto_libs? If there was something like cc_test_expected_to_fail rule, it would need those deps anyway, right?

But I can see how such rule could be useful. Our plan is to wait until we have a Skylark C++ API, and then maybe use that api and flip the actions exit code in Skylark. It'd have to be much more involved and intrusive if we did that directly in our Java C++ rules. We're busy cleaning the mess we have there already, I'd prefer not to introduce more.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CABsbf%3DHwGR%3DW81c27U3D45-%3Ds79hf86RAvg-wEf8u7hxfr%2BHow%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

-- 

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle | Registergericht und -nummer: Hamburg, HRB 86891

[Austin Schuh]

Marcel,

On Wed, Jun 21, 2017 at 2:21 AM Marcel Hlopko <hlo...@google.com> wrote:

Could you elaborate why it's not practical to depend on the crosstool, cc_libraries, and proto_libs? If there was something like cc_test_expected_to_fail rule, it would need those deps anyway, right?

I think we might be thinking different things, so let me try to clarify what I understood the suggestion to be.

The stack overflow response suggested that we essentially write a rule which would create a "dummy" bazel workspace with the files needed, and then run a shell script.  The script would cd to the generated folder, run bazel to try to build a target, and then verify that bazel failed (optionally with the right error).  The goal is to do a 'compile test'.  We'd like to verify that we get compile errors when our library is used in certain ways (a tiny bit of template metaprogramming, for example).

That technically works, but I'm concerned that that will be very inefficient, and practically impossible in practice.

For example, we have a library foo.

cc_library(

  name = 'foo',

  srcs = [

    'foo.cc',

  ],

  hdrs = [

    'foo.h',

  ],

  deps = [

    '//dep1',

    '//dep2',

    '//dep3',

    ...

  ]

)

For us, 'foo' has about 100 dependencies, and depends on hundreds more generated files.  It also is built with a custom CROSSTOOL file, which fetches a compiler using a new_http_repository to use to build the code.  There are also a significant number of dependencies in the WORKSPACE file that the library relies on.

The size and complexity of the dependency set makes it tricky in my mind to recreate the dependency set in a new repository dynamically and re-run bazel to verify that you get a compiler error.  Especially since the dependency set keeps evolving.

Am I missing a shortcut there?

Independently, as I think you might have suggested by your cc_test_expected_to_fail name, it's valuable to make sure a test fails.  That's pretty easy to do with a cc_binary and a sh_test, which we do today with Peloton's code base to great effect.

 

But I can see how such rule could be useful. Our plan is to wait until we have a Skylark C++ API, and then maybe use that api and flip the actions exit code in Skylark. It'd have to be much more involved and intrusive if we did that directly in our Java C++ rules. We're busy cleaning the mess we have there already, I'd prefer not to introduce more.

Understood.  An item on the roadmap is progress, and I completely understand not wanting to make things worse.  :)

With a C++ skylark API, this request should be pretty easy.  Essentially, generate the compile command that would have been used to build the library/binary normally using skylark.  Verify that the command fails, optionally matching a regex.

Thanks!

    Austin