Runfiles in native and Starlark rules

[Jakob Buchgraber]

Hi all,

I stumbled upon a quite unintuitive and possibly buggy behaviour when a target of a Starlark rule is added to the data attribute (runfiles) of a native rule. That is, the outputs of the Starlark rule are not added to the runfiles of the native rule unless they are added both as files and runfiles to the DefaultInfo provider. 

I naively wrote code like the below

def _foo_rule_impl(ctx):

  out_file = ctx.actions.declare_file(...)

  ...

  return [DefaultInfo(files = depset([out_file])]

foo_rule = rule(

  implementation = _foo_rule_impl,

)

BUILD:

foo_rule(

  name = "foo_tool",

  ...,

)

py_test(

  name = "bar_test",

  data = [":foo_tool"],

)

The behaviour I encountered is that the outputs of "foo_tool" are never added as runfiles to "bar_test". To make it work I needed to change the DefaultInfo provider (the line in bold) to 

return [DefaultInfo(files = depset([out_file], runfiles = ctx.runfiles([out_file]))]

That looks very wrong to me. First, conceptually it makes no sense for a rule to add its own outputs as runfiles. Second, a rule can't anticipate whether an upstream rule will add

its output to the runfiles. All rules will need to use DefaultInfo this way if runfiles are supposed to work across the Bazel ecosystem.

So is this a bug or am I missing something?

Thank you,

Jakob

P.S.: The only slightly related issue I found is https://github.com/bazelbuild/bazel/issues/9442 (cushon@ in cc)

Jakob Buchgraber

Software Engineer

Google Germany GmbH

Erika-Mann-Straße 33

80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.

    

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Jakob Buchgraber]

Short correction:

First, conceptually it makes no sense for a rule to add its own outputs as runfiles.

 

I was just made aware that this sentence can be read as offensive. I am very sorry

and would like to apologize. When writing this sentence I was trying to express that

I don't understand the reasoning behind this API and that it looks like a bug to me,

but I am not sure.

Best,

Jakob

[Daniel Wagner-Hall]

I agree that this behaviour can be unintuitive and surprising, but I don't think it's a bug.

As per https://docs.bazel.build/versions/master/skylark/lib/DefaultInfo.html#DefaultInfo:

The point of files is to propagate information to Bazel to decide what should be built if the target is actually on the command line.

The point of runfiles is to propagate information to bazel run, or to other targets about what files are needed at runtime.

These two things aren't fundamentally linked, or intended to be a subset of each other (though they often are in practice).

For example, it's reasonable (though unusual) that if a target needs some expensive-to-compute data table in order to run, then building the target to check that it compiles doesn't need to generate that data table, but building it to use it in a test does.

In short, I think that files is much less important/useful than its name suggests - it has a very general name but represents a much more narrow use-case.

But, because this is an unusual case, I would probably agree with a feature request which defaulted runfiles to ctx.runfiles(files) if one wasn't specified (and required people to explicitly set runfiles = ctx.runfiles([]) if they wanted to empty out the runfiles), rather than the defaults that we currently have. Of course, that's a slightly breaking change :)

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAGQ4vn19K%3D2x5zVfH3xQczJJdp2U1P%3DCm-DeMwzgAt9d%3DGZfZQ%40mail.gmail.com.