[proposal] Improving native.existing_rules

[Alexandre Rostovtsev]

Jon and I drafted a design doc for improving native.existing_rules() performance, which entails a low-impact but incompatible change to its API. We realize that existing_rules() is ugly and leads to brittle BUILD files. However, it currently doesn't have a good alternative (although the package validation proposal would address some of its current use cases), so we would like to fix the performance problems that its usage often brings.

The full text of the proposal can be found here: https://github.com/bazelbuild/proposals/blob/main/designs/2021-06-15-improving-native.existing_rules.md

-Alexandre.

[Alexandre Rostovtsev]

Benedek, do you have a link to an example of such a usage with version conflict resolution? It would be useful to add it to the doc.

On Sat, Jun 26, 2021 at 4:01 PM Benedek Thaler <thaler...@gmail.com> wrote:

Thanks for sharing!

Here's one additional use-case for native.existing_rules():

- From WORKSPACE files: load the transitive dependencies of direct
dependencies of known kinds, by iterating on the direct deps,
resolving version conflicts, and producing a bzl file that contains a
macro that adds the transitive deps.

Benedek

> --
> You received this message because you are subscribed to the Google Groups "bazel-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAC3c3JXLEh12Bseq6WFpWhsbg1bQSp8-AL0ftyXd%3D8o3%3DkvNhw%40mail.gmail.com.

[Lukács T. Berki]

At risk of being late to the party (the pull request to put this in the "approved" category arrived right after the beginning of my leave), I think this plan is reasonable, although I do lament the necessity of native.existing_rules() and I wish we could make package loading stateless, but that horse has probably left the barn.

Do I understand correctly that your dict re-implementation would omit all the methods that mutate the data structure and only leave get()/keys()/values()/items() and the operators that act on Dict? If so, go ahead -- it's a reasonably narrow interface and if someone really wants to mutate the resulting data structure, I'd much rather they change their code than we introduce the copy-on-write mechanism you mentioned in the design doc. 

I wonder if we could come up with a mechanism that makes the use of native.{existing_rule,existing_rules} less comfortable so that only people who really need it use it. It's a necessary wart, but still a wart, so I'd much rather it sees as little use as possible.

On Thu, Jun 24, 2021 at 7:14 PM 'Alexandre Rostovtsev' via bazel-dev <baze...@googlegroups.com> wrote:

--

You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAC3c3JXLEh12Bseq6WFpWhsbg1bQSp8-AL0ftyXd%3D8o3%3DkvNhw%40mail.gmail.com.

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891