Use of managed_directories

182 views
Skip to first unread message

Lukács T. Berki

unread,
May 9, 2022, 3:10:02 AM5/9/22
to al...@aspect.dev, bazel-discuss
Hey Alex (and everyone else),

Do you know what managed_directories() is used for these days?

My understanding is that it was added to support rules_nodejs and the documentation of node_js does recommend using it, but the same paragraphs also admit that there are probably a correctness issues lurking there and I heard somewhere (I don't remember where) that you think it's not very important anymore?

Context: I was poking around in code related to managed_directories() and had wistful thoughts about deleting it and the code that supports it. It's probably not feasible at this point, but I thought I'd at least reach out to see how bad it would be.

--
Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

Alex Eagle

unread,
May 9, 2022, 10:31:57 AM5/9/22
to Lukács T. Berki, Greg Magolan, bazel-discuss

The intent with this feature was to avoid developers having to install dependencies twice (node tooling typically *requires* that 3p dependencies are installed in the source tree, and that build system outputs go in the source tree as well). That's still a worthwhile goal, but your understanding is correct. I also wrote this doc about it which points out another flaw: having the external repository live in the users project interacts poorly with CI systems which do a clean before building, causing a repository invalidation on every build. If you're interested in more details on the problem and their mitigations I'm happy to write more. Short version is that the correctness and usability issues here don't feel worth the benefit.

I think deleting managed_directories is the best thing for Bazel. As always, the incompatible flag flip will have to be staged out over time, but assuming this was never used in google3 or Bazel itself, I imagine the only "work" required is to follow the incompatible change process to make it disabled-by-default in a major release, and then delete the feature in the subsequent release.

-Alex

Lukács T. Berki

unread,
May 9, 2022, 11:55:43 AM5/9/22
to Alex Eagle, Greg Magolan, bazel-discuss
On Mon, May 9, 2022 at 4:31 PM Alex Eagle <al...@aspect.dev> wrote:

The intent with this feature was to avoid developers having to install dependencies twice (node tooling typically *requires* that 3p dependencies are installed in the source tree, and that build system outputs go in the source tree as well). That's still a worthwhile goal, but your understanding is correct. I also wrote this doc about it which points out another flaw: having the external repository live in the users project interacts poorly with CI systems which do a clean before building, causing a repository invalidation on every build. If you're interested in more details on the problem and their mitigations I'm happy to write more. Short version is that the correctness and usability issues here don't feel worth the benefit.
Yeah, I'm with you there. Its semantics are complicated at best and the existence of a directory under the source tree that is somehow under the control of Bazel is quite unintuitive.
 

I think deleting managed_directories is the best thing for Bazel. As always, the incompatible flag flip will have to be staged out over time, but assuming this was never used in google3 or Bazel itself, I imagine the only "work" required is to follow the incompatible change process to make it disabled-by-default in a major release, and then delete the feature in the subsequent release.
 I'll write up a document then that kicks off the process and see whether anyone feels strongly otherwise.

Does the process require the flag to be disabled for a major release? TBH I don't remember off the bat, but the official web page of incompatible changes does not indicate that that is the case (we can still choose to do that if you think it's a good idea, this question is mostly about the official requirements)


-Alex


On Mon, May 9, 2022 at 12:10 AM Lukács T. Berki <lbe...@google.com> wrote:
Hey Alex (and everyone else),

Do you know what managed_directories() is used for these days?

My understanding is that it was added to support rules_nodejs and the documentation of node_js does recommend using it, but the same paragraphs also admit that there are probably a correctness issues lurking there and I heard somewhere (I don't remember where) that you think it's not very important anymore?

Context: I was poking around in code related to managed_directories() and had wistful thoughts about deleting it and the code that supports it. It's probably not feasible at this point, but I thought I'd at least reach out to see how bad it would be.

--
Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

Alex Eagle

unread,
May 9, 2022, 12:10:10 PM5/9/22
to Lukács T. Berki, Greg Magolan, bazel-discuss
Sorry, I meant that in the next major release the managed_directories feature would be disabled by default, so it takes only two releases to delete the code. The official documentation says you get to pick the length of the migration window before flipping the flag value, and in this case we'd propose to make that window shorter than one major to make the deletion happen sooner (if I'm understanding the process correctly).

Let me know what we can do in rules_nodejs to help.
-Alex

Lukács T. Berki

unread,
May 10, 2022, 6:14:02 AM5/10/22
to Alex Eagle, Greg Magolan, bazel-discuss, Xudong Yang
Yep, my understanding is that we can technically choose the length of the migration period to be 1 second, although that would probably not be very user-friendly. My initial idea is to set it to two months, but let's continue the discussion on the new and delightfully short design doc:


I also sent a pull request to Xudong to merge it into the design doc repository. Once approved, I'll also send an e-mail with a more appropriate subject to bazel-discuss in order to cause sufficient alarm to people who might depend on it so that they'll signal their dependency to us and we can work something out.
Reply all
Reply to author
Forward
0 new messages