Touching BUILD or WORKSPACE slows down the build

[Dan Fabulich]

While we're transitioning to Bazel, we're automatically generating our BUILD and WORKSPACE files from other sources.

I'm a bit surprised to see that the Bazel build gets a lot slower when we regenerate/touch the BUILD/WORKSPACE files, even when the files themselves are unchanged.

In this example: https://github.com/dfabulich/touch-build a clean build takes 11s, and rebuilding takes 0.2s, but if I touch BUILD it takes 1.5s, and if I touch WORKSPACE it takes 3.3s.

Is this expected? I feel like I'm going to work around it by making my generator skip writing the file if it's not changing anything.

$ bazel clean

INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 11.058s, Critical Path: 7.72s

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 0.167s, Critical Path: 0.00s

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 0.153s, Critical Path: 0.00s

$ touch BUILD

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 1.546s, Critical Path: 0.19s

$ touch BUILD

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 1.562s, Critical Path: 0.19s

$ touch WORKSPACE

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 3.281s, Critical Path: 1.75s

$ touch WORKSPACE

$ bazel build //:x

INFO: Found 1 target...

Target //:x up-to-date:

  bazel-bin/libx.jar

INFO: Elapsed time: 3.296s, Critical Path: 0.23s

[Damien Martin-guillerez]

The 1.5 seconds is not that surprising the BUILD file needs to be reparsed to see that nothing has changed. The 3 seconds of the WORKSPACE is a bit more surprising. Running with -s show that no action get reexecuted but it does invalidate the corresponding package.

I have 4s for both case on my mac laptop. It spend time writing the workspace status files

After running the profiler, it seems to spend 4s on the skyframe evaluator mostly in the skylark macros.

+Laurent Le Brun +Florian Weikert profiles says 800ms for a 1000 iteration loop, seems like a lot.

Also it spends 450ms in skyframe evaluation. Finally there are 11 calls to skyfunctions that take 80ms each. 

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/b756256f-44aa-4cfe-a18c-c8f54bba29ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Fabulich]

When you say "the BUILD file needs to be reparsed," is there something more going on here than parsing the file? It's only a dozen lines long, with another dozen lines in generator.bzl. That couldn't possibly take 1.5 seconds, could it?

[Damien Martin-guillerez]

It does more things.

It reevaluate the package, that is the build file which include parsing and generating of corresponding skyfunctions. From the analysis it says it reexecute 11 skyfunctions, each calls takes 80ms (I don't know which skyfunctions but 80ms is a lot). Then it spends 800ms reruning your macros (that is also too much). It seems like we have several problems in that simple example.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/964e74f6-1644-4cb7-9946-36bfd2bb1bc3%40googlegroups.com.

[Dan Fabulich]

I filed https://github.com/bazelbuild/bazel/issues/923 on this.

[Damien Martin-guillerez]

Thanks I added the profile information to the bug

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/76561284-68F4-43C8-B3A8-853CB37FAE27%40gmail.com.