Pros and cons of always-run build commands

[jvp...@g.rit.edu]

This is mainly a design-philosophy question. In Make, I have two uses for always-run build commands: 1) to provide simple aliases for common ancillary actions, like running tests, which I naturally want to happen even if the build hasn't changed (e.g. to catch intermittent failures), and 2) as a way of running a script to determine if the build file needs regenerated.

For the first case (running tests, etc), is this something that makes sense in a Ninja file? I know it's not specifically a part of what Ninja aims to do (provide for fast compilation), but it seems to be something that Ninja can provide almost for free, if you don't mind adding some slightly-hacky rules. I find that interacting with a single entry point for all of these common actions (build, test, package, install, etc) to be pretty convenient, but maybe Ninja experts have a better way? If so, I'd love to hear about it.

For the second case (regenerating the build file based on some command), I know this is explicitly against Ninja's design philosophy (based on previous messages to this list), but I nevertheless think that it has some merit. Obviously, this slows down builds, but used intelligently, I think the performance cost can remain low. Looking through the archives of this list, it seems the main reason for wanting to do this is to allow globbing in your meta-build script (ditto for me). One way that seems more in keeping with Ninja's design is to have the rule that regenerates the build.ninja file use a depfile, but that can actually force build.ninja to be regenerated *more* often than running a specialized script, since the script can analyze the globs more closely and eliminate some false positives. In any case, if builds started to get unacceptably slow due to scanning for changes in your globs, you could always remove those globs and explicitly refer to the files. Then you're not forced to, say, migrate from Make to Ninja *after* your project is huge; you can just optimize your build scripts and get on with your life. Does any of this sound sane, or am I just trying to force a square peg into a round hole here?

Apologies if this is the sort of thing that causes religious wars around here; like many others, I've been experimenting with meta-build systems, and I wanted to see what the recommended ways of doing these things in Ninja were.

Thanks,
Jim

[Matthew Woehlke]

On 2015-03-02 00:01, jvp...@g.rit.edu wrote:
> This is mainly a design-philosophy question. In Make, I have two uses for
> always-run build commands: 1) to provide simple aliases for common
> ancillary actions, like running tests, which I naturally want to happen
> even if the build hasn't changed (e.g. to catch intermittent failures), and
> 2) as a way of running a script to determine if the build file needs
> regenerated.
>
> For the first case (running tests, etc), is this something that makes sense
> in a Ninja file? I know it's not specifically a part of what Ninja aims to
> do (provide for fast compilation), but it seems to be something that Ninja
> can provide almost for free, if you don't mind adding some slightly-hacky
> rules. I find that interacting with a single entry point for all of these
> common actions (build, test, package, install, etc) to be pretty
> convenient, but maybe Ninja experts have a better way? If so, I'd love to
> hear about it.

I do believe you already get this (first case) if you are using CMake to
generate Ninja files. Presumably that means that there is a way to
express it in Ninja.

Can't you (also) achieve the second already by making build.ninja depend
on running your script? Just have it not do anything if you don't
actually need to regenerate the build.ninja; I believe that should
already be working.

--
Matthew

[Nico Weber]

The first case has been discussed a bit here (with example code snippes): https://groups.google.com/forum/#!topic/ninja-build/9HSGDHcAp5U

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jvp...@g.rit.edu]

On Monday, March 2, 2015 at 9:29:37 AM UTC-6, Matthew Woehlke wrote:

I do believe you already get this (first case) if you are using CMake to
generate Ninja files. Presumably that means that there is a way to
express it in Ninja.

Yeah, I've figured out how to do it (the easiest way seems to be to make a phony target with a name that will never refer to a real file), but I'm not exactly happy with that as a solution. Ditto with the solution Nico posted. Now that I think about it, I think this is a more general problem with the phony target in Ninja: if a phony build item is being used as an alias, that alias breaks if the phony's target actually exists on the filesystem. This is actually what .PHONY in Make was designed to fix, so it's a bit weird to see the name reused like this.

Can't you (also) achieve the second already by making build.ninja depend
on running your script? Just have it not do anything if you don't
actually need to regenerate the build.ninja; I believe that should
already be working.

Yep, I've actually managed to implement the second way (with a bit more indirection so that I can have separate configure and check-for-updates scripts): <https://github.com/jimporter/bfg9000/issues/1#issuecomment-76674176>. It too relies on the phony trick above, so I'm not exactly happy with it. I also wanted to know if other people who have encountered this problem have solved it in a fundamentally different way (e.g. by not using ninja directly, or using something like inotify to watch for changes in real-time), or even if I should just give up on this because of the performance implications.

- Jim