Introducing a special depfile variable

[Allan Odgaard]

With the recent addition of ‘deps = (gcc|msvc)’ it seems the simplest build file will now be:

rule cc
command = $CC $FLAGS -o $out -MMD -MF $out.d $in
depfile = $my_depfile
deps = gcc

build my$ source.o: cc my$ source.c
my_depfile = my$ source.o.d

There are two issues here, one is that we can’t use ‘depfile = $out.d’ because the filename may contain spaces, in which case the $out variable is escaped prior to being used internally by ninja, thus we need to use a variable that doesn’t have this magic escaping (I believe there already is one or two issues about this).

The other issue is that for the common case, we’ll now need two lines to tell ninja about our dependency file (location + format). I thought the addition of ‘deps = (gcc|msvc)’ was done to be backwards compatible, but adding this line does break when using older versions of ninja.

So my first thought was, why not introduce a new ‘gcc_depfile’ and ‘msvc_depfile’ variable so that we set both location and format in one line.

This however made me think, why set a variable at all? How about ninja scans the command for ‘$(gcc|msvc)_depfile’ and if set, it will populate that variable with a temporary filename.

So the above would instead be:

rule cc
command = $CC $FLAGS -o $out -MMD -MF $gcc_depfile $in

build my$ source.o: cc my$ source.c

Only if the command uses this variable, would ninja populate the variable and subsequently read and delete the file created, so there should be no overhead for the case where the variable is not used in the command (other than the string scan, but the shorter build file should make up for that).

[Nico Weber]

I think your generator can set `deps=` to disable depsdb for source files that contain spaces (which is probably uncommon).

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Evan Martin]

For what it's worth, with msvc you don't need to specify a depfile at all (msvc outputs dependencies to stdout while building, so Ninja just parses from there instead of using a temporary file).

Your idea (of automatically providing the path) is interesting.  I guess it would work around your spaces issue too, wouldn't it?  (In that $gcc_depfile could string expand to a quoted path like who $in does.)

Some random thoughts:

- In the case where you're not using deps mode (where the .d files need to stick around between builds), Ninja isn't really equipped to generate a long-lived path for you.  This means we might need to vary behavior (generate paths vs not) in deps vs non-deps mode.

- We don't really have a precedent for "magic" behavior like this (where accessing a variable automatically makes it both exist and have meaning).  Though maybe you could squint at it and say it's like "$in", like: "If you specify deps=gcc, then the variable $depfile is also provided by Ninja to point at a temporary file."

- We have a similar issue with rspfile, though maybe its requirements are different (you need those files to stay around if you want to debug your build).

On Fri, May 24, 2013 at 9:52 PM, Allan Odgaard <text...@gmail.com> wrote:

[Allan Odgaard]

On May 25, 2013, at 23:41, Evan Martin <mar...@danga.com> wrote:

> […] Your idea (of automatically providing the path) is interesting. I guess it would work around your spaces issue too, wouldn't it? (In that $gcc_depfile could string expand to a quoted path like who $in does.)

Yes, it would also be a fix for the spaces issue.

> Some random thoughts:
> - In the case where you're not using deps mode (where the .d files need to stick around between builds), Ninja isn't really equipped to generate a long-lived path for you. This means we might need to vary behavior (generate paths vs not) in deps vs non-deps mode.

I hadn’t considered that case. I assume though that if people want the .d file to stick around, they want to control the path themselves.

> - We don't really have a precedent for "magic" behavior like this (where accessing a variable automatically makes it both exist and have meaning). Though maybe you could squint at it and say it's like "$in", like: "If you specify deps=gcc, then the variable $depfile is also provided by Ninja to point at a temporary file."

That does remove the magic (and I generally prefer explicit, as it leaves no room for misunderstandings). It gives us 4 total cases:

1. ‘deps’ unset and ‘depfile’ unset → ninja does nothing
2. ‘deps’ unset and ‘depfile’ set → ninja uses depfile
3. ‘deps’ set and ‘depfile’ set → ninja copies to cache and unlinks
4. ‘deps’ set and ‘depfile’ unset → ninja sets $depfile for execution, copies to cache and unlinks


Here #2 would be for users that want the .d files to stick around and #3 would be legacy since #4 offers handling of shell escaping with less syntax.

Possibly #4 would only have my stated action for ‘deps = gcc’ and for ‘deps = msvc’ it would read from stdout rather than involve the temporary file (if I understood you correct).

If we conisder #3 legacy my only dislike with the above is that #2 leaves the shell escaping to the user, which is somewhat arcane. However, this is also the current situation, and I would think very few users would actually need #2, so the manual would encourage #4 and mention #2 as a possibility for those who need full control over the lifetime of their .d files.