Listing all file targets

[Allan Odgaard]

I recently updated my meta build system so that when build.ninja is regenerated it removes all files which were created using the old build file, but which no longer have a target using the new build file.

To obtain the list of targets I run:

ninja -t targets all

This does not include .d files. Is this an oversight, or is there a (philosophical) reason we wouldn’t want to consider .d files as targets? I’d of course like to request that .d files were included.

Additionally I have a minor issue with non-file targets. These can be classified into:

1. Targets that does not generate a file, for example ‘clean’.
2. Targets which generate a file based on SCM info, network downloads, database dumps, or similar. These should always be considered out-of-date.

My solution to the first case is to specify e.g. ‘$builddir/.never-exists’ as the target we build and then use the phony rule to give it a nicer name.

Perhaps it’s too much of an edge-case to justify new syntax, since there is a workaround. I just feel being explicit would be nicer. Especially for the second case, where we do generate a file, but can’t tell ninja about it, because then ninja will not re-run our command even though the file is to be considered out of date.

A minor side note, using -C with ninja outputs “changing directory” to stdout. This means a script will have to account for that if it parses output from ninja (like the ‘-t targets all’). Probably this should be stderr and/or suppressible.

[Nico Weber]

The note is needed for tools like emacs, else they won't be able to
parse error messages. It was added here:
https://github.com/martine/ninja/issues/124. As of ninja 1.0.0, this
message should only be printed when doing a build, not when using a
tool (https://github.com/martine/ninja/pull/368). So you only have to
update your ninja binary to make this part work.

Nico

[Richard Geary]

If you don't want the message printed at all, I've added a -q quiet option to suppress it (patch available in github issue #420)

[Nicolas Desprès]

On Sat, Sep 15, 2012 at 6:14 PM, Allan Odgaard <text...@gmail.com> wrote:

I recently updated my meta build system so that when build.ninja is regenerated it removes all files which were created using the old build file, but which no longer have a target using the new build file.

To obtain the list of targets I run:

    ninja -t targets all

This does not include .d files. Is this an oversight, or is there a (philosophical) reason we wouldn’t want to consider .d files as targets? I’d of course like to request that .d files were included.

It prints what you can pass as argument to ninja. It is mainly useful for implementing shell completion. See the full commit message: https://github.com/martine/ninja/commit/3ad977241b24ea8077073f1f3feecab7b8dc0b55. I think this tool is not meant to print the .d files.

If you want to remove all generated files you can use ninja -t clean or just remove the build directory.

[...]

Cheers,

--
Nicolas Desprès

[Allan Odgaard]

On Sep 27, 2012, at 10:53 AM, Nicolas Desprès <nicolas...@gmail.com> wrote:

> […] It prints what you can pass as argument to ninja. It is mainly useful for implementing shell completion.

I see (that it was written with that in mind). In this case, I suggest a variant of ‘all’, e.g.:

ninja -t targets callable # for shell completion
ninja -t targets all # return *all* targets

> […] If you want to remove all generated files you can use ninja -t clean or just remove the build directory.

I think you missed my explanation of what I was doing, quoting myself:

> I recently updated my meta build system so that when build.ninja
> is regenerated it removes all files which were created using the
> old build file, but which no longer have a target using the new
> build file.

So imagine you have a source tree and you rename “image.tiff” to “image.png”.

After this, build.ninja is automatically updated to now copy “image.png” to the build directory and “ninja -t clean” will _not_ remove “image.tiff” from the build directory (as that is now unknown).

Sure, you _could_ remove the entire build directory after such change, but these changes can happen while switching branches, or you just might forget. For example if I remove a framework from my source tree I may forget that some header file was generated from this framework (placed into the build directory) and some other source, which hasn’t been removed, includes this generated file — my build will continue to work until I remove the entire build directory (something I am unlikely to do), hence why it is important that the build system catches these things.

[Nicolas Desprès]

On Thu, Sep 27, 2012 at 12:08 PM, Allan Odgaard <text...@gmail.com> wrote:

On Sep 27, 2012, at 10:53 AM, Nicolas Desprès <nicolas...@gmail.com> wrote:

> […] It prints what you can pass as argument to ninja. It is mainly useful for implementing shell completion.

I see (that it was written with that in mind). In this case, I suggest a variant of ‘all’, e.g.:

    ninja -t targets callable # for shell completion
    ninja -t targets all      # return *all* targets

I'm not sure .d files can be considered as targets. Any way, maybe you could just remove $target.d file.

 

> […] If you want to remove all generated files you can use ninja -t clean or just remove the build directory.

I think you missed my explanation of what I was doing, quoting myself:

    > I recently updated my meta build system so that when build.ninja
    > is regenerated it removes all files which were created using the
    > old build file, but which no longer have a target using the new
    > build file.

So imagine you have a source tree and you rename “image.tiff” to “image.png”.

After this, build.ninja is automatically updated to now copy “image.png” to the build directory and “ninja -t clean” will _not_ remove “image.tiff” from the build directory (as that is now unknown).

I meant run ninja -t clean with the old ninja files.

 

Sure, you _could_ remove the entire build directory after such change, but these changes can happen while switching branches, or you just might forget. For example if I remove a framework from my source tree I may forget that some header file was generated from this framework (placed into the build directory) and some other source, which hasn’t been removed, includes this generated file — my build will continue to work until I remove the entire build directory (something I am unlikely to do), hence why it is important that the build system catches these things.

Your script could run "ninja -t clean". This tool is meant to remove generated files.

--
Nicolas Desprès

[Maxim Kalaev]

On Thursday, September 27, 2012 4:53:24 PM UTC+2, Nicolas Desprès wrote:

Your script could run "ninja -t clean". This tool is meant to remove generated files.

I believe that Allan wants to delete some of the targets, such that are not referenced in the new branch.

BTW, it is a nice feature which I'd like to use too. I am just wondering if there can be a solution better than writing a custom script for calculating the intersection between the targets. - It feels like ninja's build log could help.

[Nicolas Desprès]

Maybe the clean with the -n and -v option on.

--
Nicolas Desprès

[Nicolas Desprès]

Also, I'm not against extending the targets tool.

--
Nicolas Desprès

[Maxim Kalaev]

I've implemented a 'clean orphan targets' feature which is related to the described problem: https://github.com/martine/ninja/pull/447.
If you have a chance trying it, please let me know how it works for you.

[Allan Odgaard]

On Oct 12, 2012, at 3:17 PM, Maxim Kalaev <maxim...@gmail.com> wrote:

>> […]

> I've implemented a 'clean orphan targets' feature which is related to the described problem:

Thanks — is this based on the targets listed in the log file?

I have my build system setup to automatically recreate build.ninja if there are (potential relevant) changes in the source tree, so I assume that I should have this rebuild phase perform ‘ninja -t clean ^‘ after recreating build.ninja.

[Maxim Kalaev]

Yes & yes. It is based on information stored in .ninja_log and it only makes sense to clean orphans when a new build.ninja is generated.