Remove records from .ninja_deps

[Alfred Zien]

Hello!

As we've migrated to ninja 1.8 with support for multiple dependencies for deps = gcc, we started to have issue with dependencies graph. 

Imagine you have dependency graph like this:  A -> B -> C  and then it changes to  A -> C -> B. If that dependency information is in build.ninja file itself, there is no problem, ninja will just recompile that is needed, if that info is in depfile, ninja will yell about cycle but you can safely remove .d files for B and C, and then it will be fine. But if that information in .ninja_deps file, there is no way to remove it from there, the only way is to do a clean build.

The sample build.ninja file to reproduce the issue.

rule file_with_deps

  depfile = $out.d

  deps = gcc

  command = touch $out && echo "$out: $in" > $depfile

rule touch

  command = touch $out

build out/foo: touch

# comment after first build

build out/bar: file_with_deps out/foo

build out/baz: file_with_deps out/bar

# uncomment after first build

# build out/bar: file_with_deps out/baz

# build out/baz: file_with_deps out/foo

default out/baz

Do you know how to workaround this issue? Can this be resolved on ninja side? As I understand it is safe to ignore dependencies from depfiles in case of dependency cycles.

Thanks,

- Alfred

[David Turner]

Hello Alfred,

We encounter the same kind of issues in the Fuchsia build, and haven't found a way to work around them, except for forcing a clean build when we detect this happens :-(

It seems the root issue is that Ninja does not record that dependencies come from depfiles. If it did, it could ignore them when it determines there is an issue (e.g. output is no longer in the graph, implicit input does no longer exist), but it's difficult to say how easy that feature would be to implement, and how much it would make the code more complex / less maintainable, or introduce surprising edge cases.

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ninja-build/346683F7-1EC3-41CF-90BC-64435B3A3149%40yandex-team.ru.

[Ben Boeckel]

On Mon, May 02, 2022 at 20:46:32 +0200, 'David Turner' via ninja-build wrote:
> Hello Alfred,
>
> We encounter the same kind of issues in the Fuchsia build, and haven't
> found a way to work around them, except for forcing a clean build when we
> detect this happens :-(
>
> It seems the root issue is that Ninja does not record that dependencies
> come from depfiles. If it did, it could ignore them when it determines
> there is an issue (e.g. output is no longer in the graph, implicit input
> does no longer exist), but it's difficult to say how easy that feature
> would be to implement, and how much it would make the code more complex /
> less maintainable, or introduce surprising edge cases.

I wonder if making the information there "depend on" the build rule
itself and consider it out of date if that changes would work (after
all, a `-I` change could very well cache-invalidate a header lookup
resolution)? I don't think ninja has such a concept though. Maybe
deferring integrating the information until the build node is considered
would be enough (though this may run into the same limitations which
prevent the optimizations present in
https://github.com/ninja-build/ninja/issues/1760 from being used)?

--Ben