subninja and paths

[Vladimir Vukicevic]

Howdy,

I expected files read via subninja to have their path modified accordingly (or at least the executed rules to behave as if cwd was the location of the ninja file), but they don't seem to.  I've got a project that has a toplevel build.ninja that includes a file with some rules, and then calls subninja to include some additional rules from a subdir.  That subdir can be built as a project of its own; it has its own build.ninja with the same rules and includes the target.ninja file.

If that explanation doesn't make sense:

a/build.ninja:
  rule cc...
  rule link...
  include targets.ninja
  subninja b/c/targets.ninja

a/targets.ninja:
  build foo.o: cc foo.c
  build foo: link foo.o b/c/bar.o

a/b/c/build.ninja (identical to a/build.ninja, minus subninja line)
  rule cc...
  rule link...
  include targets.ninja

a/b/c/targets.ninja
  build bar.o: cc bar.c

I would expect that the build command for bar.o would have "b/c" as the cwd, but that's not the case; it's all executed from the toplevel.  (Doesn't even start executing, because the bar.c dependency can't be found -- it's in a/b/c/bar.c, not in a/bar.c.)

Is being able to support this type of subproject behaviour desired?  I saw the thread at https://groups.google.com/forum/#!topic/ninja-build/aqT63egYRXI talking about a similar setup (and with a similar solution for the edges), but seems like it would run into the same issue.

    - Vlad

[Nico Weber]

Hi,

On Sun, Aug 4, 2013 at 11:48 PM, Vladimir Vukicevic <vlad...@pobox.com> wrote:

Howdy,

I expected files read via subninja to have their path modified accordingly (or at least the executed rules to behave as if cwd was the location of the ninja file), but they don't seem to.  I've got a project that has a toplevel build.ninja that includes a file with some rules, and then calls subninja to include some additional rules from a subdir.  That subdir can be built as a project of its own; it has its own build.ninja with the same rules and includes the target.ninja file.

If that explanation doesn't make sense:

a/build.ninja:
  rule cc...
  rule link...
  include targets.ninja
  subninja b/c/targets.ninja

a/targets.ninja:
  build foo.o: cc foo.c
  build foo: link foo.o b/c/bar.o

a/b/c/build.ninja (identical to a/build.ninja, minus subninja line)
  rule cc...
  rule link...
  include targets.ninja

a/b/c/targets.ninja
  build bar.o: cc bar.c

I would expect that the build command for bar.o would have "b/c" as the cwd, but that's not the case; it's all executed from the toplevel.  (Doesn't even start executing, because the bar.c dependency can't be found -- it's in a/b/c/bar.c, not in a/bar.c.)

Correct, all paths are relative to the build directory. You probably build the toplevel build file with

  ninja -C a

Since all paths are relative to the build director ("a" in this case), bar.o will end up in a/ too, since the "bar.o" in "build bar.o" is relative to that build directory too – this is probably not what you want either, and I guess you have an old bar.o flying around in b/c/ which makes this work?

If you make sure all your paths are relative to your (toplevel) build directory, you might be able do build just a/b/c by doing `ninja -C . -f b/c/build.ninja` (I haven't tried that though).

(ninja generators usually set things up so that all build artifacts end up in a build directory, not next to the source files, and if you're writing `ninja -C build -f gen/b/c/build.ninja`, generating a named target for "b/c" might be more convenient.)

Nico

 

Is being able to support this type of subproject behaviour desired?  I saw the thread at https://groups.google.com/forum/#!topic/ninja-build/aqT63egYRXI talking about a similar setup (and with a similar solution for the edges), but seems like it would run into the same issue.

    - Vlad

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Richard Geary]

> (ninja generators usually set things up so that all build artifacts end up in a build directory, not next to the source files, and if you're writing `ninja -C build -f gen/b/c/build.ninja`, generating a named target for "b/c" might be more convenient.)

As someone who has written a generator, I agree this wasn't obvious to
me initially. Are there any conventions of what a ninja generator
should produce & where?
I wanted to build different default targets when running ninja from
different source directories, so putting ninja files in a build
directory seemed contradictory at first. Eventually I wrote a bash
wrapper script to launch ninja rather than extend it. It provides
ninja with a -C argument to point to the build dir, and a -f argument
to point to the current dir's ninja file (which then subninja's in to
the build dir's ninja file). Not pretty, but probably the best
solution.

Also ninja will error if you include/subninja a ninja file twice, ie.
it won't ignore the 2nd include. Thus it makes it hard to reuse
existing ninja scripts - eg. say project P depends on A,B, and both
A&B depend on D, and P,A,B & D all have their own ninja files. In this
case, you would want P to subninja in to A & B's ninja files. But you
can't, as D would be included twice.

[Nico Weber]

On Wed, Aug 21, 2013 at 5:41 PM, Richard Geary <richar...@gmail.com> wrote:

> (ninja generators usually set things up so that all build artifacts end up in a build directory, not next to the source files, and if you're writing `ninja -C build -f gen/b/c/build.ninja`, generating a named target for "b/c" might be more convenient.)

As someone who has written a generator, I agree this wasn't obvious to
me initially. Are there any conventions of what a ninja generator
should produce & where?

Not written down anywhere, as far as I know. Evan has been collecting links to various threads on this list at https://github.com/martine/ninja/wiki/Build-Patterns (I think the wiki is world-editable if someone wants to flesh this out a bit – it could need it), that's all the documentation I'm aware of.

Both cmake and gyp let ninja write build artifacts into a dedicated build folder. cmake makes you pick the configuration you want (Debug or Release) at cmake time, while gyp generates "Debug" and "Release" subfolders in the build folder (each with its own build.ninja file).

 

I wanted to build different default targets when running ninja from
different source directories, so putting ninja files in a build
directory seemed contradictory at first. Eventually I wrote a bash
wrapper script to launch ninja rather than extend it. It provides
ninja with a -C argument to point to the build dir, and a -f argument
to point to the current dir's ninja file (which then subninja's in to
the build dir's ninja file). Not pretty, but probably the best
solution.

Yes, I think that's the best you can do if you don't want to generate named phony targets for all directories.

Thinking out loud: Your generator could also write a "build.sh" into every directory that invokes ninja with the right arguments, then you could build every directory with a/b/c/build.sh.

(In my experience, folders map more or less nicely to actual targets though and I still don't understand the "build this directory" use case well, so my thinking out loud on this topic probably isn't terribly useful.)

 

Also ninja will error if you include/subninja a ninja file twice, ie.
it won't ignore the 2nd include. Thus it makes it hard to reuse
existing ninja scripts - eg. say project P depends on A,B, and both
A&B depend on D, and P,A,B & D all have their own ninja files. In this
case, you would want P to subninja in to A & B's ninja files. But you
can't, as D would be included twice.

That's probably part of pushing work into generators to keep ninja itself fast. (Which is true of ninja requiring complete knowledge of the build graph too instead of supporting some recursive make thing: that allows for better scheduling of parallelism.)

Nico