ninja status report
Evan Martin
Ninja got a lot more reaction than I had expected. I have been
surprised to see all the mail and the patches. Thanks a lot!
I have been slow to respond because I try to work on Ninja in my free
time and I haven't had a lot of that lately. I'm now at the end of
a brief vacation and about to head on a longer one so I thought I'd
write a status report, since I'm unlikely to reply to any mail soon.
Currently,
- Ninja successfully builds Chrome on Linux, usually, though I keep
finding small bugs. This actually means that gyp successfully
generates .ninja files for Linux. XCode 4 [1] is reportedly
especially bad for Chrome so I've heard some interest in making
gyp+Ninja work for Mac too, though we'd probably first make gyp
generate Makefiles before trying.
- I don't know of any other real users, but I seem to have been
getting patches for Mac/FreeBSD/Windows support. Who are you
people? :)
In the near future,
- I've heard some interest about generating .ninja files from CMake.
I don't know a lot about CMake but as far as I know it is exactly
the sort of meta-build system Ninja was designed to support.
- I've applied some patches for Mac/FreeBSD support. I don't really
have the systems to test these on, so I apologize for occasionally
breaking them. I'm considering using an extra git branch (or tag,
if that makes more sense to you; they are equivalent in git) per
platform to track a known-working revision. Or maybe that's too
heavyweight.
- I've seen a couple of patches related to Windows support. I am a
bit anxious about Windows because I worry that some of my
assumptions (like that stat() is cheap when the disk is warm, or
that it's ok to read a bunch of tiny depfiles on startup) don't hold
on Windows. (For example, perhaps a database backing like how redo
works might be more performant.) With that said, I'd love it if the
people who are interested in this working could collaborate. (I'm
sorry I haven't had time to look at the patches; I'm literally
getting on a plane in a few hours...)
Ninja mostly does what I want right now so I don't anticipate changing
it too much. But I do have a few bits to polish:
- I'm considering making implicit dependencies have the same semantics
as explicit ones (except for not appearing in $in). That is to say,
I'd remove the "file missing is still ok" rule and instead
special-case edges as discovered through depfiles. I think this might
make the semantics of Ninja simpler, though opinions are welcome.
- I'm not too happy about how there's a special "builddir" variable. I'm
still mulling over options there. For Chrome's purposes it'd be nice
to be able to generate multiple sets of build files in parallel that
use different build dirs, so perhaps the right path is making builddir
even *more* special and accepting a new value for it on the command line
(e.g. "ninja -d release/ chrome" would build the "chrome" target from
the build files found in the "release/" directory).
- I'd like to check in an "official" Python module that generates
Ninja syntax. (E.g. the class would have methods like
"write_rule()"). I could use this from gyp, and it would also aid
anyone else who wanted to generate their own project files. Ideally
I'd make this public domain so that it could just be inlined into
whichever projects need it, though maybe my work wouldn't like that.
- I collect some timing stats via the .ninja_log file; I'd like to
write some sort of visualizer for them. It also recently occurred
to me that if I logged one specific number -- "how many parallel
jobs were running when this command finished?" -- that could show
you points in your build that aren't parallelizing well.
- It's a little embarrassing how the code is styled where any object
can poke into any other one (all the fields are public). I mostly
did it this way because I wasn't certain how things would end up,
but now that they are more settled I may try to make it more OO-ish.
It's probably not worth it though.
Evan Jones
> - I don't know of any other real users, but I seem to have been
> getting patches for Mac/FreeBSD/Windows support. Who are you
> people? :)
I'm a perpetual build system hater. A few years ago I foolishly
thought I quickly hack up a tool that would actually get "precise
rebuilds" for my small C++ project. I only got as far as the build
graph traverser, so what I ended up with looks a lot like Ninja. At
this point, my small C++ project has grown, and my build system has
"interesting" bugs I don't want to fix. I've been playing with Ninja
because it seems to have a similar design, but is WAY more polished.
> - I've applied some patches for Mac/FreeBSD support. I don't really
> have the systems to test these on, so I apologize for occasionally
> breaking them. I'm considering using an extra git branch (or tag,
> if that makes more sense to you; they are equivalent in git) per
> platform to track a known-working revision. Or maybe that's too
> heavyweight.
I've been testing Ninja on my Mac and on Linux. My vote: I'm happy to
hack around the minor incompatibilities that crop up occasionally and
send you patches.
Evan
Nicolas Desprès
On Mar 23, 12:05 am, Evan Martin <mart...@danga.com> wrote:
> Hi everyone,
>
> Ninja got a lot more reaction than I had expected. I have been
> surprised to see all the mail and the patches. Thanks a lot!
>
[...]
> In the near future,
>
> - I've heard some interest about generating .ninja files from CMake.
> I don't know a lot about CMake but as far as I know it is exactly
> the sort of meta-build system Ninja was designed to support.
>
to now, I handle basic executable and shared libraries. All the work
is in a local branch on my computer and not yet published on CMake
staging repository. All the discussions happens on cmake-developers
mailing list (http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-
developers)
Here are the announcement thread when I first heard about Ninja:
http://www.cmake.org/pipermail/cmake/2011-February/042656.html
Here some discussions threads about the development of the Ninja
generator for CMake:
http://public.kitware.com/pipermail/cmake-developers/2011-April/001480.html
http://public.kitware.com/pipermail/cmake-developers/2011-April/001528.html
[...]
>
> Ninja mostly does what I want right now so I don't anticipate changing
> it too much. But I do have a few bits to polish:
>
> - I'm considering making implicit dependencies have the same semantics
> as explicit ones (except for not appearing in $in). That is to say,
> I'd remove the "file missing is still ok" rule and instead
> special-case edges as discovered through depfiles. I think this might
> make the semantics of Ninja simpler, though opinions are welcome.
dependencies to add dependencies not appearing in $in. Also I think it
would be good to be able to "slice" $in and $out since for some
command you want to interleave some flags between input files or
sometimes just remove one.
Here a use case from the doxygen integration in Ninja:
# Generate Doxygen.
rule doxygen
command = doxygen $in
description = DOXYGEN $in
doxygen_mainpage_generator = ./gen_doxygen_mainpage.sh
rule doxygen_mainpage
command = $doxygen_mainpage_generator $in > $out
description = DOXYGEN_MAINPAGE $out
build $builddir/doxygen_mainpage: doxygen_mainpage \
README HACKING COPYING | $doxygen_mainpage_generator
build doxygen: doxygen doxygen.config || $builddir/doxygen_mainpage
With this setup we don't get doxygen rebuilt if you touch README has
it ought to be. So you can fix that by
1) build doxygen: doxygen doxygen.config | $builddir/doxygen_mainpage
2) build doxygen: doxygen doxygen.config $builddir/doxygen_mainpage
both work but to be truly correct 2 should use a doxygen rule written
like that:
rule doxygen
command = doxygen $in[0]
description = DOXYGEN $in[0]
[...]
>
> - I collect some timing stats via the .ninja_log file; I'd like to
> write some sort of visualizer for them. It also recently occurred
> to me that if I logged one specific number -- "how many parallel
> jobs were running when this command finished?" -- that could show
> you points in your build that aren't parallelizing well.
>
> - It's a little embarrassing how the code is styled where any object
> can poke into any other one (all the fields are public). I mostly
> did it this way because I wasn't certain how things would end up,
> but now that they are more settled I may try to make it more OO-ish.
> It's probably not worth it though.
definitely helps people to hack Ninja.
>
> [1]http://code.google.com/p/chromium/wiki/Xcode4Tips
Nicolas Desprès
[...]
>>
>> - It's a little embarrassing how the code is styled where any object
>> can poke into any other one (all the fields are public). I mostly
>> did it this way because I wasn't certain how things would end up,
>> but now that they are more settled I may try to make it more OO-ish.
>> It's probably not worth it though.
>
> I could give a hand for this. Improving internal documentation would
> definitely helps people to hack Ninja.
>
Maybe we could just start to properly encapsulate all attributes by
making them private. This is very handy when you want to optimize
without changing the interface of your class and that will prevent
people from directly accessing the internal attributes.
--
Nicolas Desprès