Re: Ninja: 1.9 incorrect dirty targets?

[Takuto Ikuta]

Sounds like this is not a bug for gn nor ninja.

But what happens if you change rule for alink to `command = rm -f ${out} && ar rcs ${out} ${in} && touch ${out}` ?

On Tue, Sep 10, 2019 at 6:13 AM Charles Nicholson <nicho...@verily.com> wrote:

More data if it's helpful:

My hard drive is APFS, and almost all of my intermediate (compiled) files have nonzero timestamps. The file on disk has the truncated-nanosecond timestamp.

>>> os.stat('libhello.a').st_mtime_ns
1568062644000000000
>>> os.stat('libhello.hello.o').st_mtime_ns
1568062644560564461

I've also realized that I'm using /usr/bin/ar to assemble a static library, and not putting together a mac toolchain based on the detected xcode version, though /usr/lib/ranlib claims to come from Xcode. I see that CMake, when interrogating a host os, finds the current version of Xcode and uses

/Applications/Xcode-<your version>.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar

and

/Applications/Xcode-<your version>.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib

I'm not running ranlib, perhaps the version of ar that I'm using isn't sophisticated enough to know about nanosecond timestamps? I'd assume those to be the domain of the OS.

On Mon, Sep 9, 2019 at 4:31 PM Charles Nicholson <nicho...@verily.com> wrote:

I should also mention that I only see this on my Macbook Pro, and not my Linux desktop. I haven't had the chance to run it on Windows yet.

On Mon, Sep 9, 2019 at 4:27 PM Charles Nicholson <nicho...@verily.com> wrote:

I'm learning GN and my sample project has a static library (libhello.a) that depends on a single c++ file (libhello.cc).

ninja --version gives me 1.9.0, so I'm on the newest released version with sub-second timestamp accuracy.

The generated ninja script for this target looks like this:

defines =
include_dirs = -I../../../libhello/include
cflags = -ffunction-sections -fdata-sections -Wall -Wextra -pedantic -O0 -g
cflags_cc = -std=c++14
target_out_dir = obj/libhello
target_output_name = libhello

build obj/libhello/libhello.hello.o: cxx ../../../libhello/hello.cc

build obj/libhello/libhello.a: alink obj/libhello/libhello.hello.o
  arflags =
  output_extension = .a
  output_dir =

My "alink" tool is simple, doesn't use response files, just

rule alink
  command = rm -f ${out} && ar rcs ${out} ${in}
  description = ar ${target_output_name}${output_extension}

When I run my build on mac ("ninja -C out/tmp/debug"), all of my targets build as expected, and I see my files. libhello.hello.o and libhello.a have the exact same timestamp though. When I run again, ninja sees all of my static library targets as dirty and re-archives them, then re-links everything downstream of them. This wasn't necessary; everything was already built and I haven't touched any files since the last run.

Instead, if I build once and then run "ninja -d explain", I see the following:

/Users/nicholsonc/src/gntest: ./b -- -d explain
ninja: Entering directory `/Users/nicholsonc/src/gntest/out/tmp/debug'
ninja explain: output obj/libhello/libhello.a older than most recent input obj/libhello/libhello.hello.o (1568060338000000000 vs 1568060338064635267)
ninja explain: obj/libhello/libhello.a is dirty

libhello.a looks like it has an extremely suspicious timestamp to me, the final 9 digits are all zeroes. I suspect that its actual timestamp is a hair above that of libhello.hello.o, and something's truncating it down, which moves the timestamp to before the .o file.

I see this phenomenon across all of my static library targets, not just this one. The .a files have truncated timestamp files, and the .o files have high-precision ones.

Am I doing something wrong here? This is low enough in the implementation guts of ninja and maybe GN that I'm not sure exactly what the next step is, other than recompiling the ninja code and adding some printf debug statements that spit out the timestamps.

Thanks in advance for any advice,

Charles

--
To unsubscribe from this group and stop receiving emails from it, send an email to gn-dev+un...@chromium.org.

--

Takuto Ikuta

Software Engineer in Tokyo

Chrome Ops (goma team)

[Nico Weber]

Also, on macOS, `libtool` is faster than `ar rcs` – does it help if you switch to libtool?

Obligatory https://github.com/llvm/llvm-project/blob/master/llvm/utils/gn/build/toolchain/BUILD.gn#L48

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ninja-build/CALNjmMouR9NLuTWidUdTYY%3D9MxbQFS4dVrbE1uOLW3iBTamByg%40mail.gmail.com.

[Charles Nicholson]

[Charles Nicholson]

[Charles Nicholson]

I hacked up the ninja source code to have StatSingleFile print out the filename and the mtime nanoseconds, and confirmed everything that ar created had 0's for the nanosecond portion of the mtime fields. This is very clearly a "feature" of ar at least on macOS. I'm guessing there are scenarios where it makes sense for ar to explicitly set the last-modified timestamp of the output archive file rather than letting the OS take care of it?

I don't seem to have permission to run "sudo dtruss ar rcs libhello.a hello.o" on my Macbook, but I'm guessing I'd see an explicit mtime set lurking in there.

I just read through the Chromium GN code, saw that it was using libtool, and was in the middle of moving over to it when I read your email just now :) Judging by the number of mac toolchains (llvm, chromium, fuchsia) that don't have this problem with libtool, I'm guessing it's the right way to go on macOS.

Computers, am I right?

Charles

[Charles Nicholson]

I should also mention that I only see this on my Macbook Pro, and not my Linux desktop. I haven't had the chance to run it on Windows yet.

On Mon, Sep 9, 2019 at 4:27 PM Charles Nicholson <nicho...@verily.com> wrote: