Build problems

[Scott Violet]

Hello,

Twice in one day!

I'm trying to run a pinpoint job for a commit point > 1 month old. It keeps failing with build errors. The bot is using siso, and the build errors seem to happen on the remote side. Sample error output follows. The compile is failing because a generated header (a buildflag header) doesn't have the right set of defines. I believe it has a newer set of defines (perhaps from the machine building a more recent revision). Some folks mentioned this may be because chrome's deps are wrong: https://b.corp.google.com/issues/345633541 . For the pinpoint job I modified the target that generates the header https://chromium-review.googlesource.com/c/chromium/src/+/5771115 as well as moving/adding to public_deps where it may have been missing, but I still see the issue.

I also tried building (at the same commit point) locally, using siso, and couldn't replicate. Any suggestions?

Error:

../../buildtools/reclient/rewrapper -cfg=../../buildtools/reclient_cfgs/chromium-browser-clang/rewrapper_mac.cfg -inputs=build/config/unsafe_buffers_paths.txt -exec_root=/Volumes/Work/s/w/ir/cache/builder/src/ ../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF obj/base/allocator/partition_allocator/src/partition_alloc/allocator_core/partition_oom.o.d -DOFFICIAL_BUILD -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FORTIFY_SOURCE=2 -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE -D__ARM_NEON__=1 -DCR_XCODE_VERSION=1500 -DCR_CLANG_REVISION=\"llvmorg-19-init-14561-gecea8371-1\" -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCXXABI_DISABLE_VISIBILITY_ANNOTATIONS -DCR_LIBCXX_REVISION=09b99fd8ab300c93ff7b8df6688cafb27bd3db28 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DIS_PARTITION_ALLOC_IMPL -I../.. -Igen -I../../buildtools/third_party/libc++ -I../../base/allocator/partition_allocator/src -Igen/base/allocator/partition_allocator/src -Wall -Wextra -Wimplicit-fallthrough -Wextra-semi -Wunreachable-code-aggressive -Wthread-safety -Wunguarded-availability -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wno-cast-function-type -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wno-invalid-offsetof -Wno-vla-extension -Wno-thread-safety-reference-return -Wshadow -Werror -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector -fcolor-diagnostics -fmerge-all-constants -fno-sized-deallocation -fcrash-diagnostics-dir=../../tools/clang/crashreports -mllvm -instcombine-lower-dbg-declare=0 -mllvm -split-threshold-for-reg-with-hint=0 -ffp-contract=off -flto=thin -fsplit-lto-unit -mllvm -inlinehint-threshold=360 -fwhole-program-vtables -fcomplete-member-pointers --target=arm64-apple-macos -mno-outline -ffile-compilation-dir=. -no-canonical-prefixes -ftrivial-auto-var-init=pattern -fno-omit-frame-pointer -gdwarf-4 -g1 -gdwarf-aranges -isysroot ../../build/mac_files/xcode_binaries/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -mmacos-version-min=10.15 -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Xclang -add-plugin -Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang span-ctor-from-string-literal -Xclang -plugin-arg-find-bad-constructs -Xclang raw-ref-template-as-trivial-member -Xclang -plugin-arg-find-bad-constructs -Xclang check-stack-allocated -Xclang -plugin-arg-find-bad-constructs -Xclang check-allow-auto-typedefs-better-nested -Xclang -add-plugin -Xclang raw-ptr-plugin -Xclang -plugin-arg-raw-ptr-plugin -Xclang check-raw-ptr-to-stack-allocated -Xclang -plugin-arg-raw-ptr-plugin -Xclang disable-check-raw-ptr-to-stack-allocated-error -Xclang -plugin-arg-raw-ptr-plugin -Xclang raw-ptr-exclude-path=/renderer/ -Xclang -plugin-arg-raw-ptr-plugin -Xclang raw-ptr-exclude-path=../../third_party/blink/public/web/ -Xclang -plugin-arg-raw-ptr-plugin -Xclang raw-ptr-exclude-path=../../third_party/dawn/ -DUNSAFE_BUFFERS_BUILD -Xclang -add-plugin -Xclang unsafe-buffers -Xclang -plugin-arg-unsafe-buffers -Xclang ../../build/config/unsafe_buffers_paths.txt -Wexit-time-destructors -O3 -fno-math-errno -Wc++11-narrowing -Wconditional-uninitialized -Wcstring-format-directive -Wctad-maybe-unsupported -Wdeprecated-copy -Wdeprecated-copy-dtor -Wduplicate-enum -Wextra-semi -Wextra-semi-stmt -Wimplicit-fallthrough -Winconsistent-missing-destructor-override -Winvalid-offsetof -Wmissing-field-initializers -Wnon-c-typedef-for-linkage -Wpessimizing-move -Wrange-loop-analysis -Wredundant-move -Wshadow-field -Wstrict-prototypes -Wsuggest-destructor-override -Wsuggest-override -Wtautological-unsigned-zero-compare -Wunreachable-code-aggressive -Wunused-but-set-variable -Wunused-macros -pedantic -Wenum-compare-conditional -Wno-c++11-narrowing-const-reference -Wno-missing-template-arg-list-after-template-kw -std=c++20 -Wno-trigraphs -fno-exceptions -fno-rtti -nostdinc++ -isystem../../third_party/libc++/src/include -isystem../../third_party/libc++abi/src/include -fvisibility-inlines-hidden -c ../../base/allocator/partition_allocator/src/partition_alloc/partition_oom.cc -o obj/base/allocator/partition_allocator/src/partition_alloc/allocator_core/partition_oom.o
stdout:
In file included from ../../base/allocator/partition_allocator/src/partition_alloc/partition_oom.cc:8:
In file included from ../../base/allocator/partition_allocator/src/partition_alloc/oom.h:10:
In file included from ../../base/allocator/partition_allocator/src/partition_alloc/allocation_guard.h:10:
[1m../../base/allocator/partition_allocator/src/partition_alloc/partition_alloc_config.h:69:5: [0m [0;1;31merror: [0m [1mfunction-like macro 'PA_BUILDFLAG_INTERNAL_USE_STARSCAN' is not defined [0m
   69 | # [0;34mif [0m PA_BUILDFLAG(USE_STARSCAN) [0m
      | [0;1;32m    ^
[0m [1m../../base/allocator/partition_allocator/src/partition_alloc/buildflag.h:15:29: [0m [0;1;36mnote: [0mexpanded from macro 'PA_BUILDFLAG' [0m
   15 | #define PA_BUILDFLAG(flag) (PA_BUILDFLAG_CAT(PA_BUILDFLAG_INTERNAL_, flag)()) [0m
      | [0;1;32m                            ^
[0m [1m../../base/allocator/partition_allocator/src/partition_alloc/buildflag.h:14:32: [0m [0;1;36mnote: [0mexpanded from macro 'PA_BUILDFLAG_CAT' [0m
   14 | #define PA_BUILDFLAG_CAT(a, b) PA_BUILDFLAG_CAT_INDIRECT(a, b) [0m
      | [0;1;32m                               ^
[0m [1m../../base/allocator/partition_allocator/src/partition_alloc/buildflag.h:13:41: [0m [0;1;36mnote: [0mexpanded from macro 'PA_BUILDFLAG_CAT_INDIRECT' [0m
   13 | #define PA_BUILDFLAG_CAT_INDIRECT(a, b) a##b [0m
      | [0;1;32m                                        ^
[0m [1m<scratch space>:110:1: [0m [0;1;36mnote: [0mexpanded from here [0m
  110 | PA_BUILDFLAG_INTERNAL_USE_STARSCAN [0m
      | [0;1;32m^

  -Scott

[Dirk Pranke]

My only suggestion is the same one I suggested via chat: track down a trooper and coordinate a build with them, so that they can log into the bot while the build is going on and grab the header(s) as needed. This shouldn't be too hard for them to do, I think, I think they should be able to take the bot out of swarming temporarily so that you could then analyze more at leisure.

-- Dirk

--
You received this message because you are subscribed to the Google Groups "build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to build+un...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/build/CAKARY_%3DXXByeL5nz9wOfc5zzLwV2EdbFX19jHtpfY32ecPb1fg%40mail.gmail.com.

[Scott Violet]

Sorry for not including that part. I was able to get a trooper to help. They extracted the file and it is what I expect, meaning there shouldn't be a build error. I can only think of two possibilities:

1. When the trooper grabbed the file the build hadn't failed yet, so the local file was fine. Maybe it wouldn't be fine if they grabbed shortly after the failures?

2. It's the remote side that is failing, so I'm wondering if I need to grab the file from the bot siso/remote-exec is executing on.

  -Scott

[Mark Mentovai]

Can you show where that log snippet came from? Neither of your links are it. You said that you’re building at a revision over a month old, but it’d be useful to know what revision and to have other data about the build.

starscan was removed from Chrome’s base partition_allocator about a month ago at 6e9560c26595 (2024-07-12, https://chromium-review.googlesource.com/c/5678260), so what side of that revision you’re working at is relevant. Or, alternatively, the build that’s actually being done might not be happening at the revision you think.

More details are better. Log snippets should come with links to where you got them from. And links to builds should come with an excerpt of the log you’re interested in. One isn’t a substitute for the other.

--

[Scott Violet]

On Fri, Aug 9, 2024 at 4:47 PM Mark Mentovai <ma...@chromium.org> wrote:

Can you show where that log snippet came from?

Pinpoint job is here: https://pinpoint-dot-chromeperf.appspot.com/job/165753a0210000 . Both sides fail with the same error. Here's the older build: https://ci.chromium.org/ui/p/chrome/builders/try/Mac%20arm%20Builder%20Perf/76433/overview . Seeing the failures on the siso side is tricky. Here's a screenshot showing how to do it: https://screenshot.googleplex.com/B5oVV8jqrLECuuz .Commit position is 8f2f943f128343c606a7613e9492b45653c9d998 . Sorry if some of these links are google only.

 

Neither of your links are it. You said that you’re building at a revision over a month old, but it’d be useful to know what revision and to have other data about the build.

starscan was removed from Chrome’s base partition_allocator about a month ago at 6e9560c26595 (2024-07-12, https://chromium-review.googlesource.com/c/5678260), so what side of that revision you’re working at is relevant. Or, alternatively, the build that’s actually being done might not be happening at the revision you think.

More details are better. Log snippets should come with links to where you got them from. And links to builds should come with an excerpt of the log you’re interested in. One isn’t a substitute for the other.

Please let me know if you need more.

Also note that I did file a bug here: https://issues.chromium.org/u/1/issues/357145643 .

  -Scott

[Dirk Pranke]

What I would do in that case is

1) take the bot out of rotation

2) re-do the build (on the bot) to get the failure.

3) see if you can figure out whether you're getting a version of some header from some directory other than the one you're expecting. 

-- Dirk

[Scott Violet]

Thanks Dirk. I went with a different approach: have infra stand up a bot that builds locally. Not particularly satisfying, but it worked. Special thanks to infra for handling a one-off!

  -Scott