Connect stuck when compiling chromium

[L TY]

Hi, I'm compiling chrome, and the compile process stucks like:

➜ gn gen out/TEST && autoninja -C out/TEST chrome

Done. Made 12482 targets from 2001 files in 2482ms

ninja: Entering directory `out/TEST'

[1/482] ACTION //components/resources:about_credits(//build/toolchain/linux:clang_x64)^T^C

ninja: build stopped: interrupted by user.

^CTraceback (most recent call last):

  File "/PATH/TO/depot_tools/ninjalog_uploader_wrapper.py", line 118, in <module>

    sys.exit(main())

  File "/PATH/TO/depot_tools/ninjalog_uploader_wrapper.py", line 76, in main

    config = LoadConfig()

  File "/PATH/TO/depot_tools/ninjalog_uploader_wrapper.py", line 31, in LoadConfig

    'chromium-build-stats.appspot.com'),

  File "/PATH/TO/depot_tools/ninjalog_uploader.py", line 49, in IsGoogler

    _, content = h.request('https://'+server+'/should-upload', 'GET')

  File "/PATH/TO/depot_tools/third_party/httplib2/__init__.py", line 1694, in request

    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)

  File "/PATH/TO/depot_tools/third_party/httplib2/__init__.py", line 1434, in _request

    (response, content) = self._conn_request(conn, request_uri, method, body, headers)

  File "/PATH/TO/depot_tools/third_party/httplib2/__init__.py", line 1354, in _conn_request

    conn.connect()

  File "/PATH/TO/depot_tools/third_party/httplib2/__init__.py", line 1100, in connect

    sock.connect(sockaddr)

  File "/usr/lib/python2.7/socket.py", line 228, in meth

    return getattr(self._sock,name)(*args)

KeyboardInterrupt

When I try to connect to `https://chromium-build-stats.appspot.com/should-upload`, the connection is established and I got "Access Denied: You're not on corp".

Is anyone has ideas about what is happening?

[Takuto Ikuta]

I think there are 2 issues, one is that some tasks stuck in compile, and exception from ninjalog_uploader_wrapper.py

You can silence second exception via

$ ninjalog_uploader_wrapper.py opt-out

But not sure which task is stuck while build, how long did you wait for that?

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/6d8eea82-38eb-4ef7-91ea-4fcfa1c99400n%40chromium.org.

[L TY]

Thank you for your reply.

I have tried to build for several times and the longest wait took about 40 minutes before I cancel it manually.

[L TY]

And when I run ` ninjalog_uploader_wrapper.py opt-out`, it is stuck in the same place

在2020年10月21日星期三 UTC+8 下午8:46:21<tik...@chromium.org> 写道：

[Takuto Ikuta]

On Wed, Oct 21, 2020 at 11:23 PM L TY <n0b0...@gmail.com> wrote:

And when I run ` ninjalog_uploader_wrapper.py opt-out`, it is stuck in the same place

Hmm, I see.

Then could you place `ninjalog.cfg` under depot_tools dir with the following content?

```

{
  "is-googler": false,
  "countdown": 0,
  "version": 2,
  "opt-in": false
}

```

[Takuto Ikuta]

+Chromium-dev 

I don't think ninja blocks compile processes.

Do you see cpu usage of compile tasks while ninja stuck?

On Thu, Oct 22, 2020 at 2:51 PM L TY <n0b0...@gmail.com> wrote:

And when I run autoninja again, it is stuck at licenses.py again, and the v8_window is successfully compiled.

```

➜ find ./ -name "v8_window.o"
./obj/third_party/blink/renderer/bindings/core/v8/bindings_core_impl/v8_window.o

```

L TY <n0b0...@gmail.com> 于2020年10月22日周四 下午1:45写道：

I place the `ninjalog.cfg` file as you said but the building processing is still stuck.

Then I use `-v` flag in autoninja command, and it shows that the process stucks at

`[1/595] python ../../tools/licenses.py --target-os=linux --depfile gen/components/resources/about_credits.d credits gen/components/resources/about_credits.html`

When I run the python command manually in the command line, it works fine.

Then I tried remove some compiled lib to add some more tasks to build process

`rm -r out/Default/gen/third_party/blink/renderer/core/fetch` 

and compile again.

This time the build process stucks at another place:

```

[420/893] ../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF obj/third_party/blink/renderer/bindings/core/v8/bindings_core_impl/v8_window.o.d -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 -DUSE_X11=1 -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DCHROMIUM_BUILD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -DCR_CLANG_REVISION="362913-80fee257-1" -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCOMPONENT_BUILD -D_LIBCPP_ABI_UNSTABLE -D_LIBCPP_ABI_VERSION=Cr -D_LIBCPP_ENABLE_NODISCARD -DCR_LIBCXX_REVISION=361348 -DCR_SYSROOT_HASH=e7c53f04bd88d29d075bfd1f62b073aeb69cbe09 -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DBLINK_CORE_IMPLEMENTATION=1 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_32 -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_26 -DWEBP_EXTERN=extern -DGL_GLEXT_PROTOTYPES -DUSE_GLX -DUSE_EGL -DVK_NO_PROTOTYPES -DBLINK_IMPLEMENTATION=1 -DINSIDE_BLINK -DBORINGSSL_SHARED_LIBRARY -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_POSIX -DWEBRTC_LINUX -DABSL_ALLOCATOR_NOTHROW=1 -DNO_MAIN_THREAD_WRAPPING -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=uint16_t -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DHAVE_PTHREAD -DPROTOBUF_USE_DLLS -DSK_GL -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_USER_CONFIG_HEADER="../../skia/config/SkUserConfig.h" -DSK_HAS_JPEG_LIBRARY -DSK_VULKAN_HEADER="../../skia/config/SkVulkanConfig.h" -DSK_VULKAN=1 -DSKIA_DLL -DSKCMS_API=__attribute__\(\(visibility\("default"\)\)\) -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER="gpu/config/gpu_driver_bug_workaround_autogen.h" -DVK_NO_PROTOTYPES -DUSING_V8_SHARED -DV8_ENABLE_CHECKS -DV8_DEPRECATION_WARNINGS -DLEVELDB_PLATFORM_CHROMIUM=1 -DLEVELDB_PLATFORM_CHROMIUM=1 -DLEVELDB_SHARED_LIBRARY -DSUPPORT_WEBGL2_COMPUTE_CONTEXT=1 -DWTF_USE_WEBAUDIO_PFFFT=1 -DUSE_LIBJPEG_TURBO=1 -DUSING_V8_SHARED -DV8_ENABLE_CHECKS -DV8_DEPRECATION_WARNINGS -DLIBXSLT_STATIC -I../.. -Igen -I../../third_party/libyuv/include -I../../third_party/libwebp/src -I../../third_party/khronos -I../../gpu -I../../third_party/vulkan/include -Igen/third_party/dawn -I../../third_party/dawn/src/include-I../../third_party/boringssl/src/include -I../../third_party/webrtc_overrides -I../../third_party/webrtc -Igen/third_party/webrtc -I../../third_party/abseil-cpp -I../../third_party/ced/src -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../third_party/protobuf/src -I../../third_party/protobuf/src -Igen/protoc_out -I../../third_party/skia -I../../third_party/vulkan/include -I../../third_party/skia/third_party/vulkanmemoryallocator -I../../third_party/vulkan/include -I../../third_party/angle/include -I../../third_party/angle/src/common/third_party/base -Igen/angle -I../../third_party/angle/include -I../../v8/include -Igen/v8/include -I../../third_party/libwebm/source -I../../third_party/leveldatabase -I../../third_party/leveldatabase/src -I../../third_party/leveldatabase/src/include -I../../third_party/libjpeg_turbo -I../../third_party/iccjpeg -I../../third_party/libpng -I../../third_party/zlib -I../../third_party/ots/include -I../../v8/include -Igen/v8/include -I../../third_party/libxml/src/include -I../../third_party/libxml/linux/include -I../../third_party/libxslt/src -I../../third_party/snappy/src -I../../third_party/snappy/linux -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -funwind-tables -fPIC -B../../third_party/binutils/Linux_x64/Release/bin -pthread -fcolor-diagnostics -fmerge-all-constants -fcrash-diagnostics-dir=../../tools/clang/crashreports -Xclang -mllvm -Xclang -instcombine-lower-dbg-declare=0 -fcomplete-member-pointers -m64 -march=x86-64 -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -Xclang -fdebug-compilation-dir -Xclang . -no-canonical-prefixes -Wall -Werror -Wextra -Wimplicit-fallthrough -Wthread-safety -Wextra-semi -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-ignored-pragma-optimize -O0 -fno-omit-frame-pointer -fvisibility=hidden -Xclang -add-plugin -Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang check-ipc -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wexit-time-destructors -Xclang -add-plugin -Xclang blink-gc-plugin -Wglobal-constructors -g2 -gsplit-dwarf -ggnu-pubnames -isystem../../build/linux/debian_sid_amd64-sysroot/usr/include/glib-2.0 -isystem../../build/linux/debian_sid_amd64-sysroot/usr/lib/x86_64-linux-gnu/glib-2.0/include -Wconversion -Wno-float-conversion -Wno-sign-conversion -Wno-implicit-float-conversion -Wno-implicit-int-conversion -Wno-shorten-64-to-32 -Wno-header-guard -isystem../../build/linux/debian_sid_amd64-sysroot/usr/include/nss -isystem../../build/linux/debian_sid_amd64-sysroot/usr/include/nspr -DLIBXML_STATIC= -Wno-undefined-bool-conversion -Wno-tautological-undefined-compare -std=c++14 -fno-exceptions -fno-rtti -nostdinc++ -isystem../../buildtools/third_party/libc++/trunk/include -isystem../../buildtools/third_party/libc++abi/trunk/include --sysroot=../../build/linux/debian_sid_amd64-sysroot -fvisibility-inlines-hidden -c gen/third_party/blink/renderer/bindings/core/v8/v8_window.cc -o obj/third_party/blink/renderer/bindings/core/v8/bindings_core_impl/v8_window.o

```

During stuck, I check the cpu usage and confirm that the compile process stopped.

I tried to run the long command in command line manually and got some compile error I did not get during the compiling process before like:

```

../../third_party/skia/include/third_party/skcms/skcms.h:120:1: error: expected expression
SKCMS_API const skcms_TransferFunction* skcms_sRGB_TransferFunction(void);
^
<command line>:59:45: note: expanded from here
#define SKCMS_API __attribute__((visibility(default)))
                                            ^
In file included from gen/third_party/blink/renderer/bindings/core/v8/v8_window.cc:27:
In file included from gen/third_party/blink/renderer/bindings/core/v8/v8_animation.h:19:
In file included from ../../third_party/blink/renderer/core/animation/animation.h:45:
In file included from ../../third_party/blink/renderer/core/animation/document_timeline.h:42:
In file included from ../../third_party/blink/renderer/core/dom/element.h:34:
In file included from ../../third_party/blink/renderer/core/display_lock/display_lock_context.h:12:
In file included from ../../third_party/blink/renderer/core/frame/local_frame_view.h:45:
In file included from ../../third_party/blink/renderer/platform/graphics/paint/paint_record.h:8:
In file included from ../../cc/paint/paint_record.h:9:
In file included from ../../cc/paint/paint_op_buffer.h:22:
In file included from ../../cc/paint/paint_canvas.h:12:
In file included from ../../cc/paint/paint_image.h:17:
In file included from ../../third_party/skia/include/core/SkImage.h:12:
In file included from ../../third_party/skia/include/core/SkImageEncoder.h:11:
In file included from ../../third_party/skia/include/core/SkBitmap.h:11:
In file included from ../../third_party/skia/include/core/SkColor.h:11:
In file included from ../../third_party/skia/include/core/SkImageInfo.h:11:
In file included from ../../third_party/skia/include/core/SkColorSpace.h:15:

```

So do you know whether is the ninja blocking the process, or the compiling process stopped because of compile failure?

Takuto Ikuta <tik...@chromium.org> 于2020年10月22日周四 下午1:25写道：

[Takuto Ikuta]

+Chromium-dev 

Don't drop chromium-dev, I may miss your mail and you may lose a chance to get response from other contributors.

On Thu, Oct 22, 2020 at 3:04 PM L TY <n0b0...@gmail.com> wrote:

I run the command

`./mksnapshot --turbo_instruction_scheduling --target_os=linux --target_arch=x64 --embedded_src gen/v8/embedded.S --embedded_variant Default --random-seed 314159265 --startup_blob snapshot_blob.bin --native-code-counters`

and it just stuck, without any error message and information.

Is it because some part of my code went wrong, cause I have modified some code in blink and v8?

I think so if you don't see the slowness without your change.

 

L TY <n0b0...@gmail.com> 于2020年10月22日周四 下午2:00写道：

The cpu usage is very low, and The cpu is consumed by `mksnapshot` task

Takuto Ikuta <tik...@chromium.org> 于2020年10月22日周四 下午1:54写道：

[L TY]

+Chromium-dev 

Thank you for your reply.

I will double check my code and keep updating in this channel.

Takuto Ikuta <tik...@chromium.org> 于2020年10月22日周四 下午2:08写道：

[Takuto Ikuta]

Do you use autoninja?

On Thu, Oct 22, 2020 at 6:51 PM huhu sha <27786...@qq.com> wrote:

I didn't find the ninjalog.cfg file under the depot_tools folder 。this is why? 

[huhu sha]

I didn't find the ninjalog.cfg file under the depot_tools folder 。this is why? 

在2020年10月22日星期四 UTC+8 下午1:28:25<tik...@chromium.org> 写道：

[L TY]

Here is the final answer:

There are some bugs in my code, and they blocked the mksnapshot process, although the autoninja showed that the process is stuck at somewhere else

[Bruce Dawson]

Ninja doesn't print the name of the command that is currently executing (impossible, because there may be ~1,000 commands executing) - it prints the name of the command that last finished executing.

Thus, if ninja is hung you should wait until it is down to a single process (setting NINJA_SUMMARIZE_BUILD=1 will get autoninja to tell ninja to show this information) and then use OS tools to figure out what that process is.

[L TY]

Got it, thank you for your explanation XD