Mac web-test timeouts

[Luke Swiderski]

Hello,

I've made a small code change to fix a bug in devtools and now I'm trying to run web tests to make sure I didn't break anything.  I'm on a mac 11.2.3,  xcode version 12.4 (12D4e).

I've been following this document. 
 https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/testing/web_tests.md

At the initial setup, it says 
"On Mac, you probably want to strip the content_shell binary before starting the tests. If you don‘t, you’ll have 5-10 running concurrently, all stuck being examined by the OS crash reporter. This may cause other failures like timeouts where they normally don't occur.
strip ./xcodebuild/{Debug,Release}/content_shell.app/Contents/MacOS/content_shell"

I tried running the command from chromium/src, but I get an error 
" /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: can't open file: ./xcodebuild/Debug/content_shell.app/Contents/MacOS/content_shell (No such file or directory)"

I downloaded chromium and devtools frontend to an external harddrive and am using the integrated checkout, as per this doc:
https://chromium.googlesource.com/devtools/devtools-frontend/+/HEAD/docs/workflows.md

I'm wondering if because I have chromium on an external that maybe the file path for this "strip ./xcodebuild/{Debug,Release}/content_shell.app/Contents/MacOS/content_shell" command is wrong.  I tried running the web tests without it and they timeout over and over again for up to 24 hours!

If anyone could point out what I'm doing wrong, I would sure appreciate it!

[Christian Biesinger]

I'm not sure why that document has xcodebuild in the path, you
probably want out/{Debug, Release}/... instead.

Christian

> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev
> ---
> You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/34e5ce27-0ac9-4948-8f21-2e8c75a76311n%40chromium.org.

[Dirk Pranke]

Interesting. That would mean that document hasn't been updated in a really, *really* long time :) [*]

I'm not actually sure that stripping the binary is even needed these days. I certainly haven't historically done that when running tests.

If you're getting lots of tests timing out and/or crashing, and it's not because you've changed something locally that is causing crashes, then you might have something else wrong.

I'd recommend making sure you can run tests on a local checkout without any changes. You also might not want to try to run the entire suite; unless you have a Mac Pro-class machine, running the whole suite may take many hours even if things are fine.

-- Dirk

[*] ./xcodebuild was what we used in the pre-Ninja days.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAPTJ0XH5C3u9pgJufKRxuhTk7L1Y%3D-XRicy%2BwP6vkT2BKyHxNw%40mail.gmail.com.

[Luke Swiderski]

That is actually what I have been trying, I've been running the tests on the main origin branch to see what that looked like before running them on the branch with my changes.  

[Dirk Pranke]

I would try just running one test, with extra debugging enabled (e.g., `./third_party/blink/tools/run_web_tests.py --debug-rwt-logging --driver-logging fast/html/unknown-tag.html`) and see if that tells you anything further. You might also try just launching content_shell by itself and see if it is functional. If neither of those things helps enough, maybe file a bug and folks can help dig into it there?

-- Dirk

On Mon, Sep 6, 2021 at 8:47 AM Luke Swiderski <luke.sw...@gmail.com> wrote:

I tried running the tests again without any local changes.  They ran for about 24 hours and the output was thousands of messages similar to 

"Content Shell took too long to startup.

stopping Content Shell(pid 46415) timed out, killing it

[1238/1238] http/test...c-stack.js failed unexpectedly (test timed out) 34.8744s

num regressions (1238) exceeds max exit status (101)"

followed by:

"0 tests ran as expected, 1238 didn't: "

This is without any local changes, just following the docs.  Is there anything I can try differently?  

[Luke Swiderski]

I tried running the tests again without any local changes.  They ran for about 24 hours and the output was thousands of messages similar to 

"Content Shell took too long to startup.

stopping Content Shell(pid 46415) timed out, killing it

[1238/1238] http/test...c-stack.js failed unexpectedly (test timed out) 34.8744s

num regressions (1238) exceeds max exit status (101)"

followed by:

"0 tests ran as expected, 1238 didn't: "

This is without any local changes, just following the docs.  Is there anything I can try differently?  

On Thursday, September 2, 2021 at 11:24:31 PM UTC-4 Luke Swiderski wrote: