Re: [chromium-dev] Linux content_shell from chromium-browser-snapshots doesn't support GL

[Ken Russell]

(chromium-dev to bcc)

Chromium's graphics experts might be better able to help. If you can provide a minimal reproduction script or setup that various team members could run against Chromium builds, that would enable diagnosis.

-Ken

On Fri, Apr 1, 2022 at 10:53 PM Connor Clark <cja...@chromium.org> wrote:

I maintain a test that downloads a content_shell from http://commondatastorage.googleapis.com/chromium-browser-snapshots and runs run_web_tests.py web tests against the latest development version of Lighthouse (a third party library used in DevTools). This all runs in GitHub Actions. This test has been running successfully for nearly two years now, and it's been pretty useful to us in catching regressions.

We had no choice but to run these tests on Mac (Ubuntu would have been preferred) because the Ubuntu content_shell wouldn't work with web tests. Recently I started to look at this again, and this is what I found:

- These tests work fine using a locally built Linux content_shell, but not with a content_shell from the data storage repository. The error logged by the web test runner is very explicit: GL is required for it to work, and GL is not working.
- The locally built content_shell has webgl enabled (according to https://webglreport.com/), but the downloaded content_shell does not. I tried various flags but nothing worked

- On start up, the downloaded content_shell logs that it cannot initialize GL. I noticed that the Mac content_shell zip that can be downloaded from this repository comes with swiftshader and GL libraries, but the Linux one doesn't.

Is lack of GL expected for these Linux content_shells? I believe that if only GL worked, then run_web_tests.py would work too.

Here's our setup, in case anyone is interested how one can run web tests on GitHub Actions: https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/test/chromium-web-tests/README.md

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/6b2a9ad3-1d77-4597-a3d2-01b3bbc84603n%40chromium.org.

[Gregg Tavares]

Just throwing this out there. I'm sure content_shell is smaller or maybe more up to date but I've been using puppeteer for similar things since it seems to guarantee that WebGL is working and is officially supported? Whereas AFAIK content_shell is not.

[Connor Clark]

Simple repro is: download any linux content shell (http://commondatastorage.googleapis.com/chromium-browser-snapshots/Linux_x64/992876/content-shell.zip) and see that WebGL is not enabled. https://webglreport.com/
Mac content shells work fine. And so does a locally built linux content shell.

Gregg, unfortunately the web test runner only works with content shell, not chrome.

[Ken Russell]

+su...@chromium.org +Nicolas Capens (ca...@chromium.org) 

Are you expecting hardware-accelerated WebGL inside content_shell, or are you looking for SwiftShader fallback?

SwiftShader isn't bundled in the content_shell zip archives:

https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Linux_x64/992876/

You might be able to drop your own built SwiftShader into the unpacked content_shell directory.

Also, I think the --use-gpu-in-tests command line argument may have an effect with content_shell - not sure.

On the whole however our team doesn't test WebGL via Blink's web tests and content_shell, so we don't have much recent experience with this configuration. For several years we've been using the Telemetry-based harnesses in src/content/test/gpu/gpu_tests/ to bring up the entire browser because this was more reliable than content_shell and better exercises the graphics stack's shipping code paths.

As Gregg mentioned, Puppeteer is another good option nowadays which also utilizes the entire browser rather than content_shell. This would require rewriting some tests to use different primitives for privileged operations that content_shell offers via window.internals. GpuBenchmarkingExtension is what the Telemetry-based tests use:

https://source.chromium.org/chromium/chromium/src/+/main:content/renderer/gpu_benchmarking_extension.h?q=gpu_benchmarking_extension

-Ken

[Connor Clark]

Puppeteer doesn't update often enough for our use case: we use this test to successfully prevent regressions in our tool very early, and we can't do that without tracking ToT. Another problem is that the webtests runner simply doesn't work with full chrome, it only works for content shell.

> Are you expecting hardware-accelerated WebGL inside content_shell, or are you looking for SwiftShader fallback?

I think any webgl support would do: our tests aren't graphics intensive so I don't think the performance hit of swiftshader would matter.

Is there a particular reason why the Linux content shell available in the chromium-browser-snapshots bucket isn't built with webgl but MacOS is? Wondering if we can just flip a switch...

[Connor Clark]

Puppeteer doesn't update often enough for our use case: we use this test to successfully prevent regressions in our tool very early, and we can't do that without tracking ToT. Another problem is that the webtests runner simply doesn't work with full chrome, it only works for content shell.

> Are you expecting hardware-accelerated WebGL inside content_shell, or are you looking for SwiftShader fallback?

I think any webgl support would do: our tests aren't graphics intensive so I don't think the performance hit of swiftshader would matter.

Is there a particular reason why the Linux content shell available in the chromium-browser-snapshots bucket isn't built with webgl but MacOS is? Wondering if we can just flip a switch...

[Ken Russell]

On Wed, Apr 27, 2022 at 11:53 AM Connor Clark <cja...@chromium.org> wrote:

Puppeteer doesn't update often enough for our use case: we use this test to successfully prevent regressions in our tool very early, and we can't do that without tracking ToT. Another problem is that the webtests runner simply doesn't work with full chrome, it only works for content shell.

I see, and understood.

Yes, there are a lot of complexities in this area. Our team runs our tests on full Chrome for many reasons:

https://chromium.googlesource.com/chromium/src.git/+/HEAD/docs/gpu/gpu_testing.md

I think it would be nice to better unify the web tests runner with headless Chrome, to get closer to a single binary and set of behaviors. Perhaps consider filing a low priority RFE on crbug.com about that if you have more motivation for doing so.

> Are you expecting hardware-accelerated WebGL inside content_shell, or are you looking for SwiftShader fallback?

I think any webgl support would do: our tests aren't graphics intensive so I don't think the performance hit of swiftshader would matter.

Is there a particular reason why the Linux content shell available in the chromium-browser-snapshots bucket isn't built with webgl but MacOS is? Wondering if we can just flip a switch...

I don't think so. Could you please file a bug on crbug.com/ and post the bug ID here so we can add the right labels? We can almost surely fix these content shell bundles to include SwiftShader.

Thanks,

[Connor Clark]

> I don't think so. Could you please file a bug on crbug.com/ and post the bug ID here so we can add the right labels? We can almost surely fix these content shell bundles to include SwiftShader.

Here's the bug: https://bugs.chromium.org/p/chromium/issues/detail?id=1320508 . Thanks!