About the gpu black list

[yongsheng zhu]

hi,
I want to know the rule or standard on how a GPU(with its driver)
could be removed out of the black list. For example, what should the
pass rate of the webgl conformance test cases be achieved if 'webgl'
is removed from the black list? See the list in
chrome/browser/resources/software_rendering_list/
software_rendering_list.json
May I miss previous discussions. So could anyone give me some hints?

[Vangelis Kokkevis]

We use the following signals to add GPUs and drivers to our blacklist:

1. Driver release date: Old drivers are notoriously flaky and unstable so we want to encourage our users to update them. This is by far the largest contributor of blacklist hits for Chrome, especially for Windows XP users.

2. Crash rates: We actively monitor GPU process crash reports and look for clustering of crashes on particular gpu id's or driver ranges. 

3. User reports: These have been invaluable in identifying particularly problematic combinations of hardware and drivers that either grossly mis-render or cause other system malfunctions (hungs, crashes, etc)

4. Our own testing, especially via the ever expanding WebGL conformance test suite. A single test failure won't cause a GPU to land on the blacklist. We're looking for widespread failures or other serious side-effects.

These are all signals but at the end of the day, there's a judgment call to be made on whether we will backlist a particular GPU or driver based on how severe the issues with it are and how many users it will affect. We typically err on the side of caution.

I should also mention that a fair amount of effort goes into trying to work around driver issues by tweaking our code, massaging user-supplied shaders, etc.  Unfortunately it's not always possible or practical to do so.

 

Is there a particular GPU that we're currently blocking and you don't think we should be? 

Vangelis

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
   http://groups.google.com/a/chromium.org/group/chromium-dev

[Yongsheng Zhu]

On Fri, Nov 18, 2011 at 3:00 PM, Vangelis Kokkevis <vang...@google.com> wrote:

We use the following signals to add GPUs and drivers to our blacklist:

1. Driver release date: Old drivers are notoriously flaky and unstable so we want to encourage our users to update them. This is by far the largest contributor of blacklist hits for Chrome, especially for Windows XP users.

2. Crash rates: We actively monitor GPU process crash reports and look for clustering of crashes on particular gpu id's or driver ranges. 

3. User reports: These have been invaluable in identifying particularly problematic combinations of hardware and drivers that either grossly mis-render or cause other system malfunctions (hungs, crashes, etc)

4. Our own testing, especially via the ever expanding WebGL conformance test suite. A single test failure won't cause a GPU to land on the blacklist. We're looking for widespread failures or other serious side-effects.

These are all signals but at the end of the day, there's a judgment call to be made on whether we will backlist a particular GPU or driver based on how severe the issues with it are and how many users it will affect. We typically err on the side of caution.

I should also mention that a fair amount of effort goes into trying to work around driver issues by tweaking our code, massaging user-supplied shaders, etc.  Unfortunately it's not always possible or practical to do so.

 

Is there a particular GPU that we're currently blocking and you don't think we should be? 

 No, at least currently. However, I am trying to test the pass rate of webGL with the mesa Intel drivers since mesa 7.11 fixes a lot of issues for webGL. So i'm gonna try to satisfy the rules here.

 Thanks for clarification.

[yongsheng zhu]

Hi, Vangelis
We're working on solving the failure cases of WebGL conformance test
suite on Intel GPU mesa drivers, mainly for two families: Intel®
Graphics Media Accelerator 3150 and Intel® HD Graphics 2000/3000).
Here are test results for the two platforms:
Intel® Graphics Media Accelerator 3150:
Passrate: 97.8%; passed cases/total cases: 8265/8446; timeout cases:
2; OS: Ubuntu 11.10, Fedora 13; chrome: 16.0.912.63; mesa: 7.11
Intel® HD Graphics 2000/3000:
Passrate: 98.4%; passed cases/total cases: 8312/8443; timeout cases:
0; OS: Fedora 13; chrome: 16.0.912.63; mesa: >=7.11

The most failed cases are in three testing aspects: 'fbo
attachement' , 'gl-pointcoord', and 'deletion of texture, render
buffer and frame buffer'. I don't know whether they are the blocking
issue. May we get some hints?

Vangelis, so do you think it meets the requirements of the signals?
May we know if we can do anything else to help push forward to these
two families out of the GPU black list?
Thanks for your attention.

> <zhuyongsheng2...@gmail.com>wrote:

>
>
>
>
>
>
>
> > hi,
> > I want to know the rule or standard on how a GPU(with its driver)
> > could be removed out of the black list. For example, what should the
> > pass rate of the webgl conformance test cases be achieved if 'webgl'
> > is removed from the black list? See the list in
> > chrome/browser/resources/software_rendering_list/
> > software_rendering_list.json
> > May I miss previous discussions. So could anyone give me some hints?
>
> > --

> > Chromium Developers mailing list: chromium-...@chromium.org

[Zhenyao Mo]

If you can provide the output of the three failing tests, I can have a look and let you know how critical the bugs are.

Mo

Chromium Developers mailing list: chromi...@chromium.org

[Yongsheng Zhu]

That's so great.

Since it's a little big to paste them in the mail, I attach them in a zip file. See wglc.zip.

Yongsheng

[Zhenyao Mo]

Sorry I didn't make myself clear.  What I need is the whole output page content.  Not just the FAIL lines.

I need to know which exact tests fail to tell how serious the bugs are.

Mo

[John Bauman]

Also, the blacklist entry may mention a chromium bug that caused that entry to be added, so if a new driver fixes that bug it's a sign it's on the right track.

[Yongsheng Zhu]

Please see the new one.

For each gfx, there is one summary txt file and one 'details' directory to save the output of each failed case.

Any issues please let me know.

[Yongsheng Zhu]

I think it should be related to #76703: http://code.google.com/p/chromium/issues/detail?id=76703

It blocks all Intel mesa drivers. I'll post the discussion there. 

Thanks for your reminder.

[Clemens Buchacher]

That bug is supposedely fixed. Although I have found no way to test
for it. From looking at the conformance test suite, the remaining
errors do not seem very serious to me.

[Zhenyao Mo]

bcc: Chromium-dev

cc: vangelis

Yongsheng,

What's the chrome version you were using to generate the results? (you can find out in about:version).

It seems like a few bugs we already fixed, they still fail in your results.  So it's hard for me to tell which ones are chromium bugs and which ones are driver bugs.

If it's not too much trouble, could you download a dev channel chromium from here: http://www.chromium.org/getting-involved/dev-channel

Then run conformance test suite and give me the results one more time?  Also, could you also provide the about:gpu page content?

Thank you very much.

Mo

[Yongsheng Zhu]

Zhenyao,

The chrome version used to test for previous data:

Google Chrome   16.0.912.63 (Official Build 113337)

OS      Linux

WebKit  535.7 (@101565)

JavaScript      V8 3.6.6.11

"about:gpu":

Initialization time     81

Vendor Id       0x8086

Device Id       0xa011

Driver vendor   Mesa

Driver version  7.11

Driver date

Pixel shader version    1.20

Vertex shader version   1.20

GL version      1.4

GL_VENDOR       Tungsten Graphics, Inc

GL_RENDERER     Mesa DRI Intel(R) IGD x86/MMX/SSE2

GL_VERSION      1.4 Mesa 7.11

[Yongsheng Zhu]

Hi,

I firstly tested Intel GMA3150 with the dev chromium. If you think the provided information is okay, then we'll do the same for the Intel HD Graphics 2000/3000. For details, see the attached file.

Here is the summary：

8289 of 8446 passed, 2 timed out, passrate: 98.14%

"about:version"

Google Chrome   17.0.963.26 (Official Build 116225) dev

OS      Linux

WebKit  535.11 (@103967)

JavaScript      V8 3.7.12.12

"about:gpu":

Initialization time     54

Vendor Id       0x8086

Device Id       0xa011

Driver vendor   Mesa

Driver version  7.11

Driver date

Pixel shader version    1.20

Vertex shader version   1.20

GL version      1.4

GL_VENDOR       Tungsten Graphics, Inc

GL_RENDERER     Mesa DRI Intel(R) IGD x86/MMX/SSE2

GL_VERSION      1.4 Mesa 7.11

[Yongsheng Zhu]

Like you said, 24 newly passed. Please have a check of the details. Thanks.

[Yongsheng Zhu]

No, please see the previous one of this message. I didn't see your comment so just a reminder. :)
Please check my post on Jan 6, 11:26 am. It includes the details for Intel GMA3150.
Thanks.

On Sat, Jan 7, 2012 at 10:36 AM, Zhenyao Mo <z...@google.com> wrote:

You forgot about the attachment. :)

[Yongsheng Zhu]

Re-send the details in case you don't see the report mail.

[Zhenyao Mo]

The results look really promising, but can you try one more time?  I
got the wrong impression that dev channel has already progressed to
M18, but it's still M17 and way behind the top of tree.

http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?116960

Follow this link, you will see a Linux and Linux_x64 folder; click one
depending on your machine arch, and roll down all the way to the
bottom to download the newest release and then run conformance one
more time.

Really sorry that I gave you the wrong instruction and wasted your time.

Mo

[Yongsheng Zhu]

I don't find any Linux or Linux_x64 folder in the link. Could you please have a double check?

so will wait for the latest release and then re-collect the data.

Yongsheng

[Yongsheng Zhu]

hi, 

we re-test with the latest chromium on Intel GMA3150 without other changes and get a better result. See details in the attached file.

Test Summary (8641 total tests):
Tests PASSED: 8536
Tests FAILED: 105
Tests TIMED OUT: 4

Chromium    18.0.1004.0 (Developer Build 117116)                                                                                                   
OS  Linux
WebKit  535.16 (@104535)
JavaScript  V8 3.8.4.1
Flash   (Disabled)
User Agent  Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.16 (KHTML, like Gecko) Chrome/18.0.1004.0 Safari/535.16
Command Line     ./chrome --ignore-gpu-blacklist --flag-switches-begin --flag-switches-end

On Wed, Jan 11, 2012 at 2:40 AM, Zhenyao Mo <z...@google.com> wrote:

This link should work:  http://commondatastorage.googleapis.com/chromium-browser-continuous/index.html

Thank you.

Mo

[Yongsheng Zhu]

Hi, Mo

The blocking issues for Intel HD Graphics 2000-3000 were resolved. Please see the attached test results.

If any other problems, please let us know.

For other platforms like Intel GMA3150, we're continuing fixing those issues. 

Yongsheng

On Wed, Jan 18, 2012 at 5:16 AM, Zhenyao Mo <z...@google.com> wrote:

That sounds like a chromium bug.

On Mon, Jan 16, 2012 at 5:50 PM, Yongsheng Zhu

<zhuyongs...@gmail.com> wrote:
> yes, it's weird.
> I find another weird thing: If I open 'Developer Tools' and debug this case
> step by step, then it always fails even though for those platformas which
> never fail before. I tested it on windows and linux.
> Really weird... I'm keeping following it.
>
> Yongsheng
>
>
> On Sat, Jan 14, 2012 at 3:02 AM, Zhenyao Mo <z...@google.com> wrote:
>>
>> On Thu, Jan 12, 2012 at 5:52 PM, Yongsheng Zhu
>> <zhuyongs...@gmail.com> wrote:
>> > Zhenyao,
>> > For 1) & 2), we'll try to follow them.
>> > For 3), do you mean you change the webgl tests included in chromium
>> > code?
>> > I think it might be related
>> > to http://code.google.com/p/chromium/issues/detail?id=104740
>> > The root cause of these is the buffers are not recalled immediately when
>> > the
>> > frame is destroyed. In the conformance testing, the buffers for the
>> > previous
>> > tests may be not recalled when testing this case. Anyway, I'll check and
>> > retest this.
>>
>> I mean, from your results, we rendered fine with 1x2048 texture in
>> texture-size.html, but failed to renderer with 1x2048 texture in
>> gl-max-texture-dimensions.html, and rendered fine with 2048x1 tetxure
>> in gl-max-texture-dimensions.html.
>>
>> This looks kinda weird to me.
>>
>> >
>> > Thanks for your valuable feedback.
>> >
>> > Yongsheng
>> >
>> >
>> > On Fri, Jan 13, 2012 at 5:58 AM, Zhenyao Mo <z...@google.com> wrote:
>> >>
>> >> Hi Yongsheng,
>> >>
>> >> From the test results, I see a few issues with the current driver:
>> >>
>> >> 1) getUniform returns values from another uniform location instead of
>> >> the specified one
>> >> 2) gl_PointCoord & gl_PointSize support
>> >> 3) MAX sized texture support
>> >> 4) NPOT cube texture support
>> >> 5) renderbuffer handles size 0 buffers incorrectly
>> >>
>> >> I ordered them (roughly) from the most critical issue to the least.
>> >> (1) is fatal.  (2) and (3) are also pretty bad.  We might be able to
>> >> live with (4) and (5).
>> >>
>> >> As for 3), 2k x 1 texture test succeeded in one test but failed in
>> >> another, which looked kinda weird to me.  Also, I modified the test a
>> >> little bit to print out a message if failure is due to out-of-memory.
>> >> Could you re-run these tests and see if out-of-memory is the cause?
>> >>
>> >> Mo
>> >>
>> >> On Wed, Jan 11, 2012 at 1:22 AM, Yongsheng Zhu

[Zhenyao Mo]

Are these the only failing cases? If yes, that's really awesome!

Thanks for letting us know.

Mo

On Mon, Feb 6, 2012 at 12:24 AM, Yongsheng Zhu

[Yongsheng Zhu]

Yes, you're right. There are totally 17 failed cases (8843/8860). You can see that the number of failed cases in the attachment is also 17. It means all failed ones are included. 

So next, for this platform, we'll track the mesa release and provide a patch of chromium to whitewash this platform for WebGL. 

I have a question. What about other gpu features, like "accelerated_2d_canvas", "accelerated_compositing" and "multisampling"? If there's anything else to do for them, please let us know. Thanks.

Yongsheng

[Zhenyao Mo]

Multisampling is on by default, so if you play a few webgl demos, like
WebGL Aquarium and see no issues with rendering, then multisampling
should be OK.

As for accelerated_2d_canvas and accelerated_compositing, they use a
limited subset of gl functions, so usually if WebGL runs fine, they
should also be fine. However, sometimes we block them for performance
issues. I'll see if we can provide a set of test cases for each
feature.

Great progress!

Cheers,
Mo

On Mon, Feb 6, 2012 at 5:58 PM, Yongsheng Zhu

[Yongsheng Zhu]

Hi, Mo

I created an issue to whitewash the gpu device ids for sandybridge. Could you please help review it? Thanks a lot.

http://code.google.com/p/chromium/issues/detail?id=113936

Yongsheng

[Yongsheng Zhu]

Hi, Mo
We also did WebGL conformance tests for IvyBridge platforms in Linux. See the details in the attached file. Please let me know if any problems.

Thanks,
Yongsheng

[Zhenyao Mo]

The results looks good. I am fine with unblacklisting this
card/driver with antialiasing off (this is because the antialiasing
test returned wrong pixel values).

Mo

On Sun, Feb 26, 2012 at 11:48 PM, Yongsheng Zhu

[Yongsheng Zhu]

I don't find anything about 'antialiasing' in software rendering list. May I know where to disable it?  Thanks.

Yongsheng

[Zhenyao Mo]

It's multisampling.

On Tue, Feb 28, 2012 at 9:22 PM, Yongsheng Zhu

[Yongsheng Zhu]

okay, thanks a lot.

[Yongsheng Zhu]

hi, 

We're working on the removal of Cedartrail, the next generation of Netbook, from the GPU black list. The attached document is the webgl conformance test results. We know 3 kinds of blocking issues from your feedback. So we also want to know whether there are other blocking issues. I know it will take much time of you to review this. So thanks a lot for your time. 

One question, could it be possible to let us know the critical cases or checklist for WebGL so that we can do it by ourselves firstly before sending this out?

Thanks,

Yongsheng