Wrong pixel format (OpenGL) on notebooks with dual GPU on Windows

[Anthony Nemoff]

Hi,

We experienced an OpenGL display bug when running FLTK applications that use Fl_Gl_Windows (our own app, but also FLTK examples like glpuzzle, gl_overlay, fractals, etc.) on a notebook that has both an Intel integrated GPU and an NVidia GPU:

- correct display when running with the integrated GPU: everything looks fine (but obviously less performant);

- incorrect display when running with the NVidia GPU (Right-click > Run with graphics processor > High-performance NVIDIA processor): the OpenGL viewport looks pale as if it had a gamma correction.

We tried combining different modes when calling Fl_Gl_Window::mode, but it didn't fix the issue when running on NVIDIA.

After digging into FLTK code and debugging we found out that the chosen pixel format is not the same, especially the color bit depth is smaller when running on the NVidia GPU (among other minor differences).

We worked around this bug by forcing the pixel format to a hand crafted one in fl_create_gl_context instead of using the one chosen by Fl_Gl_Choice::find.

But we're looking into a cleaner solution.

We are using version 1.3.4 rev 11742.

Any ideas?

Thanks.

[Albrecht Schlosser]

On 25.01.2018 17:18 Anthony Nemoff wrote:
>
> We experienced an OpenGL display bug when running FLTK applications that
> use Fl_Gl_Windows (our own app, but also FLTK examples like glpuzzle,
> gl_overlay, fractals, etc.) on a notebook that has both an Intel
> integrated GPU and an NVidia GPU:
> - correct display when running with the integrated GPU: everything looks
> fine (but obviously less performant);
> - incorrect display when running with the NVidia GPU (Right-click > Run
> with graphics processor > High-performance NVIDIA processor): the OpenGL
> viewport looks pale as if it had a gamma correction.
>
> We tried combining different modes when calling Fl_Gl_Window::mode, but
> it didn't fix the issue when running on NVIDIA.
> After digging into FLTK code and debugging we found out that the chosen
> pixel format is not the same, especially the color bit depth is smaller
> when running on the NVidia GPU (among other minor differences).

Can you post your exact findings? I'd be interested to know what's going
on. It'd be interesting to see the resultant pixel formats you get with
both GPU's.

Please see also below for a patch that is instrumented with debug output.

> We worked around this bug by forcing the pixel format to a hand crafted
> one in fl_create_gl_context instead of using the one chosen
> by Fl_Gl_Choice::find.
> But we're looking into a cleaner solution.
> We are using version 1.3.4 rev 11742.

Thanks for the report and the debugging. Selecting the "wrong" pixel
format seems to be a known issue (STR #3119), although this STR is a
slightly different case, i.e. it's a supposedly "wrong" pixel format but
with another background.
http://www.fltk.org/str.php?L3119

> Any ideas?

Please take a look at the above mentioned STR. I prepared a patch, but
this patch has not yet been integrated into the FLTK lib. But maybe it
can help you or you can provide additional insights to select a sensible
algorithm in FLTK. Please try the patch - it can be applied cleanly to
FLTK svn current and likely to the latest snapshot as well, but you may
need to tweak it if you want to apply it to FLTK 1.3.x.

http://www.fltk.org/strfiles/3119/support_composition_v2.diff

The patch contains some debug output statements that can be helpful. As
I said, it's related to another issue, but anyway, this is what I get on
my system when I run test/glpuzzle:

$ bin/examples/glpuzzle.exe
pfd #5 supports composition: yes
... & PFD_GENERIC_FORMAT: accelerated
... Overlay Planes : 0
... Color & Depth : 32, 24
pfd #7 supports composition: yes
... & PFD_GENERIC_FORMAT: accelerated
... Overlay Planes : 0
... Color & Depth : 32, 16
current pixelformat : 5
pfd #9 supports composition: yes
... & PFD_GENERIC_FORMAT: accelerated
... Overlay Planes : 0
... Color & Depth : 32, 16
current pixelformat : 5
pfd #10 supports composition: yes
... & PFD_GENERIC_FORMAT: accelerated
... Overlay Planes : 0
... Color & Depth : 32, 24
current pixelformat : 5
pfd #38 supports composition: yes
... & PFD_GENERIC_FORMAT: generic
... Overlay Planes : 0
... Color & Depth : 32, 32
current pixelformat : 10
pfd #39 supports composition: yes
... & PFD_GENERIC_FORMAT: generic
... Overlay Planes : 0
... Color & Depth : 32, 16
current pixelformat : 10
pfd #42 supports composition: yes
... & PFD_GENERIC_FORMAT: generic
... Overlay Planes : 0
... Color & Depth : 32, 32
current pixelformat : 10
pfd #43 supports composition: yes
... & PFD_GENERIC_FORMAT: generic
... Overlay Planes : 0
... Color & Depth : 32, 16
current pixelformat : 10
PFD_SUPPORT_COMPOSITION = 0x8000
Chosen pixel format is 10
Pixel format supports composition: yes

You may want to add other tests and output statements so you can see
what's different if you run it with different GPU's.

I can't test this since I don't have different GPU's and, BTW, I believe
the algorithm can fail on a particular GPU no matter if you have one or
two, so I guess it would always fail for you if you didn't have the
Intel GPU (or switched it off by hardware/BIOS setup), but this is only
a guess and may be wrong.

I'd appreciate if you post your results here. Maybe we can find a better
algorithm that selects the "right" pixel format for you. However, I
believe that the algorithm is very fragile because the order of pixel
formats in Windows' enumeration and the selection order based on
"assumed priority" both play a role in the final decision. It's hard or
even impossible to predict the outcome...

If this all doesn't work we can try to provide a function to let the
user set a pixel format or give a list of pixel format bits ordered by
user priority, but this would IMHO be a little too low level for most
users. But if it helps...

Currently I don't have another idea how to solve the issue in FLTK. Any
hints and suggestions are welcome.

[Anthony Nemoff]

Thanks for the quick answer and the details.

Actually, we had already patched Fl_Gl_Choice for STR 3119 but I'll try this new one with the debug output to get more info.

I'll share the details of my findings as soon as I can test them again.

Having an API that gives more control on the pixel format could be useful indeed.

But before doing that, it's worth investigating why the current implementation fails under some specific environments.

Regards,

[Albrecht Schlosser]

On 26.01.2018 11:08 Anthony Nemoff wrote:
> Thanks for the quick answer and the details.

Welcome.

> Actually, we had already patched Fl_Gl_Choice for STR 3119 but I'll try
> this new one with the debug output to get more info.
> I'll share the details of my findings as soon as I can test them again.

That would be very much appreciated.

> Having an API that gives more control on the pixel format could be
> useful indeed.
> But before doing that, it's worth investigating why the current
> implementation fails under some specific environments.

I agree, and that's why I asked you to test and post your output. TIA.

You may need to add even more output statements to see what's actually
going on in your case, particularly the color bit depth if that's your
main issue. The printf's added to the patch were primarily to see why a
pixel format w/o composition (PFD_SUPPORT_COMPOSITION) was chosen (the
issue of STR #3119).

The problem(s) I see with the existing algorithm:

(1) Windows presents the available pixel formats in a list whose order
we don't know.

(2) The algorithm tries to find "better" pixel formats sequentially,
using an implicit order of preferences (priority) given by the order of
tests in the chosen algorithm.

(3) We don't know the user's actual preferences which may eventually
lead to choosing a pixel format that is suboptimal for a particular user
and/or application.

I believe that we can _miss_ a pixel format with a higher bit depth if
one of the other properties in that particular pixel format is "lower"
(i.e. usually false) than this property in the pixel format that was
chosen so far. My guess is that something like that is happening in your
case. If this is true we'd need additional output at the beginning of
the selection process about color depth and other properties so we can
see what type of pixel format is skipped - particularly if this is the
pixel format you selected in your hardcoded decision (your own patch).

Thanks for your help.

[Anthony Nemoff]

Hi,

So, I added more output information in Fl_Gl_Choice::find and ran glpuzzle on the guilty machine.

When running with the Intel GPU, the chosen PDF was:

pfd #10:

- Composition           : yes

- Hardware acceleration : yes

- Double Buffering      : yes

- Overlay Planes        : 0 overlay planes & 0 underlay planes

- Color Buffer Depth    : 32 (excluding alpha)

- RGBA Bits             :  8,  8,  8,  8

- Alpha Buffer Depth    :  8

- Accum Buffer Depth    : 64

- Accum RGBA Bits       : 16, 16, 16, 16

- Depth Buffer Depth    : 24

- Stencil Buffer Depth  :  8

- Buffer Swap Behavior  : copy

- Visible Mask          : 0

When running with the Nvidia GPU, the chosen PDF was:

pfd #90:

- Composition           : yes

- Hardware acceleration : yes

- Double Buffering      : yes

- Overlay Planes        : 0 overlay planes & 0 underlay planes

- Color Buffer Depth    : 64 (excluding alpha)

- RGBA Bits             : 16, 16, 16, 16

- Alpha Buffer Depth    : 16

- Accum Buffer Depth    :  0

- Accum RGBA Bits       :  0,  0,  0,  0

- Depth Buffer Depth    : 24

- Stencil Buffer Depth  :  8

- Buffer Swap Behavior  : exchange

- Visible Mask          : 0

PFD #10 (8 bits) is also available with Nvidia, but PFD #90 has higher color bit depth (16 bits) so it was chosen. I can post the full list of PFDs if you like.

The buffer looks as if it had a gamma of 0.45.

Also, I noticed that widgets that redraw constantly their buffer don't have the issue. For example, the fractals demo with the spinning geometry is OK as long as the rotation is ON, but once you stop the rotation the issue appears (no more redraws).

We are thinking of modifying the condition when comparing color bit depth: favor higher bit depth but not higher than a specified value.

[Albrecht Schlosser]

I'm interested to see the list, but we'd need it for both GPU's. If you
post it, please use two attachments (don't post the list inline).
Another possibility would be to put it online somewhere, maybe
pastebin.com or similar.
https://pastebin.com/

I'm also interested to see the output statements you used for the list
above so I can see exactly which variables of the PFD are used for the
list. Could you please post just that part of your code?

> The buffer looks as if it had a gamma of 0.45.

How did you find this value (0.45)?

Might be interesting to see two images of, say glpuzzle. Can you post
two screenshots of the same program with both GPU's?

> Also, I noticed that widgets that redraw constantly their buffer don't
> have the issue. For example, the fractals demo with the spinning
> geometry is OK as long as the rotation is ON, but once you stop the
> rotation the issue appears (no more redraws).

When I read this I thought of an issue with the alpha channel, but
that's more likely not the case. Or maybe it is, who knows...

> We are thinking of modifying the condition when comparing color bit
> depth: favor higher bit depth but not higher than a specified value.

And the specified value would be 8 (see below).

Some thoughts:

(1) FLTK doesn't use 16-bit colors internally, we're limited to 8-bit
colors anyway, so we wouldn't lose anything if we skipped PFD's with
more than 8 color and alpha bits. Hence we could even hard-code this:
skip PFD's that offer more than 8 color bits (at least if we do already
have 8 color bits) or something like that. I mean your proposal would be
good with a hard limit of 8 as the highest acceptable color depth.

(2) Point (1) would IMHO only cure symptoms. I don't know if a higher
color bit depth in the GPU would offer any advantages, but if it did we
should check the code at other places. Maybe we don't handle this
correctly when we "feed" the GPU with our color bits? Maybe we don't do
correct sign extension or color or (maybe more likely) alpha value
propagation from our internal 8-bit value to the GPU's 16-bit value?

Unfortunately I'm personally not an OpenGL expert and I don't have a
16-bit-color GPU to test so the only thing I can do regarding point (2)
is to ask you if you can try to debug this and maybe find the cause of
the issue elsewhere in our OpenGL related code. Or one of the other devs
who knows more about our OpenGL implementation (and maybe has a
16-bit-color GPU) can help...

Anyway, point (1) would be a sensible workaround if we can't find a
better solution with (2).

Thanks again for your help.

[Anthony Nemoff]

I'm interested to see the list, but we'd need it for both GPU's.

I have attached 3 text files with the different outputs: 

I'm also interested to see the output statements you used for the list
above so I can see exactly which variables of the PFD are used for the
list. Could you please post just that part of your code?

Here is my code to output the PFD details (then I used the function in the loop for each PFD) :

static void debug_print_pfd(int pfd_id, const PIXELFORMATDESCRIPTOR &pfd) {
    printf("pfd #%d:\n", pfd_id);
    printf("- Composition           : %s\n", (pfd.dwFlags & PFD_SUPPORT_COMPOSITION) ? "yes" : "no");
    printf("- Hardware acceleration : %s\n", (pfd.dwFlags & PFD_GENERIC_FORMAT) ? "no" : "yes");
    printf("- Double Buffering      : %s\n", (pfd.dwFlags & PFD_DOUBLEBUFFER) ? "yes" : "no");
    printf("- Overlay Planes        : %u overlay planes & %u underlay planes\n", (pfd.bReserved & 0x0F), (pfd.bReserved & 0xF0));
    printf("- Color Buffer Depth    : %2d (excluding alpha)\n", pfd.cColorBits);
    printf("- RGBA Bits             : %2d, %2d, %2d, %2d\n", pfd.cRedBits, pfd.cGreenBits, pfd.cBlueBits, pfd.cAlphaBits);
    printf("- Alpha Buffer Depth    : %2d\n", pfd.cAlphaBits);
    printf("- Accum Buffer Depth    : %2d\n", pfd.cAccumBits);
    printf("- Accum RGBA Bits       : %2d, %2d, %2d, %2d\n", pfd.cAccumRedBits, pfd.cAccumGreenBits, pfd.cAccumBlueBits, pfd.cAccumAlphaBits);
    printf("- Depth Buffer Depth    : %2d\n", pfd.cDepthBits);
    printf("- Stencil Buffer Depth  : %2d\n", pfd.cStencilBits);
    printf("- Buffer Swap Behavior  : %s\n", (pfd.dwFlags & PFD_SWAP_COPY) ? "copy" : ((pfd.dwFlags & PFD_SWAP_EXCHANGE) ? "exchange" : "unknown"));
    printf("- Aux Buffers Count     : %d\n", pfd.cAuxBuffers);
    printf("- Visible Mask          : %d\n", pfd.dwVisibleMask);
}

How did you find this value (0.45)?

0.45 is the "standard" gamma value used when encoding images to disk (or buffers). Then a value of 2.2 is used when displaying it on a display, to restore it.

So I just tried applying a gamma of 0.45 to the correct render to see if it matched the incorrect render.

I have attached 2 screenshots:

- one with both renders side to side

- same screenshot with gamma correction on each render.

I had the opportunity to test the "vanilla" glpuzzle on another desktop machine also equipped with a Nvidia GPU that also supports 16 bits buffers:

- the display was OK, which is confusing compared to the results on the laptop;

- the PFD had the same values as the PFD chosen on the laptop.

(1) FLTK doesn't use 16-bit colors internally, we're limited to 8-bit
colors anyway, so we wouldn't lose anything if we skipped PFD's with
more than 8 color and alpha bits. Hence we could even hard-code this:
skip PFD's that offer more than 8 color bits (at least if we do already
have 8 color bits) or something like that. I mean your proposal would be
good with a hard limit of 8 as the highest acceptable color depth.

Looks good to me as a first fix.

In fact, I changed my fix to follow this idea rather than hand-crafting a PFD and forcing it. In Fl_Gl_Choice::find the test now is:

// otherwise more bit planes is better, but no more than 32 (8 bits per channel):
else if (pfd.cColorBits > 32 || chosen_pfd.cColorBits > pfd.cColorBits) continue;

 

(2) Point (1) would IMHO only cure symptoms. I don't know if a higher
color bit depth in the GPU would offer any advantages, but if it did we
should check the code at other places. Maybe we don't handle this
correctly when we "feed" the GPU with our color bits? Maybe we don't do
correct sign extension or color or (maybe more likely) alpha value
propagation from our internal 8-bit value to the GPU's 16-bit value?

Unfortunately I'm personally not an OpenGL expert and I don't have a
16-bit-color GPU to test so the only thing I can do regarding point (2)
is to ask you if you can try to debug this and maybe find the cause of
the issue elsewhere in our OpenGL related code. Or one of the other devs
who knows more about our OpenGL implementation (and maybe has a
16-bit-color GPU) can help...

We are investigating this on our side, but we will most likely stick to 8 bits for now.

Thanks. 

[Albrecht Schlosser]

On 02.02.2018 15:52 Anthony Nemoff wrote:
> I'm interested to see the list, but we'd need it for both GPU's.
>
> I have attached 3 text files with the different outputs:
>
>
> I'm also interested to see the output statements you used for the list
> above so I can see exactly which variables of the PFD are used for the
> list. Could you please post just that part of your code?
>
>
> Here is my code to output the PFD details (then I used the function in
> the loop for each PFD) :

[...]

Thanks for the code, the text files, the images, and the explanation of
what you did and how you found the gamma values. The lists are very
helpful in finding the order of PFD's and maybe a better algorithm,
independent of your actual issue.

> I had the opportunity to test the "vanilla" glpuzzle on another desktop
> machine also equipped with a Nvidia GPU that also supports 16 bits buffers:
> - the display was OK, which is confusing compared to the results on the
> laptop;
> - the PFD had the same values as the PFD chosen on the laptop.

Yes, this is a "known" problem: such display and OpenGL related issues
are often caused by the drivers or their setups. Is there any chance you
can compare the Nvidia drivers (at least their dates) or that you can
update these drivers to newer versions? Maybe this would resolve the
problem for you w/o any other work.

Is there a GPU (Nvidia) driver setup that allows to limit the driver to
8-bit colors?

> (1) FLTK doesn't use 16-bit colors internally, we're limited to 8-bit
> colors anyway, so we wouldn't lose anything if we skipped PFD's with
> more than 8 color and alpha bits. Hence we could even hard-code this:
> skip PFD's that offer more than 8 color bits (at least if we do already
> have 8 color bits) or something like that. I mean your proposal
> would be
> good with a hard limit of 8 as the highest acceptable color depth.
>
>
> Looks good to me as a first fix.
> In fact, I changed my fix to follow this idea rather than hand-crafting
> a PFD and forcing it. In Fl_Gl_Choice::find the test now is:
>
> |
> // otherwise more bit planes is better, but no more than 32 (8 bits per
> channel):
> elseif(pfd.cColorBits >32||chosen_pfd.cColorBits >pfd.cColorBits)continue;
> |

Thanks for this patch. I'm not sure whether we should implement it,
although I believe it wouldn't do any harm. Maybe we should...

Looking forward to the findings of your Nvidia drivers and versions.

> (2) Point (1) would IMHO only cure symptoms. I don't know if a higher
> color bit depth in the GPU would offer any advantages, but if it did we
> should check the code at other places. Maybe we don't handle this
> correctly when we "feed" the GPU with our color bits? Maybe we don't do

> correct sign extension or color or (maybe more likely) alphvalue

> propagation from our internal 8-bit value to the GPU's 16-bit value?
>
> Unfortunately I'm personally not an OpenGL expert and I don't have a
> 16-bit-color GPU to test so the only thing I can do regarding point (2)
> is to ask you if you can try to debug this and maybe find the cause of
> the issue elsewhere in our OpenGL related code. Or one of the other
> devs
> who knows more about our OpenGL implementation (and maybe has a
> 16-bit-color GPU) can help...
>
>
> We are investigating this on our side, but we will most likely stick to
> 8 bits for now.

I suggest to check the Nvidia drivers first, but I'd appreciate if you
could find something related to 16-bit vs. 8-bit colors in the FLTK code
or the confirmation that "nothing is wrong with it" (as far as you can
tell, of course).

Thanks in advance.

[imm]

With the usual caveat that I probably don't know what I'm talking about...

On 2 February 2018 at 14:52, Anthony Nemoff wrote:
>

> So I just tried applying a gamma of 0.45 to the correct render to see if it
> matched the incorrect render.
> I have attached 2 screenshots:
> - one with both renders side to side
> - same screenshot with gamma correction on each render.

OK - it looks to me that the screenshots with the gamma "adjustments"
applied, seemed to be showing us that the gamma is indeed being
misapplied to one of the GPU drivers (presumably the Nvidia one in
these screenshots?)

This machine doesn't have an Nvidia driver so I can't check, but the
last time I looked the Nvidia config GUI had settings for adjusting
the gamma for the driver (on a per-monitor basis?)

So it seems feasible that the fltk GL stuff is potentially "OK" and
the different appearance is arising from some unexpected local setting
that doesn't affect the intel GPU driver?

> I had the opportunity to test the "vanilla" glpuzzle on another desktop
> machine also equipped with a Nvidia GPU that also supports 16 bits buffers:
> - the display was OK, which is confusing compared to the results on the
> laptop;
> - the PFD had the same values as the PFD chosen on the laptop.

Which might support the hypothesis that the problem is related to a
local driver setting, then?

>> (1) FLTK doesn't use 16-bit colors internally, we're limited to 8-bit
>> colors anyway, so we wouldn't lose anything if we skipped PFD's with
>> more than 8 color and alpha bits. Hence we could even hard-code this:
>> skip PFD's that offer more than 8 color bits (at least if we do already
>> have 8 color bits) or something like that. I mean your proposal would be
>> good with a hard limit of 8 as the highest acceptable color depth.
>

Though some folks do need higher bit-depths (e.g. Robin Rowe for one)
so we should consider how that would work.

Perhaps a setting that defaults to an 8-bits per element depth, but
with an API to request deeper colours if they are needed?

[Anthony Nemoff]

So it seems feasible that the fltk GL stuff is potentially "OK" and
the different appearance is arising from some unexpected local setting
that doesn't affect the intel GPU driver?

In fact, we faced a new scenario that reinforces the idea that the display or driver settings are guilty.

When using the same laptop with dual GPUs with and a secondary screen (using HDMI), the issue disappeared when using the 2nd monitor as the main display.

When I will have more time I'll look into the display and drivers settings.

 

Perhaps a setting that defaults to an 8-bits per element depth, but
with an API to request deeper colours if they are needed?

Something like that, yes.