using system mesa

[Paweł Hajdan, Jr.]

I did a quick experiment to try adding an option to use system mesa for Linux distros, but ran into issues. I'd have some questions:

1. Why some gyp targets like ui/gl/gl.gyp:gl hardcode mesa include paths instead of depending on some target from third_party/mesa/mesa.gyp?

2. What's the purpose of ui/gl/generate_bindings.py?

3. Why are symbol names mirroring mesa naming scheme being put into gfx namespace in generated out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h ? This results in name clashes like below (when trying system mesa):

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:40:43: error: previous declaration of ‘GLboolean (* gfx::g_OSMesaGetColorBuffer)(osmesa_context*, GLint*, GLint*, GLint*, void**)’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:41:43: error: previous declaration of ‘void (* (* gfx::g_OSMesaGetProcAddress)(const char*))()’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:42:39: error: previous declaration of ‘void (* gfx::g_OSMesaColorClamp)(GLboolean)’

make: *** [out/Debug/obj.target/gl/ui/gl/gl_surface_linux.o] Error 1

[Paweł Hajdan, Jr.]

ping

[Antoine Labour]

On Mon, Oct 22, 2012 at 12:03 PM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

I did a quick experiment to try adding an option to use system mesa for Linux distros, but ran into issues.

I think we have some fair amount of mods in third_party/mesa.

 

I'd have some questions:

1. Why some gyp targets like ui/gl/gl.gyp:gl hardcode mesa include paths instead of depending on some target from third_party/mesa/mesa.gyp?

That could probably be fixed, though right now mesa.gyp doesn't have a target that would export the path without compiling something. Adding a deps would make us compile osmesa when we don't really need it (see below).

 

2. What's the purpose of ui/gl/generate_bindings.py?

We need to choose the GL implementation at run-time because different systems have different GL support (e.g. native vs ANGLE on windows, GLX vs EGL on linux/Chrome OS etc., osmesa for tests etc.), so we need to dynamically load the GL library.

We don't want this fact to leak all over the GPU code, so we generate wrappers around the GL libraries that hide the symbol indirection. That's what generate_bindings.py does.

3. Why are symbol names mirroring mesa naming scheme being put into gfx namespace in generated out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h ? This results in name clashes like below (when trying system mesa):

We reuse GL/GLX/EGL/WGL/MESA names so that the calling code is clear about what it calls.

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:40:43: error: previous declaration of ‘GLboolean (* gfx::g_OSMesaGetColorBuffer)(osmesa_context*, GLint*, GLint*, GLint*, void**)’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:41:43: error: previous declaration of ‘void (* (* gfx::g_OSMesaGetProcAddress)(const char*))()’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:42:39: error: previous declaration of ‘void (* gfx::g_OSMesaColorClamp)(GLboolean)’

make: *** [out/Debug/obj.target/gl/ui/gl/gl_surface_linux.o] Error 1

I suspect some files depend on include order. Ideally we wouldn't need to include osmesa headers at all, however enums are not covered by generate_bindings, so I suspect that's why osmesa.h is still included.

Antoine

 

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Paweł Hajdan, Jr.]

On Mon, Oct 29, 2012 at 3:16 PM, Antoine Labour <pi...@google.com> wrote:

On Mon, Oct 22, 2012 at 12:03 PM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

I did a quick experiment to try adding an option to use system mesa for Linux distros, but ran into issues.

I think we have some fair amount of mods in third_party/mesa.

Have you sent them upstream? If not, I'd be happy to help with doing the upstreaming. This should help with the maintenance, right? :)

 

I'd have some questions:

1. Why some gyp targets like ui/gl/gl.gyp:gl hardcode mesa include paths instead of depending on some target from third_party/mesa/mesa.gyp?

That could probably be fixed, though right now mesa.gyp doesn't have a target that would export the path without compiling something. Adding a deps would make us compile osmesa when we don't really need it (see below).

Would you be OK with me just adding a target that would set the right include paths? This should avoid the problem with unnecessarily pulling in osmesa, while bringing the code closer to my goal.

By the way, is osmesa used only for testing?

 

2. What's the purpose of ui/gl/generate_bindings.py?

We need to choose the GL implementation at run-time because different systems have different GL support (e.g. native vs ANGLE on windows, GLX vs EGL on linux/Chrome OS etc., osmesa for tests etc.), so we need to dynamically load the GL library.

We don't want this fact to leak all over the GPU code, so we generate wrappers around the GL libraries that hide the symbol indirection. That's what generate_bindings.py does.

Got it. When adding option to use system ffmpeg, I've noticed it's easier to just bypass the stubs/bindings completely. Does the above rationale still make sense assuming we know at compile time which GL library we're going to use?

 

3. Why are symbol names mirroring mesa naming scheme being put into gfx namespace in generated out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h ? This results in name clashes like below (when trying system mesa):

We reuse GL/GLX/EGL/WGL/MESA names so that the calling code is clear about what it calls.

Sorry, I think that doesn't answer the original question. To solve that, you'd probably introduce different prefixes or something, but putting the names in Chrome-specific gfx namespace doesn't seem to address the above issue at all. :-/

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:40:43: error: previous declaration of ‘GLboolean (* gfx::g_OSMesaGetColorBuffer)(osmesa_context*, GLint*, GLint*, GLint*, void**)’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:41:43: error: previous declaration of ‘void (* (* gfx::g_OSMesaGetProcAddress)(const char*))()’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:42:39: error: previous declaration of ‘void (* gfx::g_OSMesaColorClamp)(GLboolean)’

make: *** [out/Debug/obj.target/gl/ui/gl/gl_surface_linux.o] Error 1

I suspect some files depend on include order. Ideally we wouldn't need to include osmesa headers at all, however enums are not covered by generate_bindings, so I suspect that's why osmesa.h is still included.

I don't think so. I rather think it's having the same symbols declared in both gfx and no namespace. Maybe with your answers to the above questions I'll figure out something more.

Thanks,

Paweł

[Stéphane Marchesin]

On Thu, Nov 8, 2012 at 9:29 AM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

On Mon, Oct 29, 2012 at 3:16 PM, Antoine Labour <pi...@google.com> wrote:

On Mon, Oct 22, 2012 at 12:03 PM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

I did a quick experiment to try adding an option to use system mesa for Linux distros, but ran into issues.

I think we have some fair amount of mods in third_party/mesa.

Have you sent them upstream? If not, I'd be happy to help with doing the upstreaming. This should help with the maintenance, right? :)

Well, I think it's more complicated than just upstreaming. A lot of changes aren't upstreamable as-is. Others apply to parts of mesa which have changed since then. You'll also hit new issues which aren't present in Chrome's mesa but are there on git mesa.

That said, if you want to work on this, I'll be happy to help/review the patches.

Stéphane

[Antoine Labour]

On Thu, Nov 8, 2012 at 9:29 AM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

On Mon, Oct 29, 2012 at 3:16 PM, Antoine Labour <pi...@google.com> wrote:

On Mon, Oct 22, 2012 at 12:03 PM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

I did a quick experiment to try adding an option to use system mesa for Linux distros, but ran into issues.

I think we have some fair amount of mods in third_party/mesa.

Have you sent them upstream? If not, I'd be happy to help with doing the upstreaming. This should help with the maintenance, right? :)

 

I'd have some questions:

1. Why some gyp targets like ui/gl/gl.gyp:gl hardcode mesa include paths instead of depending on some target from third_party/mesa/mesa.gyp?

That could probably be fixed, though right now mesa.gyp doesn't have a target that would export the path without compiling something. Adding a deps would make us compile osmesa when we don't really need it (see below).

Would you be OK with me just adding a target that would set the right include paths? This should avoid the problem with unnecessarily pulling in osmesa, while bringing the code closer to my goal.

I don't mind.

 

By the way, is osmesa used only for testing?

Mostly testing/bots and to do development on some machines that don't have a working GL stack (e.g. Aura requires a GPU).

 

2. What's the purpose of ui/gl/generate_bindings.py?

We need to choose the GL implementation at run-time because different systems have different GL support (e.g. native vs ANGLE on windows, GLX vs EGL on linux/Chrome OS etc., osmesa for tests etc.), so we need to dynamically load the GL library.

We don't want this fact to leak all over the GPU code, so we generate wrappers around the GL libraries that hide the symbol indirection. That's what generate_bindings.py does.

Got it. When adding option to use system ffmpeg, I've noticed it's easier to just bypass the stubs/bindings completely. Does the above rationale still make sense assuming we know at compile time which GL library we're going to use?

How can you know at compile time what GPU you have on the target machine, and whether or not it supports GL?

The bindings have changed recently, and I believe we now rely on the indirection for more than the different GL library, but as well to implement GPU context virtualization. gman@ would know.

*If* you can make it work without adding complexity that creeps everywhere on the call sites, I don't mind. But please remember that using system libraries is an explicit non-goal of the chromium project, while we can accommodate local fixes, we do not want to add complexity in the code to support this unsupported use case.

 

 

3. Why are symbol names mirroring mesa naming scheme being put into gfx namespace in generated out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h ? This results in name clashes like below (when trying system mesa):

We reuse GL/GLX/EGL/WGL/MESA names so that the calling code is clear about what it calls.

Sorry, I think that doesn't answer the original question. To solve that, you'd probably introduce different prefixes or something, but putting the names in Chrome-specific gfx namespace doesn't seem to address the above issue at all. :-/

gl_bindings_autogen_osmesa.h has some types and other things in the gfx namespace, that don't conflict with osmesa headers. It does also define macros that wrap getting to the bindings, that have the same names as the osmesa entrypoints. These are the ones that conflict if they are included before the osmesa headers, which is why I think the issue is with include order. E.g. OSMesaGetProcAddress expands to (or used to, bindings have changed recently) gfx::g_OSMesaGetProcAddress, so if the osmesa.h header is included after, you end up with an illegal declaration like the following.

The use of the gfx namespace has nothing to do *at all* with your issues.

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:40:43: error: previous declaration of ‘GLboolean (* gfx::g_OSMesaGetColorBuffer)(osmesa_context*, GLint*, GLint*, GLint*, void**)’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:267:44: error: ‘void (* gfx::g_OSMesaGetProcAddress(const char*))()’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:41:43: error: previous declaration of ‘void (* (* gfx::g_OSMesaGetProcAddress)(const char*))()’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ should have been declared inside ‘gfx’

/usr/include/GL/osmesa.h:276:34: error: ‘void gfx::g_OSMesaColorClamp(GLboolean)’ redeclared as different kind of symbol

out/Debug/obj/gen/ui/gl/gl_bindings_autogen_osmesa.h:42:39: error: previous declaration of ‘void (* gfx::g_OSMesaColorClamp)(GLboolean)’

make: *** [out/Debug/obj.target/gl/ui/gl/gl_surface_linux.o] Error 1

I suspect some files depend on include order. Ideally we wouldn't need to include osmesa headers at all, however enums are not covered by generate_bindings, so I suspect that's why osmesa.h is still included.

I don't think so. I rather think it's having the same symbols declared in both gfx and no namespace. Maybe with your answers to the above questions I'll figure out something more.

No, I don't think there's any issue with a symbol being declared in the gfx and no namespace. If there was, it would cause issues at the call site, not when including the header.

Antoine

[Gregg Tavares (社用)]

I'm kind of lost what the questions are anymore but ....

As piman said, the stuff in ui/gl is abstracting/redirecting all these various system APIs through our own stubs so that at runtime we can choose to run on different graphics stacks without recompilng. Right now on any platform you can use --use-gl=osmesa and the calls go glXXX like glTexImage2D will go to our own osmesa where as without that flag they'll likely go to the OpenGL driver on mac/linux and to OpenGL ES 2.0 on android/windows (supplied by ANGLE on Windows)

All the code in chrome in the GPU process goes though this indirection. That code looks like it's calling the normal 'glXXX' calls but is actually going through function pointers to the runtime selected API. This is done by including ui/gl/gl_bindings.h anywhere we want to call gl.

Additionally an option was recently added were we can virtualize multiple simulated contexts on top of one real context. In that case the function pointers in ui/gl point the virtual implementation and the virtual implementation calls the old function pointers to whatever implementations were selected at runtime. 

gl_bindings.h includes a bunch of macros that redefine all the gl functions so

   glBindTexture(...)

becomes

   ::gfx::g_current_gl_context->glBindTextureFn

Some of the errors mentioned above may have gone away based on the virtual context changes or at least changed as there are less global pointers now.

I'm not sure if that answers your questions or not.