Data parsimony and WebGL shaders
37 views
Skip to first unread message
Bruce Sherwood
May 17, 2012, 11:33:57 AM5/17/12
to dis...@sfcomplex.org
I've had a lot of trouble with WebGL shader code working on my Windows
machine and failing on a Macbook Pro and Ubuntu, all with NVIDIA
graphics and drivers.
I did a lot of work to ensure that I send to each of the many shaders
only that data that is actually referenced and used by that particular
shader, and this change made mouse picking and transparency work again
on all platforms. Moral of the tale: don't send anything from CPU to
shaders that the shaders aren't going to use, even though this works
okay on Windows. It seems to me that the shader compilers in the Mac
and Ubuntu drivers are wrong, but at least now I know what to do to
preserve cross-platform compatibility.
In other news, over the last couple of weeks I had intensive
discussions with David Scherer about the GlowScript API for creating
your own models out of triangles or quads and dynamically modifying
them to change the shape of the object. I'm working toward a major
release, 1.0. Some time ago I posted an example of a pulse moving
along a rug, which has been rewritten with the new API:
http://www.glowscript.org/#/user/Bruce_Sherwood/folder/Pub/program/Rug
To take the rug as a specific example, I create a set of vertices
covering the rug in a rectangular pattern. I then create quad objects
for each group of 4 vertices, with vertices internal to the rug shared
among 4 adjacent quads. The animation of the pulse running along the
rug is then done simply by modifying the vertices, with no need to
think about the quads that make the display. Take a look at the
CoffeeScript source code for the rug; it's now quite simple.
Bruce
machine and failing on a Macbook Pro and Ubuntu, all with NVIDIA
graphics and drivers.
I did a lot of work to ensure that I send to each of the many shaders
only that data that is actually referenced and used by that particular
shader, and this change made mouse picking and transparency work again
on all platforms. Moral of the tale: don't send anything from CPU to
shaders that the shaders aren't going to use, even though this works
okay on Windows. It seems to me that the shader compilers in the Mac
and Ubuntu drivers are wrong, but at least now I know what to do to
preserve cross-platform compatibility.
In other news, over the last couple of weeks I had intensive
discussions with David Scherer about the GlowScript API for creating
your own models out of triangles or quads and dynamically modifying
them to change the shape of the object. I'm working toward a major
release, 1.0. Some time ago I posted an example of a pulse moving
along a rug, which has been rewritten with the new API:
http://www.glowscript.org/#/user/Bruce_Sherwood/folder/Pub/program/Rug
To take the rug as a specific example, I create a set of vertices
covering the rug in a rectangular pattern. I then create quad objects
for each group of 4 vertices, with vertices internal to the rug shared
among 4 adjacent quads. The animation of the pulse running along the
rug is then done simply by modifying the vertices, with no need to
think about the quads that make the display. Take a look at the
CoffeeScript source code for the rug; it's now quite simple.
Bruce
Bruce Sherwood
May 17, 2012, 11:46:55 AM5/17/12
to dis...@sfcomplex.org
David Scherer made this interesting comment about shader compilers:
"If I had to guess, on Windows you are using Angle
(http://code.google.com/p/angleproject/issues/list), a library that
maps opengl to directx that is being developed at least largely for
webgl, and thus has a big focus on correctness, robustness, security.
On the other platforms you are probably using the hardware opengl
drivers, which have a big focus on running the latest games really
fast.
If you ask me, the OpenGL design decision to make the shader compiler
part of the driver was insane."
"If I had to guess, on Windows you are using Angle
(http://code.google.com/p/angleproject/issues/list), a library that
maps opengl to directx that is being developed at least largely for
webgl, and thus has a big focus on correctness, robustness, security.
On the other platforms you are probably using the hardware opengl
drivers, which have a big focus on running the latest games really
fast.
If you ask me, the OpenGL design decision to make the shader compiler
part of the driver was insane."
Edward Angel
May 17, 2012, 12:02:09 PM5/17/12
to dis...@sfcomplex.org
It is true that some of the Windows implementations are coverting everything to Direct X, a strategy that is typical MS and as insane as anything anyone else might try.
The supposed downside of having the drivers compile the shaders is the time it takes compared to downloading compiled code but that is less of a problem these days and makes portability much easier. Nvidia made a big deal of the fact that Cg used compiled shader code but it's not clear it matters to many people.
With all the issues you've had pushing the boundaries with WebGL, it's not been clear where the problem really lies. Is it drivers or something in the browsers or the memory architecture of particular GPUs. Fortunately, the future should be a lot better since WebGL is taking off.
Ed
__________
Ed Angel
Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico
1017 Sierra Pinon
Professor Emeritus of Computer Science, University of New Mexico
1017 Sierra Pinon
--
You received this message because you are subscribed to the Santa Fe Complex "discuss" group.
To post to this group, send email to dis...@sfcomplex.org
To unsubscribe from this group, send email to
discuss+u...@sfcomplex.org
For more options, visit this group at
http://groups.google.com/a/sfcomplex.org/group/discuss
Owen Densmore
May 17, 2012, 12:16:11 PM5/17/12
to dis...@sfcomplex.org
Fascinating!
I looked at the CS code and it made a lot of sense. What wasn't clear is the GPU interactions.
So in terms of data sent to the shader, I'm guessing you send just the vertices to your vertex attribute buffer, so you don't have to store multiple copies of the same vertex in the GPU. You then have a pretty sophisticated binding of these vertices to the image for texture drawing. Is that right? .. or do you have to replicate the vertices to keep the shaders simple?
Just as a point of interest, where *are* the shaders? My guess is that GlowScript has a set of standard shaders and separate program objects for standard use.
CS/GS certainly are very readable.
-- Owen
Bruce
Bruce Sherwood
May 17, 2012, 3:19:24 PM5/17/12
to dis...@sfcomplex.org
Some time ago Owen pointed out to me that it is much better for
various reasons to have many small special-purpose shader programs
rather than a few that contain many if statements for the various
situations. GlowScript now uses a total of 13 shader programs, 8
vertex shaders and 5 fragment shaders. There are three major
categories:
1) Display of opaque objects, of three types (fixed solids such as
box, sphere, etc.; curves -- a sequence of points; triangles and quads
for dynamic shapes)
2) Display of transparent objects by depth peeling (currently curves
cannot be transparent); see
http://www.khronos.org/message_boards/viewtopic.php?f=44&t=4905
3) Mouse picking (by assigning false colors to objects and using no
lighting calculations, so that the front-most pixel under the mouse
has a false color representing its id number)
The GlowScript files are found at
https://bitbucket.org/davidscherer/glowscript. In the Source folder,
lib/glow/WebGLRenderer.js has the CPU code for driving the shaders,
and the shaders are in Source/shaders (the experimental shaders
extent_vertex and final_fragment are not currently used).
A vertex object in CPU memory has arrays of position, normal, color,
opacity, texel coordinates, bumpmap coordinates, shininess, and
emissive (looks like it glows). The constructor for a vertex object
adds this information to Float32Arrays used for rendering.
As with all GlowScript objects, changing any attribute of a vertex has
the side effect of setting a changed flag. At the start of every
render, we run through the list of changed objects (box, sphere,
vertex, whatever) and update information to be passed to the GPUs.
This means that a user program can change the position of an object
many times between renders, but we pay the price of updating the GPU
data just once, at the start of the next render. (This is the same
scheme that is used by VPython.)
In the case of changes to a vertex, that means updating the
Float32Arrays. We then run through the triangle and quad objects and
categorize them as to whether they are totally opaque or have
transparent elements, and whether they have textures and/or bumpmaps,
and for each category we construct a list of vertex indices. Then for
example to render all of the opaque objects that have the same
texture, we just give the GPUs pointers to the vertex information and
to the index list for these objects (converted to Uint16Array), so
that all of the triangles and quads (which are rendered as though they
were two triangles) get rendered as one object, which is very fast.
I should mention that a position or color Float32Array might be many
thousands of vertices long, but if you give a Uint16Array index set of
just 6 numbers, corresponding to two triangles, it is the list of the
Uint16Array that governs, even though the indices point into giant
arrays. The length of the index array need not match the length of the
arrays pointed into.
There are some performance optimizations that I haven't done yet. In
particular, just to get going, I reload all of the Float32Array data
every render cycle using gl.bufferData, but I should reload just the
changed data using gl.bufferSubData.
Bruce
various reasons to have many small special-purpose shader programs
rather than a few that contain many if statements for the various
situations. GlowScript now uses a total of 13 shader programs, 8
vertex shaders and 5 fragment shaders. There are three major
categories:
1) Display of opaque objects, of three types (fixed solids such as
box, sphere, etc.; curves -- a sequence of points; triangles and quads
for dynamic shapes)
2) Display of transparent objects by depth peeling (currently curves
cannot be transparent); see
http://www.khronos.org/message_boards/viewtopic.php?f=44&t=4905
3) Mouse picking (by assigning false colors to objects and using no
lighting calculations, so that the front-most pixel under the mouse
has a false color representing its id number)
The GlowScript files are found at
https://bitbucket.org/davidscherer/glowscript. In the Source folder,
lib/glow/WebGLRenderer.js has the CPU code for driving the shaders,
and the shaders are in Source/shaders (the experimental shaders
extent_vertex and final_fragment are not currently used).
A vertex object in CPU memory has arrays of position, normal, color,
opacity, texel coordinates, bumpmap coordinates, shininess, and
emissive (looks like it glows). The constructor for a vertex object
adds this information to Float32Arrays used for rendering.
As with all GlowScript objects, changing any attribute of a vertex has
the side effect of setting a changed flag. At the start of every
render, we run through the list of changed objects (box, sphere,
vertex, whatever) and update information to be passed to the GPUs.
This means that a user program can change the position of an object
many times between renders, but we pay the price of updating the GPU
data just once, at the start of the next render. (This is the same
scheme that is used by VPython.)
In the case of changes to a vertex, that means updating the
Float32Arrays. We then run through the triangle and quad objects and
categorize them as to whether they are totally opaque or have
transparent elements, and whether they have textures and/or bumpmaps,
and for each category we construct a list of vertex indices. Then for
example to render all of the opaque objects that have the same
texture, we just give the GPUs pointers to the vertex information and
to the index list for these objects (converted to Uint16Array), so
that all of the triangles and quads (which are rendered as though they
were two triangles) get rendered as one object, which is very fast.
I should mention that a position or color Float32Array might be many
thousands of vertices long, but if you give a Uint16Array index set of
just 6 numbers, corresponding to two triangles, it is the list of the
Uint16Array that governs, even though the indices point into giant
arrays. The length of the index array need not match the length of the
arrays pointed into.
There are some performance optimizations that I haven't done yet. In
particular, just to get going, I reload all of the Float32Array data
every render cycle using gl.bufferData, but I should reload just the
changed data using gl.bufferSubData.
Bruce
Bruce Sherwood
May 17, 2012, 3:24:32 PM5/17/12
to dis...@sfcomplex.org
I should add that I'm working toward an imminent "1.0" release that
will include the vertex machinery and some other significant
enhancements, including a "compound' object originally designed by
Scherer: You give a list of (existing) objects and all of the
triangles are merged into one list to be rendered very efficiently as
one object (and with the same position/color/axis/size/up attributes
of the built-in objects).
Bruce
will include the vertex machinery and some other significant
enhancements, including a "compound' object originally designed by
Scherer: You give a list of (existing) objects and all of the
triangles are merged into one list to be rendered very efficiently as
one object (and with the same position/color/axis/size/up attributes
of the built-in objects).
Bruce
Bruce Sherwood
May 17, 2012, 3:28:49 PM5/17/12
to dis...@sfcomplex.org
The evidence strongly suggests that it is the compiler in the drivers.
The problems I saw on Mac and Ubuntu were present on multiple browsers
on those platforms, and they went away when (as of yesterday) I
stopped sending to a shader data that the particular shader didn't
use. Also, the architecture of the three NVIDIA cards on my three
machines is presumably similar.
Bruce
The problems I saw on Mac and Ubuntu were present on multiple browsers
on those platforms, and they went away when (as of yesterday) I
stopped sending to a shader data that the particular shader didn't
use. Also, the architecture of the three NVIDIA cards on my three
machines is presumably similar.
Bruce
Bruce Sherwood
May 17, 2012, 3:51:08 PM5/17/12
to dis...@sfcomplex.org
I see that I didn't really address your question about "where are the
shaders". Currently to build a new version of GlowScript, which runs
in the Google App Engine, I run a short Python program which reads up
all the shader programs from the shader folder and puts them into one
minimized file, from which they are read at execution time.
I'll mention that for an official (non-dev) release, we package up all
the files, in minimized form, with a version number. You may have
noticed a header line "GlowScript 0.7" or some such for each user
program. If you specify an older version, it should still run, because
we keep the old libraries by version number.
Bruce
On Thu, May 17, 2012 at 10:16 AM, Owen Densmore <ow...@backspaces.net> wrote:
> Just as a point of interest, where *are* the shaders? My guess is that
> GlowScript has a set of standard shaders and separate program objects for
> standard use.
>
> -- Owen
shaders". Currently to build a new version of GlowScript, which runs
in the Google App Engine, I run a short Python program which reads up
all the shader programs from the shader folder and puts them into one
minimized file, from which they are read at execution time.
I'll mention that for an official (non-dev) release, we package up all
the files, in minimized form, with a version number. You may have
noticed a header line "GlowScript 0.7" or some such for each user
program. If you specify an older version, it should still run, because
we keep the old libraries by version number.
Bruce
On Thu, May 17, 2012 at 10:16 AM, Owen Densmore <ow...@backspaces.net> wrote:
> Just as a point of interest, where *are* the shaders? My guess is that
> GlowScript has a set of standard shaders and separate program objects for
> standard use.
>
Reply all
Reply to author
Forward
0 new messages