opengl es textures
Johann Koenig
I'm trying to write a very simple video player. No audio, just pushing
a full screen image. It's currently working with one sample at 256x144,
but I can't get any other sizes working. The problem is any calls to
glTexImage2D with different width/height return GL_INVALID_VALUE but I
don't know why. From /android/frameworks/base/opengl/libagl/texture.cpp:
void glTexImage2D(
GLenum target, GLint level, GLint internalformat,
GLsizei width, GLsizei height, GLint border,
GLenum format, GLenum type, const GLvoid *pixels)
{
<...>
if (width<0 || height<0 || border!=0 || level < 0) {
ogles_error(c, GL_INVALID_VALUE);
return;
}
<...>
border and level are both 0, and printing out width and height before
shows that they're both greater than 0, too.
From the openGL docs online:
GL_INVALID_VALUE is generated if width or height is less than 0 or
greater than 2 + GL_MAX_TEXTURE_SIZE.
GL_INVALID_VALUE is generated if non-power-of-two textures are not
supported and the width or height cannot be represented as 2 k + 2
border for some integer value of k.
For the first case, GL_MAX_TEXTURE_SIZE is 0x0D33 (3379) and I'm only
trying to work with files up to the screen size: 480 and 320 (G1).
For the second case, the 256x144 sample works, and only one of the
dimensions is a power of 2. The dimensions are always even, and the
border is always 0, so that shouldn't be a problem either.
The code I'm working with is a mix of san-angeles
and /android/frameworks/base/opengl/tests/textures/textures.c
Very basically:
DemoActivity.java is pretty much unchanged. onDrawFrame() calls
nativeRenderer() calls appRender().
<code to generate an RGB 565 image>
width/height are from onSurfaceChanged - 480/270 for G1
int workspace[] = {0, 0, width, height};
glBindTexture(GL_TEXTURE_2D, 0);
glTexParameteriv(GL_TEXTURE_2D, GL_TEXTURE_CROP_RECT_OES, workspace);
glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexEnvx(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
glEnable(GL_TEXTURE_2D);
glColor4f(1,1,1,1);
glClear(GL_COLOR_BUFFER_BIT);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width-10,
height-10, 0, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, rgb);
glDrawTexiOES(5, 5, 0, width, height);
and for error checking there is a macro:
#define CHECK_GL_ERROR() {\
GLint err = glGetError();\
if (err != GL_NO_ERROR)\
__android_log_print(ANDROID_LOG_INFO, "SanAngeles", "opengl
error: %04x (line %d)", err, __LINE__);\ }
It's called after every openGL call, but left out here for clarity.
So the end result is a light gray screen and:
I/SanAngeles( 3544): opengl error: 0501 (line 220)
where 0x0501 corresponds to
./build/platforms/android-4/common/include/GLES/gl.h:#define
GL_INVALID_VALUE 0x0501
and line 220 refers the glTexImage2D() call.
If anyone has any insight or knows of a better way to draw frames to
the screen (in YUV format would be awesome, save a blit), I would
really appreciate hearing it.
Thanks,
--
-Johann Koenig
Robert Green
of 2. I actually would assume that it won't work normally because
both height and width should have to be a power of 2. Here's how I'd
try to do it:
For each frame draw,
get the texture-to-be-bitmap data
scale/stretch it to 256x256 or 512x512 using the fastest possible
algorithm
glTexImage2D the data (runs faster than glSubTexImage2D if you want a
full replacement)
draw it at the original dimensions but bound from 0,0 to 1,1 on the
UV, which will anamorphically draw it back to its original aspect
ratio.
That should work on all devices and should be fast enough for 30FPS
playback, so long as your scale method is quick enough.
You probably don't need linear filtering. Consider using nearest
neighbor if you run into fill-rate issues on a device.
It's normal to squish/stretch an image into a power-of-two texture and
then squish/stretch it back to fit the screen. 256 will obviously be
quicker but 512 is what you'll want for good full-screen quality.
Johann Koenig
Robert Green <rbgr...@gmail.com> wrote:
> Perhaps that sample is working because the width dimension is a power
> of 2. I actually would assume that it won't work normally because
> both height and width should have to be a power of 2. Here's how I'd
> try to do it:
>
> For each frame draw,
> get the texture-to-be-bitmap data
> scale/stretch it to 256x256 or 512x512 using the fastest possible
> algorithm
> glTexImage2D the data (runs faster than glSubTexImage2D if you want a
> full replacement)
> draw it at the original dimensions but bound from 0,0 to 1,1 on the
> UV, which will anamorphically draw it back to its original aspect
> ratio.
>
> That should work on all devices and should be fast enough for 30FPS
> playback, so long as your scale method is quick enough.
Work just got me an Nexus One to do this on. Interestingly, I still get
the 0x0501/GL_INVALID_VALUE error from glSubTexImage2D, but it puts the
image on the screen anyway. Unfortunately, it's much too slow for a
640x480 image. The decode/blit takes about 15ms (using gettimeofday)
and the glTexImage2D takes about 45ms or more. I thought it was a
reconfiguration problem, and moved all the gl* calls except
glClear/glTexImage2D/glDrawTexiOES to the init function so they'll only
be called once, but it didn't make any difference at all.
At ~60ms/frame, I'm getting about 15fps. Dropping the image size speeds
things up significantly, but I was really hoping to play full size. Is
OpenGL the best way to do
this?
Just tried a 512x480 image on the N1, and still getting the 0x0501
error. If I cut it down to 512x256, the error finally goes away. Also,
now going back to the 256x144 file, I get the error again. But it still
shows the image.
> You probably don't need linear filtering. Consider using nearest
> neighbor if you run into fill-rate issues on a device.
I think that's my problem right now (fill-rate). I took your suggestion
and changed GL_TEXTURE_(MIN|MAG)_FILTER to GL_NEAREST, but it seems
about the same. Any other techniques to speed this up?
> It's normal to squish/stretch an image into a power-of-two texture and
> then squish/stretch it back to fit the screen. 256 will obviously be
> quicker but 512 is what you'll want for good full-screen quality.
I'll make up some pre-distorted source videos to try that out. Thanks
for all the suggestions.
--
-Johann Koenig
Johann Koenig
Johann Koenig <johann...@gmail.com> wrote:
> At ~60ms/frame, I'm getting about 15fps. Dropping the image size
> speeds things up significantly, but I was really hoping to play full
> size. Is OpenGL the best way to do this?
Interestingly, it gets much much worse when I skip the decode:
static char zeros[512*256*2];
static char ones[512*256*2];
static init = 0;
static color = 1;
if (!init)
{
memset(zeros, 0x0F, 512*256*2);
memset(ones, 0xF0, 512*256*2);
init = 1;
}
glClear(GL_COLOR_BUFFER_BIT);
if(color)
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 512, 256, 0, GL_RGB,
GL_UNSIGNED_SHORT_5_6_5, zeros);
else
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 512, 256, 0, GL_RGB,
GL_UNSIGNED_SHORT_5_6_5, ones);
color = !color;
glDrawTexiOES(0, 0, 0, width, height);
That gives at least 60ms per frame, up to 100, but usually hovering
between the two. Obviously I'm doing something terribly wrong in the
OpenGL calls. What is it though? With a 640x480 decode + YUV->RGB blit
+ cropping down to 512x256 (not scaling, just cropping), it can almost
stay under 30ms/frame. Why would displaying static colors take longer?
Tried malloc'ing the zeros/ones instead, but same results. They're
8 byte aligned and glPixelStorei(GL_PACK_ALIGNMENT, 8).
--
-Johann Koenig
Robert Green
that on a variety of devices because while each dimension is a power
of two, the image itself is not and some chips won't support that. To
be totally safe, you need to use a 256x256 or a 512x512 but not a
combination of the dimensions like you're doing.
all of your lag will be in memory copying, image decoding and
uploading the texture to vram. Actually drawing it should take very
little time at all assuming you're ortho and drawing using DrawTexiOES
or just drawing a big quad.
I'm surprised that glTexImage2D is taking that long. I've never
benched it but I can't believe it's not faster on an N1. There could
be a better way to do this that I'm unaware of. I haven't messed with
a lot of the gl extensions so I don't know what else is out there.
Can you try the following two tests for me?
Benchmark glTexImage2D for a 512x256 vs a 512x512 vs 256x256. One
would think that it's a direct correlation to the amount of memory
being copied so 256x256 will be fastest and 512x512 will be slowest
but we should make sure that's the order and that 512x256 isn't
horribly slow for some odd reason.
Just upload the one texture on init and don't do it again every
frame. Make sure that with just one frame, your draw code is running
at the speed you think it should be. I expect 60FPS for what you're
doing. If you're not getting that, something is surely wrong and you
should investigate that before trying the dynamic texturing again.
Also, I totally don't understand why the static values are slower.
That makes no sense to me.
On Jan 27, 10:50 am, Johann Koenig <johann.koe...@gmail.com> wrote:
> On Wed, 27 Jan 2010 09:19:50 -0500
>
Johann Koenig
Robert Green <rbgr...@gmail.com> wrote:
> Does 512x256 perform better than 512x512 for you? You'll want to test
> that on a variety of devices because while each dimension is a power
> of two, the image itself is not and some chips won't support that. To
> be totally safe, you need to use a 256x256 or a 512x512 but not a
> combination of the dimensions like you're doing.
Ok, I'll check that. I'm currently only worried about the N1.
> all of your lag will be in memory copying, image decoding and
> uploading the texture to vram. Actually drawing it should take very
> little time at all assuming you're ortho and drawing using DrawTexiOES
> or just drawing a big quad.
Which memory copy? The YUV->RGB blit copies the image from the output
of the decoder into the buffer that gets handed to glTexImage2D and is
very very fast: it doesn't even show up at 1ms resolution for 256x144,
and is still less than 10ms for 640x480. It can be sped up, but it's
not a concern at the moment. The 120+ms to upload a 512x512 texture
(see below) is.
I'm assuming I'm ortho. Do I need to specify that? The image shows up
flat and undistorted. All I use is glDrawTexiOES
> I'm surprised that glTexImage2D is taking that long. I've never
> benched it but I can't believe it's not faster on an N1. There could
> be a better way to do this that I'm unaware of. I haven't messed with
> a lot of the gl extensions so I don't know what else is out there.
>
> Can you try the following two tests for me?
>
> Benchmark glTexImage2D for a 512x256 vs a 512x512 vs 256x256. One
> would think that it's a direct correlation to the amount of memory
> being copied so 256x256 will be fastest and 512x512 will be slowest
> but we should make sure that's the order and that 512x256 isn't
> horribly slow for some odd reason.
Attached is basic.c which is the bare minimum to get an image on the
screen (as far as I'm aware). Also attached is bench.txt, which shows
the results. Not heartening:
256x256 => mid 30s
512x256 => high 60s (with a jump to 100+?)
256x512 => high 60s
512x512 => 130-150
So they seem to scale appropriately.
> Just upload the one texture on init and don't do it again every
> frame. Make sure that with just one frame, your draw code is running
> at the speed you think it should be. I expect 60FPS for what you're
> doing. If you're not getting that, something is surely wrong and you
> should investigate that before trying the dynamic texturing again.
Which calls would run every time? Just glDrawTexiOES? or nothing at
all? When I tried this before (testing a pause fxn), the image would
flicker if I didn't make any openGL calls. It looked like it was
swapping it for the previous image and back. In order to make it
smooth, I had to re-upload the previous frame with glTexImage2D.
I guess, ideally, I'd like access to those buffers it's using so the
blit could use them for output.
If you mean the decode stuff, yeah, it's under 15ms for decode and blit
at 640x480, with appropriate gains for dropping frame size.
> Also, I totally don't understand why the static values are slower.
> That makes no sense to me.
Same here. I thought maybe the image was cached because it had just
been touched, but I got the same results memcpy'ing one buffer into
another right before the glTexImage2D call.
--
-Johann Koenig
Johann Koenig
Johann Koenig <johann...@gmail.com> wrote:
> Attached is basic.c which is the bare minimum to get an image on the
> screen (as far as I'm aware). Also attached is bench.txt, which shows
> the results. Not heartening:
> 256x256 => mid 30s
> 512x256 => high 60s (with a jump to 100+?)
> 256x512 => high 60s
> 512x512 => 130-150
>
> So they seem to scale appropriately.
Just checked out the BootAnimation code [1], and it's pretty similar to
what I'm running. This guy [2] has a full screen animation for it. It
appears to be running at a little over 20fps on the Nexus based on
adjusting the framerate in desc.txt [3] and seeing how far it gets.
The pngs in the file are 240x427 (portrait). I had checked before if
things got faster running in portrait mode, but hadn't any luck. Just
checked again (using AndroidManifest.xml to lock orientation) but the
numbers are the same.
20fps is 50ms/frame. 240x427 = 102480
That matches pretty closely with 512x256 (131072) running in the high
60s.
And both of those sizes are actually doubled, because RGB_565 is 2
bytes/pixel. Maybe using a compressed format would help. Scratch that,
don't see much support for COMPRESSED in the GLES headers.
Guess that's about the limit of the interface, unfortunately.
[1] android/frameworks/base/cmds/bootanimation/BootAnimation.cpp
[2]
http://www.droidforums.net/forum/droid-hacks/15515-dr-who-boot-animation.html
[3]
http://m.droidforums.net/forum/droid-hacks/9163-desc-txt-boot-animations-works-like.html
--
-Johann Koenig
Robert Green
worry about draw time, it can't be nearly as bad as these upload
numbers you've posted.
I'm actually surprised by these. I didn't think it would be this bad.
So what I was saying earlier was to glTexImage2D ONCE but then every
frame you still need to draw to the buffer so you'd still have
glDrawTexiOES or you'll get the flickering problem you were saying.
It's still flipping between a front and back buffer.
At 2 bytes per pixel:
256x256 = 128k
512x256 = 256k
512x512 = 512k
So if it's purely a system ram to vram copy lag, then I should think
that the times you'll see will directly reflect these numbers and that
seems to be exactly what you're reporting. I just can't believe it's
so slow to copy a half megabyte of data from system ram to vram.
That's crazy. I was expecting a few milliseconds but over 100ms seems
extreme.
I just put in some logs on one of my games and on the N1, a 512x512
texture uploads in 30ms consistently. 256x256 in 15ms and 128x128 in
7-8ms. You are testing on a device, not on an emulator, correct? I'm
uploading mine through Java just using RGB_565 bitmaps.
I believe that the N1 (Qualcomm) supports ATITC as the compressed
texture format and PowerVR chips like the Droid support PVRTC.
On Jan 28, 3:25 pm, Johann Koenig <johann.koe...@gmail.com> wrote:
> On Thu, 28 Jan 2010 11:14:33 -0500
>
> [2]http://www.droidforums.net/forum/droid-hacks/15515-dr-who-boot-animat...
>
> [3]http://m.droidforums.net/forum/droid-hacks/9163-desc-txt-boot-animati...
> --
> -Johann Koenig
Johann Koenig
Robert Green <rbgr...@gmail.com> wrote:
> I just put in some logs on one of my games and on the N1, a 512x512
> texture uploads in 30ms consistently. 256x256 in 15ms and 128x128 in
> 7-8ms. You are testing on a device, not on an emulator, correct? I'm
> uploading mine through Java just using RGB_565 bitmaps.
Strangely enough, I was getting worse times for the sample posted
than for the actual application which included the decode/blit. All of
this was running on the device because it needed neon instructions.
In any case, there was no way an OpenGL texture would work for this.
I switched to an example [1] using direct access to the surface. Being
able to blit straight into the screen buffer got the results I needed.
Of course, none of this works from the NDK, so I'll leave it at that.
Thanks for your help.
[1]
http://freepine.blogspot.com/2009/02/code-snippet-for-drawing-red-block-in-c.html
--
-Johann Koenig
guich
In begining of January, i was trying to do the same thing, and someone
told me that the time to get the texture into the vram would be just
too much.
I ended up using a java bitmap. I posted the code this week in another
thread in the Android Developers forum.
I was looking at the C code you mentioned (draw a red block in c++),
but i was unable to find the headers in ui/*. Do you know where are
these?
You said:
"Being able to blit straight into the screen buffer got the results I
needed."
Can you share the code and/or give the directions?
best regards
guich
Johann Koenig
guich <guih...@gmail.com> wrote:
> Hi Johann,
>
> In begining of January, i was trying to do the same thing, and someone
> told me that the time to get the texture into the vram would be just
> too much.
That is what I found too.
> I ended up using a java bitmap. I posted the code this week in another
> thread in the Android Developers forum.
Yeah, I saw your code. I don't think I have enough overhead to be
moving data across the jni bridge though.
> I was looking at the C code you mentioned (draw a red block in c++),
> but i was unable to find the headers in ui/*. Do you know where are
> these?
Are you looking at the SDK/NDK or the AOSP? These are not officially
available, which is why I said "none of this works from the NDK"
In AOSP you can find them in frameworks/base/include/ui
> You said:
>
> "Being able to blit straight into the screen buffer got the results I
> needed."
>
> Can you share the code and/or give the directions?
The code gets a surface: (lines 54/55)
Surface::SurfaceInfo info;
status_t err = surface->lock(&info, &dirtyRegion);
then sets the bitmap pixels to point to it: (line 66)
bitmap.setPixels(info.bits);
Looking back at how I used it, I don't know if the bitmap was even
necessary. I just end up writing straight to info.bits. The 'blit' is
the conversion from one image format (YUV) to another (RGB_565). I used
info.bits as the output buffer.
But again, this is well outside the scope of this list and is not really
useful for writing distributable packages. android-porting would
probably be more appropriate. I just wanted to provide a little closure
in case anyone finds this in the archives.
--
-Johann Koenig