Squeak VM on iPad/IPhone gets open/GL rendering

[John M McIntosh]

On friday night I created an open/GL renderer to work with Squeak screen updates on the iPhone/iPad.

Unfortunately it runs slows than the tiled Core Animation render that I had work out earlier in the year
with help from the Apple graphics engineers.

So if anyone has a fair amount of knowledge about Open/GL ES maybe they could contact me and
we'll see if we can improve things?

--
===========================================================================
John M. McIntosh <joh...@smalltalkconsulting.com> Twitter: squeaker68882
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================

[John M McIntosh]

Spent a few more hours today reading and then decided the test case I had from Bert spent 90% of it's time in the interpreter's
copyLoop logic. Toss that for the fall back of using bouncing atoms so I could get a better feel of the performance differences.

So after some optimization, on the iPad, the CALayer pushes about 38 fps, the OpenGL code pushes 49 fps (which is limited by Morphic loop).

In checking Instruments the 10% of the time taken in CALayer for memcpy of data out of the UIImages is gone when I do open/GL
However if the drawing area becomes too big then the Open/GL drawing is slower, trade offs, trade offs.

Also the byte alignment plays a part, that I have to explore more. Still this is significant progress and we'll see what happens this week.

A quick check on an iPhone 3GS shows CALayer pushes about 27fps, OpenGL does 43 fps. So much bigger win, I think we'll keep it.

Sadly it doesn't fly on an iPod Gen 1 device (3.1.3) Zero clues.. It's possible of course there isn't isn't enough memory to go around, but
given I've two different implementation classes I can choose the viable one at run time...

[John M McIntosh]

I figured out we can't reasonably do open/GL arbitrary sized textures on the first gen iPhone or iPod Touch.
So I check for iOS 3.1.x and fall back to using the CALayer render, the wonders of polymorphism.

I also commented out some of the nifty eToys on the iPad features lurking in the VM, like rotate me to get a keyboard,
and also checked and confirmed that rotation of both types of renders with the view as a plain view or
embedded in a scrolling view worked as expected.

At this point I'm somewhat done the open/GL optimization.

The implementation is to:

Ccreate a texture and populate via glTexImage2D using apple's extension APPLE_texture_2D_limited_npot
at view surface creation time.

Then on a ioForceDisplayUpdate (implicit or explicit) we take the union of the rectangles observed
in ioShowDisplayOnWindow and then use the

glTexSubImage2D

to push the bytes one row at a time by calculating the offset into the bytes found in the Display special object,
the texture then is drawn to the screen. I note we create the full sized glTexImage2D only once at startup time.

An alternate choice was to use kEAGLDrawablePropertyRetainedBacking=YES and set the
glTexImage2D & glTexSubImage2D pair for each ioForceDisplayUpdate using the subrectangle.
But I found the Open/GL fps had a lot of jitter, so it seemed less animation friendly than creating the
glTexImage2D once and doing the glTexSubImage2D on each ioForceDisplayUpdate.

Someone mutters, GL_FRAMEBUFFER_OES but at this point someone can pay me to explore faster
alternatives.

[Steve Wart]

I guess in 2010 it's probably still not save to assume that all
platforms support OpenGL (Android?) but is it realistic to hope that
one day OpenGL could be a rendering option for other VMs too?

Downloading now...

Thank you John.