What about supporting just Stage3D (which is now shipping in Flash 11)?
I agree about not supporting the legacy Flash drawing APIs.
What I can do is, after you submit your changes, I can suggest any
neccessary API changes that might allow the flash backend to render
more efficiently.
-Ray
I'm strongly in favor of this. I want to spend more time helping debug
your new canvas backend, but I got yoinked off onto a wacky side
project. I'll hopefully be able to get back to at least part-time
PlayN development soon.
> - replace the java2d rendering code with lwjgl, and
I don't have strong opinions on this front, other than a mild aversion
because it will make it impossible to deploy PlayN games as unsigned
applets. If it weren't for Minecraft's kajillion dollars, I would
write applets off completely. I suppose Minecraft was a signed applet
that used LWJGL anyway, so maybe I shouldn't care.
> - unify the two (three after making the lwjgl change) opengl rendering
> backends with a single shared one.
I think this is going to be impossible due to the pervasive use of
typed arrays (WebGL), versus NIO array buffers (LWJGL and Android GL).
Maybe we can do some GWT magic here and use NIO buffers everywhere and
have those magically be rewritten as typed arrays during compilation,
but I'm wary of introducing yet another abstraction over buffers.
> Now let's add two more goals to the mix:
> - The ability to add "direct-paint" layers
I'm strongly in favor of this. I have dozens of canvas or surface
layer uses that I am dying to rewrite to render directly.
> - 3D layers (both surface-backed and direct-paint).
This seems like a giant distraction to me, but I don't want to rain on
anyone's parade.
> And for anyone (Matt?) relying on dom/css performance as a stopgap for
> supporting iOS devices, how important do you think this is for your
> projects? And do you think there is a realistic way to get acceptable
> performance from direct rendering on this platform (which I presume means
> "with canvas"), perhaps by reducing the resolution and possibly scaling up
> with an accelerated transform?
I've been doing a lot of testing of my new game on an iPhone 4 with
the DOM backend, and I find it to be painfully sluggish, and my game
is not even remotely graphically intensive. This is in Mobile Safari
as well which uses the JavaScript JIT compiler. If you actually wanted
to ship an app on iOS using the HTML5 backend, you'd have to use
something like PhoneGap and give up the JavaScript JIT and take a
great big performance hit on top of the already crappy performance.
While it would be unfortunate for things to become even more sluggish
by replacing DOM with Canvas, I don't think it's viable to ship a game
on iOS using HTML5. Either we come up with a native compilation
approach for iOS or we don't support it. Things might be decent on an
iPhone 4S or iPad 2, but it's a pretty sorry state of affairs if we
can't do super simple 2D graphics on anything except the fastest
multi-core iOS devices.
I'm strongly in favor of switching to a simpler backend where the core
requirement is the ability to do accelerated drawing/blitting into a
frame buffer. That's the right architecture for future development and
it will give us a good balance of simplicity of the backend
implementations and room to maneuver so that we can achieve good
performance across all the supported platforms.
First off, I don't really care if this feature is kept or not, frankly
it sounds more like smoke and mirrors than a feature at this point, so
I'm fine with axing it.
However, I just want to clarify one thing, the skill-set difference
between consuming a library that utilizes Flash and writing a Flash
cross compiler are massively different! I wouldn't look towards the
lack of volunteers as indicating a lack of interest. Frankly, outside
of the Google-sphere, the number of people with experience writing GWT
back-ends is like... 6. That's one of the major problems with PlayN
from the whole expecting external developers to chip in part, first
off, the Java ecosystem ( Ant... Maven... Eclipse... on it goes ) has
an exorbitant mental cost of entry to start with, but then on top of
that, you are built fairly heavily on proprietary ( open, but
proprietary ) Google tech to start with. Unlike most other open source
projects, there simply aren't too many developers, even experienced
Java developers, that can easily jump in and help.
TL;DR, a Flash back in is either going to be a Google driven feature,
and Adobe driven feature, or something you should drop completely.
Does the resurrection include improving the AS3 output as detailed in
GwtFlashTodo?
Sounds awesome. I understand that the flash backend feels like a low
priority for many here, as I assume your focus is on the html5 target,
hence flash feels a bit redundant. But for us, who make social games
with facebook and google plus as our main market, it is an important
target. Rough estimates based on our google analytics tracking on our
other games indicate that around 30% of our users right now do not
have support for canvas. Giving up 30% of our users would be a
crucially big hit to us.
I fully understand though that maintaining something like that is up
to us the users, and I would love to do it, but right now we are still
in the game logic phase, but down the line I am sure we can try to
contribute. But just as Mike points out, the initial investment to get
a grasp of how it all works is huge, and it is not my area of
expertise, but it by no means we have no interest in it. And I would
be more than happy to work off trunk of gwt, if that is what is needed
to have a working flash target, not a unreasonable requirement imo.
Would also just like to thank everyone that works hard on playn and
has gotten it to where it is now, and I will for sure contribute more
as we go more into the polish and bug fix part of our project!
/Simen
On Mon, Dec 5, 2011 at 11:34 AM, Joel Webber <j...@google.com> wrote:
> - replace the java2d rendering code with lwjgl, andI don't have strong opinions on this front, other than a mild aversion
because it will make it impossible to deploy PlayN games as unsigned
applets. If it weren't for Minecraft's kajillion dollars, I would
write applets off completely. I suppose Minecraft was a signed applet
that used LWJGL anyway, so maybe I shouldn't care.
> - unify the two (three after making the lwjgl change) opengl renderingI think this is going to be impossible due to the pervasive use of
> backends with a single shared one.
typed arrays (WebGL), versus NIO array buffers (LWJGL and Android GL).
Maybe we can do some GWT magic here and use NIO buffers everywhere and
have those magically be rewritten as typed arrays during compilation,
but I'm wary of introducing yet another abstraction over buffers.
> - 3D layers (both surface-backed and direct-paint).This seems like a giant distraction to me, but I don't want to rain on
anyone's parade.
Ooh, awesome! I'd love to see that make its way into
user/super/com/google/gwt/emul/java/nio.
This is especially poignant, given that I recently spent six hours
debugging a problem that came down to the fact that in
src/com/google/gwt/typedarrays/client/Uint8Array.java and friends, the
subarray methods:
public final native Uint8Array subarray(int offset, int length) /*-{
return this.subarray(offset, length);
}-*/;
are wrong.
The subarray methods take start+end, not offset+length. Of course,
once things are compiled into JavaScript, you get zero useful feedback
when all of your code has gone horribly wrong because your typed
arrays are truncated to zero length arrays (or weird length arrays
when length happens to be greater than offset), and JavaScript returns
NaN instead of doing something useful (like throwing an exception)
when you ask for an array element that's out of bounds. Yay for sloppy
languages, they're so user friendly.
> Yikes! The nio stuff in there was forked from the old gwt-quake code ages
> ago, and in many cases the interfaces have actually evolved over time
> (yeah, yay for sloppy languages -- at least you don't find out until the
> world collapses around your ears, long after the offending call is off the
> stack...).
>
> Stefan's done a lot of cleanup work on the quake code since then, so it
> should be in a much better state. It will probably be a while before I can
> get around to this, as I need to sort out all that Canvas[ImageLayer] stuff
> first, but once it's in a reasonable state (and we can finally deprecate
> CanvasLayer), this will be next on my list.
I'm playing around with this stuff at the moment. I looked briefly in
the gwt-quake source code and I'm wondering if it's not expensive to
create a new typed WebGl array out of a nio array in every draw call?
As I see it at the moment the reason for emulating nio was to make the
port to WebGl much easier. But is it needed in a fresh enviroment?
Instead of emulating java.nio I created a generic 'Buffer' interface
which is implemented in the target platform. Like PlayN does it for
many other things.
The 3D library that we wrote for SK
(http://github.com/threerings/clyde) uses NIO in 42 source files (out
of 728), with the following breakdown:
23 import java.nio.FloatBuffer;
19 import java.nio.ByteBuffer;
14 import java.nio.IntBuffer;
11 import java.nio.ShortBuffer;
6 import java.nio.DoubleBuffer;
4 import java.nio.LongBuffer;
4 import java.nio.CharBuffer;
2 import java.nio.ByteOrder;
1 import java.nio.charset.Charset;
1 import java.nio.channels.FileChannel;
There's essentially no use of NIO in the main game code (modulo some
Steam integration that uses a combination of NIO and native code).
So it's probably not that big a deal. It would probably take a couple
of hours to change all the uses of NIO directly over to TypedArrays or
some other TypedArray-like buffer abstraction (maybe even less with
some judicious use of Perl).
That said, something like SK is so complex that we would probably not
want the intermediating layer of PlayN and would instead just port
Clyde directly to Android or GWT+WebGL. Experimenting with the former
is on my long list of things to do. Though after seeing Chrome's
performance lately, maybe I should also experiment with the latter. :)
Hi Joel,
I've been prototyping a game and testing it a lot with the DOM backend
on an iPhone 4. I've been seeing weird freezing behavior in the
display, and I've pared things down sufficiently to determine that
it's just the DOM that's not updating, rather than PlayN freezing.
Basically update() and paint() are getting called, but no visible
changes are taking place to the DOM for varying freeze durations up to
2-3000ms. It seems to be related to receiving touch events, since the
touch events are what trigger it.
I'm not particularly motivated to debug it further, since the DOM
backend can't go away soon enough, as far as I'm concerned. And the
problem does not happen with the new Canvas backend. I did put
together a simple demonstration:
http://samskivert.com/playn/touchtest-dom/
http://samskivert.com/playn/touchtest-canvas/
To see the failure, just click the button or click around anywhere on
the page, and it will periodically freeze. You can see that the game
update loop is not frozen by enabling debug logging in iOS Safari, and
you'll see that even when it's frozen, the JavaScript log messages
that are being sent ~10 times a second are accumulating.
Anyhow, that's all just FYI. What I really want to ask is whether
you've got fixes to the Canvas backend lying around uncommitted,
because I'm going to start looking into those bugs soonish if you
haven't. Also, I am wondering if you want to restructure your canvas
patch now that you want to preserve the DOM backend. Perhaps not,
since the DOM stuff should probably get new class names rather than
the Canvas stuff, which is more rightfully called, e.g.
HtmlGraphicsCanvas.java.
I'm aiming to make my prototype available to some testers this week,
so I want to sort out this pesky Canvas issue. But I think I can just
apply your canvas patch to a branch on the Three Rings PlayN
repository, and switch our build server to use that branch for now.
That way if you totally revamp the patch before ultimately pushing it
upstream, I won't have a big messy merge to do later.
I don't actually use rotation in my prototype at the moment, so the
existing canvas patch works just fine, but I'll still try to spend
some time looking into the rotation bugs this week just to help get
that code into the mainline sooner.
On Fri, Dec 16, 2011 at 8:15 AM, Joel Webber <j...@google.com> wrote:The 3D library that we wrote for SK
> Michael might have some opinion on this -- how much easier would it be,
> e.g., to port Spiral Knights to PlayN if its eventual 3D support used the
> nio interfaces?
(http://github.com/threerings/clyde) uses NIO in 42 source files (out
of 728), with the following breakdown:
23 import java.nio.FloatBuffer;
19 import java.nio.ByteBuffer;
14 import java.nio.IntBuffer;
11 import java.nio.ShortBuffer;
6 import java.nio.DoubleBuffer;
4 import java.nio.LongBuffer;
4 import java.nio.CharBuffer;
2 import java.nio.ByteOrder;
1 import java.nio.charset.Charset;
1 import java.nio.channels.FileChannel;
There's essentially no use of NIO in the main game code (modulo some
Steam integration that uses a combination of NIO and native code).
So it's probably not that big a deal. It would probably take a couple
of hours to change all the uses of NIO directly over to TypedArrays or
some other TypedArray-like buffer abstraction (maybe even less with
some judicious use of Perl).
That said, something like SK is so complex that we would probably not
want the intermediating layer of PlayN and would instead just port
Clyde directly to Android or GWT+WebGL. Experimenting with the former
is on my long list of things to do. Though after seeing Chrome's
performance lately, maybe I should also experiment with the latter. :)
Anyhow, that's all just FYI. What I really want to ask is whether
you've got fixes to the Canvas backend lying around uncommitted,
because I'm going to start looking into those bugs soonish if you
haven't. Also, I am wondering if you want to restructure your canvas
patch now that you want to preserve the DOM backend. Perhaps not,
since the DOM stuff should probably get new class names rather than
the Canvas stuff, which is more rightfully called, e.g.
HtmlGraphicsCanvas.java.
I'm aiming to make my prototype available to some testers this week,
so I want to sort out this pesky Canvas issue. But I think I can just
apply your canvas patch to a branch on the Three Rings PlayN
repository, and switch our build server to use that branch for now.
That way if you totally revamp the patch before ultimately pushing it
upstream, I won't have a big messy merge to do later.
I don't actually use rotation in my prototype at the moment, so the
existing canvas patch works just fine, but I'll still try to spend
some time looking into the rotation bugs this week just to help get
that code into the mainline sooner.
I've been prototyping a game and testing it a lot with the DOM backend
on an iPhone 4. I've been seeing weird freezing behavior in the
display, and I've pared things down sufficiently to determine that
it's just the DOM that's not updating, rather than PlayN freezing.
Basically update() and paint() are getting called, but no visible
changes are taking place to the DOM for varying freeze durations up to
2-3000ms. It seems to be related to receiving touch events, since the
touch events are what trigger it.
On 2011-12-19, at 11:57 AM, Michael Bayne wrote:I've been prototyping a game and testing it a lot with the DOM backend
on an iPhone 4. I've been seeing weird freezing behavior in the
display, and I've pared things down sufficiently to determine that
it's just the DOM that's not updating, rather than PlayN freezing.
Basically update() and paint() are getting called, but no visible
changes are taking place to the DOM for varying freeze durations up to
2-3000ms. It seems to be related to receiving touch events, since the
touch events are what trigger it.
Two things to keep in mind when developing for iOS:1) You have to cancel all touchstart/touchmove/touchend events. If you don't, Safari triggers the smooth scrolling logic, which stalls the rendering engine until the scroll process has finished.
2) Don't mutate the DOM at all within the touch event handlers. Queue up the events and re-run them on update(). Safari's rendering process can stall anywhere from 2-5s if you do too much DOM mutation inside of a touch event.
Le 19 décembre 2011 14:49, Matt Mastracci <mat...@mastracci.com> a écrit :On 2011-12-19, at 11:57 AM, Michael Bayne wrote:I've been prototyping a game and testing it a lot with the DOM backend
on an iPhone 4. I've been seeing weird freezing behavior in the
display, and I've pared things down sufficiently to determine that
it's just the DOM that's not updating, rather than PlayN freezing.
Basically update() and paint() are getting called, but no visible
changes are taking place to the DOM for varying freeze durations up to
2-3000ms. It seems to be related to receiving touch events, since the
touch events are what trigger it.
Two things to keep in mind when developing for iOS:1) You have to cancel all touchstart/touchmove/touchend events. If you don't, Safari triggers the smooth scrolling logic, which stalls the rendering engine until the scroll process has finished.Would it be reasonable for the library to do this automatically, or would that likely screw up other interactions? Sure would be nice if we didn't have to keep giving this advice.
2) Don't mutate the DOM at all within the touch event handlers. Queue up the events and re-run them on update(). Safari's rendering process can stall anywhere from 2-5s if you do too much DOM mutation inside of a touch event.Did you ever run into this in the course of "normal" playn code? I *believe* all the regular code paths work hard not to touch dom/css during the update() phase -- all mutations should be deferred until paint(). The idea is that it should be ok to mutate layers in update() if you really want (or are just being lazy and/or not bothering with interpolation), but user-visible changes should only happen during paint().
I am doing that.
> 2) Don't mutate the DOM at all within the touch event handlers. Queue up the
> events and re-run them on update(). Safari's rendering process can stall
> anywhere from 2-5s if you do too much DOM mutation inside of a touch event.
Perhaps this is the source of the problem, though my simple test case
just redraws a single button. Rerendering that button is apparently
enough to cause the delays.
On 2011-12-19, at 12:58 PM, Joel Webber wrote:Le 19 décembre 2011 14:49, Matt Mastracci <mat...@mastracci.com> a écrit :
On 2011-12-19, at 11:57 AM, Michael Bayne wrote:I've been prototyping a game and testing it a lot with the DOM backend
on an iPhone 4. I've been seeing weird freezing behavior in the
display, and I've pared things down sufficiently to determine that
it's just the DOM that's not updating, rather than PlayN freezing.
Basically update() and paint() are getting called, but no visible
changes are taking place to the DOM for varying freeze durations up to
2-3000ms. It seems to be related to receiving touch events, since the
touch events are what trigger it.
Two things to keep in mind when developing for iOS:1) You have to cancel all touchstart/touchmove/touchend events. If you don't, Safari triggers the smooth scrolling logic, which stalls the rendering engine until the scroll process has finished.Would it be reasonable for the library to do this automatically, or would that likely screw up other interactions? Sure would be nice if we didn't have to keep giving this advice.I'd prefer to have PlayN do it automatically. If we call setPreventDefault(true) on the events returned by the touch handlers, the code will take care of this and user has the ability to undo it (with the caveat that it will probably screw everything up on Mobile Safari).
2) Don't mutate the DOM at all within the touch event handlers. Queue up the events and re-run them on update(). Safari's rendering process can stall anywhere from 2-5s if you do too much DOM mutation inside of a touch event.Did you ever run into this in the course of "normal" playn code? I *believe* all the regular code paths work hard not to touch dom/css during the update() phase -- all mutations should be deferred until paint(). The idea is that it should be ok to mutate layers in update() if you really want (or are just being lazy and/or not bothering with interpolation), but user-visible changes should only happen during paint().The HTML DOM stuff is half immediate direct-mutation, half done post-paint(). Layer children are added/removed from the DOM immediately, setting the image of a layer is immediate, setting alpha is immediate, translation/rotation/visibility are handled in a method called after paint() is done.I haven't seen any performance issues around this in update() or paint() (both of which are done in setTimeout callback on iOS). It seems like the touch event handlers are just 'magical'. I haven't had the time to dig into the problem to see if it's just the excessive DOM mutation, or if it's just taking too long to process the event-- even if it's just pure scripting/offscreen canvas manipulation-- that causes the problems.(as an aside: I have been bit by a slightly different problem as a result of this: when I add new layers to the tree as a result of a physics event, but don't position them until paint(), I sometimes get a sprite flash at (0,0) that shows the sprite in the default position)I hadn't noticed this before, but the DOM code re-computes the CSS styles after each paint() call, whether they've changed or not. It might be worth investigating on my end whether a dirty flag would make this more performant. No need to re-compute CSS styles on layers that haven't been directly mutated between paint() calls. I don't know if WebKit is smart enough to avoid re-compositing if the current transformation matrix is just re-set via CSS properties.
Le 19 décembre 2011 15:45, Matt Mastracci <mat...@mastracci.com> a écrit :I'd prefer to have PlayN do it automatically. If we call setPreventDefault(true) on the events returned by the touch handlers, the code will take care of this and user has the ability to undo it (with the caveat that it will probably screw everything up on Mobile Safari).Will it just stop you from uselessly touch-scrolling the browser window? If so, that actually sounds like a good thing. It's really disconcerting when you're able to "drag" a browser window in which dragging serves no purpose.
The HTML DOM stuff is half immediate direct-mutation, half done post-paint(). Layer children are added/removed from the DOM immediately, setting the image of a layer is immediate, setting alpha is immediate, translation/rotation/visibility are handled in a method called after paint() is done.
Damn it, you're right (side note: I hate trying to keep two scene graphs in sync; it's just nasty). If we really *had* to, we could batch up all DOM mutations, though that could get a bit ugly. Please let me know if you find any evidence (or even suspect) that the lack of batching is causing problems.
Can we not make this the default? My prototype performs much better
using the Canvas backend on iPhone/iPad Mobile Safari. Probably our
games differ in their performance characteristics, but I think it's
not a guarantee that the DOM backend is preferable on iOS. Plus the
Canvas backend is already "more correct" than the DOM backend.
Perhaps we can allow this use of DOM on iOS to be configured on HtmlPlatform?
Sorry, by "this" I mean defaulting to DOM on iOS. I'm fine with
automatically preventing default on touch events.
Oh, yeah, this is on iOS5, which has hardware accelerated Canvas. I
have an iPad 1 as well (running iOS 5), so I'll test on that when I
get home, to be sure that things perform reasonably.
I'm not averse to defaulting to DOM on iOS4, though the number of
people that ship a PlayN game for iOS4 using the HTML5 backend may
never exceed one. I certainly aim to have a native iOS solution by
hook or crook before I ship the game I'm working on now.
Are you planning to ship your game PhoneGap-style, or is it going to
be deployed via the web?
On Tue, Dec 20, 2011 at 9:31 AM, Matt Mastracci <mat...@mastracci.com> wrote:Are you running an iPad 2 and/or iOS5, or making use of surface layers? Iknow for sure that canvas runs like molasses on iPad1/iOS4 - it doesn't seemto be capable of running faster than 2-5 fps for an extremely trivial game.The same game is reasonably smooth in Canvas/iPad2/iOS5, however.
Oh, yeah, this is on iOS5, which has hardware accelerated Canvas. I
have an iPad 1 as well (running iOS 5), so I'll test on that when I
get home, to be sure that things perform reasonably.
I'm not averse to defaulting to DOM on iOS4, though the number of
people that ship a PlayN game for iOS4 using the HTML5 backend may
never exceed one. I certainly aim to have a native iOS solution by
hook or crook before I ship the game I'm working on now.
Are you planning to ship your game PhoneGap-style, or is it going to
be deployed via the web?
On 2011-12-20, at 12:57 PM, Michael Bayne wrote:On Tue, Dec 20, 2011 at 9:31 AM, Matt Mastracci <mat...@mastracci.com> wrote:
Are you running an iPad 2 and/or iOS5, or making use of surface layers? Iknow for sure that canvas runs like molasses on iPad1/iOS4 - it doesn't seem
to be capable of running faster than 2-5 fps for an extremely trivial game.The same game is reasonably smooth in Canvas/iPad2/iOS5, however.
Oh, yeah, this is on iOS5, which has hardware accelerated Canvas. I
have an iPad 1 as well (running iOS 5), so I'll test on that when I
get home, to be sure that things perform reasonably.Great! I'd be interested to know if it's "iPad 1" or "iOS4" that causes the brutal performance in canvas mode. The inability to downgrade iDevices from iOS[n] to iOS[x<n] is a big frustration for me - I can only test iPad1 in iOS4, or iPad2 in iOS5.
I'm not averse to defaulting to DOM on iOS4, though the number of
people that ship a PlayN game for iOS4 using the HTML5 backend may
never exceed one. I certainly aim to have a native iOS solution by
hook or crook before I ship the game I'm working on now.
Are you planning to ship your game PhoneGap-style, or is it going to
be deployed via the web?
Initially via the web, but (possibly) PhoneGap-style in the future. I need to dig into the JS JIT situation more before then: I can't recall whether Apple started accelerating JS on iOS4 or 5. If we're currently getting acceleration on iOS4 and we lose it by using a PhoneGap wrapper, I might not be able to go that route.