2.0 Rendering Architecture
22 views
Skip to first unread message
Alex Iskander
Sep 15, 2010, 2:58:17 PM9/15/10
to sproutc...@googlegroups.com
I spent a couple hours over the weekend writing up, with some input from Charles and Colin, an idea of how we could upgrade the SC rendering architecture. The thought is that, now that we are working on 2.0, perfect 100% backwards-compatibility is not so necessary, we can do a lot more.
Among other things, this design:
- takes renderers (renaming them Cells, as per Charles' suggestion) and makes them more usable and much more powerful.
- It does this while simplifying the architecture itself, and, I think, would potentially boost performance.
- Offers low-level control if needed.
- Largely separates SC's view layer from the DOM. This in theory allows for entire view trees to exist within canvas, etc., yet for the rendering code to stay the same (drawText, etc. use primitive cells (renderers) that are defined differently in a Canvas-based theme than a DOM-based theme).
Thoughts? Suggestions? Ideas?
Alex
Darcy Murphy
Sep 15, 2010, 9:53:57 PM9/15/10
to SproutCore Developers
I like the concept. I think that having a universal API for rendering
content is a smart idea. That said, I was wondering, what about SVG?
content is a smart idea. That said, I was wondering, what about SVG?
Alex Iskander
Sep 15, 2010, 10:00:18 PM9/15/10
to sproutc...@googlegroups.com
SVG works under a similar concept to DOM, I think. My non-DOM concepts are "straw men" to help design the API. :)
Alex
Michael Cohen
Sep 15, 2010, 11:42:56 PM9/15/10
to SproutCore Developers
The concept seems interesting. It certainly allows for improved
flexibility of how views are rendered beyond HTML. By any chance,
would you happen to have any diagrams that depict the proposed
architecture?
Regarding SVG, yes, it works like DOM where each SVG element is part
of the DOM structure and therefore acts like DOM elements. However,
there's a difference with pre-HTML5 SVG and the new HTML5 SVG.
For pre-HTML5 SVG, there are a few ways to render. One method is to
use the embed or object HTML tags to include an SVG image. That
approach isn't great for when you want to dynamically create SVG
images on the fly, like I'm sure most people would like to do in
SproutCore :-). Another approach is to have SVG directly part of the
document by representing it as a string of (X)HTML syntax that can be
interpreted by the browser's rendered. But for (X)HTML approach to
work you need to have the document's content type to be "application/
xhtml+xml" and that must come from the HTTP response or else the
browser won't recognize the SVG syntax. Commonly, web pages served up
by a server are "text/html", as is the case with sc-server. In
addition, you also have to have special meta doc headers to indicate
the use of XHTML. It's a headache. A final approach is to instead use
JavaScript to call the document object's createElementNS method to
dynamically generate SVG elements. The benefit of this is that you
don't have to do anything special to the HTTP response or document
itself since createElementNS is W3C DOM compliant. The downside is
that you can't represent the SVG and text and use innerHTML to render
the SVG. And as we all know, touching the DOM is sloooow.
In HTML5, the SVG syntax is part of the normal HTML document structure
and is implicitly recognized by the browser's rendered. So no funny
XML/XHTML things to do. And because the SVG syntax is build into the
browser's lexicon, you can use innerHTML and get the speed
performance. This is where Microsoft has an advantage with VML in that
the VML syntax was directly recognized by the IE browser, so you could
use innerHTML.
What would this all mean for the proposed SC2.0 rendering API? Well,
for newer HTML5 compliant browsers, not much I guess since the
DOMRenderContext would fit the bill. But for pre-HTML5 browsers,
that's an open question at the moment. Something that may be of
interest is to incorporate the SproutCore Sai framework that handles
the rendering of SVG and VML and update it to work with the new SC2.0
rendering API. As of now, Sai hooks into the view's didCreateLayer and
didUpdateLayer methods in order to trigger when to create and update a
special Sai canvas that renders either SVG or VML. Sai will be updated
to work with HTML5 SVG, that way innerHTML would be used to create the
SVG elements instead of using createElementNS as is being done right
now. Perhaps if DOMRenderContext were updated to have pre- and post-
rendering hooks, it would make it easy to make the necessary updates.
In any case, that was just a quick idea that was currently on the top
of my head :-).
Mike
flexibility of how views are rendered beyond HTML. By any chance,
would you happen to have any diagrams that depict the proposed
architecture?
Regarding SVG, yes, it works like DOM where each SVG element is part
of the DOM structure and therefore acts like DOM elements. However,
there's a difference with pre-HTML5 SVG and the new HTML5 SVG.
For pre-HTML5 SVG, there are a few ways to render. One method is to
use the embed or object HTML tags to include an SVG image. That
approach isn't great for when you want to dynamically create SVG
images on the fly, like I'm sure most people would like to do in
SproutCore :-). Another approach is to have SVG directly part of the
document by representing it as a string of (X)HTML syntax that can be
interpreted by the browser's rendered. But for (X)HTML approach to
work you need to have the document's content type to be "application/
xhtml+xml" and that must come from the HTTP response or else the
browser won't recognize the SVG syntax. Commonly, web pages served up
by a server are "text/html", as is the case with sc-server. In
addition, you also have to have special meta doc headers to indicate
the use of XHTML. It's a headache. A final approach is to instead use
JavaScript to call the document object's createElementNS method to
dynamically generate SVG elements. The benefit of this is that you
don't have to do anything special to the HTTP response or document
itself since createElementNS is W3C DOM compliant. The downside is
that you can't represent the SVG and text and use innerHTML to render
the SVG. And as we all know, touching the DOM is sloooow.
In HTML5, the SVG syntax is part of the normal HTML document structure
and is implicitly recognized by the browser's rendered. So no funny
XML/XHTML things to do. And because the SVG syntax is build into the
browser's lexicon, you can use innerHTML and get the speed
performance. This is where Microsoft has an advantage with VML in that
the VML syntax was directly recognized by the IE browser, so you could
use innerHTML.
What would this all mean for the proposed SC2.0 rendering API? Well,
for newer HTML5 compliant browsers, not much I guess since the
DOMRenderContext would fit the bill. But for pre-HTML5 browsers,
that's an open question at the moment. Something that may be of
interest is to incorporate the SproutCore Sai framework that handles
the rendering of SVG and VML and update it to work with the new SC2.0
rendering API. As of now, Sai hooks into the view's didCreateLayer and
didUpdateLayer methods in order to trigger when to create and update a
special Sai canvas that renders either SVG or VML. Sai will be updated
to work with HTML5 SVG, that way innerHTML would be used to create the
SVG elements instead of using createElementNS as is being done right
now. Perhaps if DOMRenderContext were updated to have pre- and post-
rendering hooks, it would make it easy to make the necessary updates.
In any case, that was just a quick idea that was currently on the top
of my head :-).
Mike
Darcy Murphy
Sep 16, 2010, 3:15:39 PM9/16/10
to SproutCore Developers
I came across this project today that renders SC views via RaphaëlJS,
an SVG library. http://github.com/rklancer/RaphaelViews
SVG sounds like it could be a pain, but I know Firefox will eventually
support inline SVG, and IE9 supports SVG natively (probably in an
isolated fashion but I'd have to check), so it's obviously going to be
important tech in the future. I'm thinking that if SproutCore is to
have a leg up on the competition we should build towards that future.
That said, HTML is probably the sanest bet at this point, but the
upside is that the two should be fairly compatible on an abstract
level.
P.S. application/xhtml+xml was never understood by IE (though I'm
unsure about 9), which is why sites are served as text/html.
On Sep 15, 10:42 pm, Michael Cohen <michael.lee.co...@gmail.com>
wrote:
an SVG library. http://github.com/rklancer/RaphaelViews
SVG sounds like it could be a pain, but I know Firefox will eventually
support inline SVG, and IE9 supports SVG natively (probably in an
isolated fashion but I'd have to check), so it's obviously going to be
important tech in the future. I'm thinking that if SproutCore is to
have a leg up on the competition we should build towards that future.
That said, HTML is probably the sanest bet at this point, but the
upside is that the two should be fairly compatible on an abstract
level.
P.S. application/xhtml+xml was never understood by IE (though I'm
unsure about 9), which is why sites are served as text/html.
On Sep 15, 10:42 pm, Michael Cohen <michael.lee.co...@gmail.com>
wrote:
Richard Klancer
Sep 16, 2010, 3:52:13 PM9/16/10
to sproutc...@googlegroups.com
On Thu, Sep 16, 2010 at 3:15 PM, Darcy Murphy <mrdarc...@gmail.com> wrote:
> I came across this project today that renders SC views via RaphaëlJS,
> an SVG library. http://github.com/rklancer/RaphaelViews
> I came across this project today that renders SC views via RaphaëlJS,
> an SVG library. http://github.com/rklancer/RaphaelViews
Just so you know, I'm using this project for my own purposes, but
(excepting a merge commit and doc update I made after i saw this
message) I've also put its active development on the back burner while
I wait to see what happens with Sai, which Michael Cohen describes
above. As Michael described, the advantage of using
createElement/createElementNS as RaphaelJS does is that it steps
around the SVG <-> text/html issues that have held back SVG for so
long; the disadvantage is that innerHtml is so much faster.
That a decision I reached when Evin Grano 'pre-announced' Sai. Now
that he and Michael been actively developing it, I'll see if I can
help.
On the other hand, with my project (and, above all, leveraging Alex's
work on CollectionFastPath) I am able to render scatter plots that are
CollectionView subclasses and that handle mouse events without custom
code. So I was happy about that...
--Richard
Michael Cohen
Sep 16, 2010, 8:28:52 PM9/16/10
to SproutCore Developers
The RaphaelViews project is certainly interesting. Sai also leverages
RaphaelJS, but instead of using the Raphael code as is, it went
through a massive refactoring. (Check out the core_svg.js and
core_vml.js in the Sai project.) SVG and VML shapes are wrapped in an
shape element that ultimately derives from SC.Object, so you get the
advantages of binding, observing and so on for all shapes represented
in a Sai canvas. This also means that each Sai element can be part of
the responder chain when handling user actions (mouseup, mousedown,
etc).
Going back to the proposed SC2.0 API renderer, in might be worth while
modularizing the graphics support for things like circles, paths, and
so on into a separate mixin that way the interface would be
consistent. Both the DOMRenderContext and CanvasRenderContext could
have it mixed since they both ultimately allow rendering of vector
graphics in one way or another where as the NativeRenderContext would
not.
Something else to keep in mind is not just the basic rendering of
vector graphics in either SVG, VML and canvas, but also how to access
the shapes and how they respond to events. Where SVG and VML create
DOM elements that you can ultimately hook in event handlers for and
access via the core query functionality, there is no such thing for
the shapes rendered in the HTML5 canvas. The user is ultimately
responsible for making up their own functionality to detect a user
event on a specific shape and accessing those shapes. Having one
holistic, unified approach to handle accessing and event handling of
shapes would be considerably helpful to those looking to use either
the DOMRenderContext or CanvasRenderContext. The main challenge is
really creating a scene graph for shapes put into an HTML5 canvas and
having the logic in place to go through the scene graph to know what
shape is to respond to an incoming event. For SVG and VML, the browser
already does it implicitly. Food for thought, right ;-)? We've been
looking at updating Sai to also handle the canvas tag, so it might be
something worth looking into the updating SC rending API.
http://github.com/etgryphon/sai
Mike
On Sep 16, 3:52 pm, Richard Klancer <r...@pobox.com> wrote:
RaphaelJS, but instead of using the Raphael code as is, it went
through a massive refactoring. (Check out the core_svg.js and
core_vml.js in the Sai project.) SVG and VML shapes are wrapped in an
shape element that ultimately derives from SC.Object, so you get the
advantages of binding, observing and so on for all shapes represented
in a Sai canvas. This also means that each Sai element can be part of
the responder chain when handling user actions (mouseup, mousedown,
etc).
Going back to the proposed SC2.0 API renderer, in might be worth while
modularizing the graphics support for things like circles, paths, and
so on into a separate mixin that way the interface would be
consistent. Both the DOMRenderContext and CanvasRenderContext could
have it mixed since they both ultimately allow rendering of vector
graphics in one way or another where as the NativeRenderContext would
not.
Something else to keep in mind is not just the basic rendering of
vector graphics in either SVG, VML and canvas, but also how to access
the shapes and how they respond to events. Where SVG and VML create
DOM elements that you can ultimately hook in event handlers for and
access via the core query functionality, there is no such thing for
the shapes rendered in the HTML5 canvas. The user is ultimately
responsible for making up their own functionality to detect a user
event on a specific shape and accessing those shapes. Having one
holistic, unified approach to handle accessing and event handling of
shapes would be considerably helpful to those looking to use either
the DOMRenderContext or CanvasRenderContext. The main challenge is
really creating a scene graph for shapes put into an HTML5 canvas and
having the logic in place to go through the scene graph to know what
shape is to respond to an incoming event. For SVG and VML, the browser
already does it implicitly. Food for thought, right ;-)? We've been
looking at updating Sai to also handle the canvas tag, so it might be
something worth looking into the updating SC rending API.
http://github.com/etgryphon/sai
Mike
On Sep 16, 3:52 pm, Richard Klancer <r...@pobox.com> wrote:
> On Thu, Sep 16, 2010 at 3:15 PM, Darcy Murphy <mrdarcymur...@gmail.com> wrote:
> > I came across this project today that renders SC views via RaphaëlJS,
> > an SVG library.http://github.com/rklancer/RaphaelViews
> > I came across this project today that renders SC views via RaphaëlJS,
Alex Iskander
Sep 16, 2010, 10:56:05 PM9/16/10
to sproutc...@googlegroups.com
Doing it as a mixin is an interesting idea.
In my proposal, it was intended that they'd be defined in the base RenderContext, and that they'd basically call drawCell.
Alex
Michael Cohen
Sep 16, 2010, 11:51:48 PM9/16/10
to SproutCore Developers
Having the graphics functionality in the root RenderContext class is
fine so long as all derived types have support for all the graphical
elements. For those render context's that do not support a full
graphics set then those graphic methods exposed through the RC
interface are essentially no-ops. I guess it comes down to trying to
only expose functionality that is intended for actual use.
You mentioned that the render context decouples the SC view's layer
from the underlying representation, and that in theory this would
allow the entire view tree to be represented within an HTML5 canvas.
That's good from a presentation perspective, but it does not address
the behavioral aspect when trying to interact with the views in the
canvas. Again, when rendering as HTML, the view's and it various DOM
element parts can be all monitored for user actions via listeners
using standard hooks and ultimately allow for events to be propagated
to the correct responder/view. However, this is not the case for when
all the views are rendered in the canvas. The canvas DOM element is
seen as a single entity by the browser, so when events occur on the
canvas there is no target to the corresponding graphic within the
canvas. You have to create the code to determine what was actually the
target of the event through the use of x-y coordinates. Therefore, in
order to have consistent behavior when views are rendered as HTML as
when they are rendered as graphics canvas elements, the SC may have to
be enhanced to deal with event notification when using
CanvasRenderContext. This isn't even taking into account when both
DOMRenderContent and CanvasRenderContext are both used to render views
within the application.
Basically, the expansion of the render architecture is good, but it
only accounts for part of the needed architectural solution. Event
propagation would also need to be fully handled and normalized in
accordance to what rendering technique is being used.
Mike
fine so long as all derived types have support for all the graphical
elements. For those render context's that do not support a full
graphics set then those graphic methods exposed through the RC
interface are essentially no-ops. I guess it comes down to trying to
only expose functionality that is intended for actual use.
You mentioned that the render context decouples the SC view's layer
from the underlying representation, and that in theory this would
allow the entire view tree to be represented within an HTML5 canvas.
That's good from a presentation perspective, but it does not address
the behavioral aspect when trying to interact with the views in the
canvas. Again, when rendering as HTML, the view's and it various DOM
element parts can be all monitored for user actions via listeners
using standard hooks and ultimately allow for events to be propagated
to the correct responder/view. However, this is not the case for when
all the views are rendered in the canvas. The canvas DOM element is
seen as a single entity by the browser, so when events occur on the
canvas there is no target to the corresponding graphic within the
canvas. You have to create the code to determine what was actually the
target of the event through the use of x-y coordinates. Therefore, in
order to have consistent behavior when views are rendered as HTML as
when they are rendered as graphics canvas elements, the SC may have to
be enhanced to deal with event notification when using
CanvasRenderContext. This isn't even taking into account when both
DOMRenderContent and CanvasRenderContext are both used to render views
within the application.
Basically, the expansion of the render architecture is good, but it
only accounts for part of the needed architectural solution. Event
propagation would also need to be fully handled and normalized in
accordance to what rendering technique is being used.
Mike
Alex Iskander
Sep 17, 2010, 12:03:49 AM9/17/10
to sproutc...@googlegroups.com
Agreed there. This design is not so we can actually have a CanvasRenderContext and SVGRenderContext and all that. It is merely to refresh the rendering architecture. These factors are just kept in mind to ensure good design and great flexibility. If someone wants to actually implement, there will be other work to do in the responders, event handling, etc.
Alex
Reply all
Reply to author
Forward
0 new messages