Flutter Engine port on WebGL / Canvas 2D

[Benjamin Thong Chee Wai]

Hi,

Thank you all for an amazing product.

I would like to attempt a web-based Flutter Engine, seeing that dart:ui is the only dependency and that WebGL/Canvas2D seems to expose the required Skia / Text layer functionality.

However I'm lost in the depth and width of the Flutter/Engine source repository and would deeply appreciate some pointers and documentation to get me going.

Thank you.

Regards,

Benjamin

[Adam Barth]

On Wed, Nov 22, 2017 at 2:16 AM Benjamin Thong Chee Wai <benjam...@gmail.com> wrote:

Thank you all for an amazing product.

Thanks!

I would like to attempt a web-based Flutter Engine, seeing that dart:ui is the only dependency

That's correct.

 

and that WebGL/Canvas2D seems to expose the required Skia / Text layer functionality.

That's less clear, although it might be so.  In developing the dart:ui API, we've haven't been that much attention to what is exposed on the web through WebGL/Canvas2D, so it's likely we've included some features that are difficult to implement using the web APIs.  You'd need to study each feature carefully to see what its analog would be.

However I'm lost in the depth and width of the Flutter/Engine source repository and would deeply appreciate some pointers and documentation to get me going.

The dart:ui API is defined in this directory:

https://github.com/flutter/engine/tree/master/lib/ui

There are a few approaches to this project that you might consider:

1) You could try compiling the C++ Flutter Engine to WebAssembly.  In this approach, you'd get the full fidelity text engine and use WebGL as the backend.  You'd probably skip Canvas2D as the WebAssembly you'd get would know how to transform text and other 2D commands into WebGL.

2) You could write a new implementation of dart:iu and use Canvas2D to implement the https://docs.flutter.io/flutter/dart-ui/Canvas-class.html interface, including the https://docs.flutter.io/flutter/dart-ui/Canvas/drawParagraph.html interface.  Using this approach, you'd need to make sure that Canvas2D's text features are powerful enough to implement the Flutter Engine's text features.  For https://docs.flutter.io/flutter/dart-ui/SceneBuilder-class.html, you could either use Canvas2D or the DOM.  I'd probably recommend using the DOM so you can take advantage of the browser's compositor to composite the scene.

3) Finally, you could use the DOM to implement the text features of dart:ui.  The advantage of this approach is that the DOM offers a more featureful text engine than Canvas2D.  However, you'll run into some difficulty measuring text accurately with high performance because your text measurement will be tied into the browser's rendering pipeline.  There's also more trickiness involved when the Flutter app interleaves text operations with other 2D operations (especially https://docs.flutter.io/flutter/dart-ui/Canvas/saveLayer.html) because you'll need to use Canvas2D to implement some of the 2D operations.

My rough guess is that (2) is the most promising approach.  The most difficult thing to do well will be accessibility, but https://wicg.github.io/aom/spec/ might help with that once it's widely available.

Hope that helps,

Adam

[jaap.ve...@gmail.com]

You might want to take a look at this project:

https://github.com/jban332/flur

[Adam Barth]

Wow, that's pretty cool.

Adam

--
You received this message because you are subscribed to the Google Groups "Flutter Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benjamin Thong Chee Wai]

Thank you for the pointers Adam, really appreciate it.

How about the lower level stuff like runApp, scheduling a frame/widget, the render loop, etc.

I have some game development background and I assume it is very much like a game scene-graph that gets drawn in a render loop?

Could you perhaps point out where the low level entry point is into Flutter, where everything gets initialised and kicks off the render loop?

Also the flow (or parallel flows) of the main components and the source paths of each component?

I've seen numerous Flutter presentations but it's all very high level and conceptual while the code base seems like a beast to the uninitiated like myself 😇. 

Thank you again for your patience 🤓

[Benjamin Thong Chee Wai]

Ah I missed the earlier post on https://github.com/jban332/flur as I was responding via Email.

Thank you @Jaap! Is that your repo?

Will dig into it. Thank you both for your help!

[Adam Barth]

On Wed, Nov 22, 2017 at 7:29 PM Benjamin Thong Chee Wai <benjam...@gmail.com> wrote:

Thank you for the pointers Adam, really appreciate it.

How about the lower level stuff like runApp, scheduling a frame/widget, the render loop, etc.

These arrive at https://docs.flutter.io/flutter/dart-ui/Window/scheduleFrame.html, which calls back via https://docs.flutter.io/flutter/dart-ui/Window/onBeginFrame.html when the engine determines that it's time to produce another frame.  It should map fairly directly to https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame .

I have some game development background and I assume it is very much like a game scene-graph that gets drawn in a render loop?

Correct.  The main difference is that the render loop in games tend to run fully throttle, but UI toolkits typically throttle back under load.  The scheduleFrame/onBeginFrame mechanism provides that throttle, just as requestAnimationFrame does on the web.

Could you perhaps point out where the low level entry point is into Flutter, where everything gets initialised and kicks off the render loop?

The entry points into Flutter are defined in this file:

https://github.com/flutter/engine/blob/master/lib/ui/hooks.dart

There isn't a loop per se.  It's an asynchronous process wherein Flutter requests to produce a another frame with scheduleFrame and the engine gives the green light sometime later via onBeginFrame.

The starting point for that process is just main.  Typically, main calls runApp, which eventually calls scheduleFrame and returns quickly and waits for onBeginFrame.

 

Also the flow (or parallel flows) of the main components and the source paths of each component?

That's a complex question.  https://docs.flutter.io/flutter/rendering/PipelineOwner-class.html is a good place to start exploring the docs.

I've seen numerous Flutter presentations but it's all very high level and conceptual while the code base seems like a beast to the uninitiated like myself 😇.

Take a look at the various classes with "binding" or "pipeline" in their name.  They're the glue between the framework and the engine.

Thank you again for your patience 🤓

Good luck and enjoy!

Adam