canvas_ui - a proof of concept web implementation for flutter's dart:ui layer

[Benjamin Strauß]

Hi,

so, over the weekend I created this repository called canvas_ui. The goal is basically to implement flutter's dart:ui layer ontop of web/canvas while retaining a 100% compatible API. The best case scenario would be that the 'dart:ui' import inside flutter could just be changed to 'packages:canvas_ui' (configurable imports?). I know there is other stuff that needs to be done (mojo services, etc.) but the rendering part of dart:ui should be a good starting point. Most of the drawing calls can directly be mapped to canvas drawing calls. The native calls have been removed and everything currently throws UnimplementedErrors. It is not an easy task and it still is very early. This is basically just a proof of concept that I'm hacking on in my spare time, so any pull requests or ideas are welcome. :)

[Benjamin Strauß]

repo link: https://github.com/b-strauss/canvas_ui

[Eric Seidel]

I looked at this briefly a while back. To compile the framework via
JS you'll need to patch dart2js to allow defining new package: files
e.g. https://codereview.chromium.org/2098493002 but there are also
features of Dart which we use (e.g. super_mixins) which only exist in
th VM and not in dart2js (yet). I never got past that point.

Interested to hear what you learn.

> --
> You received this message because you are subscribed to the Google Groups
> "Flutter Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to flutter-dev...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Raju Bitter]

Hey Benjamin,
Interesting work. Based on my experience personal experience it’s easier to focus on doing one thing right, which means good performance on mobile - and there are two operating systems with many different Android versions already. Getting good performance out of mobile applications with very useful features like hot reloading for the most common code changes is not easy, as you can see if you look at the complexity of something like React Native.

If you want to support web/browser environment, there are immediately features of Dart which cannot be used, but which are very valuable when creating a framework like Flutter. And another big problem is how to deal with multithreaded rendering in a browser environment. Rik Arends, creator of Makepad & DreemGL (both browser frameworks rendering to WebGL in the browser) spent the last 4 years getting performance out of the WebGL in the browser. Check this presentation he did last week, he talks about how he got incredible performance out of his system: https://www.youtube.com/watch?v=tVTWdFE6-O0

Text rendering in GL world is complex, especially if you want to achieve the same quality of text rendering you have working in the browser (e.g. HarfBuzz for Chrome). If you use HTML/div for text, you’ll end up with multiple canvas instances to enable z-ordering. It’s just going to be a lot of work.

Using an existing API/library like StageXL might make things easier, since a lot of the problems have been solved there already (http://www.stagexl.org/). The StageXL folks mention  Flutter support in their roadmap:

https://pub.dartlang.org/packages/stagexl "Support for Flutter (native runtime environment for Android).”

Of course their idea is to get the StageXL API up and running on top of Flutter.

Personally I think it would be useful to have Flutter running in the browser, as long as it doesn't make the Android & iOS support less powerful.
Cheers, Raju

[rdewi...@gmail.com]

I'd like to mention that from a non-technical view, it would be very interesting to see web support in Flutter. My expection would be MANY more webdev's would come to the flutter and dart community, and therefore many things will accelerate many factors.

Not saying that it is a good idea from a technical view, I have no clue. I could life with having it in beta release for long time because of some under-performing components as speed/opengl is not important to every webapp.

Timing on more Dart support/community may be critical though ;-)

[Benjamin Strauß]

I don't plan to use a WebGLRenderingContext to implement dart:ui. WebGL has no path or text rendering API and I know from projects such as PixiJS or StageXL that this would be out of my scope.

I thought about using PixiJS or StageXL as backend, but I think the dart:ui API is (relatively) simple enough to directly implement it ontop of a CanvasRenderingContext2D.In fact, dart:ui looks very similar to canvas2d. Of course, calculating stuff like intrinsic paragraph sizes can't be mapped directly and will not be trivial, but this was also intended as a learning project. :)

[Adam Barth]

Yeah, the text metrics are going to be the trickiest part of this approach.  Looks like a interesting project.