Saving a Canvas as a PNG or JPEG.

[Simon Lightfoot]

Hi everyone,

I'm attempting to render part of the Widget hierarchy or Canvas to an image file, say as a PNG or JPEG. I know how I can do this on Android. But of course since Flutter renders all the widgets I'm at a loss. It looks like I can use a `PaintingContext` to render a RenderBox/RenderObject to a Canvas. So really its just getting a Canvas to Image on disk. I found a SO post on the topic. https://stackoverflow.com/questions/45680088/how-to-save-a-flutter-canvas-as-a-bitmap-image but it seems like the `Image.toByteData` is still yet to be implemented. See this issue #11648.

Currently on the dart side I am doing the following.

var recorder = new ui.PictureRecorder();
var canvas = new Canvas(recorder, new Rect.fromLTWH(0.0, 0.0, 200.0, 200.0));
var painter = new TestCustomPainter(50.0, 50.0);
painter.paint(canvas, const Size(200.00, 200.00));
var pic = recorder.endRecording();
var image = pic.toImage(200, 200);
print(sprintf("%dx%d", [image.width, image.height]));

I see the output ui.Image size so its there. But I have not way from Dart to get that ui.Image saved to disk. 

I have looked at the Flutter engine and have found https://github.com/flutter/engine/blob/master/shell/common/picture_serializer.cc which looks to save a Skia SkPicture to a path. It looks like https://github.com/flutter/engine/blob/master/lib/ui/painting/picture.h native backing of the dart Picture class has a SkPicture internally. So I don't see any reason why I couldn't get a Picture from my PictureRecorder and then call the SerializePicture to save the image. Either that or find out how to implement the SkPixelSerializer/PngPixelSerializer on a SkStreamMemory and have it output a byte array. This would essentially be a `toByteData` for Picture. Not exactly the request for the outstanding 

Anyone got any tips on how I might go about this? modifying the engine to perform this action?

Thanks,

Simon

[Seth Ladd]

Thanks for reaching out!

Would https://pub.dartlang.org/packages/image be helpful? (that library is all in Dart, so there may be a better/faster way to do this via Skia APIs, not sure)

Where do you want to save the image, specifically? (local file system, media gallery, firebase cloud storage, etc...)

--
You received this message because you are subscribed to the Google Groups "Flutter Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eric Seidel]

There are contributing instructions for the engine here:

https://github.com/flutter/engine/blob/master/CONTRIBUTING.md

If I were going to do this, I'd probably start by just trying to add Image.toByteData as described in #11648.  I suspect this almost entirely boils down to an API design question.  The amount of code to add APIs to dart:ui is typically pretty small, e.g. https://github.com/flutter/engine/pull/4222

I'm not entirely sure what the right API design here is.  We've typically tried to expose Skia's APIs directly through dart:ui as much as possible, but the SkData and SkPixelSerializer abstractions here would probably be overkill.

One way to proceed would be to just hack something together in your local engine and post a PR for review.  Having a concrete proposal would likely move the issue to resolution pretty quickly.

Hope that helps?

Eric

--
You received this message because you are subscribed to the Google Groups "Flutter Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev...@googlegroups.com.

[Simon Lightfoot]

Thanks guys,

Seth, Thanks for the Image library, I'm not sure its the best direction for what I need though. 

Eric, Great advise, I feel like a bit of an idiot for not looking at the contrib guide.

I'll post back my PR to the mailing list, if I get that far! ;)

Simon

On Wednesday, 1 November 2017 22:45:23 UTC, eseidel wrote:

There are contributing instructions for the engine here:

https://github.com/flutter/engine/blob/master/CONTRIBUTING.md

If I were going to do this, I'd probably start by just trying to add Image.toByteData as described in #11648.  I suspect this almost entirely boils down to an API design question.  The amount of code to add APIs to dart:ui is typically pretty small, e.g. https://github.com/flutter/engine/pull/4222

I'm not entirely sure what the right API design here is.  We've typically tried to expose Skia's APIs directly through dart:ui as much as possible, but the SkData and SkPixelSerializer abstractions here would probably be overkill.

One way to proceed would be to just hack something together in your local engine and post a PR for review.  Having a concrete proposal would likely move the issue to resolution pretty quickly.

Hope that helps?

Eric

[Idan Aizik-Nissim]

Any news about this?

I'm also looking into saving the picture_recorder Picture.toImage result as PNG/JPG to app the folder

[Todd Volkert]

https://github.com/flutter/engine/pull/4762, currently under review, adds Image.toByteData()

--
You received this message because you are subscribed to the Google Groups "Flutter Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev+unsubscribe@googlegroups.com.

[Inas Luthfi]

Looks like this has been merged

Hopefully on flutter dart sdk, this api will be exposed soon, so we can save image to disk (for signature, paint app, ect)

[Todd Volkert]

FYI, this API landed in version `0.2.10` on the dev channel.

[Todd Volkert]

For those following along, this API is likely to change before the next beta release.  The newly added dependency on image encoding in the Flutter engine increased our release binary size by an unacceptable amount.  The final API will likely return raw, unencoded, image bytes -- and it will be up to the caller to perform the encoding (using something like package:image or a platform plugin).

-T

[Inas Luthfi]

I see, it is good to know that flutter devs are striving to make binary size as minimum as possible. Thanks Todd. Any ETA for next beta?

So, after we got unencoded image bytes then encode it using image package, is it safe to say that we can write (save) it as an image file on android and ios?

Looks like this is a good entry for Flutter cookbook. I hope there is a member of community who is much more knowledgeable than me willing to share his/her knowledge to us newbies! :)

[pix...@gmail.com]

Makes sense, the majority wins.

It's a loss for those who need to encode images.

They're left with a much slower Dart implementation, which they must run in an Isolate and copy bytes around (engine -> main thread -> isolate -> main thread).

(Still waiting on transferable ByteData).

Or implement it in native code (Still need to copy bytes around, when Skia has everything that is needed).

Isn't this a good use-case for a modular Flutter engine?

More Skia features could be exposed (opt-in image encoders, all kinds of front-ends and back-ends, ...) but stripped off when --release.

Some kind of tree-shake for the engine :P

[Todd Volkert]

On Thu, Apr 12, 2018 at 7:46 PM, Inas Luthfi <nas...@gmail.com> wrote:

I see, it is good to know that flutter devs are striving to make binary size as minimum as possible. Thanks Todd. Any ETA for next beta?

We try to release to the beta channel fairly regularly, but there's no firm date for when the next release will be.

 

So, after we got unencoded image bytes then encode it using image package, is it safe to say that we can write (save) it as an image file on android and ios?

Yep.

 

Looks like this is a good entry for Flutter cookbook. I hope there is a member of community who is much more knowledgeable than me willing to share his/her knowledge to us newbies! :)

On Friday, April 13, 2018 at 5:39:46 AM UTC+7, Todd Volkert wrote:

For those following along, this API is likely to change before the next beta release.  The newly added dependency on image encoding in the Flutter engine increased our release binary size by an unacceptable amount.  The final API will likely return raw, unencoded, image bytes -- and it will be up to the caller to perform the encoding (using something like package:image or a platform plugin).

-T

[Todd Volkert]

I agree that it's not as nice an API for those who do need to do image encoding.  But the ~90K in release binary size just wasn't worth it for the majority of apps that don't need image encoding, I'm afraid.

I'd be interested to hear your thoughts on what a more modular engine would look like from an API standpoint. If you could file an issue at https://github.com/flutter/flutter/issues/new with suggestions, that'd be great.

Thanks!

-T

--

[pix...@gmail.com]

Hi,

Although I use Dart quite a lot, I only recently started experimenting with Flutter.

I'm not familiar enough to make an informed proposal and I'm not actually in need of such a feature, so take this with a grain of salt.

I imagine a modular engine would require some kind of preprocessor annotations and compile-time list of opt-in features, that would conditionally include/exclude code from both the engine and the SDK.

I think I saw something similar being proposed but I couldn't find it, not sure if it was a Flutter or Dart issue.

For example:

[if(feature='Flutter.Image.encodeImage')] enum EncodedImageFormat {   [if(feature='Flutter.Image.encodeImage.jpeg')]   JPEG,   [if(feature='Flutter.Image.encodeImage.png')]   PNG,   [if(feature='Flutter.Image.encodeImage.webp')]   WEBP, }

class Image ... {
  [if(feature='Flutter.Image.toByteData')]
  Future<ByteData> toByteData({EncodingFormat format: const EncodingFormat.jpeg()}) { ... }
 
  [if(feature='Flutter.Image.encodeImage')]
  void encodeImage(EncodedImageFormat format, int quality, ImageEncoderCallback callback) { ... }
}

The methods would only exist if the features are explicitly enabled.

If the user tries to call a method, the Analyzer would complain that the feature X is not enabled, instead of complaining that the method does not exist.

How many kBs does toByteData add to a release build? Wouldn't be a problem anymore.

Flutter could evolve with less restrictions and maybe add OpenGL/Vulkan bindings, PDF front and back-ends, etc ...

I can imagine other approaches to a "modular" API but they would be a lot less pleasant to work with.

My 2 cts ... :P

To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev...@googlegroups.com.

[Todd Volkert]

FYI, the ability to get an encoded PNG in Image.toByteData() was added back in flutter/engine#5060.  This is now available on the "dev" channel, version v0.3.3.

Cheers,

-T

To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev+unsubscribe@googlegroups.com.

[Ged Wed]

I really like those 2 cents !!!

[Matt Carroll]

I also like that recommendation.

I was talking with Chinmay about the possibility of bringing in more SKIA capabilities.  One of the concerns was unnecessary bloat for apps that don't need the capabilities.  If we can optionally include those capabilities then Flutter could really open up the power of UI transformations and other rendering tools for those that need them and keep the cost down for those that don't.

[Ged Wed]

I guess use cases are important here, so we know what to expose....

I was annoyed about the lack on being able to get a video stream of the screen. The image access now allows me to feed that to a video encoder at least.

i also hit road block on printing. I wanted to get the screen but have it rendered at a high resolution so i could send it out to a printer. 

[Todd Volkert]

Regarding getting access to a video, I believe you want to follow https://github.com/flutter/flutter/issues/8245

[Ged Wed]

Awesome, thanks Todd. 

Speaking of use cases, i commented on that issue at the very bottom. I needed a way to mask text globally for privacy reasons for users.

I know i can just use a global variable and make the widgets respect that and mask text.

But it feels also like the sort of thing that can be told to Skia itself and it just does it. Then you dont need to make all your widgets hardcoded to this, but instead just have a global to tell skia to render all text with a mask.