Help: PaintRecord over Mojo

[Nasser Al-shawwa]

I need to serialize a PaintRecord and send it from the browser process to a service process over mojo. I found several places in Chrome code that do this by converting it to SkPicture first, but it would make my code more maintainable if I can avoid this. Here's what I tried:

// Browser process
const cc::PaintOpBuffer& buffer = browser_paint_record.buffer();
size_t buffer_size = buffer.bytes_used();
std::unique_ptr<uint8_t, base::AlignedFreeDeleter> memory =
    cc::PaintOpWriter::AllocateAlignedBuffer<uint8_t>(buffer_size);
cc::SimpleBufferSerializer serializer(memory.get(), buffer_size,
                                      cc::PaintOp::SerializeOptions());
serializer.Serialize(buffer);

// memory should now contain the paint op buffer. 

// Serialize to a memory region to pass via mojo

base::MappedReadOnlyRegion region_mapping =
    base::ReadOnlySharedMemoryRegion::Create(buffer_size);
std::memcpy(region_mapping.mapping.memory(), memory.get(), buffer_size);
base::ReadOnlySharedMemoryRegion region = std::move(region_mapping.region);

// Service process
base::ReadOnlySharedMemoryMapping mapping = region.Map();
auto serialized_content = mapping.GetMemoryAsSpan<uint8_t>();
std::vector<uint8_t> scratch_buffer;
cc::PaintOpBuffer::DeserializeOptions de_options{.scratch_buffer =
                                                       scratch_buffer};
sk_sp<cc::PaintOpBuffer> out_buffer = cc::PaintOpBuffer::MakeFromMemory(
    serialized_content.data(), serialized_content.size(), de_options);
cc::PaintRecord service_paint_record = out_buffer->ReleaseAsRecord();

This code fails at `MakeFromMemory`, where out_buffer returned is null. Can someone let me know what I'm doing wrong?

Best,

Nasser

[Nasser Al-shawwa]

+Dominique Fauteux-Chapleau 

[Vladimir Levin]

Hey,

It's possible that your buffer isn't aligned correctly and this fails: https://source.chromium.org/chromium/chromium/src/+/main:cc/paint/paint_op_reader.cc;l=130;drc=5108636c70c0b08fdbeb57de2640a22e138f6685

In general, I'd debug and see which of the deserialization steps fails, which can narrow down the problem.

Note that things like images and fonts (iirc?) don't go through this path and instead use the transfer cache to move images, which may not work with this approach for your service.

+Khushal Sagar +Xianzhu Wang who may have more insights

Thanks,

Vlad

[Vladimir Levin]

Khushal points out that images are also supported for printing, so whatever path that takes should also work for you

[Xianzhu Wang]

This buffer needs to be aligned to 16 bytes to be used for deserialization with enable_security_constraints=false by default:

auto serialized_content = mapping.GetMemoryAsSpan<uint8_t>();

I'm not sure if alignment in the buffer is possible. This comment says currently enable_security_constraints needs to be false (thus paint records and images are not supported) if the caller can't control the alignment. If alignment is not possible and paint records and images are needed, we may need to add another mode (in which e.g. we copy the unaligned image data to an aligned buffer) in addition to enable_security_constraints being true and false.

[Khushal Sagar]

Is there a design doc or more context clarifying how this data will be used in the browser process? We don't allow decoding of fonts/images in a trusted process. Understanding the use-case will help us in advising better.

It looks like printing is indeed converting to an SkPicture first and then using skia hooks for fonts/images serialization here.

[Khushal Sagar]

Left a comment with some code pointers on the doc.

On Mon, Dec 2, 2024 at 3:36 PM Nasser Al-shawwa <alsh...@google.com> wrote:

The reason behind my question is because I observed that printing converts to an SkPicture first, as you noted: I am doing the same with my implementation of watermarking for print. The idea is this: I'll render the object I need onto an cc::PaintRecord-backed cc::PaintCanvas in the browser process. The font rendering is done using gfx::RenderText, which takes care of any font decoding. Then I serialize the record and play it back in the trusted process. I've updated the design in go/cbe-dpr-watermarking-print-dd based on a (gnarly) prototype I prepared and uploaded to https://crrev.com/c/6020991.

I ran into some limitations based on the cc::PaintCanvas and cc::PaintRecord interfaces: 

1. You cannot obtain a SkCanvas from cc::PaintCanvas.

2. You cannot create a cc::PaintRecord from an SkPicture. 

3. You can convert a cc::PaintRecord to an SkPicture.

So by drawing onto the PaintRecord, I can draw on both SkCanvas and PaintCanvas. The easiest thing for me to do as an interim solution is to serialize an SkPicture before passing to the trusted process and call it a day, but that would slightly complicate things (as you can see me discuss in the design). 

Regarding alignment, I'm not familiar with that level of detail yet. I'll try to debug and report back with follow up questions.

Nasser

[Nasser Al-shawwa]

The reason behind my question is because I observed that printing converts to an SkPicture first, as you noted: I am doing the same with my implementation of watermarking for print. The idea is this: I'll render the object I need onto an cc::PaintRecord-backed cc::PaintCanvas in the browser process. The font rendering is done using gfx::RenderText, which takes care of any font decoding. Then I serialize the record and play it back in the trusted process. I've updated the design in go/cbe-dpr-watermarking-print-dd based on a (gnarly) prototype I prepared and uploaded to https://crrev.com/c/6020991.

I ran into some limitations based on the cc::PaintCanvas and cc::PaintRecord interfaces: 

1. You cannot obtain a SkCanvas from cc::PaintCanvas.

2. You cannot create a cc::PaintRecord from an SkPicture. 

3. You can convert a cc::PaintRecord to an SkPicture.

So by drawing onto the PaintRecord, I can draw on both SkCanvas and PaintCanvas. The easiest thing for me to do as an interim solution is to serialize an SkPicture before passing to the trusted process and call it a day, but that would slightly complicate things (as you can see me discuss in the design). 

Regarding alignment, I'm not familiar with that level of detail yet. I'll try to debug and report back with follow up questions.

Nasser

On Mon, Dec 2, 2024 at 2:41 PM Khushal Sagar <khusha...@google.com> wrote:

[Nasser Al-shawwa]

Regarding your comments, this is great! Basically, the approach which I was implementing as an interim solution turns out to be optimal. A developer's dream come true!  

FYI, here is the CL that does what you suggest (tests have not been updated as of this e-mail so everything will appear broken, so please consider it a WIP): https://crrev.com/c/6062565. 

Best,

Nasser