canvaskit-wasm: drawParagraphs performance, and converting SKParagraph to SKTextBlob

71 views
Skip to first unread message

Alan Song

unread,
Mar 12, 2024, 8:03:34 PMMar 12
to skia-discuss

As discussed in a few places https://github.com/flutter/flutter/issues/66614  https://github.com/flutter/flutter/issues/58478, paragraph is both memory intensive and cpu intensive. This makes it hard to cache (memory overflow) or create on the fly (frame drops). It can be somewhat mitigated by caching surface snapshots that contain rendered results of the paragraphs, but I wonder if there’s a way do make it faster.


Also, drawParagraph is much slower compared to canvas2d fillText as seen in this benchmark https://stackblitz.com/edit/vitejs-vite-gxijha?file=index.ts where we've pre-extracted the positions of text runs and call fillText to render the same paragraph.


From what I gather, SkParagraph is designed for Flutter, https://groups.google.com/g/skia-discuss/c/fdHz3n4MJEo/m/-NKr9zkOAQAJ and isn’t used in Chromium. Chromium uses it’s own text shaping and converts glyphs into SKTextBlobs and call drawTextBlobs.


So I wonder if we can use SKParagraph for shaping and create SKTextBlobs with the shaped result, if the resulting TextBlobs aren’t too expensive to cache. Here’s my naive attempt: https://stackblitz.com/edit/vitejs-vite-bv7evn?file=index.ts


```
const textBlobs: TextBlob[] = [];
const lines = paragraph.getShapedLines();
for (const line of lines) {
  for (const { glyphs, positions } of line.runs) {
    const rsx = new Array(glyphs.length).fill(0).flatMap((_, i) => {
      const x = positions[2 * i];
      const y = positions[2 * i + 1];
      return [1, 0, x, y];
    });
    const blob = CanvasKit.TextBlob.MakeFromRSXformGlyphs(glyphs, rsx, font);
    textBlobs.push(blob);
  }
}

```

result.png

The immediate problem is that hinting is all over the place. Maybe the `positions` in GlyphRun doesn’t include hinting information? 

The other problem would be lack of `typeface` property in GlyphRun, which is necessary to know which exact fallback font was picked for the glyphs. 


I also found this ticket https://github.com/flutter/flutter/issues/81224 in flutter suggesting to move from drawParagraph to drawGlyphs, but it was closed without clarification what the alternative was. 


I’m curious whether this could be a viable approach - converting paragraphs to text blobs and caching them. SkParagraph would be used as a shaper, which is very hard to implement in js.


Alan Song

unread,
Mar 12, 2024, 8:06:55 PMMar 12
to skia-discuss
Sorry I must have brain-farted. By `hinting` I meant `kerning`.

Alan Song

unread,
Mar 21, 2024, 8:55:16 PMMar 21
to skia-discuss

Seems the correct way is to cache the paragraph as a SkPicture. It seems to achieve the goal of caching the text blobs, as drawPicture stack trace shows exactly drawTextBlob. https://stackblitz.com/edit/vitejs-vite-xvvk9q?file=memory-paragraph.ts,memory-skpicture.ts

draw-picture-of-paragraph.png

For 50k paragraphs, storing paragraphs themselves cost ~1615MB or ~33KB each. Storing pictures cost ~470MB or ~10KB each.


Unfortunately drawPicture of paragraphs are just as slow - but at least we can afford to hold on to the pictures to avoid overhead of paragraph creation.

Reply all
Reply to author
Forward
0 new messages