Fl_Image_Surface, alpha, caching fl_draw

[Evan Laforge]

Hi, I recently had the clever idea to accelerate drawing of text in my
app by caching the output of the fl_draw in an image. I used
Fl_Image_Surface, and it almost works... except the output has no
alpha channel. Since the text is always black, I made a copy of the
image and inferred an alpha with `0xff - avg(r, g, b)` but the results
are noticeably thinner than the native antialiased fl_draw output.
The black is still entirely opaque as it should be, but the
transparent parts are more transparent.

Is the alpha channel non-linear in some way? Or is `1 - avg(r, g, b)`
too simplistic?

And of course, is there some better way than manually copying the
image and inferring an alpha channel? When the OS drew the text in
the first place, surely it was using alpha... is there any way to get
my hands on that? I notice it uses fl_read_image(), which does have
an alpha argument... but it's just a hardcoded value. Tracking down
the OS X implementation, I wind up at a CGContextRef, and it looks
like these may or may not have alpha info. In fact,
Fl_Cocoa_Screen_Driver.cxx looks like it does actually use a depth=4,
but fl_read_image is confusing. It looks like if its alpha arg is
non-zero and read_win_rectangle returns an image with alpha, then it
leaves the alpha intact, but copies it into another Fl_RGB_Image, so
we can get the array out, so we can make it into another Fl_RGB_Image,
in Fl_Quartz_Image_Surface_Driver::image... which always passes
alpha=0.

Regardless of the alpha situation, wouldn't it be faster and simpler
for fl_read_image to directly return the Fl_RGB_Image from the
relevant screen driver, instead of making multiple copies and fiddling
with the alpha in complicated ways? I assume this arose due to
history, but maybe we could have a direct path and leave fl_read_image
for backward compatibility.

Alternately, I'm sure caching text drawing is a common thing to do...
has anyone done something like this, and can share how they did it?

I'm using fltk-1.4 on OS X.

[Ian MacArthur]

On Thursday, 3 June 2021 at 07:24:13 UTC+1 qdunkan wrote:

Hi, I recently had the clever idea to accelerate drawing of text in my
app by caching the output of the fl_draw in an image. I used
Fl_Image_Surface, and it almost works... except the output has no
alpha channel. Since the text is always black, I made a copy of the
image and inferred an alpha with `0xff - avg(r, g, b)` but the results
are noticeably thinner than the native antialiased fl_draw output.
The black is still entirely opaque as it should be, but the
transparent parts are more transparent.

Hi Evan,

Ummm... text rendering is often pretty swift, on most platforms, so I'd need to see some actual hard metrics before I bothered with caching.

That said, if you have text rendered via pango (rather than just XFT) on *nix systems, that is slower than "stock" if it has to do a lot of font substitutions to render the full text.

 

Is the alpha channel non-linear in some way? Or is `1 - avg(r, g, b)`
too simplistic?

Probably a bit too simplistic...

Is this just for a "regular window? It's not for a GL window or anything?

The text for GL windows is already cached, since there's no "GL native" mechanism for rendering strings of text. 

WIN32 has a GL text interface that works pretty well, so we do use that directly, but on both OSX/macOS and X11 we use a scheme where we render the AA text into a cache of little GL buffers and then reuse those cached text images when redrawing those strings.

This works pretty well, supports the "full" Unicode space (at least as well as the fonts I have, anyway) and seems reasonably swift in practice - so if you are in a GL window, that should already be cached and I doubt there'd be much benefit from a further level of caching there.

If this is for a non-GL context, then you could try looking at the GL caching scheme and see how the textures are constructed there, in case that is informative for what you are truing to do. 

The X11 case is probably the more generic (the OSX path uses some Apple-isms, or at least used to, I'm not certain if it still does) and at some point in the past I understood that code (since I wrote chunks of it) but it is complicated...

[Bill Spitzak]

If your image consists entirely of black text where it is transparent everywhere it is not black, then in fact the color channels would be black everywhere. What you are getting on the edges of your letters sounds like it is gray (with a partial alpha channel) and thus lighter than black, and this is why the letters look thinner.

--
You received this message because you are subscribed to the Google Groups "fltk.general" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkgeneral...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fltkgeneral/CACbaDy5H_yHhPur%3DZJUQhO-kM%2B6wzZJumBu_4vNzm%3Dnrb51oVA%40mail.gmail.com.

[Manolo]

Hi Evan,

I, too, after Ian, doubt it's possible to accelerate text drawing on macOS by caching images.

But, FLTK 1.4 contains the code to get the proper alpha channel of antialiased text, because it's used

to draw text on GL scenes.

Use this virtual member function of class Fl_Gl_Window_Driver :

  virtual char *alpha_mask_for_string(const char *str, int n, int w, int h, Fl_Fontsize fs);

where

  str is the UTF8 text to draw, n is its length

  w, h is the size of the desired image containing the drawn text

  fs is the font size (the current font is used)

This function returns a char array of size w * h containing the adequate alpha channel for the text.

The effect of this function as antialiased text is visible with any text-drawing, GL-using FLTK program (e.g., test/cube).

The macOS implementation of this virtual function, Fl_Cocoa_Gl_Window_Driver::alpha_mask_for_string,

builds a CGContextRef for a CGBitmapContext which contains a depth-4 pre-multiplied byte array

that can also be used to create an adequate depth-4 Fl_RGB_Image object.

[duncan]

Use this virtual member function of class Fl_Gl_Window_Driver :

  virtual char *alpha_mask_for_string(const char *str, int n, int w, int h, Fl_Fontsize fs);

where

  str is the UTF8 text to draw, n is its length

 . . .

I looked at this and immediately thought, "is n the size in characters, or bytes?" and

then wondered whether it would make sense, over time, to rename "str" variables as

"utf8" and "n" variables as "nbytes" so it would be more obvious and self documenting.

Just a thought

[Evan Laforge]

On Thu, Jun 3, 2021 at 1:40 AM Ian MacArthur wrote:
> Hi Evan,
>
> Ummm... text rendering is often pretty swift, on most platforms, so I'd need to see some actual hard metrics before I bothered with caching.
> That said, if you have text rendered via pango (rather than just XFT) on *nix systems, that is slower than "stock" if it has to do a lot of font substitutions to render the full text.

Sure, I've been doing this a lot lately, but I'm clumsy with this kind
of thing so mistakes are possible. And in fact, I took a second look
and I think you are correct. Text drawing takes up around 0.00002 and
0.00004 per (small) string, and though I'm calling fl_draw 1542 times
to draw at one zoom level (admittedly it's a lot), the total time is
only 0.0393, which is not much compared to the whole time of 0.7528.

So I think I wrongly accused text drawing here. I was misled by my
metrics which didn't adequately bracket it, and tempted by caching
since there are many small text fragments drawn from a small
vocabulary. Caching may still be useful, but it's clearly not the big
problem here.

Some further digging shows that most of the time is taken up by
Fl_RGB_Surface::image (0.0288), with a smaller chunk taken by
allocating the Fl_RGB_Surface in the first place (0.0152). I'm a bit
surprised that even allocation takes so long, but these wind up being
7-16mb chunks, so they're oversized... this is not for text, but the
result of an earlier attempt to speed up scrolling by caching window
contents. That attempt worked, but has clearly has side-effects at
high zoom levels where the overdraw is extreme. My original plan was
to split the scroll cache into tiles so I'm not drawing much outside
the visible area, and it looks like I'll have to do that after all.

Still, the apparent multiple copies in the Fl_Surface_Image -> driver
chain is interesting, especially if the allocation is so expensive.
I'll look into if it's really doing copies, and if so, if it would be
possible to reduce that.

On Thu, Jun 3, 2021 at 11:29 AM Bill Spitzak wrote:
>
> If your image consists entirely of black text where it is transparent everywhere it is not black, then in fact the color channels would be black everywhere. What you are getting on the edges of your letters sounds like it is gray (with a partial alpha channel) and thus lighter than black, and this is why the letters look thinner.

I'll bet that's it! Since the black text has been drawn on a white
background, it's effectively premultiplied against white, and by using
it as alpha I'm applying it twice. I set the image to all black and
used `0xff - red` and got exactly what I expected. Unfortunately I'll
probably discard this caching attempt based on the input from Ian, but
I'll keep this in mind if I decide to cache text after all.