FLTK + libdecor + Wayland

[Manolo]

The ideal behaviour for libdecor would be for it to be completely insensitive to
what plugin is at work to draw window titlebars. Overall, libdecor is very
close to achieving this behaviour. Noticeably, libdecor completely hides
to its user the identity of the running plugin. This strict decision
can be enforced only if libdecor fully achieves the ideal mentionned above.

In practice, FLTK has been confronted to two situations where libdecor
does not provide FLTK with information it needs and where that information depends
on the plugin identity. They are :

1) Dealing with minimum window size requirements that vary from plugin to plugin:
* the Cairo plugin, after a simple modification, accepts any window size;
* the GTK plugin does not run well with width < 134;
* in SSD mode, the KDE/Plasma desktop apparently enforces a minimum window width
of about 134 pixels.

2) Read window titlebars. libdecor provides nothing of this kind. FLTK adds code
that provides the required information in CSD mode and for the Cairo and GTK plugins.
FLTK needs to be aware of what plugin runs to provide this information.

FLTK defines 2 functions to identify whether libdecor is in CSD or SSD mode and
the identity of the running plugin, and to read titlebars. They are :
- const char *fl_get_libdecor_plugin_description() identifies the plugin as a
character string using a complicated procedure because it's well hidden by libdecor;
- unsigned char *fl_libdecor_titlebar_buffer() allows FLTK to read window titlebars
when in CSD mode. NULL is returned when in SSD mode.

These are the improvements to libdecor I suggest so FLTK could rely on libdecor
to get all needed information without having to hack the identity of what plugin runs.

===============================================

*** About minimum window size ***

In order to be able to prevent error messages for short window sizes
FLTK needs to be informed of what is the minimum accepted window size
for the loaded plugin. FLTK does that by identifying what is the loaded
plugin and using custom values adequate for each plugin. FLTK uses
fl_get_libdecor_plugin_description() to determine what is the loaded plugin.

I see two alternative ways to improve that :

1)The plugins are required not to enforce a minimum window size (i.e., accept down to 1x1)
- This would require substantial changes in libdecor-gtk.c because many warnings
are sent to stderr when width < 134 (OK for height). This is what Albrecht advocates.
- This creates a problem with SSD mode under KDE/Plasma which seems to impose
window width ⪆ 134 because libdecor has no control about what happens in SSD mode.

or

2) Each plugin has its own set of minimum window sizes. It becomes necessary
to have an API for the libdecor user to be informed of this minimum value.

Proposed API :
Arguments
        pwidth: [out] points to minimum window width supported by the plugin
        pheight: [out] points to minimum window height supported by the plugin
       
LIBDECOR_EXPORT void
libdecor_minimum_window_size(struct libdecor_frame *frame, int *pwidth, int *pheight);


===============================================


*** About the possibility to read the window titlebar as a pixel buffer ***

This is a function needed by FLTK that libdecor does not provide at all.
FLTK adds fl_libdecor_titlebar_buffer() for this feature.
FLTK needs to be aware of what plugin has been loaded to be in the position
of making fl_libdecor_titlebar_buffer() work. FLTK uses fl_get_libdecor_plugin_description()
for this purpose. Then, FLTK adds 2 functions based on data structures defined,
not in a public way, in libdecor-cairo.c and libdecor-gtk.c that provide the
pixel buffer information for each plugin.

My proposition would be to extend the libdecor API to provide this feature
(read window titlebars) in a plugin-independent way.

Proposed new API in libdecor.h
/**
Return value: NULL or start of a byte array containing pixel data
of the decorated window's titlebar. Pixel data are in one of the Wayland-defined
wl_shm_format 's.
Arguments
        frame: identifies the decorated window
        pwidth: [out] points to width of image
        pstride: [out] points to number of bytes per image row
        psize: [out] points to total number of bytes in image
        pbuf_fmt: [out] points to identification of image pixel format
*/
LIBDECOR_EXPORT const char *
libdecor_titlebar_wl_buffer(
        struct libdecor_frame *frame,
        int *pwidth, int *pstride, int *psize, enum wl_shm_format *pbuf_fmt)


Proposed implementation of this API :
[I still need to run this proposed implementation to check it's OK]


* Additional member of struct libdecor_plugin_interface :
        .frame_titlebar_wl_buffer

* In libdecor.c :

LIBDECOR_EXPORT const char *
libdecor_titlebar_wl_buffer(
        struct libdecor_frame *frame,
        int *pwidth, int *pstride, int *psize, enum wl_shm_format *pbuf_fmt)
{
        struct libdecor_frame_private *frame_priv = frame->priv;
        struct libdecor_plugin *plugin = frame_priv->context->plugin;

        if (!plugin->priv->iface->frame_titlebar_wl_buffer) return NULL;
        return plugin->priv->iface->frame_titlebar_wl_buffer(plugin, frame,
                        pwidth, pstride, psize, pbuf_fmt);
}


* Example implementation inside libdecor-cairo.c :

  * Additional assignment in initializer of cairo_plugin_iface :
        .frame_titlebar_wl_buffer = libdecor_plugin_cairo_titlebar_wl_buffer,

static const char *
libdecor_plugin_cairo_titlebar_wl_buffer (
        struct libdecor_frame *frame,
        int *pwidth,
        int *pstride,
        int *psize,
        enum wl_shm_format *pbuf_fmt)
{
  struct libdecor_frame_cairo *lfc = (struct libdecor_frame_cairo *)frame;
  struct border_component *bc = &lfc->title_bar.title;
  struct buffer *buffer = bc->server.buffer;
  if (!buffer) return NULL;
  *pwidth = buffer->buffer_width;
  *pstride = cairo_format_stride_for_width(CAIRO_FORMAT_ARGB32, buffer->buffer_width);
  *psize = buffer->buffer_height * *pstride;
  *pbuf_fmt = WL_SHM_FORMAT_ARGB8888;
  return (const char*)buffer->data;
}


* Example implementation inside libdecor-gtk.c :

  * Additional assignment in initializer of gtk_plugin_iface :
        .frame_titlebar_wl_buffer = libdecor_plugin_gtk_titlebar_wl_buffer,

static const char *
libdecor_plugin_gtk_titlebar_wl_buffer (
        struct libdecor_frame *frame,
        int *pwidth,
        int *pstride,
        int *psize,
        enum wl_shm_format *pbuf_fmt)
{
  struct libdecor_frame_gtk *lfg = (struct libdecor_frame_gtk *)frame;
  struct border_component *bc = &lfg->headerbar;
  struct buffer *buffer = bc->buffer;
  if (!buffer) return NULL;
  *pwidth = buffer->buffer_width;
  *pstride = cairo_format_stride_for_width(CAIRO_FORMAT_ARGB32, buffer->buffer_width);
  *psize = buffer->buffer_height * *pstride;
  *pbuf_fmt = WL_SHM_FORMAT_ARGB8888;
  return (const char*)buffer->data;

}

[Albrecht Schlosser]

Dear FLTK devs

I feel inclined to add some more context to Manolo's statements for other readers. We discussed the implications enforced by using libdecor on the program that uses libdecor or the GUI toolkit (FLTK) using libdecor in previous private mails.

Basically, libdecor and its plugins enforce minimum window sizes in several ways by overriding FLTK's requests. This means that you can't create a window with a size of for instance 80x80 pixels (with decorations). A statement like

  win->size_range(10, 10, 100, 100);

would effectively be overridden by libdecor and/or its plugins which enforce several *different* minimum window sizes, particularly the window width but also the window height. The latter is IMHO more than absurd.

This also affects of course window resizing by the user, i.e. a program running on X11 with the above size_range() could be resized as expected whereas the same program running under Wayland with libdecor would depend on libdecor and/or its currently loaded plugin which enforces other arbitrary minimum sizes.

We found that the Cairo plugin had more or less hard coded minimum sizes w = 134, h = 24 (IIRC).

The Gtk plugin does not have any hard coded limits (IIRC) but whenever the window width is reduced below 124 its (Gtk) drawing functions emit warnings on stderr as Manolo wrote. The "magic number" 124 results from adding all title bar widget widths (label + 3 buttons). When the label width drops below 1 (while the buttons keep their sizes) the first Gtk widget issues the warnings because the label widget has width 0 and can't be drawn.

My statement: these are bugs in libdecor and its plugins and this behavior is not acceptable.

So far I was not aware of similar restrictions in SSD mode (KDE) as Manolo mentioned.

The current *WORKAROUND* in FLTK is to determine ( in other words: *GUESS* ) the minimum width requirements of each known plugin and to set FLTK's own restrictions on minimum window widths to avoid the strange behavior (e.g. warning messages) of libdecor and its plugins.

From my point of view it's not only absurd that libdecor imposes restrictions on minimum window sizes, it's really a no-go. Nobody can know how such plugins would change over time and we can never know which libdecor plugins will be invented in the future.

You should know: libdecor's design goal is that desktop environments should provide their own plugins in the future which should help program and toolkit developers to draw the window decorations in the appropriate desktop style.

That said, I had a discussion with the Gtk plugin developer who said (in my own words) that we (FLTK) should make sure that all the decorations can be drawn by the plugin which (a) requires said minimum window widths and (b) we (FLTK) can not know in advance. No further comment on this. Anybody else?

My clear point is that the window decoration library MUST NOT impose restrictions on window sizes. Never. Even if the decorations (i.e. minimze, maximize, and close buttons) would become invisible, this is not the responsibility of the window decoration library, it's solely the responsibility of the program developer. The GUI toolkit (FLTK) should be transparent (FLTK itself *IS* transparent) but the usage of the current libdecor library would break this - and thus existing programs.

I would also say that this is not the right place to start such a discussion with all these implementation details Manolo mentioned below. I asked him for his opinion before I would start an issue on the libdecor development site.

Now that he posted his answer here, what do you FLTK devs and other readers think about this issue?

I apologize for this intentional top posting. I'm leaving Manolo's post below unchanged.