RFC: API for screen sizes on mobile devices

[melcher....@googlemail.com]

Hi there,

I am hoping for a good idea to manage screen size on Android. The physical resolution and density can be pretty much anything (for example, the Galaxy Note 10 Plus has 6.8-inch at 3040x1440 pixel), but apps are often written for a fixed scale and can request a different pixel buffer resolution that is then scaled up by hardware at screen refresh time. 

Also, there should be an API to limit apps to landscape or portrait mode.

I was thinking about something like:

Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_ONLY);

Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_WIDTH, 1024);

Device: MOBILE, DESKTOP, ...

Screen: LANDSCAPE_ONLY, PORTRAIT_ONLY, LANDSCAPE_WIDTH/HEIGHT, PORTRAIT_WIDTH/HEIGHT, ...

This API avoids #ifdef __ANDROID__ and can be extended later without braking the ABI. Unsupported hints are simply ignored. 

Using Fl::screen_scale() is similar in that it would allow super high resolution rendering on Android, but it would use up a lot more resources (time and energy), and may slow down rendering considerably.

What do you think?

 - Matthias

[Manolo]

On Wednesday, September 2, 2020 at 10:21:58 AM UTC+2 Matthias wrote:

Hi there,

I am hoping for a good idea to manage screen size on Android. The physical resolution and density can be pretty much anything (for example, the Galaxy Note 10 Plus has 6.8-inch at 3040x1440 pixel), but apps are often written for a fixed scale and can request a different pixel buffer resolution that is then scaled up by hardware at screen refresh time. 

Does that mean "the developer writes the app for a pixel size s/he decides, and the hardware efficiently scales the

bitmap produced by the app to the true display size"?

In that case, the developer must respect the width/height ratio of the display.

Also, there should be an API to limit apps to landscape or portrait mode.

I was thinking about something like:

Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_ONLY);

Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_WIDTH, 1024);

Device: MOBILE, DESKTOP, ...

I'm not sure to understand that line.

Screen: LANDSCAPE_ONLY, PORTRAIT_ONLY, LANDSCAPE_WIDTH/HEIGHT, PORTRAIT_WIDTH/HEIGHT, ...

Is the intention to use only  WIDTH or only HEIGHT, and infer the other dimension considering the hardware display size?

Isn't an API to get the hardware width/height ratio necessary?

This API avoids #ifdef __ANDROID__ and can be extended later without breaking the ABI. Unsupported hints are simply ignored. 

Using Fl::screen_scale() is similar in that it would allow super high resolution rendering on Android, but it would use up a lot more resources (time and energy), and may slow down rendering considerably.

Does that mean "FLTK builds a bitmap the size of the display hardware, but transmits scaled units to the FLTK user. Just like

under Windows on a HighDPI screen."?

Is it sure this would be slower, since no bitmap scaling should be performed?

[Albrecht Schlosser]

On 9/2/20 10:21 AM 'melcher...' via fltk.coredev wrote:

> I am hoping for a good idea to manage screen size on Android. The
> physical resolution and density can be pretty much anything (for
> example, the Galaxy Note 10 Plus has 6.8-inch at 3040x1440 pixel), but
> apps are often written for a fixed scale and can request a different
> pixel buffer resolution that is then scaled up by hardware at screen
> refresh time.
>
> Also, there should be an API to limit apps to landscape or portrait mode.
>
> I was thinking about something like:
> Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_ONLY);
> Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_WIDTH, 1024);
>
> Device: MOBILE, DESKTOP, ...
> Screen: LANDSCAPE_ONLY, PORTRAIT_ONLY, LANDSCAPE_WIDTH/HEIGHT,
> PORTRAIT_WIDTH/HEIGHT, ...
>
> This API avoids #ifdef __ANDROID__ and can be extended later without
> braking the ABI. Unsupported hints are simply ignored.

I'm -1 on such API's as Fl::hint(int, int, int...). They are confusing
and hard to implement (need switch(..) etc.). Also, extension of the API
using other datatypes would be possible but less obvious.

I'd propose to introduce a new (static?) object type like Fl_Screen or
similar with virtual methods that can easily be extended (one method per
logical function). The methods above could then be:

Fl_Screen::format(FL_SCREEN::LANDSCAPE_ONLY);

Fl_Screen::landscape_width(1024);

or (the latter):

Fl_Screen::width(FL_SCREEN::LANDSCAPE, 1024);
Fl_Screen::width(FL_SCREEN::PORTRAIT, 600);

The device type would not be necessary because it would be "known" by
the (virtual) method implementation.

As proposed by you, such virtual methods would be ignored (i.e. not
implemented at all) on devices that don't support them. The base class
would implement a sensible default.

This is consistent with the current driver model and getter methods
would be very simple to call, supposedly only in driver related code parts.

Just a thought...

[melcher....@googlemail.com]

Manolo schrieb am Mittwoch, 2. September 2020 um 11:38:20 UTC+2:

On Wednesday, September 2, 2020 at 10:21:58 AM UTC+2 Matthias wrote:

...but apps are often written for a fixed scale and can request a different pixel buffer resolution that is then scaled up by hardware at screen refresh time. 

Does that mean "the developer writes the app for a pixel size s/he decides, and the hardware efficiently scales the

bitmap produced by the app to the true display size"?

Yes, the hardware composites the screen content from 8 or so layered hardware buffers, each one can be scaled arbitrarily.

  

In that case, the developer must respect the width/height ratio of the display.

Yes, pixels must always be square. You would only set the WIDTH *or* the HEIGHT, and the driver would calculate the rest.

Also, there should be an API to limit apps to landscape or portrait mode.

I was thinking about something like:

Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_ONLY);

Fl::hint(FL_MOBILE_DEVICES, FL_SCREEN_LANDSCAPE_WIDTH, 1024);

Device: MOBILE, DESKTOP, ...

I'm not sure to understand that line.

Sorry, too abbreviated. There could be different hints for different host types. For example:

Fl::hint(FL_MOBILE_DEVICES, FL_APP_FULLSCREEN);

Fl::hint(FL_DESKTOP_DEVICES, FL_APP_WINDOWED);

 

Using Fl::screen_scale() is similar in that it would allow super high resolution rendering on Android, but it would use up a lot more resources (time and energy), and may slow down rendering considerably.

Does that mean "FLTK builds a bitmap the size of the display hardware, but transmits scaled units to the FLTK user. Just like

under Windows on a HighDPI screen."?

Is it sure this would be slower, since no bitmap scaling should be performed?

 

Imagine old CRT displays. You could request 800x600 mode or 640x480, or whatever, and it would always fill the entire screen. By keeping Fl::screen_scale() to 1.0 and having the Android compositor scale the screen in real time, we can make life quite easy (and fast). 

[melcher....@googlemail.com]

Albrecht Schlosser schrieb am Mittwoch, 2. September 2020 um 13:27:49 UTC+2:

On 9/2/20 10:21 AM 'melcher...' via fltk.coredev wrote:
I'm -1 on such API's as Fl::hint(int, int, int...). They are confusing
and hard to implement (need switch(..) etc.). Also, extension of the API
using other datatypes would be possible but less obvious.

I'd propose to introduce a new (static?) object type like Fl_Screen or
similar with virtual methods that can easily be extended (one method per
logical function).

An Fl_Screen class array with some accessor would also solve the increasing number of Fl::screen_...() methods.

 

The methods above could then be:

Fl_Screen::format(FL_SCREEN::LANDSCAPE_ONLY);
Fl_Screen::landscape_width(1024);

Preferably inside a if (platform==Android) { ... }. On a desktop machine, those calls alone would be misleading, even if they are not implemented in the driver. But yeah, ok. I see your point.

The device type would not be necessary because it would be "known" by
the (virtual) method implementation.

(...) 

This is consistent with the current driver model and getter methods
would be very simple to call, supposedly only in driver related code parts.

OK, makes sense. I'll have to implement an automated version of this first anyway, and then I will get back to a possible API.