FLTK on Android in progress

[Georg Potthast]

I am working on getting FLTK to run on Android devices. This is still in early stages but I was able to display an FLTK window on an Android device already.

 

Here is a screenshot of the Android emulator running an FLTK program:  https://sourceforge.net/p/fltk-dos/wiki/FLTK%20on%20Android%20screenshot/

 

This project is based on the Android NDK (native development kit) and the JNI interface. This allows to compile C/C++ code to libraries that can be called from an Android JAVA program. So you make a tiny JAVA program and a bigger C/C++ library with your code.

 

My project is based on the Allegro graphics library which can be used with Android. I cross-compiled Microwindows/NXlib for Android and wrote screen, mouse and keyboard drivers so the Allegro graphics library can be used as a backend for NXlib. Microwindows had old Allegro drivers so I started from those. Then I patched FLTK to work with NXlib and cross-compiled it for Android. After that I could compile FLTK programs for Android. These get compiled to shared libraries which link FLTK, NXlib and Allegro and are called from a tiny JAVA program.

 

There is still a lot to do to, truetype fonts support, optimize rendering speed,  etc., etc. After I get the main issues fixed I will post the code on the Microwindows Github site together with an application note.

Georg

[Ian MacArthur]

Hi Georg,

That’s pretty cool - though quite a mixture of parts; Nxlib + Allegro + FLTK to get there.
Impressed that it works!



But isn’t that always the way; You wait ages for an Android port, then two come along at once.

(For those who aren’t aware, the thread "[fltk.coredev] Porting to new OS” over in fltk-dev is about reworking the fltk back end to ease implementing an Android-native back-end to FLTK.)

--
Ian

[Melcher Matthias]

> On Jan 29, 2016, at 6:24 PM, Georg Potthast <dos...@googlemail.com> wrote:
>
>
> I am working on getting FLTK to run on Android devices. This is still in early stages but I was able to display an FLTK window on an Android device already.
>
> Here is a screenshot of the Android emulator running an FLTK program: https://sourceforge.net/p/fltk-dos/wiki/FLTK%20on%20Android%20screenshot/

Hi Georg,

This is great news.

As Ian already mentioned, we are currently rearranging the graphics and window manager concept to allow easy porting of FLTK to any platform, first and foremost Android. The core library will depend on very few graphics calls to provide a crude interface. The more parts of the driver are implemented, the faster and better FLTK will run.

At this point, we can create widgets on OpenGL windows. Translated to Android, we could relatively easily port FLTK right on top of JNI Native context, either through Android calls or OpenGL ES calls, keeping FLTK tiny, yet blazing fast.

Feel free to take a look at what is going on in the core.dev group. Maybe you can join in at some point and help to interface the build system to JNI?

- Matthias

[Greg Ercolano]

On 01/29/16 09:24, Georg Potthast wrote:
> I am working on getting FLTK to run on Android devices. [..]

> Here is a screenshot of the Android emulator running an FLTK program:
> https://sourceforge.net/p/fltk-dos/wiki/FLTK%20on%20Android%20screenshot/

Wow!!

So can one actually make + release android apps in C/C++ now?

Last I read up on the subject, there was a mandate (from Google?)
that java be used for all apps, so that an app can work on all
Android hardware with a single app file.

Compiling C/C++ programs meant compiling for specific cpu
targets, which mean several executables would have to be
made to support the entire Android hardware universe.

I recall there was a strong demand though from developers
for the ability to write time critical stuff in C/C++,
which meant the door was maybe opening for apps completely
written in C/C++.

I would think the Android dev environment could cross compile
a C/C++ app for 'all known android hardware', and make this
into some kind of "package" so that app stores could easily
give end users the correct native binary for their particular
hardware..? Or maybe there's some other way Google is coming
up with, like maybe a bytecode format + virtual machine
for C/C++ apps?

Just thinking aloud; curious what the direction is for
Android app development in C/C++.

[Georg Potthast]

Greg,

the JNI (Java native interface) was developed to allow Java applications to access libraries written in C/C++. JNI has been part of Java for a long time.

So, as I mentioned, you have to compile the FLTK program to a shared library and not to an elf executable. Then you need a "stub" Java program that calls your FLTK program which is compiled to a shared library. Once the FLTK shared library "program" is called it works (within limits) like a normal FLTK program. In addition you link libfltk.a etc. to this project.

Georg

[Georg Potthast]

I had started with this Android project before the thread mentioned by Ian and Mathias started. Currently I would like to "finish" it before starting with anything else.

[Greg Ercolano]

On 01/29/16 10:09, Georg Potthast wrote:
> the JNI (Java native interface) was developed to allow Java applications to access libraries written in C/C++. JNI has been part of Java for a long time.

Ah..

> So, as I mentioned, you have to compile the FLTK program to a shared library and not to an elf executable. Then you need a "stub" Java program that calls your FLTK program which is compiled to a shared library. Once the FLTK shared library "program" is called it works (within limits) like a normal FLTK program. In addition you link libfltk.a etc. to this project.

Neat -- thanks for the clarification.

So I take it the "shared library" version of the fltk app would have a good chance
of running on most Android hardware then?

I ask because even shared libs, as I understand them, still contain cpu-specific machine code.
But perhaps cpu variations aren't the issue on Android hardware as much as hardware variations
that shared libs can solve, e.g. graphics/audio/etc.

Anyway, very glad you're doing this.. good luck, keep us posted on progress..!

[Georg Potthast]

There are different toolchains available for the different supported CPUs. So far I have only compiled for ARM.
This worked on my available ARM devices. But it would need to be compiled with a different toolchain if the target
is e.g. x86_64.

One thing I still have to explore are the different screen sizes of the various devices and how to take care of that
in the JNI library.

[Melcher Matthias]

> On Jan 29, 2016, at 7:32 PM, Greg Ercolano <erco_...@seriss.com> wrote:
> So I take it the "shared library" version of the fltk app would have a good chance
> of running on most Android hardware then?
>
> I ask because even shared libs, as I understand them, still contain cpu-specific machine code.

The Android NDK (Native Developer Kit) does provide a pure native app interface now for a year or so. Under the hood, it still does exactly what Georg says, but reduces the amount of developer work a little bit.

Android apps are distributed in .apk files. These are archives that contain several XML files that describe the app, one or more compiled Java classes, and possibly a bunch of shared libraries. You can add the same library multiple times, but compiled for different CPUs. The OS will make sure that the right library is loaded at run-time.

The Java Native Engine (JNI) can request libraries at run time. A certain syntax in the "C" interface is used to semi-automatically create stubs between Java VM and native code. To a certain degree, you can jump back between Java and native code.

For the longest time, certain functionality (i.e. Sound) was only available through the Java interface. It's getting better though slowly. Pressure from game developers to get native access to everything has been great.

[Georg Potthast]

 To a certain degree, you can jump back between Java and native code.

I used that to bring up the Android keyboard from the native code and remove it again.
 

For the longest time, certain functionality (i.e. Sound) was only available through the Java interface. It's getting better though slowly. Pressure from game developers to get native access to everything has been great.

Allegro does support sound on Android. You can cross-compile libogg, libvorbis or lib FLAC to play sound on Android with Allegro.

[Melcher Matthias]

> On Jan 29, 2016, at 7:13 PM, Georg Potthast <dos...@googlemail.com> wrote:
>
>
> I had started with this Android project before the thread mentioned by Ian and Mathias started. Currently I would like to "finish" it before starting with anything else.
>

I just looked around on the GitHub page a bit more. That is quite amazing work.

I did not mean to ask that you stop your work. To the contrary. The FLTK core driver system will not be written over night, and in it's simplest for, it will not have such a detailed window system as on other platforms.

I am just happy to see that we are broadening the list of target devices. ;-)

[Georg Potthast]

Am Freitag, 29. Januar 2016 20:42:19 UTC+1 schrieb Matthias Melcher:

I am just happy to see that we are broadening the list of target devices. ;-)

Allegro does support the iPhone too. I do not have a Mac to compile applications for the iPhone, but I think you could port FLTK to the iPhone with Microwindows/Allegro as well.

Georg

[Cyantree Guo]

Great job!

I port fltk to ios and it's almost release. Just for reference only. Link: https://github.com/cyantreeguo/minifltk-1.3

在 2016年1月30日星期六 UTC+8上午3:58:05，Georg Potthast写道：

[Rob McDonald]

I'm really excited to see all this activity porting FLTK to mobile platforms.  Thanks much for all that work.

How are these efforts handling touch and gestures?  Does the OS perform the gesture interpretation -- then sending pinch and rotate events?  Or are you implementing a gesture layer in FLTK, or in your own app?

It has been pretty frustrating to see the fragmentation of approaches to touch and gestures along with the intentional incompatibility policies from the major players.  Didn't we learn the value of portability decades ago?  There is great opportunity here for FLTK (and others) to provide a unifying layer for touch and gestures across desktop and mobile.

Rob

[Georg Potthast]

I use touch events as mouse events. A tab on the screen is a left button click. If you touch the display and move your finger you move the mouse.

[Rob McDonald]

On Monday, February 8, 2016 at 12:27:47 PM UTC-8, Georg Potthast wrote:

I use touch events as mouse events. A tab on the screen is a left button click. If you touch the display and move your finger you move the mouse.

Sure, that is a good start, and most touch devices have long passed those events through as-if they came from a mouse.

However, can you handle pinch-zoom?  Or two-finger rotate?  Two- or three-figure drags?  Variable pressure contact or stylus? 

These are the kinds of things that we're all becoming used to on our phones and tablets, but where desktop software is being left behind.

Rob

[Georg Potthast]

I could extend the supported touch events, but currently FLTK just supports mouse input as far as I am aware.

[Albrecht Schlosser]

On 09.02.2016 11:36 Georg Potthast wrote:

> I could extend the supported touch events, but currently FLTK just
> supports mouse input as far as I am aware.

FYI: Recently Matt added a new event FL_ZOOM_GESTURE = 26, but IIRC only
for Mac OS X. I don't know if or how that works though.

[Georg Potthast]

X11 does have these touch events: XI_TouchBegin, XI_TouchUpdate and XI_TouchEnd. Maybe one could add these to FLTK.

[MacArthur, Ian (Finmeccanica, UK)]

> X11 does have these touch events: XI_TouchBegin, XI_TouchUpdate and
> XI_TouchEnd. Maybe one could add these to FLTK.


> Am Dienstag, 9. Februar 2016 11:54:09 UTC+1 schrieb Albrecht Schlosser:

At the app level, "high level" abstractions of the touch interactions, like FL_ZOOM_GESTURE, seem like they would be a "better" place to be than lower-level events (XI_TouchBegin, XI_TouchUpdate and XI_TouchEnd)

So, does anybody know what "touch events" each host provides?

Is there any easy congruence we can map to? (I.e. do they all provide a "zoom" event, or do we need to track multiple events and decide for ourselves whether it is a zoom or etc.?)




Selex ES Ltd
Registered Office: Sigma House, Christopher Martin Road, Basildon, Essex SS14 3EL
A company registered in England & Wales. Company no. 02426132
********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

[Georg Potthast]

Here is an overview what gestures Android supports:

https://www.google.com/design/spec/patterns/gestures.html#

As you can see a gesture with two fingers cannot be constructed by FLTK from XI_TouchBegin, XI_TouchUpdate and XI_TouchEnd.

Since FLTK relies on X11 I think support for these three events would do it for now. However, I will use mouse emulation so I do not rely on these.

[Ian MacArthur]

On Wed Feb 10 2016 17:37:45, 'Georg Potthast' via fltk.general wrote:

Here is an overview what gestures Android supports:

https://www.google.com/design/spec/patterns/gestures.html#

As you can see a gesture with two fingers cannot be constructed by FLTK from XI_TouchBegin, XI_TouchUpdate and XI_TouchEnd.

You say “cannot” here, but did you mean “can”?

So far as I understand it (which is not very much) each XI_Touch* message has a “details” parameter that is different for each finger, or mouse, or whatever in your system, so you can process multiple XI_Touch streams at the same time.

Since FLTK relies on X11 I think support for these three events would do it for now. However, I will use mouse emulation so I do not rely on these.

For Win32 the equivalent might be... https://software.intel.com/en-us/articles/handling-touch-input-in-windows-8-applications

[Ian MacArthur]

On Wed Feb 10 2016 22:26:12, Ian MacArthur wrote:

For Win32 the equivalent might be... https://software.intel.com/en-us/articles/handling-touch-input-in-windows-8-applications

Or it may be much more (unnecessarily) complicated than that...

That initial link mainly is about the WM_POINTER API, which seems to be the “new” way since Win8. Or maybe I’m wrong.

It seems there are also WM_GESTURE and WM_TOUCH API’s inherited from earlier iterations of Windows, that provide similar, overlapping functionality in similar but different ways.

And which may be deprecated. 

But the “preferred” way may not work on “older" systems (Win7, not exactly older for my money...)

So... multi-touch on WinXX anyone?