Gesture progress for 1.4?

[Rob McDonald]

Is anyone working towards gesture & multitouch support for FLTK 1.4 or beyond?

My understanding is that part of the difficulty is that OSs do not detect and trigger a uniform set of gesture events.  Likewise, processing multi-touch events into gestures in-application is problematic.  So little progress is made.

Most of what I've seen in this area is relevant to phones and tablets (iOS and Android), where FLTK is still mostly focused on the desktop.

So, I know it is hard -- but the FLTK team keeps surprising -- and the benefits are pretty obvious.  So, I figure it is worth an ask...  Is anyone working on this?

Rob

[Albrecht Schlosser]

On 10/6/21 12:11 AM Rob McDonald wrote:
> Is anyone working towards gesture & multitouch support for FLTK 1.4 or
> beyond?
>
> My understanding is that part of the difficulty is that OSs do not

> detect and trigger a uniform set of gesture events.  ...

>
> So, I know it is hard -- but the FLTK team keeps surprising -- and the
> benefits are pretty obvious.  So, I figure it is worth an ask...  Is
> anyone working on this?
>

I think there's something already available for macOS but I don't know
how complete / working this code is.

From FL/Enumerations.H:

  /** The user has made a zoom/pinch/magnification gesture (Mac OS
platform only).
      The Fl::event_dy() method can be used to find magnification amount,
      Fl::event_x() and Fl::event_y() are set as well.
      This event is sent to the window's handle method.
     */
  FL_ZOOM_GESTURE       = 26,


I'm personally not working on this and I'm too busy with other stuff to
start yet another "project". I don't know of any other dev working on
touch and gesture events.

[Rob McDonald]

I think there's something already available for macOS but I don't know
how complete / working this code is.

  FL_ZOOM_GESTURE       = 26,

Thanks for pointing that out, I was unaware of it.

It works for me.

 I'd encourage whoever implemented this to continue to support other MacOS supported gestures.  Hopefully in time it could also grow to include any Windows and Linux gestures.

 

I'm personally not working on this and I'm too busy with other stuff to
start yet another "project". I don't know of any other dev working on
touch and gesture events.

Understood.  Thanks,

Rob

 

[Rob McDonald]

Thanks to Albrecht's direction, I parroted this commit by Matthias Melcher to add support for a couple other MacOS gestures, two-finger rotate and two-finger scroll.  I can add three finger swype, but my application does not need it, so I'll hold off to see how this is received first.

Changes pushed up here:

https://github.com/ramcdona/fltk/tree/MacOS_Gestures

This was done in a monkey-see monkey-do way -- I do not claim to have any understanding of Apple's API's.  I have never used Objective-C and will continue to make that claim after doing this work.  I followed Matthias' example and read this bit of Apple documentation as well as a few other bits of their terrible API docs...

In the end, it works for my application and is pretty cool.  It would be great if other OS support could be similarly added.

The rotate gesture is pretty straightforward.  Apple returns the rotation angle in floating point degrees.  I followed Matthias' example and multiplied by 1000.0 and returned it the dy integer.

I also added handling for two-finger scroll on the trackpad.  Apple actually fires the same event for this as for a mouse scroll wheel -- but with tiny magnitude dx,dy such that it seems like nothing is actually happening.  In my application (involving 3D manipulation), I wanted to have different behavior for the scroll wheel (zoom) as for two-finger scroll (pan).  So, I needed to differentiate between these different kinds of scrolls.

I eventually found NSEventSubtype which is equal to NSEventSubtypeMouseEvent when the event originates from a mouse rather than other hardware.  There seemed to be other similar things to check for slightly different API versions.  I don't know if we want to add a bunch of conditionals to know which variable to test.

With this change, FLTK will issue separate FL_MOUSEWHEEL and FL_SCROLL_GESTURE events depending on this value.  Existing applications should be OK with this because the dx,dy returned by two-finger scroll via FL_MOUSEWHEEL is probably getting ignored (unless you do some really angry scrolling in the y-direction only).

I wasn't sure if the float s = Fl::screen_driver()->scale(0); scale value needs to be applied to the scroll gesture.  I'm not sure where it comes from and how I would get it to have different values for testing on my machine.

I also wasn't sure if and where to add to unit tests, test applications, documentation, etc.

Finally, as this adds some new event types, I am not sure how to add protections to existing applications that still need to be able to compile against older versions of FLTK that do not have these defined.

Rob

[Rob McDonald]

This

https://docs.microsoft.com/en-us/windows/win32/wintouch/windows-touch-gestures-overview

Looks like the documentation to support this kind of thing on Windows.  I don't have any Windows touch hardware to develop this.

Rob

[Albrecht Schlosser]

On 10/9/21 7:22 AM Rob McDonald wrote:

Thanks to Albrecht's direction, I parroted this commit by Matthias Melcher to add support for a couple other MacOS gestures, two-finger rotate and two-finger scroll.  I can add three finger swype, but my application does not need it, so I'll hold off to see how this is received first.

Changes pushed up here:

https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Thanks, I took a quick look ... looks good, thank you very much!

I still need to test it though.

This was done in a monkey-see monkey-do way -- I do not claim to have any understanding of Apple's API's.  I have never used Objective-C and will continue to make that claim after doing this work.  I followed Matthias' example and read this bit of Apple documentation as well as a few other bits of their terrible API docs...

Yes, as a macOS / Objective-C beginner I find these docs terrible as well. If you know the basics it might be better to handle though.

In the end, it works for my application and is pretty cool.  It would be great if other OS support could be similarly added.

I took a look into the Windows documentation you posted in another message. I couldn't resist and tried what I could get. Took me (way too many) hours but I'm now at a point where I get the gesture messages and I can interpret the data partly - not yet finished though.

The tricky part will be to integrate it with the FLTK event system (for now I'm only logging the messages) and to create the *correct* FLTK event data to have identical events and associated data on all platforms. Example:

WM_MOUSEHWHEEL message (526), delta = -5
WM_MOUSEHWHEEL message (526), delta = -2
Gesture GID_BEGIN        (1): flags=00, args=0, pos=(1398, 280)
Gesture GID_ROTATE       (5): flags=01, args=39413, pos=(1334, 525)
    rotation angle =    1.27 (  73.01 degrees)
Gesture GID_ROTATE       (5): flags=00, args=32186, pos=(1334, 525)
    rotation angle =   -0.11 (  -6.39 degrees)
Gesture GID_ROTATE       (5): flags=00, args=32164, pos=(1334, 525)
    rotation angle =   -0.12 (  -6.63 degrees)
...
Gesture GID_ROTATE       (5): flags=00, args=34298, pos=(1327, 526)
    rotation angle =    0.29 (  16.81 degrees)
Gesture GID_ROTATE       (5): flags=04, args=34298, pos=(1327, 526)
    rotation angle =    0.29 (  16.81 degrees)
Gesture GID_END          (2): flags=00, args=0, pos=(1328, 406)
Gesture GID_BEGIN        (1): flags=00, args=0, pos=(1196, 321)
Gesture GID_PAN          (4): flags=01, args=143, pos=(1190, 394)
Gesture GID_PAN          (4): flags=00, args=142, pos=(1286, 394)
Gesture GID_PAN          (4): flags=00, args=142, pos=(1300, 394)
...
Gesture GID_PAN          (4): flags=00, args=142, pos=(1410, 394)
Gesture GID_PAN          (4): flags=04, args=142, pos=(1410, 394)
Gesture GID_END          (2): flags=00, args=0, pos=(1410, 399)
...
Gesture GID_BEGIN        (1): flags=00, args=0, pos=(1447, 427)
Gesture GID_ZOOM         (3): flags=01, args=81, pos=(1414, 437)
Gesture GID_ZOOM         (3): flags=00, args=174, pos=(1445, 439)
WM_MOUSEWHEEL message (522), delta = 120
Gesture GID_ZOOM         (3): flags=00, args=179, pos=(1443, 439)
Gesture GID_ZOOM         (3): flags=00, args=186, pos=(1439, 439)
Gesture GID_ZOOM         (3): flags=00, args=191, pos=(1437, 440)
Gesture GID_ZOOM         (3): flags=00, args=197, pos=(1434, 440)
Gesture GID_ZOOM         (3): flags=00, args=202, pos=(1431, 440)
Gesture GID_ZOOM         (3): flags=00, args=208, pos=(1428, 441)
WM_MOUSEWHEEL message (522), delta = 120
Gesture GID_ZOOM         (3): flags=00, args=212, pos=(1426, 441)
Gesture GID_ZOOM         (3): flags=00, args=218, pos=(1423, 442)
Gesture GID_ZOOM         (3): flags=00, args=222, pos=(1421, 442)
...
Gesture GID_ZOOM         (3): flags=04, args=269, pos=(1397, 452)
Gesture GID_END          (2): flags=00, args=0, pos=(1397, 452)
Gesture GID_BEGIN        (1): flags=00, args=0, pos=(1562, 437)
Gesture GID_ZOOM         (3): flags=01, args=321, pos=(1402, 471)
Gesture GID_ZOOM         (3): flags=00, args=273, pos=(1382, 481)
...

I thought I might get these events from my notebook's touchpad but this didn't work. However, the touch*screen* worked fine (see logs above).

Note that the two WM_MOUSEHWHEEL messages at the beginning of the log are really that. However, I don't know where exactly the other ewo interspersed WM_MOUSEHWHEEL messages came from. I think that's a Windows "feature", some gestures are sent as normal mousewheel events for backwards compatibility (more tests required).

The rotate gesture is pretty straightforward.  Apple returns the rotation angle in floating point degrees.  I followed Matthias' example and multiplied by 1000.0 and returned it the dy integer.

Of course the rotation angle must have the same units and orientation on all platforms etc..

How is the "rotation angle" on macOS defined? Is it relative (per event) or is it some kind of absolute angle? Clockwise or counter-clockwise? My observations on Windows seem to indicate that it's the absolute angle of the two fingers, 0° = "north", increasing counter-clockwise (more tests needed, I didn't find it in the docs - docs are AFAICT incomplete and confusing).

BTW: after reading the macOS docs I believe that the zoom value issued on macOS by FLTK's FL_ZOOM_GESTURE event is not "correct" (whatever "correct" means here). The docs say you need to add 1 to the "magnification" value (you can still multiply with 1000). But what exactly is that value?

Some questions:
(1) What zoom values do you see on your macOS system? Do they make sense?
(2) What hardware did you use?
(3) I have a macBook Air with touch-/trackpad (no external device). Does it work with this config?

I also added handling for two-finger scroll on the trackpad.  Apple actually fires the same event for this as for a mouse scroll wheel -- but with tiny magnitude dx,dy such that it seems like nothing is actually happening.  In my application (involving 3D manipulation), I wanted to have different behavior for the scroll wheel (zoom) as for two-finger scroll (pan).  So, I needed to differentiate between these different kinds of scrolls.

I eventually found NSEventSubtype which is equal to NSEventSubtypeMouseEvent when the event originates from a mouse rather than other hardware.  There seemed to be other similar things to check for slightly different API versions.  I don't know if we want to add a bunch of conditionals to know which variable to test.

With this change, FLTK will issue separate FL_MOUSEWHEEL and FL_SCROLL_GESTURE events depending on this value.  Existing applications should be OK with this because the dx,dy returned by two-finger scroll via FL_MOUSEWHEEL is probably getting ignored (unless you do some really angry scrolling in the y-direction only).

I think here we have an issue with backwards compatibility. If I read your statements correctly (but this may be not the case) then "splitting" in different event types may be an issue. IIRC I used my built-in (macBook Air) touchpad with two-finger scrolling as a replacement for scrolling with the mousewheel - because I don't use a mouse with my macBook. If this would be "converted" to FL_SCROLL_GESTURE then it could break existing apps that rely on FL_MOUSEWHEEL simulation. But, again, this needs testing...

I wasn't sure if the float s = Fl::screen_driver()->scale(0); scale value needs to be applied to the scroll gesture.  I'm not sure where it comes from and how I would get it to have different values for testing on my machine.

You can define the env variable FLTK_SCALING_FACTOR=1.5 or similar before starting your program. Then all your FLTK programs should start zoomed by 150% and you could use cmd/+/- on a Mac to change scaling (ctrl/+/- on other systems).

I also wasn't sure if and where to add to unit tests, test applications, documentation, etc.

Eventually we'll need that. A test program (test/gestures.cxx ?) would be helpful if we could demonstrate the zoom, pan, scroll, and rotate gestures in one cross-platform program.

Finally, as this adds some new event types, I am not sure how to add protections to existing applications that still need to be able to compile against older versions of FLTK that do not have these defined.

That's not an issue. Additional events are ignored by existing programs. But changing the semantics like changing a working event to another one (which would be ignored) is problematic (see my notes above).

The other thing we do really need is the fl_eventnames array which you did already update in your patch. Thanks.

PS: it's been a long day, I'll be offline shortly, don't expect any quick replies today (it's ~11pm (Sat.) here).

[Albrecht Schlosser]

On 10/9/21 10:58 PM Albrecht Schlosser wrote:

On 10/9/21 7:22 AM Rob McDonald wrote:

Thanks to Albrecht's direction, I parroted this commit by Matthias Melcher to add support for a couple other MacOS gestures, two-finger rotate and two-finger scroll.  I can add three finger swype, but my application does not need it, so I'll hold off to see how this is received first.

Changes pushed up here:

https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Thanks, I took a quick look ... looks good, thank you very much!

I still need to test it though.

Unfortunately I can't test it. My MacBook Air doesn't generate gesture events on the built-in touchpad (trackpad), I don't have an external tablet device for testing, and the MacBook Air doesn't have a touchscreen. :-(

I took a look into the Windows documentation you posted in another message. I couldn't resist and tried what I could get. Took me (way too many) hours but I'm now at a point where I get the gesture messages and I can interpret the data partly - not yet finished though.

The tricky part will be to integrate it with the FLTK event system (for now I'm only logging the messages) and to create the *correct* FLTK event data to have identical events and associated data on all platforms. ...

My present test code can be found here:

https://github.com/Albrecht-S/fltk/tree/windows-gestures

Unfortunately I can't test this on my Linux system with my Virtualbox Windows VM because it (Virtualbox) doesn't seem to propagate touch events to the Windows VM. The only way I have now for further tests is to reboot my Linux/Window dual-boot system into Windows - which I try to avoid as much as possible.

The next part would be to create a suitable cross-platform FLTK event system that is able to provide the user with compatible events on all platforms.

I don't know yet how Linux/Unix provides multi-touch events to the user program, i.e. FLTK.

[Albrecht Schlosser]

Progress report:

On 10/10/21 3:17 PM Albrecht Schlosser wrote:

On 10/9/21 10:58 PM Albrecht Schlosser wrote:

On 10/9/21 7:22 AM Rob McDonald wrote:

Thanks to Albrecht's direction, I parroted this commit by Matthias Melcher to add support for a couple other MacOS gestures, two-finger rotate and two-finger scroll.  I can add three finger swype, but my application does not need it, so I'll hold off to see how this is received first.

Changes pushed up here:

https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Thanks, I took a quick look ... looks good, thank you very much!

I still need to test it though.

Unfortunately I can't test it. My MacBook Air doesn't generate gesture events on the built-in touchpad (trackpad), I don't have an external tablet device for testing, and the MacBook Air doesn't have a touchscreen. :-(

Sorry, that statement above was an error. Testing with the built-in touchpad works fine. :-(

I can now see the events and proceed...

One issue I noticed so far is as I wrote before: using the touchpad with two-finger scroll gestures does no longer scroll in the FLTK test apps with the proposed macOS patch. Mouse wheel spinning does still scroll fine. This is because the touchpad gestures send the event "FL_SCROLL_GESTURE" which is unknown/ignored rather than sending "FL_MOUSEWHEEL" which would do the scrolling in, for instance, Fl_Text_Display in test/editor.

This is something we need to find a solution for (backwards compatibility).

The next part would be to create a suitable cross-platform FLTK event system that is able to provide the user with compatible events on all platforms.

To clarify the above statement: before we can develop cross-platform events for multi-touch gestures we need to investigate what the different platforms offer. AFAICT the Windows and macOS platforms are slightly different (Linux not yet investigated).

We need a specification that can be used for all platforms. As I wrote before, the events, types, variables and values (units, magnitude, etc.) should be as close as possible on all supported platforms.

So far I can imagine two generally different approaches:

(1) We define only one FL_MULTI_TOUCH[_GESTURE] (?) event (like Microsoft does) and send different event types (pan, zoom, rotate, ...) together with some corresponding variables (maybe a struct). The user code needs to handle only one additional event but needs to dispatch the different types.

(2) We define one FLTK event per multi-touch event type. This is like the early "FL_ZOOM_GESTURE" event and the proposed patch has been done with additional events: "FL_ZOOM_GESTURE", "FL_ROTATE_GESTURE", "FL_SCROLL_GESTURE", ... and maybe more in the future.

Currently I tend to prefer (1), but I'd like to read about others' opinions.

I don't know yet how Linux/Unix provides multi-touch events to the user program, i.e. FLTK.

This is under investigation but links to documentation (or even code snippets) would be appreciated.

I'll keep you updated.

[Ian MacArthur]

On 10 Oct 2021, at 19:07, Albrecht Schlosser wrote:
>

>> I don't know yet how Linux/Unix provides multi-touch events to the user program, i.e. FLTK.
>
> This is under investigation but links to documentation (or even code snippets) would be appreciated.
>

This is not something I know about - I did (briefly) look at how X11 was proposing to handle multi-touch (which was the precursor to gesture events, in effect) but that was a long time ago and I never got anywhere useful.

Also, just to cheer everybody up, my understanding (so may well be wrong!) is that X11 and Wayland take fundamentally different approaches to this so we probably need to handle both at some point.
(My understanding, such as it is, is that X11 strives to maintain backwards compatibility with legacy pointer events, but Wayland has gone all-in on a new touch event interface.)

For X11, there’s an Xinput Extension for it, which I have not read... um, might be: https://www.x.org/releases/X11R7.7/doc/inputproto/XI2proto.txt or something of that ilk.

No idea what Wayland does.

[Rob McDonald]

On Saturday, October 9, 2021 at 1:58:37 PM UTC-7 Albrecht Schlosser wrote:

Thanks, I took a quick look ... looks good, thank you very much!

I still need to test it though.

Thanks for checking it out.

 

I took a look into the Windows documentation you posted in another message. I couldn't resist and tried what I could get. Took me (way too many) hours but I'm now at a point where I get the gesture messages and I can interpret the data partly - not yet finished though.

This looks like a great start to me.   

The tricky part will be to integrate it with the FLTK event system (for now I'm only logging the messages) and to create the *correct* FLTK event data to have identical events and associated data on all platforms. Example:

Agreed, we want to make all platforms behave as much the same as possible to a FLTK developer / user. 

Of course the rotation angle must have the same units and orientation on all platforms etc..

How is the "rotation angle" on macOS defined? Is it relative (per event) or is it some kind of absolute angle? Clockwise or counter-clockwise? My observations on Windows seem to indicate that it's the absolute angle of the two fingers, 0° = "north", increasing counter-clockwise (more tests needed, I didn't find it in the docs - docs are AFAICT incomplete and confusing).

I believe the angle on MacOS is always a delta.  I'll have to dig back in to verify the sign convention.

BTW: after reading the macOS docs I believe that the zoom value issued on macOS by FLTK's FL_ZOOM_GESTURE event is not "correct" (whatever "correct" means here). The docs say you need to add 1 to the "magnification" value (you can still multiply with 1000). But what exactly is that value?

The pinch zoom also gave deltas 

Some questions:
(1) What zoom values do you see on your macOS system? Do they make sense?

I'll go back and set up a test program to look into this.  I needed some sign flips to make zoom work right on my machine -- but I'm feeding it into an OpenGL zoom parameter that may be counter intuitive.

 

(2) What hardware did you use?

I'm testing with my touchpad on my 15" 2017 MacBook Pro.

 

(3) I have a macBook Air with touch-/trackpad (no external device). Does it work with this config?

I would expect it should work with that.

 

I think here we have an issue with backwards compatibility. If I read your statements correctly (but this may be not the case) then "splitting" in different event types may be an issue. IIRC I used my built-in (macBook Air) touchpad with two-finger scrolling as a replacement for scrolling with the mousewheel - because I don't use a mouse with my macBook. If this would be "converted" to FL_SCROLL_GESTURE then it could break existing apps that rely on FL_MOUSEWHEEL simulation. But, again, this needs testing...

I agree, this will need to be addressed for compatibility.

In my (poor) testing, I was not seeing two finger scroll events coming through as useful mousescroll events (I must have been wrong).  So, I didn't think we were losing anything.

I think it is valuable to be able to have an application specify that it wants to treat a scroll gesture differently from a mouse wheel scroll -- while keeping the default the same for compatibility.  Are there examples in FLTK of specifying this kind of behavior?

You can define the env variable FLTK_SCALING_FACTOR=1.5 or similar before starting your program. Then all your FLTK programs should start zoomed by 150% and you could use cmd/+/- on a Mac to change scaling (ctrl/+/- on other systems).

OK, I wasn't sure it was _that_ scale factor.

On a screen-based touch system, it makes sense for the events to be somehow in 'pixels' -- but I have no idea what the scale is for a trackpad on a Mac.

Ideally, in a 3D manipulation program, if you use two-finger scroll for Pan, you would want the object to track with your fingers -- like you were physically moving it.  No matter the zoom level, etc. 

Do we have a 3D OpenGL test program?  It might provide a good example to make a test case out of.

We could also do a 2D version -- pan, zoom, and rotate some simple objects on a screen.  Perhaps an image manipulation, or a simple map interface.

Finally, as this adds some new event types, I am not sure how to add protections to existing applications that still need to be able to compile against older versions of FLTK that do not have these defined.

That's not an issue. Additional events are ignored by existing programs. But changing the semantics like changing a working event to another one (which would be ignored) is problematic (see my notes above).

I meant for the situation where my open source application is sometimes compiled by end users on old versions of FLTK bundled with Linux distributions.   I'd like for these cases to still build and work - even if they lose this functionality.

I suppose a simple

#ifndef FL_ROTATE_GESTURE
#define FL_ROTATE_GESTURE=28
#endif

somewhere in my application will work.

[Rob McDonald]

On Sunday, October 10, 2021 at 11:07:43 AM UTC-7 Albrecht Schlosser wrote:

Progress report:

Unfortunately I can't test it. My MacBook Air doesn't generate gesture events on the built-in touchpad (trackpad), I don't have an external tablet device for testing, and the MacBook Air doesn't have a touchscreen. :-(

Sorry, that statement above was an error. Testing with the built-in touchpad works fine. :-(

I can now see the events and proceed...

Great! 

 

One issue I noticed so far is as I wrote before: using the touchpad with two-finger scroll gestures does no longer scroll in the FLTK test apps with the proposed macOS patch. Mouse wheel spinning does still scroll fine. This is because the touchpad gestures send the event "FL_SCROLL_GESTURE" which is unknown/ignored rather than sending "FL_MOUSEWHEEL" which would do the scrolling in, for instance, Fl_Text_Display in test/editor.

This is something we need to find a solution for (backwards compatibility).

Agreed. 

To clarify the above statement: before we can develop cross-platform events for multi-touch gestures we need to investigate what the different platforms offer. AFAICT the Windows and macOS platforms are slightly different (Linux not yet investigated).

We need a specification that can be used for all platforms. As I wrote before, the events, types, variables and values (units, magnitude, etc.) should be as close as possible on all supported platforms.

So far I can imagine two generally different approaches:

(1) We define only one FL_MULTI_TOUCH[_GESTURE] (?) event (like Microsoft does) and send different event types (pan, zoom, rotate, ...) together with some corresponding variables (maybe a struct). The user code needs to handle only one additional event but needs to dispatch the different types.

(2) We define one FLTK event per multi-touch event type. This is like the early "FL_ZOOM_GESTURE" event and the proposed patch has been done with additional events: "FL_ZOOM_GESTURE", "FL_ROTATE_GESTURE", "FL_SCROLL_GESTURE", ... and maybe more in the future.

Currently I tend to prefer (1), but I'd like to read about others' opinions.

I don't see much of a reason to prefer one over the other.

If we do (2), then we'll still have to bundle enough information to let the application determine what kind of app.  I see this weakly analogous to PUSH/DRAG/RELEASE separate from BUTTON 1,2,3 etc.  However, there we are clearly building out a matrix of capabilities -- each event can be triggered for each button.  In this case, swype, scroll, zoom, pinch, etc. seem like pretty separate events to me.

Is there an expense to having many events?

This is under investigation but links to documentation (or even code snippets) would be appreciated.

I'll keep you updated.

For X11, It looks like there are a variety of user-space tools that do gesture recognition and then pass them on to the window manager or application.  At the surface, I do not see a clear winner.

For Wayland, it seems that they use libinput https://wayland.freedesktop.org/libinput/doc/latest/gestures.html# -- but it proudly says that Wayland does not provide touch screen gestures, just touch pads.  Seems like an artificial wall to me...

Rob

 

[Bill Spitzak]

I certainly think the events should continue to produce scroll events so that scrolling in existing applications still work. You can add a new api if there is a need for an application to distinguish what produced the scrolling event.

--
You received this message because you are subscribed to the Google Groups "fltk.coredev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkcoredev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fltkcoredev/63300AF9-10DD-4DE4-B598-B049CCBB3F70%40gmail.com.

[Albrecht Schlosser]

Thanks for the pointer. This appears to provide a touch API but I'm not
yet done reading all the documetation. Hard stuff.

> No idea what Wayland does.

Wayland seems to use libinput but they say this should only be used
inside the Wayland compositor. The client "should not know" about this.
I did not yet investigate how Wayland would provide the client with
gestures. After all I'm not sure if it's feasible to implement the
multi-touch stuff for X11.

I'm now concentrating on a suitable API using the FLTK event system.
I've also checked libinput to see how they're doing it. Once the API is
clear (spec and documentation) the implementation on Windows and macOS
seems straight-forward. After that let's see how X11 comes into play...

[Albrecht Schlosser]

On 10/11/21 6:14 PM Bill Spitzak wrote:
> I certainly think the events should continue to produce scroll events
> so that scrolling in existing applications still work. You can add a
> new api if there is a need for an application to distinguish what
> produced the scrolling event.

I agree. Backwards compatibility is the most important fact for this
change. I'm still investigating the options.

Windows and macOS appear to be straight-forward and relatively simple
because they provide a "gesture" API, i.e. the OS analyzes the raw touch
events and provides the application with the result of a gesture (pan,
zoom, etc.).

I still don't know how to use X11 though, I didn't find such a "gesture
API" yet, but I'm still investigating. I wouldn't like to analyze raw
touch events to construct the relevant gestures. This seems to be hard
to do and I don't want to reinvent the wheel.

An X11 example program that uses multi-touch gestures would be very helpful.

[Albrecht Schlosser]

On 10/11/21 4:28 AM Rob McDonald wrote:

On Saturday, October 9, 2021 at 1:58:37 PM UTC-7 Albrecht Schlosser wrote:

Thanks, I took a quick look ... looks good, thank you very much!

I still need to test it though.

Thanks for checking it out.

Welcome.

I took a look into the Windows documentation you posted in another message. I couldn't resist and tried what I could get. Took me (way too many) hours but I'm now at a point where I get the gesture messages and I can interpret the data partly - not yet finished though.

This looks like a great start to me.

Yes, once the event recognition works and you know what kind of info the OS provides with the system events the rest is straight-forward.

The tricky part will be to integrate it with the FLTK event system (for now I'm only logging the messages) and to create the *correct* FLTK event data to have identical events and associated data on all platforms. Example:

Agreed, we want to make all platforms behave as much the same as possible to a FLTK developer / user.

Thanks for all your input, this is very helpful. I'm only commenting on parts of it but I considered the rest as well.

BTW: after reading the macOS docs I believe that the zoom value issued on macOS by FLTK's FL_ZOOM_GESTURE event is not "correct" (whatever "correct" means here). The docs say you need to add 1 to the "magnification" value (you can still multiply with 1000). But what exactly is that value?

The pinch zoom also gave deltas 

Some questions:
(1) What zoom values do you see on your macOS system? Do they make sense?

I'll go back and set up a test program to look into this.  I needed some sign flips to make zoom work right on my machine -- but I'm feeding it into an OpenGL zoom parameter that may be counter intuitive.

If you are already doing this, could you please post the source code here? Otherwise please drop me a note and I'll do it myself (I'd like to avoid duplicated work).

Note that I modified the zoom value in my test environment in two ways:
 - add 1.0 to the raw zoom value as described in the macOS documentation (why could they not do this???)
 - multiply zoom and rotation by 100,000 rather than 1,000 for better granularity (this will be obsolete with the "new" event variables)

Adding 1.0 to the raw zoom value is essential and a bug fix (the 1.3 implementation got it wrong).

In the future I'll add new float/double Fl::event_* variables for gestures so the user doesn't need to "convert" the Fl::e_dy() value to something usable.

Please try my modified code ( https://github.com/Albrecht-S/fltk/commits/MacOS_Gestures ) and let me know what you think about it. See also the commit message about other changes (I also fixed indenting partly which I didn't document).

I think it is valuable to be able to have an application specify that it wants to treat a scroll gesture differently from a mouse wheel scroll -- while keeping the default the same for compatibility.  Are there examples in FLTK of specifying this kind of behavior?

FLTK has the Fl::option() interface but this is something I'd like to avoid here.

I'm thinking of some new Fl::event_* variables we'd better use for these new events, maybe a struct that can hold all necessary information. This could have a flags member with such info. Or something totally different, maybe. Still thinking...

Ideally, in a 3D manipulation program, if you use two-finger scroll for Pan, you would want the object to track with your fingers -- like you were physically moving it.  No matter the zoom level, etc. 

Do we have a 3D OpenGL test program?  It might provide a good example to make a test case out of.

Let's first think of a basic FLTK (2D) test program. This is necessary for demonstration of the functionality and to get things right.

We could also do a 2D version -- pan, zoom, and rotate some simple objects on a screen.  Perhaps an image manipulation, or a simple map interface.

Yep, something like that. I was thinking of "some simple objects" which you can move, zoom and rotate (and pan the entire scene).

Finally, as this adds some new event types, I am not sure how to add protections to existing applications that still need to be able to compile against older versions of FLTK that do not have these defined.

That's not an issue. Additional events are ignored by existing programs. But changing the semantics like changing a working event to another one (which would be ignored) is problematic (see my notes above).

I meant for the situation where my open source application is sometimes compiled by end users on old versions of FLTK bundled with Linux distributions.   I'd like for these cases to still build and work - even if they lose this functionality.

Ah, yes, I didn't think of that, but that's of course important.

I suppose a simple

#ifndef FL_ROTATE_GESTURE
#define FL_ROTATE_GESTURE=28
#endif

somewhere in my application will work.

I'd rather disable the entire part which wouldn't work if you want to be compatible with both FLTK 1.3 and 1.4, for instance in handle() like this:


  switch(event) {
    case FL_PUSH:
      ...
#if FL_API_VERSION < 10400
    case FL_ZOOM_GESTURE: // only in FLTK 1.3
      ...
#else
    case FL_ROTATE_GESTURE:
      ...
    case FL_TOUCH_GESTURE:
      ...
#endif


I'm assuming here that the old (1.3) version would deliver the data different than the 1.4 version. Using Fl::event_dy() etc. is likely not what we want in 1.4. However, there's no decision yet, this is only a potential example. I'm working on a better (and compatible) solution.

[Albrecht Schlosser]

On 10/11/21 8:14 PM Albrecht Schlosser wrote:

On 10/11/21 4:28 AM Rob McDonald wrote:

The pinch zoom also gave deltas ...

I'll go back and set up a test program to look into this.  I needed some sign flips to make zoom work right on my machine -- but I'm feeding it into an OpenGL zoom parameter that may be counter intuitive.

Well, I wrote a small demo program (test/gestures.cxx). We can certainly improve it, but it works as expected (by me) on macOS with my latest changes in my private fork:

https://github.com/Albrecht-S/fltk/commits/MacOS_Gestures

You can either use CMake or fltk-config to build the demo program (Makefiles not updated).

Changes: positive angles in FL_ROTATE_GESTURE are now clockwise.

FL_ZOOM_GESTURE is in this test version not compatible with the 1.3 version. This is intentional and will be fixed in a later version.

Have fun! Feedback would be appreciated.

[imacarthur]

On Saturday, 9 October 2021 at 21:58:37 UTC+1 Albrecht Schlosser wrote:

Note that the two WM_MOUSEHWHEEL messages at the beginning of the log are really that. However, I don't know where exactly the other ewo interspersed WM_MOUSEHWHEEL messages came from. I think that's a Windows "feature", some gestures are sent as normal mousewheel events for backwards compatibility (more tests required).

Yes, from the subsequent discussion, it does seem very likely that the WM_MOUSEWHEEL events embedded in the GID_ZOOM stream may well be there to support legacy applications.

And therefore... do we need to do something analogous in our gesture event handling...?

[Albrecht Schlosser]

Yes, I think so.

Windows gesture support in general is documented here:
https://docs.microsoft.com/en-us/windows/win32/wintouch/windows-touch-gestures-overview#gestures-overview

The MS docs on "Legacy Support" can be found on the same page:
https://docs.microsoft.com/en-us/windows/win32/wintouch/windows-touch-gestures-overview#legacy-support

Interesting part: "The pan gesture maps to using the scroll wheel: WM_VSCROLL, WM_HSCROLL".

We don't handle these messages currently but I'm pretty sure I saw (vertical) WM_MOUSEWHEEL and (horizontal) WM_MOUSEHWHEEL messages originating from pan gestures. I'll check that out.

On macOS the system handles the pan gestures by sending  "mousewheel" events but, as Rob discovered and proposed in his patch, you can distinguish pan gestures from real mousewheel events and you *can* create different FLTK events. I think we need to keep the "legacy" behavior (FL_MOUSEWHEEL) but we can maybe add a flag so user programs know the origin (if necessary).

Another possibility would be to call an FLTK method that sets an option how to deal with such messages. Old programs that don't call this would get the "legacy" messages, new programs could opt in to use the "new" (gesture) messages. Or something like that. However, I wouldn't want to do that and I'm looking for a better way.

[Albrecht Schlosser]

On 10/12/21 12:57 AM Albrecht Schlosser wrote:

I wrote a small demo program (test/gestures.cxx). We can certainly improve it, but it works as expected (by me) on macOS with my latest changes in my private fork:

https://github.com/Albrecht-S/fltk/commits/MacOS_Gestures

You can either use CMake or fltk-config to build the demo program (Makefiles not updated).

Changes: positive angles in FL_ROTATE_GESTURE are now clockwise.

FL_ZOOM_GESTURE is in this test version not compatible with the 1.3 version. This is intentional and will be fixed in a later version.

The next iteration has been pushed to my branch (link above). I added another dial widget to the demo program and updated the handling (dispatching) of the new gesture events so they can be handled by the widget below the mouse like FL_MOUSEWHEEL events.

For those wanting to test: this is all only for macOS but I'll work on the Windows implementation soon (may take two days or so, don't hold your breath).

[Rob McDonald]

Albrecht,

You've made great progress -- sorry I haven't been able to keep up.

I attempted to add a gesture responsive drawing to your test program -- but haven't quite gotten it working.  I don't normally work with FLTK on this level, so I'm probably doing something trivial wrong.

You should be able to see what I am trying to accomplish very quickly and hopefully find my error.

https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Rob

[Ian MacArthur]

On 11 Oct 2021, at 23:57, Albrecht Schlosser wrote:
>
> Have fun! Feedback would be appreciated.
>

Just “for the record”, works pretty well on my old 2010 iMac (10.13) and seems to be fine.

[Albrecht Schlosser]

On 10/12/21 7:13 PM Rob McDonald wrote:
> Albrecht,
>
> You've made great progress -- sorry I haven't been able to keep up.

Don't worry; thanks for your support.

> I attempted to add a gesture responsive drawing to your test program
> -- but haven't quite gotten it working.  I don't normally work with
> FLTK on this level, so I'm probably doing something trivial wrong.
>
> You should be able to see what I am trying to accomplish very quickly
> and hopefully find my error.
>
> https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Yep, found it. The only "real" issue is in draw(): you're pushing the
matrix at the wrong place in the code. You need to move
'fl_push_matrix();' up before 'fl_translate(...);' and it should work
... with some minor issues. Cute!

What I saw so far: the rotation is "inverted". I'm leaving this one up
to you (unless this was intended).

There was also one problem with event delivery (missing FL_ZOOM_GESTURE
in Fl_Group::handle(). I added it and now it works - at least as I'd expect.

See attached patch for both issues.

PS: Please don't top-post here, this makes the history of messages
confusing. Thanks

[Rob McDonald]

On Tuesday, October 12, 2021 at 3:13:23 PM UTC-7 Albrecht Schlosser wrote:

On 10/12/21 7:13 PM Rob McDonald wrote:
> https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Yep, found it. The only "real" issue is in draw(): you're pushing the
matrix at the wrong place in the code. You need to move
'fl_push_matrix();' up before 'fl_translate(...);' and it should work
... with some minor issues. Cute!

That would have taken me a long time to figure out -- drawing the first time works, but I suppose it quickly out-runs the stack depth of 4 and stops working...  Thanks much for the tip - makes perfect sense. 

What I saw so far: the rotation is "inverted". I'm leaving this one up
to you (unless this was intended).

No, and the 'scroll' to pan was also inverted.

Since it wasn't working, I wasn't able to test out the sign convention -- so no surprise they were wrong.

FLTK's fl_rotate() is positive CCW whereas FL_ROTATE_GESTURE is positive CW.  The sign flip is expected.

The handling of FL_SCROLL_GESTURE had a negative inherited from the FL_MOUSEWHEEL code -- since it seemed quite counterintuitive for this use, I removed it there.

There was also one problem with event delivery (missing FL_ZOOM_GESTURE
in Fl_Group::handle(). I added it and now it works - at least as I'd expect.

Thanks 

I've updated my branch with all these fixes together.  It makes for a little 2D app that responds in all the ways I would want it to -- rotate, pan, zoom.

It looks like MacOS and Windows both also support a swype gesture -- I hadn't looked into it because I don't see a use in my application.  However, as long as we're at it, we should probably be complete.

Rob

[Albrecht Schlosser]

On 10/13/21 2:20 AM Rob McDonald wrote:

On Tuesday, October 12, 2021 at 3:13:23 PM UTC-7 Albrecht Schlosser wrote:

On 10/12/21 7:13 PM Rob McDonald wrote:
> https://github.com/ramcdona/fltk/tree/MacOS_Gestures

Yep, found it. The only "real" issue is in draw(): you're pushing the
matrix at the wrong place in the code. You need to move
'fl_push_matrix();' up before 'fl_translate(...);' and it should work
... with some minor issues. Cute!

That would have taken me a long time to figure out -- drawing the first time works, but I suppose it quickly out-runs the stack depth of 4 and stops working...

Hmm, not really. You can imagin fl_push_matrix as "save" and fl_pop_matrix as "restore" functions, respectively. There was nothing wrong with the stack (which is, BTW, 32 matrices deep, not only 4) because you used push and pop correctly in pairs. But you used transformation calls before you saved the current matrix and these transformations accumulated.

FLTK's fl_rotate() is positive CCW whereas FL_ROTATE_GESTURE is positive CW.  The sign flip is expected.

OK, I see.

The handling of FL_SCROLL_GESTURE had a negative inherited from the FL_MOUSEWHEEL code -- since it seemed quite counterintuitive for this use, I removed it there.

This one is tricky on macOS. You can use the system preferences for trackball and mouse, respectively, to enable "natural scrolling". No matter which one you use, both trackball and mouse are affected! I would have expected that these are different settings, and when I switched to "natural" my Mac did what I expected with the trackpad but the mouse scrolling change was not expected. With this *user preference* the sign of the scrolling delta is inverted (again, in both cases). It's not yet clear to me which sign convention we should apply taking this into account. Do (can) we access the system preference to compensate this? Is it useful to do so? I don't know.

I assume your setting is "natural" - please try to use your program after switching "natural scrollling" off. Is it still intuitive? I don't think so.

I've updated my branch with all these fixes together.  It makes for a little 2D app that responds in all the ways I would want it to -- rotate, pan, zoom.

Yep, that's great. Your addition is really helpful. Thanks. I pulled all your changes into my own fork.

It looks like MacOS and Windows both also support a swype gesture -- I hadn't looked into it because I don't see a use in my application.  However, as long as we're at it, we should probably be complete.

Yes, probably. My next step will be to add the Windows implementation. Once that is done we can see which sign conventions we should use and how "cross platform" compatible this all will be. Windows seem to call the swipe gesture "Flicks" if I interpret their docs correctly. It's hard for a non-native English speaker like me to interpret these gestures and the particular naming in the docs.
https://docs.microsoft.com/en-us/windows/win32/wintouch/windows-touch-gestures-overview

The macOS docs state: "Three fingers brushing across the trackpad surface in a common direction is a swipe gesture."
https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/EventOverview/HandlingTouchEvents/HandlingTouchEvents.html


Finally, one important note: please don't rely on this experimental code for your "real" applications. I'm sure we will change the API before we go into production (release). For instance, one major change would be to add new Fl_event_*() accessor methods for a better interface to gestures. Some of these variables obviously need to be floats or doubles so the user doesn't need to divide by 1,000 or 100,000 or any other factor.

[Rob McDonald]

On Wednesday, October 13, 2021 at 8:17:18 AM UTC-7 Albrecht Schlosser wrote:

The handling of FL_SCROLL_GESTURE had a negative inherited from the FL_MOUSEWHEEL code -- since it seemed quite counterintuitive for this use, I removed it there.

This one is tricky on macOS. You can use the system preferences for trackball and mouse, respectively, to enable "natural scrolling". No matter which one you use, both trackball and mouse are affected! I would have expected that these are different settings, and when I switched to "natural" my Mac did what I expected with the trackpad but the mouse scrolling change was not expected. With this *user preference* the sign of the scrolling delta is inverted (again, in both cases). It's not yet clear to me which sign convention we should apply taking this into account. Do (can) we access the system preference to compensate this? Is it useful to do so? I don't know.

I assume your setting is "natural" - please try to use your program after switching "natural scrollling" off. Is it still intuitive? I don't think so.

Yes, while I hated 'natural' when they first introduced it, I also hate using computers where people set 'weird' settings (swap L/R buttons, etc).  So I forced myself to get used to it.

I'm sure these biases are particular to me -- when I was in high school, I did repair at a small computer store.  Using random customer's computers made me appreciate default settings.  Then much later in life, as a professor, I was reminded of this when I need to help students on their own laptop -- why can't everybody leave shift, alt, control, and return in the same place (within one language layout of course)....

I think trackpad scrolling (and eventually touch-screen) will need to be able to detect the 'natural' scroll setting.  In a touch-screen environment, you certainly don't want to click-and-drag something and have it fly the wrong way...

Yep, that's great. Your addition is really helpful. Thanks. I pulled all your changes into my own fork.

Great -- feel free to squash the mess down to something palatable for inclusion when the time comes. 

Yes, probably. My next step will be to add the Windows implementation. Once that is done we can see which sign conventions we should use and how "cross platform" compatible this all will be. Windows seem to call the swipe gesture "Flicks" if I interpret their docs correctly. It's hard for a non-native English speaker like me to interpret these gestures and the particular naming in the docs.
https://docs.microsoft.com/en-us/windows/win32/wintouch/windows-touch-gestures-overview

The macOS docs state: "Three fingers brushing across the trackpad surface in a common direction is a swipe gesture."
https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/EventOverview/HandlingTouchEvents/HandlingTouchEvents.html

It may not be practical to map all OS gestures to a common set of FLTK gestures.  While that would be ideal (if all OS's supported a wide set of gestures), it would not be a useful feature if there is not a useful common set.

Should we map Windows one-finger Flick and MacOS three-finger swype to the same event?  I say probably not.  Although they could reasonably be used for the same purpose (say turning pages in a virtual book), what would we do if MacOS later supports a one-finger flick and Windows a 3-finger swype?

Another approach would be to add support for all OS supported gestures -- mapping common gestures to common events.  That allows the application to decide to interpret flick and swype in the same way (or not).

I see this as analogous to the two-finger scroll being different from mouse wheel scroll -- it is nice for a default behavior for them to be the same -- but there is cool opportunity in separating them out.

 

Finally, one important note: please don't rely on this experimental code for your "real" applications. I'm sure we will change the API before we go into production (release). For instance, one major change would be to add new Fl_event_*() accessor methods for a better interface to gestures. Some of these variables obviously need to be floats or doubles so the user doesn't need to divide by 1,000 or 100,000 or any other factor.

Understood.

Rob

 

[Albrecht Schlosser]

On 10/13/21 6:48 PM Rob McDonald wrote:

I think trackpad scrolling (and eventually touch-screen) will need to be able to detect the 'natural' scroll setting.  In a touch-screen environment, you certainly don't want to click-and-drag something and have it fly the wrong way...

As I was told Apple doesn't produce systems (MacBooks) with touch-screens. Are touch-screens supported by macOS? If not we won't know how to deal with them until Apple will produce and/or support systems with touch-screens. I'm saying this for a reason: if we design an API today it may be wrong in the future once touch-screens are working with Apple systems. Currently I know that my tests with my notebook's touch-screen worked on Windows.

It may not be practical to map all OS gestures to a common set of FLTK gestures.  While that would be ideal (if all OS's supported a wide set of gestures), it would not be a useful feature if there is not a useful common set.

Should we map Windows one-finger Flick and MacOS three-finger swype to the same event?  I say probably not.  Although they could reasonably be used for the same purpose (say turning pages in a virtual book), what would we do if MacOS later supports a one-finger flick and Windows a 3-finger swype?

Another approach would be to add support for all OS supported gestures -- mapping common gestures to common events.  That allows the application to decide to interpret flick and swype in the same way (or not).

Well put, all good points. My conclusion from your writing is that we should indeed let "different things" be separate FLTK events. The one-finger-flick vs. three-finger-swipe argument is compelling.

For all new (and documented) events we can tell the user that they can and should handle FL_FLICK and FL_SWIPE events in the same 'case' statement, for instance. With a note that this may change in the future...

I see this as analogous to the two-finger scroll being different from mouse wheel scroll -- it is nice for a default behavior for them to be the same -- but there is cool opportunity in separating them out.

Agreed. However this is a little different because of backwards compatibility. Currently all FLTK programs that handle FL_MOUSEWHEEL events continue to work with trackpad scroll events if these are mapped to FL_MOUSEWHEEL events. As soon as we separate them out programs need to be modified. That's why I think we need a backwards compatible way to deal with these two event types. Let's see ...

Here's a new idea (untested): we could first send an FL_SCROLL_GESTURE event and if this is not handled by any widget, send an FL_MOUSEWHEEL event. This way "new" programs can handle the FL_SCROLL_GESTURE event and "old" programs can still handle FL_MOUSEWHEEL events. It's similar to FL_KEYBOARD events that are converted to FL_SHORTCUT (if not handled) and finally to FL_SHORTCUT with flipped case. Does that sound sensible?

[Rob McDonald]

On Wednesday, October 13, 2021 at 1:58:14 PM UTC-7 Albrecht Schlosser wrote:

On 10/13/21 6:48 PM Rob McDonald wrote:

I think trackpad scrolling (and eventually touch-screen) will need to be able to detect the 'natural' scroll setting.  In a touch-screen environment, you certainly don't want to click-and-drag something and have it fly the wrong way...

As I was told Apple doesn't produce systems (MacBooks) with touch-screens. Are touch-screens supported by macOS? If not we won't know how to deal with them until Apple will produce and/or support systems with touch-screens. I'm saying this for a reason: if we design an API today it may be wrong in the future once touch-screens are working with Apple systems. Currently I know that my tests with my notebook's touch-screen worked on Windows.

Agreed.  I expect that someday Apple will support touch screens.

I don't have a touch screen Windows machine -- I'm glad that you have such a machine to test on.

I have used various forms of SMART Boards in the past -- a multi-touch surface paired with a monitor or projector.  These usually send one-finger events as a mouse (movement, click, click-n-drag) -- or if special software is loaded, as proprietary events.  I haven't used one of these in years, but if I was marketing one for MacOS today, I would set it up to work with existing gestures.  In short - there may be a way to plug a touch screen into a Mac that isn't official Mac hardware.

These devices are commonly used in educational settings, but are also great for any collaborative / creative work environments (think two people standing at an interactive touch whiteboard).  I've used them successfully doing collaborative design reviews with 3D CAD.

 

Well put, all good points. My conclusion from your writing is that we should indeed let "different things" be separate FLTK events. The one-finger-flick vs. three-finger-swipe argument is compelling.

For all new (and documented) events we can tell the user that they can and should handle FL_FLICK and FL_SWIPE events in the same 'case' statement, for instance. With a note that this may change in the future...

Agreed -- and you guessed my preference too.

 

I see this as analogous to the two-finger scroll being different from mouse wheel scroll -- it is nice for a default behavior for them to be the same -- but there is cool opportunity in separating them out.

Agreed. However this is a little different because of backwards compatibility. Currently all FLTK programs that handle FL_MOUSEWHEEL events continue to work with trackpad scroll events if these are mapped to FL_MOUSEWHEEL events. As soon as we separate them out programs need to be modified. That's why I think we need a backwards compatible way to deal with these two event types. Let's see ...

Compatibility is merely a matter of foresight.  If we make them the same event now and want to change it later, we then have a compatibility problem (:

 

 

Here's a new idea (untested): we could first send an FL_SCROLL_GESTURE event and if this is not handled by any widget, send an FL_MOUSEWHEEL event. This way "new" programs can handle the FL_SCROLL_GESTURE event and "old" programs can still handle FL_MOUSEWHEEL events. It's similar to FL_KEYBOARD events that are converted to FL_SHORTCUT (if not handled) and finally to FL_SHORTCUT with flipped case. Does that sound sensible?

Devious and clever -- I never would have thought of this.

If that works, I think it will be a great solution.

Under all circumstances, I think a "real mousewheel" should always send "real mousewheel" events.  I.e. we should differentiate the event type - only send gesture scroll events if the OS event originates from a non-mousehweel source -- then re-send as a mousewheel if nothing down the line handled the gesture event.

Rob

 

[Albrecht Schlosser]

On  10/13/21 11:54 PM Rob McDonald wrote:

On Wednesday, October 13, 2021 at 1:58:14 PM UTC-7 Albrecht Schlosser wrote:

On 10/13/21 6:48 PM Rob McDonald wrote:

I don't have a touch screen Windows machine -- I'm glad that you have such a machine to test on.

Although I need to boot my (dual boot) working Linux machine into Windows for that. I hate to do this but it's a way to develop and test Windows software. Unfortunately touch gestures don't work with my Windows (Virtualbox) VM under Linux. :-(

I see this as analogous to the two-finger scroll being different from mouse wheel scroll -- it is nice for a default behavior for them to be the same -- but there is cool opportunity in separating them out.

Agreed. However this is a little different because of backwards compatibility. Currently all FLTK programs that handle FL_MOUSEWHEEL events continue to work with trackpad scroll events if these are mapped to FL_MOUSEWHEEL events. As soon as we separate them out programs need to be modified. That's why I think we need a backwards compatible way to deal with these two event types. Let's see ...

Compatibility is merely a matter of foresight.  If we make them the same event now and want to change it later, we then have a compatibility problem (:

Yep, agreed.

[Albrecht:] Here's a new idea (untested): we could first send an FL_SCROLL_GESTURE event and if this is not handled by any widget, send an FL_MOUSEWHEEL event. This way "new" programs can handle the FL_SCROLL_GESTURE event and "old" programs can still handle FL_MOUSEWHEEL events. It's similar to FL_KEYBOARD events that are converted to FL_SHORTCUT (if not handled) and finally to FL_SHORTCUT with flipped case. Does that sound sensible?

[Rob:] Devious and clever -- I never would have thought of this.

If that works, I think it will be a great solution.

I hope it will. We have the precedent of FL_KEYBOARD and FL_SHORTCUT, so I believe it should (a) be doable and (b) work for users. It's a little overhead to send an event that nobody uses (if both FL_SCROLL_GESTURE *and* FL_MOUSEWHEEL are not used). But that's a small price.

Under all circumstances, I think a "real mousewheel" should always send "real mousewheel" events.  I.e. we should differentiate the event type - only send gesture scroll events if the OS event originates from a non-mousehweel source -- then re-send as a mousewheel if nothing down the line handled the gesture event.

Yes, sure. If this was not clear: I wouldn't want to change mousewheel events as such. Real mousewheel events shall keep sending FL_MOUSEWHEEL events, nothing else. This idea was only about your changes to send trackpad scroll events as a different type of event which is good. The idea described above should make it possible in a backwards compatible way, and only this new event type should be handled by sending FL_SCROLL_GESTURE first and then FL_MOUSEWHEEL as a fallback.

[Bill Spitzak]

The multiple event types idea sounds good. This could be done for all gestures, for instance if "flick" is unrecognized you might still want to send a mouse click.

--
You received this message because you are subscribed to the Google Groups "fltk.coredev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkcoredev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fltkcoredev/dc3f77e3-ba4b-be0a-dd7f-e6bffab08da3%40online.de.

[melcher....@googlemail.com]

[Albrecht:] Here's a new idea (untested): we could first send an FL_SCROLL_GESTURE event and if this is not handled by any widget, send an FL_MOUSEWHEEL event. This way "new" programs can handle the FL_SCROLL_GESTURE event and "old" programs can still handle FL_MOUSEWHEEL events. It's similar to FL_KEYBOARD events that are converted to FL_SHORTCUT (if not handled) and finally to FL_SHORTCUT with flipped case. Does that sound sensible?

I hope it will. We have the precedent of FL_KEYBOARD and FL_SHORTCUT, so I believe it should (a) be doable and (b) work for users. It's a little overhead to send an event that nobody uses (if both FL_SCROLL_GESTURE *and* FL_MOUSEWHEEL are not used). But that's a small price.

Yes, this is tested and works. The overhead is not an issue, considering the speed of the human motion in relation to the speed of a modern CPU. I did implement that many moons ago for a Wacom tablet pen input: at first, it would send a FL_PEN_DOWN, and if that wasn't handled by any widget, an FL_PUSH was sent with the same coordinates (followed by FL_MOVE, FL_RELEASE, vs. FL_PEN_MOVE and FL_PEN_RELEASE). 

 

[Albrecht Schlosser]

Hi Matthias,

great to see you "back here" again.

Thanks for your confirmation. Unfortunately I've been too busy lately to proceed with this on Windows (before I can see how this is done best cross-platform) but I'll go ahead with this soon.

If you or anybody else have a good idea (and docs and/or code) how to use multi touch gestures on Linux this would be appreciated as well.

[Rob McDonald]

On Thursday, November 18, 2021 at 7:11:37 AM UTC-8 Albrecht Schlosser wrote:

Hi Matthias,

great to see you "back here" again.

Thanks for your confirmation. Unfortunately I've been too busy lately to proceed with this on Windows (before I can see how this is done best cross-platform) but I'll go ahead with this soon.

If you or anybody else have a good idea (and docs and/or code) how to use multi touch gestures on Linux this would be appreciated as well.

I don't have any great answers here, but from my research, it looks like there will be separate solutions for X11 and Wayland and most X11 solutions will require a 3rd party library.

Rob