Is handling simultaneous Left-click and Right-click drags supported?

[Harris Lau]

When implementing the Fl_Widget::Handle() function, I am trying to handle simultaneous left-click drags and right-click drags, however, the events I get is not what I would expect.

Action Expected Event Actual Event
Left-click down FL_PUSH FL_PUSH
Right-click down FL_PUSH FL_PUSH
Dragging FL_DRAG FL_DRAG
Left-click up FL_RELEASE FL_RELEASE
More dragging FL_DRAG > FL_MOVE <
Right-click up FL_RELEASE FL_RELEASE
This is preventing my code from working properly since I'm still holding the click and dragging but the event doesn't represent it.

[Greg Ercolano]

    Sounds like you might not be returning 1 when receiving FL_PUSH events.
    From the FLTK programming manual on "Handling Events", emphasis in red:

---

FL_DRAG

The mouse has moved with a button held down. The current button state is in Fl::event_state(). The mouse position is in Fl::event_x() and Fl::event_y().

In order to receive FL_DRAG events, the widget must return non-zero when handling FL_PUSH.

---

[Harris Lau]

Hi, I am returning non-zero when handling FL_PUSH, here is how I implement the handle function:

int handle(int event) {
    printf("Event was %s (%d)\n", fl_eventnames[event], event); // For illustrating the bug.
    switch(event) {
    case FL_ENTER:
        break;
    case FL_PUSH:
        if (Fl::event_button()>1)
            printf("Right Click\n");
        else
            printf("Left Click\n");
        break;
    case FL_DRAG:
        break;
    case FL_RELEASE:
        break;
    case FL_MOVE:
        break;
    default:
        return 0;
        break;
    }
    return 1;
}

The problem appears when I _hold down_ both left and right click, then release one of them, then continue to drag. I am still dragging with a click but I receive FL_MOVE instead.

[Greg Ercolano]

    Oh, I see -- ya, that sounds like that might not be supported,
    but I defer to others.

    Usually drags involve a left-drag OR a right-drag, not both
    at the same time.

    FWIW I'm including a fully compileable app. I rewrote your handle()
    a little bit for clarity, while it still demonstrates your issue.

---

#include <FL/Fl.H>
#include <FL/Fl_Window.H>
#include <FL/Fl_Widget.H>
#include <FL/names.h>   // fl_eventnames[]
#include <FL/fl_draw.H>
#include <stdio.h>

class MyWidget : public Fl_Widget {
public:
    MyWidget(int X,int Y,int W,int H) : Fl_Widget(X,Y,W,H) { }
    int handle(int event) {
        int ret = Fl_Widget::handle(event);                         // let Fl_Widget access events too

        printf("Event was %s (%d)\n", fl_eventnames[event], event); // For illustrating the bug.
        switch(event) {
            case FL_ENTER:

                return 1;

            case FL_PUSH:
                if (Fl::event_button()>1) printf("Right Click\n");
                else                      printf("Left Click\n");

                return 1;
            case FL_DRAG:
                return 1;
            case FL_RELEASE:
                return 1;
            case FL_MOVE:
                return 1;
        }
        return ret;     // return Fl_Widget::handle()'s value
    }
    void draw() {
      fl_color(color());
      fl_rectf(x(),y(),w(),h());
    }
};

int main() {
    Fl_Window win(300,300);
    MyWidget widget(10,10,100,100);
    widget.color(FL_RED);
    win.show();
    return Fl::run();
}

[Ian MacArthur]

I think the crux is that the Left-click up is triggering a FL_RELEASE event (as expected) and that thereafter the fltk internal state thinks that the subsequent mouse motions are FL_MOVE rather than FL_DRAG, because the  FL_RELEASE event (indeed _any_  FL_RELEASE event) ends the drag operation state...

That might be a tricky thing to resolve robustly in the fltk core, TBH.

It might be possible to "fix" it locally in your handle method - if you can increment a count  for the FL_PUSH events and decrement it for the FL_RELEASE events, you might be able to infer "locally" that the FL_MOVE is really still an FL_DRAG for your purposes and respond accordingly. Don't know if that will work, but it seems like it might.

Or, when you get the FL_MOVE, check the event state to see if any buttons are still down, and if they are, treat the FL_MOVE as a FL_DRAG (that may be more robust than trying to keep count, since it does not depend on the order or timing of event delivery.)

[Ian MacArthur]

Yeah - that might work - don't know if this is any good for your use-case, but here's Greg's example tweaked to track the button state, and this does seem like it might be an option:

========= fltk-config --compile drag-test.cxx ===========

#include <FL/Fl.H>
#include <FL/Fl_Window.H>
#include <FL/Fl_Widget.H>
#include <FL/names.h>   // fl_eventnames[]
#include <FL/fl_draw.H>
#include <stdio.h>

class MyWidget : public Fl_Widget {
public:
    MyWidget(int X,int Y,int W,int H) :

        Fl_Widget(X,Y,W,H),
        button_L (false),
        button_R (false)

    { }

    int handle(int event) {
        int ret = Fl_Widget::handle(event);                         // let Fl_Widget access events too

//        printf("Event was %s (%d)\n", fl_eventnames[event], event); // For illustrating the bug.

        switch(event) {
            case FL_ENTER:
                return 1;

            case FL_PUSH: {
                int bt_event = Fl::event_button();
                switch (bt_event)
                {
                    case FL_LEFT_MOUSE:
                    button_L = true;
                    break;

                    case FL_MIDDLE_MOUSE:
                    break;

                    case FL_RIGHT_MOUSE:
                    button_R = true;
                    break;
                }
                return 1;
            }

            case FL_DRAG:
                printf("Event was %s\n", fl_eventnames[event]);
                return 1;

            case FL_RELEASE: {
                int bt_event = Fl::event_button();
                switch (bt_event)
                {
                    case FL_LEFT_MOUSE:
                    button_L = false;
                    break;

                    case FL_MIDDLE_MOUSE:
                    break;

                    case FL_RIGHT_MOUSE:
                    button_R = false;
                    break;
                }
                return 1;
            }

            case FL_MOVE:
                if (button_L || button_R) {
                    printf ("Fake DRAG\n");
                }
                else {
                    printf("Event was %s\n", fl_eventnames[event]);

                }
                return 1;
        }
        return ret;     // return Fl_Widget::handle()'s value
    }
    void draw() {
      fl_color(color());
      fl_rectf(x(),y(),w(),h());
    }

    private:
    bool button_L;
    bool button_R;
};

int main() {
    Fl_Window win(300, 300);
    MyWidget widget(10, 10, 200, 200);

    widget.color(FL_RED);
    win.show();
    return Fl::run();
}

// end of file //

[Albrecht Schlosser]

On 7/12/23 12:44 Ian MacArthur wrote:

Yeah - that might work - ... here's Greg's example tweaked to track the button state, and this does seem like it might be an option:

(code removed)

Ian, on which platform did you test this code?

I noticed that it works well on Windows (cross-compiled, executed by `wine`) but it doesn't work correctly on Linux. The problem is - AFAICT - within FLTK event delivery under Linux because Fl::event_button() does not always return the correct value. Note that it should return one for left and 3 for right button.

When I do the same on Windows and Linux, i.e.

- click left and hold,
- click right,
- release right,
- release left button

I see on Windows (as expected):

Event was FL_PUSH (1)
    ... Fl::event_button() = 0x1
Event was FL_PUSH (1)
    ... Fl::event_button() = 0x3
Event was FL_RELEASE (2)
    ... Fl::event_button() = 0x3
Event was FL_MOVE (11)
Fake DRAG
Event was FL_RELEASE (2)
    ... Fl::event_button() = 0x1    # correct
Event was FL_MOVE (11)              # system event, as expected
Event was FL_MOVE                   # correct


However, on Linux I see:

Event was FL_PUSH (1)
    ... Fl::event_button() = 0x1
Event was FL_PUSH (1)
    ... Fl::event_button() = 0x3
Event was FL_RELEASE (2)
    ... Fl::event_button() = 0x3
Event was FL_MOVE (11)
Fake DRAG
Event was FL_RELEASE (2)
    ... Fl::event_button() = 0x3   # wrong, should be 0x1
Event was FL_MOVE (11)             # system event, as expected
Fake DRAG                          # wrong!


Ian's program appears to be correct for this kind of application but the internal button state seems to be wrong under Linux. This happens always when I click both the left and right button(s) together, no matter in which order.

This means that we have *two* issues:

(1) Wrong button state delivery on Linux when two or more buttons are pressed simultaneously and then released. This prevents Ian's workaround to work correctly on Linux.

(2) The supposedly wrong termination of the "drag" status when one of two buttons is released and the other button is still down. This is what the OP reported.

The former should be fixed independently of the latter.

The latter might be hard to fix, as Ian wrote before. We might want to check it though.

[Ian MacArthur]

On Wednesday, 12 July 2023 at 15:25:02 UTC+1 Albrecht Schlosser wrote:

Ian, on which platform did you test this code?

Windows-10. (It's what I have access to here.)

TBH I assumed it would Just Work elsewhere, but...

 

I noticed that it works well on Windows (cross-compiled, executed by `wine`) but it doesn't work correctly on Linux. The problem is - AFAICT - within FLTK event delivery under Linux because Fl::event_button() does not always return the correct value. Note that it should return one for left and 3 for right button.

Ah...
OK, that's a problem.

And, now that you state it, I think it is not a new problem, I think this (or variations of it) have been observed in the past...

If you are using my code, it might be interesting to see what happens if you: 

- click left and hold, drag,
- click MIDDLE,
- release  MIDDLE,
- release left button

And also:

- click RIGHT and hold, drag,
- click MIDDLE,
- release  MIDDLE, 
- release  RIGHT  button

I specifically separated out the middle and right button cases because (with the OP / Greg code) I could get odd behaviours by clicking the middle button (the original code treated right and middle as "the same button" when they are not so I got release events for "the wrong" button...)

This means that we have *two* issues:

(1) Wrong button state delivery on Linux when two or more buttons are pressed simultaneously and then released. This prevents Ian's workaround to work correctly on Linux.

I'm not sure this is new, now I think about it. 

ISTR it relates to the way that the X11 system (or such...) handles event delivery. I think it is like the way that keyboard key-up / key-down  modifiers keys issue, where you can't reliably see the state of the modifier keys until you get the next key-down.

I have a (very vague) recollection that with the mouse you don't know which mouse buttons are down until the next push event, or something like that, so the release event often just reports the last pushed button... Or something.

Anyway, my recollection was that it was a Hard Thing to fix.

Are you using X11 or Wayland? 

Maybe Wayland is better?

 

(2) The supposedly wrong termination of the "drag" status when one of two buttons is released and the other button is still down. This is what the OP reported.

And if we do this, we need to be aware how many buttons there are - just the 2 or 3 we currently support might not trap all the edge cases since modern mice (and the USB HID protocol) support a lot more buttons, potentially.

 

[Bill Spitzak]

It is supposed to produce DRAG events if any mouse buttons are down. This is a bug. FLTK accurately tracks the state of all the buttons so this should be fixable.

--
You received this message because you are subscribed to the Google Groups "fltk.general" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkgeneral...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fltkgeneral/6badefc3-75f0-4758-81b0-4fa340464b50n%40googlegroups.com.

[Albrecht Schlosser]

On 7/12/23 16:57 Bill Spitzak wrote:
> It is supposed to produce DRAG events if any mouse buttons are down.
> This is a bug. FLTK accurately tracks the state of all the buttons so
> this should be fixable.

Thanks, Bill. Your comments are always appreciated, particularly when
regarding the design (i.e. what *should* FLTK do).

I'm now going to look into these issues ...

[Albrecht Schlosser]

On 7/12/23 16:50 Ian MacArthur wrote:

On Wednesday, 12 July 2023 at 15:25:02 UTC+1 Albrecht Schlosser wrote:

Ian, on which platform did you test this code?

Windows-10. (It's what I have access to here.)

TBH I assumed it would Just Work elsewhere, but...

Sure, it should. My primary test system is Linux, which is good since we find such bugs easier if we're using different platforms.

 

I noticed that it works well on Windows (cross-compiled, executed by `wine`) but it doesn't work correctly on Linux. The problem is - AFAICT - within FLTK event delivery under Linux because Fl::event_button() does not always return the correct value. Note that it should return one for left and 3 for right button.

Ah...
OK, that's a problem.

And, now that you state it, I think it is not a new problem, I think this (or variations of it) have been observed in the past...

Hmm, maybe. But if such behavior is platform dependent, then it's really bad.

If you are using my code, it might be interesting to see what happens if you: 

- click left and hold, drag,
- click MIDDLE,
- release  MIDDLE,
- release left button

And also:

- click RIGHT and hold, drag,
- click MIDDLE,
- release  MIDDLE, 
- release  RIGHT  button

The behavior is "the same" (similar), i.e. whenever two buttons are pressed and released, at least one FL_RELEASE event returns the wrong value for Fl::event_button().

You can see the issue in our test/handle_events.cxx demo program as well. No matter which button I release first, it always returns the wrong button state.

Example (1 down, 2 down, 1 up, 2 up), returns (1 down, 2 down, 2 up, 2 up):

[ 62, win(0,0,240,240), screen 0, scale 100%] FL_PUSH            at (  68,   57), button 1 down
[ 63, win(0,0,240,240), screen 0, scale 100%] FL_PUSH            at (  68,   57), button 2 down
[ 64, win(0,0,240,240), screen 0, scale 100%] FL_DRAG            at (  68,   57)
[ 65, win(0,0,240,240), screen 0, scale 100%] FL_DRAG            at (  69,   57)
[ 66, win(0,0,240,240), screen 0, scale 100%] FL_DRAG            at (  69,   58)
[ 67, win(0,0,240,240), screen 0, scale 100%] FL_RELEASE         at (  69,   58), button 2 up
[ 68, win(0,0,240,240), screen 0, scale 100%] FL_MOVE            at (  69,   58)
[ 69, win(0,0,240,240), screen 0, scale 100%] FL_RELEASE         at (  69,   58), button 2 up
[ 70, win(0,0,240,240), screen 0, scale 100%] FL_MOVE            at (  69,   58)

I specifically separated out the middle and right button cases because (with the OP / Greg code) I could get odd behaviours by clicking the middle button (the original code treated right and middle as "the same button" when they are not so I got release events for "the wrong" button...)

I don't see any difference in the behavior whether I use the left/middle or the left/right combo (see above).

This means that we have *two* issues:

(1) Wrong button state delivery on Linux when two or more buttons are pressed simultaneously and then released. This prevents Ian's workaround to work correctly on Linux.

I'm not sure this is new, now I think about it. 

ISTR it relates to the way that the X11 system (or such...) handles event delivery. I think it is like the way that keyboard key-up / key-down  modifiers keys issue, where you can't reliably see the state of the modifier keys until you get the next key-down.

I remember this regarding keyboard events, but I was not aware of such an issue with mouse buttons.

I have a (very vague) recollection that with the mouse you don't know which mouse buttons are down until the next push event, or something like that, so the release event often just reports the last pushed button... Or something.

Anyway, my recollection was that it was a Hard Thing to fix.

Maybe, I'll look into it and report what I can find.

Are you using X11 or Wayland?

Good question. Really good question !!!

Maybe Wayland is better?

Nope, actually I *was* using Wayland which exhibited this issue.

Testing with X11 shows that X11 is correct!


@Manolo: can you please take a look why Wayland reports the wrong buttons in Fl::event_button() for FL_RELEASE events when two or more buttons have been pressed? As I wrote above, test/handle_events.cxx work well to test this.


However, the premature termination of FL_DRAG is still an issue under X11:

[ 86, win(10,80,240,240), screen 0, scale 100%] FL_PUSH            at (  94,   51), button 1 down
[ 87, win(10,80,240,240), screen 0, scale 100%] FL_PUSH            at (  94,   51), button 3 down
[ 88, win(10,80,240,240), screen 0, scale 100%] FL_DRAG            at (  95,   51)
[ 89, win(10,80,240,240), screen 0, scale 100%] FL_RELEASE         at (  95,   51), button 3 up
[ 90, win(10,80,240,240), screen 0, scale 100%] FL_MOVE            at (  95,   51)
[ 91, win(10,80,240,240), screen 0, scale 100%] FL_RELEASE         at (  95,   51), button 1 up
[ 92, win(10,80,240,240), screen 0, scale 100%] FL_MOVE            at (  95,   51)

As one can see, the correct buttons are reported but the first FL_RELEASE terminates the drag operation.

(2) The supposedly wrong termination of the "drag" status when one of two buttons is released and the other button is still down. This is what the OP reported.

And if we do this, we need to be aware how many buttons there are - just the 2 or 3 we currently support might not trap all the edge cases since modern mice (and the USB HID protocol) support a lot more buttons, potentially.

Some of these "buttons" are used for scrolling IIRC, at least under X11. I don't think that button numbers > 3 should be used for dragging (in FLTK). This would likely surprise users.

However, using buttons 1 to 3 seems appropriate and we should try to fix this bug if we can - or document the wrong behavior if we can't fix it. My tests on Windows, macOS, X11 and Wayland showed that this particular behavior is the same on all supported platforms.

[Albrecht Schlosser]

On 7/12/23 17:36 Albrecht Schlosser wrote:
> @Manolo: can you please take a look why Wayland reports the wrong
> buttons in Fl::event_button() for FL_RELEASE events when two or more
> buttons have been pressed? As I wrote above, test/handle_events.cxx

> works well to test this.

Fixed in commit 858c3cad869593765fa81f417432d2a0d0e60af5.

Manolo, sorry for the request, I couldn't resist to look into it and
just found and fixed it. The fault was not to set the button number on
FL_RELEASE events at all.

[Albrecht Schlosser]

Hmm, after looking at my fix again I believe it's not correct (setting
Fl::e_state seems wrong). It fixes the issue at hand but may have other
unwanted side effects.

I'll check it again. Sorry for the noise.

[Ian MacArthur]

On Wednesday, 12 July 2023 at 16:37:01 UTC+1 Albrecht Schlosser wrote:

The behavior is "the same" (similar), i.e. whenever two buttons are pressed and released, at least one FL_RELEASE event returns the wrong value for Fl::event_button().

You can see the issue in our test/handle_events.cxx demo program as well. No matter which button I release first, it always returns the wrong button state.

OK, so it's a consistent misbehaviour on the Linux set-up then, which may be useful to know...

 

I'm not sure this is new, now I think about it. 

ISTR it relates to the way that the X11 system (or such...) handles event delivery. I think it is like the way that keyboard key-up / key-down  modifiers keys issue, where you can't reliably see the state of the modifier keys until you get the next key-down.

I remember this regarding keyboard events, but I was not aware of such an issue with mouse buttons.

It's entirely plausible I am misremembering this... I'm sure there was something about FL_DRAG but I am less sure (now) that I remember what.

Are you using X11 or Wayland?

Good question. Really good question !!!

Maybe Wayland is better?

Nope, actually I *was* using Wayland which exhibited this issue.

Testing with X11 shows that X11 is correct!

 Oh!

OK, that was not what I expected; I thought there was something odd with X11 and assumed Wayland (which doesn't have the client/server round trip) would be better, but not so.

However, the premature termination of FL_DRAG is still an issue under X11: My tests on Windows, macOS, X11 and Wayland showed that this particular behavior is the same on all supported platforms.

Yes. Which means it is something we are doing internally, I imagine. Which is unfortunate...

 

[Albrecht Schlosser]

OK, I'm confident that I fixed the wrong behavior in commit 44840af076569ab6c692d7dcbcc70022d1d40087.

This also fixes the previous wrong behavior (before my last commit) which reset the mouse button state in Fl::e_state whenever a button press or release was handled. See the commit log why this was wrong:

commit 44840af076569ab6c692d7dcbcc70022d1d40087

    Wayland: keep mouse button state across push/release events
    
    Notes:
    
    (1) Fl::e_state holds the current state of all mouse buttons which is
        returned by Fl::event_buttons() - "plural form".
    
    (2) Fl::e_keysym holds the "key" of the current event which can be a
        mouse button, returned by Fl::event_button() - "singular form".

I hope I didn't miss anything. My tests with a modified demo program (Erco's demo with Ian's mods plus mine) seems to indicate that this is correct across platforms (macOS not yet tested).

Note: this does not fix the erroneous termination of FL_DRAG events.

[Ian MacArthur]

On Wednesday, 12 July 2023 at 18:03:13 UTC+1 Albrecht Schlosser wrote:

OK, I'm confident that I fixed the wrong behavior in commit 44840af076569ab6c692d7dcbcc70022d1d40087.

This also fixes the previous wrong behavior (before my last commit) which reset the mouse button state in Fl::e_state whenever a button press or release was handled. See the commit log why this was wrong:
commit 44840af076569ab6c692d7dcbcc70022d1d40087

Looks OK to me - tested under WSL with X11 and Wayland (I'm on a WIn11 box now...) and seems to be good. 

Also saw the issue under Wayland prior to the commits. (X11 was OK though, or at least the same as Win32.)

The FL_DRAG issue remains, but the workaround might be enough to get the OP going, I guess, in the short term.

[Albrecht Schlosser]

On 7/12/23 18:13 Ian MacArthur wrote:

On Wednesday, 12 July 2023 at 16:37:01 UTC+1 Albrecht Schlosser wrote:

However, the premature termination of FL_DRAG is still an issue under X11: My tests on Windows, macOS, X11 and Wayland showed that this particular behavior is the same on all supported platforms.

To clarify: the first part of the above statement mentions X11 explicitly which is not entirely correct. The following part of the statement is correct: this misbehavior is consistent across platforms.

Yes. Which means it is something we are doing internally, I imagine. Which is unfortunate...

Or maybe it's fortunate because we can fix it.

I imagine we could send FL_DRAG rather than FL_MOVE whenever at least one mouse button is (still) down. However, it's probably not as simple as that because we need the widget context: the widget that responded with 1 on FL_PUSH and therefore gets subsequent FL_DRAG events (or something like that). Needs some investigation, and maybe it requires changes in platform specific code.

[Albrecht Schlosser]

On 7/12/23 19:43 Albrecht Schlosser wrote:
> I imagine we could send FL_DRAG rather than FL_MOVE whenever at least
> one mouse button is (still) down. However, it's probably not as simple
> as that because we need the widget context: the widget that responded
> with 1 on FL_PUSH and therefore gets subsequent FL_DRAG events (or
> something like that). Needs some investigation, and maybe it requires
> changes in platform specific code.

I forgot to mention: commit 03913f32e0010076474d311758c388561eeac01e
extends test/handle_events.cxx to show the currently pressed mouse
buttons in some events, particularly FL_MOVE and FL_DRAG. This can be
used to verify that we currently send FL_MOVE events after the *first*
mouse button has been released.

[Albrecht Schlosser]

Commit 12592753166337548c9f80f7247803070433b2fd

    Keep sending FL_DRAG until all mouse buttons are released
    ...


fixes the issue AFAICT. All modifications are in common (cross platform)
code.

Tested on all supported platforms (Windows, macOS, Linux/X11, and
Linux/Wayland) using test/handle_events.cxx.

I don't think that this minor behavior change has much potential to
cause unexpected side effects since it affects only dragging with more
than one pressed mouse button. The order of pressing and releasing
buttons does NOT affect the behavior - the only change is that one must
release *all* mouse buttons to finish dragging.

Just in case someone might be affected by this behavioral change: it is
possible to make this optional by yet another Fl::option or something
like that.


@OP (Harris Lau): please confirm that this fixes the issue for you.
Thanks in advance.