canvas% mouse event handling difference in Racket CS?

[schle...@gmail.com]

I have a racket gui app that uses a canvas% with overridden on-event and on-paint methods.

When the user hovers over drawn elements the on-paint is called via (send this refresh)

to display the element under the cursor with a selection/outline.

Recently I noticed that this has gotten extremely slow.

It seems to me this might be a difference between BC and CS, but I haven't checked with different versions in depth yet. (just from the behavior/performance I remember it had in the past)

In the past a call to (send this refresh) seemed to be processed concurrently in regard to on-event.

Now it seems like the first (send this refresh) eagerly triggers the on-paint and this on-paint somehow blocks the processing of on-event until the on-paint is finished, after that 1 more mouse event is processed re-triggering the on-paint.

Effectively redrawing for every mouse event, causing the app to draw old uninteresting frames (because the mouse events aren't processed fast enough and only the last position is interesting for me).

Currently I have implemented a workaround:

(define (oneoff-timer interval thk)
  (new timer%
       [notify-callback thk]
       [interval interval]
       [just-once? #t]))

(define (debounce interval thk)
  (define timer #f)
  (define (fire)
    (set! timer #f)
    (thk))
  (define (trigger)
    (unless timer
      (set! timer (oneoff-timer interval fire))))
  trigger)

;; within the canvas impl
(define dirty! (debounce 50 (thunk (send this refresh))))

;; within on-event calling (dirty!) instead of (send this refresh) directly

;; this effectively waits at least 50 ms before trying to refresh thus allowing most on-event

;; calls to complete before on-paint is executed the first/next time, thus only drawing the last frame or a few in-between frames if the mouse is moved for a long time

I may try to construct a minimal example, but wanted to put the info out there, because the behavior seems so different from before.

Tested version: Racket v8.1 [cs] linux

Simon

[schle...@gmail.com]

I tested with Racket v7.8 bc and it was also slow.

I also tested it with a year old version of my code and added similar `time` calls around the drawing code

and it behaved like the recent version redrawing for all the mouse events, only difference is that the on-paint takes around 23ms instead of 50ms.

So the older code takes half the time.

So maybe the debounce is the right solution, because I can't find a version where something similar happens automatically.

I also wonder why the drawing has gotten so slow, maybe the memoization in the app fails somewhere and it repeatedly tries to load data from the sqlite db.

Is the debounce the "right" way to do this?

Maybe someone has a clever solution to collapse the calls to refresh less manually.

I was under the impression refresh was doing that automatically but maybe at a different granularity that isn't working for my app/wanted performance.

Now I also tested old app code with new racket and the draw is around 23ms there too.

So the extra slowness seems to be a problem in the application logic introduced in the new code.

And it seems that it just made it slow enough to make me notice that a debounce is useful for snappier user-feedback.

So the next step for me is to look at the differences between the old and new code.

[schle...@gmail.com]

The difference between 23ms and 50ms was that the memoization wasn't doing its job.

I removed some calls to get-brush and set-brush, only calling set-brush when the color changes. (only writing the value not reading it)

Specialized a function that returned an ignored argument, to not calculate that argument.

With that I got to 10-19 ms, together with the debounce it now is very snappy again, so I am satisfied.

The debounce often collapses 10-30 mouse-events/on-paint calls into one.

The code could be changed to precompute more stuff (needing more memory) or only redraw the region close to the cursor, but that would take to much programmer time for now.

[Alex Harsányi]

On Saturday, May 22, 2021 at 12:42:30 PM UTC+8 schle...@gmail.com wrote:

Maybe someone has a clever solution to collapse the calls to refresh less manually.

I experienced a similar problem as you, where I have to refresh a canvas on a mouse event and the paint method is somewhat slow (it does have a lot of drawing to do in my case, so I expect it to be slow).  I am not sure if this is a regression in the draw performance, but it seems that the canvas% will un `on-paint` for each `refresh` call it receives (unless I am missing something).  This means that sending a refresh on mouse events will result in many unnecessary redraws of the canvas.

Not sure if my solution is "clever", but it involves setting a flag on the first refresh and only clearing after the draw operation is completed.  The internal state is still updated for each mouse event, but refresh is only called on the canvas if the flag is not set (that is if there is no outstanding refresh call).  This way, there is no unnecessary delay for the refresh, and the on-paint always draws the latest version of the internal state, which may have been updated several times between the time refresh was initially called.

Also, if you choose to use a timer% object, you don't need to create one each time, they can be reused.

Alex.

[schle...@gmail.com]

I tried your flag version too, I like it, it works without delay and is simple.

But theoretically you can have situations where the drawn output doesn't fully represent the current state (if some things were already done and the data was changed afterwards).

Unless I misunderstand your solution.

Practically that is often not an issue.

I think for my use case I prefer the timer solution, because although it has delay, that delay can be tweaked as a tradeoff between delay and cpu use.

And I can tweak it so that it feels snappy and racket doesn't go to 100% cpu use, drawing frames I don't perceive.

It also always does a full redraw eventually after things have changed.

(not really relevant for that mouse event case though, but if I were to draw background color based on mouse position it might be)

The timer code is based on other old autosave code:

I think initially I reused the timer, but then needed another flag to know whether the timer is already running.

Also I didn't want the timer to hang around for a bunch of text fields which get modified rarely and its existence doubles as flag whether it is running.

Actually for autosave textfields that part is very important that always the latest version is saved.

So many different tradeoffs possible ;)

Thank you, for your perspective!

Simon

[Jens Axel Søgaard]

Den lør. 22. maj 2021 kl. 05.37 skrev schle...@gmail.com <schle...@gmail.com>:

I have a racket gui app that uses a canvas% with overridden on-event and on-paint methods.

When the user hovers over drawn elements the on-paint is called via (send this refresh)

to display the element under the cursor with a selection/outline.

Recently I noticed that this has gotten extremely slow.

It seems to me this might be a difference between BC and CS, but I haven't checked with different versions in depth yet. (just from the behavior/performance I remember it had in the past)

In the past a call to (send this refresh) seemed to be processed concurrently in regard to on-event.

Now it seems like the first (send this refresh) eagerly triggers the on-paint and this on-paint somehow blocks the processing of on-event until the on-paint is finished, after that 1 more mouse event is processed re-triggering the on-paint.

Effectively redrawing for every mouse event, causing the app to draw old uninteresting frames (because the mouse events aren't processed fast enough and only the last position is interesting for me).

I was looking at the code for racket/draw and spotted this:

    ;; The Racket BC can handle concurrent callbacks in different Racket
    ;; threads, because it copies the C stack in and out to implement
    ;; threads. The Racket CS cannot do that, so callbacks have to be
    ;; atomic. At the same time, we need some atomic callbacks to be able
    ;; to escape with an exception.

It matches your observations.

https://github.com/racket/draw/blob/master/draw-lib/racket/draw/unsafe/callback.rkt

/Jens Axel
 

[schle...@gmail.com]

Actually the flag version does not work for me.

(I thought it did because on-paint was fast enough not to be noticed that it was sequentially processing every mouse event with a paint call)

#(struct:v2 109 80) ;; mouse pos
draw  ;; beginning of draw
cpu time: 8 real time: 8 gc time: 0  ;; duration / end of draw
#(struct:v2 110 66)
draw
cpu time: 9 real time: 9 gc time: 0
#(struct:v2 110 64)
draw
cpu time: 8 real time: 8 gc time: 0
#(struct:v2 110 62)
draw
cpu time: 8 real time: 8 gc time: 0
#(struct:v2 112 61)
draw
cpu time: 9 real time: 9 gc time: 0
#(struct:v2 126 42)
draw
cpu time: 8 real time: 8 gc time: 0
...

The debounce works because it creates a time-window in which on-paint isn't running allowing all mouse events that arrive in that window to be processed before on-paint blocks processing of on-event.

#(struct:v2 51 371)
#(struct:v2 49 364)
#(struct:v2 44 346)
#(struct:v2 44 344)
#(struct:v2 44 343)
#(struct:v2 44 341)
#(struct:v2 42 340)
#(struct:v2 42 338)
#(struct:v2 42 336)
#(struct:v2 42 334)
draw
cpu time: 8 real time: 8 gc time: 0
#(struct:v2 42 329)
#(struct:v2 42 324)
#(struct:v2 42 319)
#(struct:v2 42 298)
#(struct:v2 42 297)
#(struct:v2 42 295)
#(struct:v2 42 294)
#(struct:v2 44 292)
#(struct:v2 44 290)
draw
cpu time: 8 real time: 8 gc time: 0
#(struct:v2 44 285)
#(struct:v2 45 282)
#(struct:v2 47 279)
#(struct:v2 50 270)
#(struct:v2 51 270)
draw
cpu time: 8 real time: 8 gc time: 0
#(struct:v2 51 269)
#(struct:v2 52 268)
draw
cpu time: 8 real time: 8 gc time: 0

...

[sidenote: the difference to yesterdays timings is that more other stuff was running on my box, I think]

The debounce makes it work somewhat similar to a "traditional single threaded game-eventloop", the difference being that the game-loop would process the queue of n events that arrived for that frame until it is empty and then do other calculations and finish the frame by rendering, while the debounce just specifies a fixed time for event and other processing before rendering takes place.

The mouse event can't run while on-paint runs and the window event priority ensures that on-paint always has a higher prio than mouse-events, this means that we can only process multiple mouse-events in one batch when they don't immediately add a on-paint/refresh event: https://docs.racket-lang.org/gui/windowing-overview.html?q=eventspace#%28part._.Event_.Types_and_.Priorities%29

I constructed a minimal example, while playing around with it I found a variant of flag that works for me:

Instead of resetting the flag at the end of on-paint, I use queue-callback to add a low priority callback that resets it,

because of event-handling priorities this means that the mouse events can be processed before the flag is reset. (I think this my new favorite solution for a lot of mouse move cases)

https://gist.github.com/SimonLSchlee/1a748b5a93b86aea2fc15045cad2be50

The example lets you switch between the 3 modes I have currently added, it is a canvas that changes the color of a grid of rounded rectangles based on mouse position,

it also prints mouse position and draw timings to the console.

Simon