Flutter windows custom game loop rendering

[michael.s.humphreys]

I posted this issued on stackoverflow a couple of weeks ago and still need help with it. I'm hoping to use flutter for a windows desktop game but my experiments show some janking in a very pared back game loop . The question on stackoverflow includes example code for flutter, for flutter using the flame package, and for a javascript version. The javascript version runs perfectly smoothly but the two flutter versions don't and I can't figure out if the reason is something in my code or something in the flutter framework. Any help would be much appreciated.

https://stackoverflow.com/questions/65066393/flutter-desktop-custom-game-loop-appears-to-drop-frames

[Stefan Matthias Aust]

Did you test a debug build or a release build? The latter might reduce the janking.

I run you demo on macOS and it looks mostly fine. I think, you shouldn't ignore the Duration passed to the game loop function but I added this code to detect missed frames and there are none. Still, it seems that from time to time the movement stutters.

Duration last;

void gameLoop(Duration now) {

  if (last == null) last = now;
  if ((now - last) > Duration(milliseconds: 17)) print(now - last);
  last = now;
  ...

Therefore, I'd guess that the lower level Windows specific problem.

Assuming that Flutter animations work as expected, I'd recommend to redo you example using a Flutter AnimationController and a Stack with a Positioned Image. If that doesn't show jank, it is worth looking into your low level code, again. Otherwise, it's out of your control. Unfortunately.

BTW, I wrote this article -> https://medium.com/icnh/flutter-without-flutter-15177c91d066 demonstrating a very similar approach and I also wrote this code -> https://dartpad.dev/10f5a01a8d16e24e5814c703b6e374d4? to use a RenderBox as a kind of middle ground between Flutter widgets and SceneBuilders which is much easier to integrate with Flutter. Perhaps that approach better works for you.

Stefan

On Thu, Dec 24, 2020 at 7:05 PM michael.s.humphreys <michael.s...@gmail.com> wrote:

I posted this issued on stackoverflow a couple of weeks ago and still need help with it. I'm hoping to use flutter for a windows desktop game but my experiments show some janking in a very pared back game loop . The question on stackoverflow includes example code for flutter, for flutter using the flame package, and for a javascript version. The javascript version runs perfectly smoothly but the two flutter versions don't and I can't figure out if the reason is something in my code or something in the flutter framework. Any help would be much appreciated.

https://stackoverflow.com/questions/65066393/flutter-desktop-custom-game-loop-appears-to-drop-frames

--
You received this message because you are subscribed to the Google Groups "Flutter Development (flutter-dev)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flutter-dev/f2a879d6-93b8-422d-8190-21e04efd1b76n%40googlegroups.com.

[michael.s.humphreys]

Thanks Stefan. I've been running into this problem with release builds. I agree with using the duration passed to the gameloop function but I don't think this is the source of this problem that I have because the javascript version doesn't use the timestamp and it runs noticeably smoother. I did not know how to implement your suggestion of using a Flutter AnimationController and a Stack with a Positioned Image  but I did adapt the dartpad code you posted which runs with the same slightly janky motion in my sprite image version.

[michael.s.humphreys]

One other thought - the javascript version runs buttery smooth on Windows using window.requestAnimationFrame. I assume Flutter harnesses this under the hood?

[michael.s.humphreys]

I added a console print line to print the difference in milliseconds between each frame measured using the timestamp in the loop callback and also independently using a DateTime.now() timestamp which is taken at the start of the loop callback. Interestingly the loop timestamp delta is usually a millisecond or two more than the now() timestamp delta but then once every 11 to 15 frames the loop timestamp delta is 14 or 15 milliseconds shorter than the now() timestamp delta. Not sure if this is relevant to the problem I'm having but it seems to suggest that each of these odd frames there is something happening after the loop timestamp is made that takes about 14 milliseconds until the loop callback is entered. Perhaps garbage collection?

[Stefan Matthias Aust]

On Fri, Dec 25, 2020 at 3:23 PM michael.s.humphreys <michael.s...@gmail.com> wrote:

I did not know how to implement your suggestion of using a Flutter AnimationController and a Stack with a Positioned Image

I meant something like this: https://dartpad.dev/c6e9e26e33b1d3b78562d7329fd9275c 

This looks smooth when compiled as a macOS application.

Regarding your other mails: I don't think that garbage collection will cause the jank, but you can check by running a profile version of your application using the dev tools and tracing the GCs on the tool's logging view. I agree that a Browser canvas-based implementation is smooth because of requestAnimationFrame which the browser is supposed to call each time it is ready to display something, is still double-buffered and doesn't directly paint. I never looked into the way the Flutter engine draws something but my guess is that it isn't double-buffered in the same way but simply asks the platform to display the recorded Skia commands.

Here's one additional observation: If I run the application twice, both animations are in sync. If I however grab one application and move it around, that application shortly stops animating, probably because of how the platform window triggers a screen redraw. If I remember correctly, macOS wants all screen drawing happen in the main UI thread which automatically gets a higher priority when the user is interacting with the window so that everything is smooth, but Flutter runs in a background non-UI thread by default which might then get a lower priority and drops frames. But that's pure guess work.

Stefan

[michael.s.humphreys]

Thanks Stefan

I ran your code in dartpad (Chome on Windows) and it was perfectly smooth. Then I loaded the same code into IntelliJ and ran 'flutter build windows' and ran the .exe file which produced the same output but does not look smooth to me. Perhaps it is the lack of double-buffering like you suggest.

Is there anyone from the Dart/Flutter team who could comment on this please?

Michael

[Stefan Matthias Aust]

There's one thing, you might want to try. According to some YT video I recently watched, clearing the whole canvas as shown in this line is important for a smooth animation:

https://github.com/flutter/flutter/blob/master/examples/layers/raw/canvas.dart#L26

I don't know whether that's cargo cult programming or a worthwhile tipp but it might be worth the try.

Stefan

PS: https://www.youtube.com/watch?v=uefxeR23R_E

--
You received this message because you are subscribed to the Google Groups "Flutter Development (flutter-dev)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/flutter-dev/141a013c-db10-4946-94c6-63d48a76cf42n%40googlegroups.com.

[michael.s.humphreys]

Thanks Stefan. There is something interesting here. The code from the video is at codepen. It runs beautifully smoothly. But if I remove the line c.drawPaint(Paint()..color=Color(0)); then it runs with a lag. This is what is said in the video as well. It is hard to understand because Color(0) is transparent so shouldn't have any effect in a newly created canvas. 

When I build the code for Windows Desktop it runs the same regardless of whether that line is in or out, but not as smoothly as it does in the browser.

[Stefan Matthias Aust]

On Wed, Dec 30, 2020 at 5:00 PM michael.s.humphreys <michael.s...@gmail.com> wrote:

Thanks Stefan. There is something interesting here. The code from the video is at codepen. It runs beautifully smoothly. But if I remove the line c.drawPaint(Paint()..color=Color(0)); then it runs with a lag. This is what is said in the video as well. It is hard to understand because Color(0) is transparent so shouldn't have any effect in a newly created canvas. 

Well, Flutter Web surely uses a canvas and requestWindowFrame and therefore benefits from the superior Browser implementation.

When I build the code for Windows Desktop it runs the same regardless of whether that line is in or out, but not as smoothly as it does in the browser.

Looking at https://github.com/flutter/engine/blob/master/examples/glfw/FlutterEmbedderGLFW.cc#L100 it looks like the platform independent part of the Flutter engine has a callback function that is called once the flutter engine wants the platform shell to display a new frame. As far as I know, the quoted code is a very basic example how to embed the Flutter engine, using GLFW as an example. This is an OpenGL-based library I don't know much of, but looking at the documentation, the quoted line is the command to swap display buffers, that is this is the "usual" double buffering approach to game engines. Depending on the default configuration of GLFW, this swap occurs during vsync – I don't know. There is a glfwSwapInterval() function that can be used to configure this, but that function isn't used in the example.

I tried to find the same function on the macOS platform code (because I don't know anything about Windows and at least a little bit about macOS) here https://github.com/flutter/engine/blob/fff415d517d641a9e750d4a2c7e643262e130068/shell/platform/darwin/macos/framework/Source/FlutterEngine.mm#L139 and here https://github.com/flutter/engine/blob/a713174aa388a72cd1aeaf0fccfeaf0b32c1bc7d/shell/platform/darwin/macos/framework/Source/FlutterEngine.mm#L543 which calls "[_resizeSynchronizer requestCommit];" which does strange things. It uses mutexes to do stuff, probably to make sure that normal screen updates don't interfere with updates caused by the user resizing the window (something which cannot happen on phones). This code can throw away frames which eventually might cause stuttering animations. At least on macOS, this also schedules the screen update on the window's main thread which might cause further delays.Then https://github.com/flutter/engine/blob/be7f80efc090a0e6cf3f1e3527b326c5435becc7/shell/platform/darwin/macos/framework/Source/FlutterView.mm#L57 is called. This does some low-level pointer switching but I don't see vsync'ing this operation. Perhaps the macOS CoreAnimation framework guarantees this, but I'm not sure. My feeling is that OpenGL is even more low level.

BTW, the Windows code is here https://github.com/flutter/engine/blob/8ceb3ed4baf3922cc3e6cb41cb9d028bc6c160a1/shell/platform/windows/flutter_windows_engine.cc#L46

Stefan

[michael.s.humphreys]

For those interested, I posted an issue here: Flutter issue

[Benedicte Roussel]

Maybe a question of frame rate?

'

30p and 60p: frame rate used in particular in the United States and Japan on TV channels and camcorders, in countries which used the NTSC system in analog. But this rate is in fact also very present in all the countries of the world, on the computer equipment which is often produced by countries which inherit the NTSC, we therefore also find this rate everywhere in the world on the screens of PCs, smartphones, tablets, and as a standard for video games.

--

You received this message because you are subscribed to the Google Groups "Flutter Development (flutter-dev)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flutter-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/flutter-dev/d6e54bcd-b088-48e9-9c3a-f1e11f1ef951n%40googlegroups.com.

[Benedicte Roussel]

see also this discussion about delta

Javascript time-based animation and requestAnimationFrame

Javascript time-based animation and requestAnimationFrame I have been playing around with canvas and animation, with HTML5 games in mind specifically and quickly learnt t...

Now you know I jumped into your post most by curiosity and do not know the subject, then your dartpad is full of null safety problems, so maybe start by that.

To view this discussion on the web visit https://groups.google.com/d/msgid/flutter-dev/250360888.4296624.1614274337934%40mail.yahoo.com.

[michael.s.humphreys]

Thanks @Bened. I'm running on a gaming laptop with 144 hz refresh rate but as you can see from the issue I filed the flutter team was able to replicate the problem on their set up. Thanks for the link to requestAnimationFrame but the javascript version I'm running doesn't show the lag/stutter, only the Flutter version so I don't think it helps. Re the null safety problems, there is a slider setting on dartpad at the bottom to switch null safety on or off. If switched off it shows no errors. However I see that the dartpad code doesn't seem to run for me (although it used to ) so I need to investigate this.

[Bened]

Thank you @Michael for your explanation;

Running -d windows and -d chrome I could not see any difference, maybe I am blind.

I wanted to share my release files (Release.zip) 

NB: I did not add any  Visual C++ redistributables as I can open the executable on my pc and as I am not sure to understand the doc (this one)

BUT my zip has been rejected:

Sorry, we were unable to deliver your message to the following address.

<flutt...@googlegroups.com>:

552: 5.7.0 This message was blocked because its content presents a potential

5.7.0 security issue. Please visit

5.7.0  https://support.google.com/mail/?p=BlockedMessage to review our

5.7.0 message content and attachment content guidelines. m17si606159lfg.0 - gsmtp

so I am sending it to your mail if that can help

[Benedicte Roussel]

well, the same with your Gmail address, so if you need it it will be through we share or ... if you do not need it just skip :), 

That must be a Gmail policy:

<michael.     re...@gmail.com>:

552: 5.7.0 This message was blocked because its content presents a potential

5.7.0 security issue. Please visit

5.7.0  https://support.google.com/mail/?p=BlockedMessage to review our

5.7.0 message content and attachment content guidelines. f14si17058495wrt.423 - gsmtp

Nevertheless, behaviours on my sides are the same in the browser and in the desktop version. So maybe it comes from your build

https://groups.google.com/d/msgid/flutter-dev/ee031474-2837-4f22-b349-4326c540044cn%40googlegroups.com

.

[Bened]

Anyway and for information, the exe files transfer will expire on March 7, 2021, I did not send it through a pro option

[Jitendra Kumar Kumawat]

Hello, My Dear Friend! Please like and subscribe to this youtube channel! Here you learn : 1: Creating a Modern Promo Video in After Effect 2: Modern Mobile Application Development in Flutter  

3: Modern Web App with React-Hooks. 4: Modern Video Editing 5: Principle of Animation 6: Creating JavaScript Games. https://youtu.be/cWnMXlFL_Qs

To view this discussion on the web visit https://groups.google.com/d/msgid/flutter-dev/9cc7307f-c05a-4f54-a3c3-164cc5d19e4fn%40googlegroups.com.