FXImageSink memory efficiency improvements

7 views
Skip to first unread message

Michael

unread,
Dec 31, 2023, 7:56:04 PM12/31/23
to gstreamer-java
Hello,

A suggested efficiency improvement for FXImageSink - happy to submit a PR if this makes sense.

At present FXImageSink is creating a fresh image each frame: https://github.com/gstreamer-java/gst1-java-fx/blob/8e603842be93b4f2be9e059dedb0fdd3c2245601/src/main/java/org/freedesktop/gstreamer/fx/FXImageSink.java#L201

While this generally works ok enough from a raw performance perspective, it means the old WritableImage is discarded each frame, which then needs to be garbage collected. This can end up forcing frequent, full garbage collections in some cases which then causes frequent stutters in video playback (particularly with higher resolution 4k videos, especially when there's not an abundance of memory available.)

What's working far better for my use case is this instead:

if (image.get().getWidth() != width || image.get().getHeight() != height) {
image.set(new WritableImage(width, height));
}

((WritableImage) image.get()).getPixelWriter().setPixels(0, 0, width, height, PixelFormat.getByteBgraPreInstance(), activeBuffer.map(false), width * 4);

...which will only create a new WritableImage if there's dimension changes, and otherwise will just use the PixelWriter on the existing image, which means it's not discarding one each frame.

If I've missed anything that means the above isn't viable though, very happy to take any criticism!

Many thanks,

Michael
Reply all
Reply to author
Forward
0 new messages