Ram usage keep increasing when using FXImageSink

[Alvin]

Hello Neil,

While I use this library, I noticed something wrong with it, when I use FXImageSink from your examples. It show the streams, however after I leave it for 1 hour, the ram usage gradually increased. So I tried your example and I found similar issue. But if I comment out the 
imageSink.requestFrameSize(640, 480);
Then the ram usage will not growing and stay the same.

Here is the ram usage before, it showing 1.36 G when I start the jar, you can see the res column shows 279M


Here is the ram usage after I leave it for a while, maybe around 15 - 30 minutes, the res is increase into 310M


If this keeps up, then after the ram usage almost full, it will go to swap memory and after the swap full. the pc will freeze and I have to hard restart it.
I also test the gstreamer pipeline to make sure if the bug is from gstremer itself. But the ram usage is not increasing.
Do you have any Idea why the ram usage keep growing? Thank you

My specification
- Ubuntu 18.04 LTS
- Gstreamer 1.14.5
- JavaFx 13.02
- Java Liberica 13

This the library that I use to implement gstreamer in java
- https://github.com/gstreamer-java/gst1-java-fx
- https://github.com/gstreamer-java/gst1-java-core

[Neil C Smith]

On Wed, 1 Dec 2021 at 09:33, Alvin <idea.vi...@gmail.com> wrote:
> While I use this library, I noticed something wrong with it, when I use FXImageSink from your examples. It show the streams, however after I leave it for 1 hour, the ram usage gradually increased. So I tried your example and I found similar issue. But if I comment out the
> imageSink.requestFrameSize(640, 480);
> Then the ram usage will not growing and stay the same.

> ...

> I also test the gstreamer pipeline to make sure if the bug is from gstremer itself. But the ram usage is not increasing.
> Do you have any Idea why the ram usage keep growing? Thank you
>
> My specification
> - Ubuntu 18.04 LTS
> - Gstreamer 1.14.5
> - JavaFx 13.02
> - Java Liberica 13

I cannot replicate your issue on current Ubuntu 20.04 dev machine. I
did see a memory leak on an older machine that I was first developing
this on, but that took about a minute to max out memory and crash the
machine. This was caused by an interaction between JavaFX and the GPU
driver. I'd have to look back through some email archives to find the
exact trigger though.

What's your GPU?

Are you using a JDK with JavaFX included? If so, don't - use the
mechanism from the examples to include JavaFX. I would also try with
updated versions of both JavaFX and JDK (although you'll need to
update the Gradle wrapper in the example for JDK 17 - working on that)

Best wishes,

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD

[Alvin]

> Are you using a JDK with JavaFX included? 

Yes, for now I'm using Java Liberica since it also included javafx.

> I cannot replicate your issue on current Ubuntu 20.04 dev machine.

what version of gstreamer that you're using? as far as I know, the latest gstreamer version in ubuntu 20.04 is 1.16.x,
Can you try in your pc with ubuntu 18.04 to reproduce it? Since the latest gstreamer version in that ubuntu is 1.14.5

> What's your GPU?
Here is my current pc CPU, since I only have GPU onboard right now. I also use 4 GB ram

Here is my current pc right now. after I left it for around 2 - 3 hours

As you can see, the res is already grow into 388M, if I left it for one days, usually I already see it around 750 - 1000M before it going to swap.
It just bothering me, why after I comment this code
imageSink.requestFrameSize(640, 480)

the ram usage just stable? Are appsink with videoconvert and videoscale will cause native leak? I just don't understand about that,
Thank you

[Neil C Smith]

On Wed, 1 Dec 2021 at 12:40, Alvin <idea.vi...@gmail.com> wrote:
> > Are you using a JDK with JavaFX included?
> Yes, for now I'm using Java Liberica since it also included javafx.

As I said, please try with a JDK without JavaFX. The JavaFX media
module has a cut-down version of GStreamer inside it. Rule that out
as the problem.

> > I cannot replicate your issue on current Ubuntu 20.04 dev machine.
> what version of gstreamer that you're using? as far as I know, the latest gstreamer version in ubuntu 20.04 is 1.16.x,
> Can you try in your pc with ubuntu 18.04 to reproduce it? Since the latest gstreamer version in that ubuntu is 1.14.5

Sorry, I no longer have easy access to a machine with 18.04, and I
cannot put free time into setting this up for testing.

> the ram usage just stable? Are appsink with videoconvert and videoscale will cause native leak? I just don't understand about that,

Check if the GStreamer VAAPI package is installed. Possibly uninstall
and try without that and see if that helps. Certainly I've seen
quirks with Intel GPU and VAAPI.

[Alvin]

> As I said, please try with a JDK without JavaFX. The JavaFX media
> module has a cut-down version of GStreamer inside it. Rule that out
> as the problem.

Well I also make some mistakes before by combining gstreamer with javafx mediaplayer, causing it to sudden force close. So when I implement this library, I also use your FXPlayer as my current Mediaplayer.

>  Sorry, I no longer have easy access to a machine with 18.04, and I
> cannot put free time into setting this up for testing.

> It's okay, I will try in my pc for now.

> Check if the GStreamer VAAPI package is installed. Possibly uninstall
> and try without that and see if that helps. Certainly I've seen
> quirks with Intel GPU and VAAPI.

Yes, I also have this problem for a while. In my experience, because of vaapi plugin, once the gstreamer try to load, my java program will suddenly crash. I already uninstalled the vaapi plugin before I test the ram usage.


For now, I'll try using openjdk 13 to see if the ram still increasing or not. Anyway, thank you for you help.

Alvin

[Alvin]

Hello Neil,

I already use openjdk 13 with javafx 13 from gluonhq for your example project again and when I did it, the ram still growing.
I also use ubuntu 20.04 LTS with latest gstreamer, 1.16

Here is the screenshot of the htop
This is when I just run the program


This is after I left it for 3 - 4 hours


Here is the gstreamer version that I used for the test


This is the example that I used
https://github.com/gstreamer-java/gst1-java-examples/blob/master/FXCamera/src/main/java/org/freedesktop/gstreamer/examples/FXCamera.java

Like I said before, If I comment this line
imageSink.requestFrameSize(640, 480);
The ram usage will stable.

So, is that because of the javafx, or the jni that cause the ram increasing? Thank you


Alvin

[Neil C Smith]

On Tue, 7 Dec 2021 at 17:02, Alvin <idea.vi...@gmail.com> wrote:
> I already use openjdk 13 with javafx 13 from gluonhq for your example project again and when I did it, the ram still growing.

Please try without OpenJDK 13 and definitely without the JavaFX SDK on
the module path.

Use OpenJDK 11 (ideally from a source other than the distro - eg.
Adopt or Zulu).

Run the example with ./gradlew run

The Gradle build uses the JavaFX plugin to pull in only the required
dependencies automatically.

Also, are you really running this as root? Again, try without doing this.

If the symptoms are still replicable under those conditions, please
add as much hardware details as you can. And if possible try with
alternative (or no) input device. Also try running VisualVM against
the running process and see if you can figure out where the memory is
in use - Java or native - and try manually triggering GC and seeing if
that helps.

Best wishes,