GStreamer screen capture and possible memory leak

[Davide Perini]

Hi all,
I'm using the mighty GStreamer Java to capture the screen.

I have a feature in my software that let you start and stop the screen
capture.

Pipeline is started in this way:
https://github.com/sblantipodi/firefly_luciferin/blob/d2b7ef45ea2d817c6330fc1fd26eb73e171c6cfe/src/main/java/org/dpsoftware/grabber/GrabberManager.java#L87

this is the main class for GStreamer:
https://github.com/sblantipodi/firefly_luciferin/blob/master/src/main/java/org/dpsoftware/grabber/GStreamerGrabber.java

and this is the way how I stop the screen capture:
pipeline.stop();

I noticed that if I start and stop the pipeline continuosly, every time
I start it, my software sucks 20MB of RAM more.
If I do it many times, the software sucks a lot of RAM.

Is this "a memory leak" somewhere?
Am I stopping the screen recording correctly? Should I set some object
to null when I stop the pipeline?

Thanks
Davide

[Neil C Smith]

On Wed, 8 Feb 2023 at 10:06, Davide Perini <perini...@dpsoftware.org> wrote:
> I noticed that if I start and stop the pipeline continuosly, every time
> I start it, my software sucks 20MB of RAM more.
> If I do it many times, the software sucks a lot of RAM.
>
> Is this "a memory leak" somewhere?
> Am I stopping the screen recording correctly? Should I set some object
> to null when I stop the pipeline?

Are you creating a pipeline each time? If so, and if you can, reuse
the existing pipeline.

This is kind of the expected behaviour. I assume the Pipeline
reference is overwritten or set to null? The problem is likely that
the native resources will not be released until the Java garbage
collector runs. And GStreamer native resources don't impact the Java
heap, so you can use up a lot of off-heap resources before the garbage
collector runs.

All native objects in the bindings can also be explicitly disposed.
One catch with that with Pipeline at the moment is that there might be
messages queued on the Bus that need to complete after calling stop().

You could look at doing (think this is correct - not at a dev machine) -

pipe.stop();
Gst.invokeLater(pipe::dispose);

This works in most cases by adding the disposal at the end of the
queue of bus messages. We should possibly change the behaviour of
dispose() on pipelines to do something like this automatically.
However there are some edge cases to consider.

I might have missed where the ScheduledExecutorService comes from in
your code. If it's not Gst.getExecutor() I would look at using that
instead - running things on that executor is recommended in most
cases.

Best wishes,

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD

[Davide Perini]

Hello Neil,

nice to hear from you again, hope you are doing well. :)

Thank you for taking the time to answer me, I really appreciate it.

I tried all your suggestions and it works like a charm, as always.

I tried disposing the pipeline and all the related objects along with using the Gst.getExecutor().

I noticed that disposing the pipeline helps but disposing all the related objects before disposing the pipeline solves the memory leak completely.

Thank you once more Neil.

Kind regards,
Davide

[Neil C Smith]

Hi,

On Thu, 9 Feb 2023 at 22:40, Davide Perini <perini...@dpsoftware.org> wrote:
> Thank you for taking the time to answer me, I really appreciate it.
>
> I tried all your suggestions and it works like a charm, as always.

You're welcome! Good to know.

> I noticed that disposing the pipeline helps but disposing all the related objects before disposing the pipeline solves the memory leak completely.

Yes, disposing any elements you've created when you're done with the
pipeline ensures fast resource cleanup. You can also dispose of
elements immediately after adding to the pipeline if you don't need to
retain a reference and they have no listeners - that clears the Java
reference, but not the native one while the pipeline is active.

All native objects are AutoCloseable too, so you can use them with
try-with-resources.

I've been working on an idea for memory scopes that would dispose of
all elements related to a pipeline automatically, and handle the bus
cleanup correctly. Unfortunately my time for the library is very
limited at the moment as all our current client work is on other
things. Need to find some free time!

[Davide Perini]

Very interesting, thanks for the explanation Neil.

Your idea would be awesome for a future update but I understand that it
requires time.
I don't think that it's that urgent, the lib works awesome as it is for now,
future updates will be the icing on the cake :)

Best wishes you too :)

Bye
Davide