How to shutdown the pipeline (safely). ( Wait all native resources complete successfuly )

[Tiago Frazao]

Hi Everyone,

I have an issue to shutdown the pipeline safely.
If the user clicks quickly on the same button (start/stop) some threads are not disposed (GstBus), causing retention of devices (camera) permanently.
As a workaround I had to include a "random sleep of 2 seconds" before a trigger pipeline.dispose(). However, it does not look  a good solution, so I decided to ask for help.

Thanks everyone in advance.

Details > 

I have a class "GstStreammerPipeline" that contains (basically) 3 methods :
   1 - play() -- // start the pipeline.
   2 - stop() -- // stop the pipeline (problematic)
   3- termination() // This one is used as a signal to proceed with pipeline.dispose when a EOS is received.

public class GstStreamerPipeline {

  public GstStreamerPipeline(String pipelineDescription) {

     this.pipeline = (Pipeline) Gst.parseLaunch(pipelineDescription);
     this.terminationSignal = new CountDownLatch(1);
     this.pipeline.getBus().connect((Bus.EOS) (e) -> termination());
  }

  public synchronized void play() {
    this.pipeline.ready();
    this.pipeline.getState();
    this.pipeline.play();
    this.pipeline.getState();
    logger.debug("[{}] - Pipeline has started", uuid);
  }

  public synchronized void stop() {

    if (stopped)
      return;
   
    this.pipeline.sendEvent(new EOSEvent());
    this.pipeline.postMessage(new EOSMessage(pipeline));
    this.pipeline.postMessage(new EOSMessage(pipeline.getBus()));

    try {

       // Wait for the termination signal.
       terminationSignal.await();
       if (!stopped) {
         // TODO : Workaround : I have to add a hardcode "waiting 2 seconds", otherwise Some threads (GstBus) stucks forever and lock the camera device).
         // When this happens, I have to kill the application to release the resources.
         // BusError => Could not open audio device for recording. Device is being used by another application.

         // Work around ==> Thread.sleep(2000);
         pipeline.stop();
         // pipeline.dispose() # also does not work
         Gst.invokeLater(pipeline::dispose);
         stopped = true;

       }

       logger.debug("[{}] - Pipeline has stopped", uuid);
    } catch (InterruptedException e) {
       logger.warn("[{}] - Pipeline stop was interrupted", uuid);
    }
  }

  public void termination() {
    // Release termination signal.
    terminationSignal.countDown();
  }

}

1 - All GstBus Daemon Threads were created on the first click.
2 - I couldn't find any way to "release" these threads.
3 - It looks like java disposes the pipeline without waiting for "confirmation" from the native gstreamer.

[Neil C Smith]

On Wed, 7 Jun 2023 at 09:58, Tiago Frazao <tiago....@gmail.com> wrote:
> Hi Everyone,

Hi, and welcome! No need to post twice - first posts are moderated -
all messages will go through now.

> public GstStreamerPipeline(String pipelineDescription) {

If your project is open-source, share a link to it, or at least more
information about the pipeline and surrounding code.

If it's closed source, you can email me off list for support.

Few comments ....

> this.terminationSignal = new CountDownLatch(1);
> ...
> public synchronized void play() {

The use of synchronized and CountDownLatch suggests you're not
interacting with GStreamer using the library's executor
(Gst::getExecutor / Gst::invokeLater).

It is best to treat this like you would for example the EDT in Swing -
do all interaction after Gst::init through the executor.

> this.pipeline.sendEvent(new EOSEvent());
> this.pipeline.postMessage(new EOSMessage(pipeline));
> this.pipeline.postMessage(new EOSMessage(pipeline.getBus()));

This is odd!

Does your pipeline actually require an EOS to shutdown cleanly? Just
post the event and wait for the EOS to propagate through the streaming
thread(s) by listening on Bus.EOS.

Directly posting the message on the pipeline or bus means your
listener is probably not doing what you think it's doing!

> // TODO : Workaround : I have to add a hardcode "waiting 2 seconds", otherwise Some threads (GstBus) stucks forever and lock the camera device).
> // When this happens, I have to kill the application to release the resources.
> // BusError => Could not open audio device for recording. Device is being used by another application.
>
> // Work around ==> Thread.sleep(2000);
> pipeline.stop();

GstBus threads aren't the issue. It's possible this is due to not
waiting correctly on the EOS. It's possibly something to do with how
the GStreamer element or OS closes the device, and whether it's
synchronous. Hard to say in the abstract.

> // pipeline.dispose() # also does not work
> Gst.invokeLater(pipeline::dispose);

You can usually let the JVM garbage collect the pipeline for you, and
the library will clean up. If you do need to dispose manually for
some reason, then running in invokeLater after stop is (currently)
required. If you understand how GStreamer works, and what calling
stop does, then you might realise why that's currently the case.
Calling stop will cause messages to be queued on the bus - you cannot
dispose of the pipeline until after the messages have been processed.
Using invokeLater queues the disposal after messages posted during
stop.

There is a plan to allow scopes for objects that would simplify the
pipeline cleanup - that's currently pending on client work or
sponsorship for it though.

> 1 - All GstBus Daemon Threads were created on the first click.
> 2 - I couldn't find any way to "release" these threads.

These are Java side proxies for native GStreamer threads. Nothing to
do, nothing to release!

> 3 - It looks like java disposes the pipeline without waiting for "confirmation" from the native gstreamer.

What do you mean? You do need to make sure a reference to any
pipeline is kept on the Java side, or when it goes out of scope the
garbage collector will eventually dispose of it, including trying to
dispose on the native side. On the other hand, if you're talking
about calling pipeline::dispose, then this point is covered above.

Best wishes,

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD

[Tiago Frazao]

Hi Neil,

Thanks for all the information.

I applied all recommendations :

  1 - Let GC dispose the pipeline
  2- I have removed all synchronized methods (It was no longer necessary)

  3- I kept only  pipeline.sendEvent(EOS) and the terminate method just execute: {  this.pipeline.stop(); this.pipeline = null; }

You are right, It looks like the OS disposes the device asynchronously and when I try to start a new pipeline using the same /dev/video, then I get the error "Could not open audio device for recording. Device is being used by another application."

I will "catch" all errors registered in the pipeline and revert the UI button state to the initial state "Off", then the user can re-try.

Just 1 question,

You mentioned that It is necessary to use Gst Executor for all Gst operations. Eg. (Gst:invokeLater(pipeline:dispose))
Should I wrap all commands on Gst:invokeLater ? I am saying it because all existing examples on the documentation does not use "invokeLater" for all pipeline method calls. 

In case I really need to call all pipeline methods inside of invokeLater, is it also necessary for the following calls :

  1- this.pipeline.sendEvent(new EOSEvent());
 2- this.pipeline.getElementByName(sink).set(property, value);

 3- this.pipeline.ready();
 4- this.pipeline.getState();
 5- this.pipeline.play();
 6- this.pipeline.getState();
 7- this.pipeline.stop();
 8- this.pipeline.getBus().connect((Bus.EOS) (e) -> termination());

Best Regards
Tiago F. Frazao

--
You received this message because you are subscribed to the Google Groups "gstreamer-java" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gstreamer-jav...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gstreamer-java/CAPxOS5Eb0oboDPrV_gw_%2Bxt9LHcHHhxkkVV8V2TsiV%2BRa6L8eg%40mail.gmail.com.

[Neil C Smith]

On Wed, 7 Jun 2023 at 16:02, Tiago Frazao <tiago....@gmail.com> wrote:
> You are right, It looks like the OS disposes the device asynchronously and when I try to start a new pipeline using the same /dev/video, then I get the error "Could not open audio device for recording. Device is being used by another application."
> I will "catch" all errors registered in the pipeline and revert the UI button state to the initial state "Off", then the user can re-try.

That's annoying then! Maybe there's a way to poll it. You could also
schedule a number of retries using Gst.getExecutor() - it's actually a
scheduled executor.

> Just 1 question,
> You mentioned that It is necessary to use Gst Executor for all Gst operations. Eg. (Gst:invokeLater(pipeline:dispose))
> Should I wrap all commands on Gst:invokeLater ? I am saying it because all existing examples on the documentation does not use "invokeLater" for all pipeline method calls.
> In case I really need to call all pipeline methods inside of invokeLater, is it also necessary for the following calls :

It's recommended, if not strictly necessary. Bus messages are also
delivered on that executor, so it helps to keep things interleaved
correctly.

Yes, the examples probably should be updated with that in mind. Using
the main thread to set things up works OK, particularly before any
pipeline starts, but they kind of give the wrong impression of the
right approach! :-)