No video streaming with PlayBin

[Ahmet Yakar]

Hello guys,

I am beginner for Gstreamer. I am trying to show rtsp stream on JFrame but all I see a blank screen. Sound is coming but there is no video streaming. If I can achieve this with your help, my next step will be doing this by constructing a pipeline element by element. 

I am using;

Gstreamer 1.12.4

gst1-java-core 0.9.1

jdk1.8.0_144

on Wİndows 10 Pro.

import org.freedesktop.gstreamer.Gst;
import org.freedesktop.gstreamer.elements.PlayBin;


import javax.swing.*;
import java.awt.*;
import java.net.URI;
import java.net.URISyntaxException;




public class Test2 {


    private static SimpleVideoComponent vc;
    private static PlayBin playBin;


    public static void main(String[] args) {


        System.setProperty("jna.library.path", "C:\\gstreamer\\1.0\\x86_64\\bin");


        Gst.init();
        EventQueue.invokeLater(() -> {


            vc = new SimpleVideoComponent();
            playBin = new PlayBin("VideoPlayer");


            try {
                playBin.setURI(new URI("rtsp://10.0.30.66/stream2"));
            } catch (URISyntaxException e) {
                e.printStackTrace();
            }


            JFrame fr = new JFrame("Camera Test");
            vc.setPreferredSize(new Dimension(330, 240));
            vc.setSize(new Dimension(330, 240));
            playBin.setVideoSink(vc.getElement());
            playBin.play();


            fr.pack();
            fr.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
            fr.setVisible(true);
        });


        Gst.main();
    }
}

Thank you,

[Neil C Smith]

Hi,

On Wed, 13 Dec 2017, 09:12 Ahmet Yakar, <ahmety...@gmail.com> wrote:

I am beginner for Gstreamer. I am trying to show rtsp stream on JFrame but all I see a blank screen.

Have you setup GStreamer in the Windows path? Just setting the jna library path is not enough on Windows as GStreamer will not find its plugins. However, I'm surprised you hear anything.

Another option as you get sound is that you chose the typical (default) rather than complete GStreamer installation? On Windows the default installation leaves out some useful plugins. 

Setting the path up in the os is the easiest way to get started, but you can also do it using the JNA Platform library - see my code at https://github.com/praxis-live/praxis/blob/master/praxis.video.gst1/src/net/neilcsmith/praxis/video/gst1/components/GStreamerLibrary.java#L50

There are other GStreamer env variables you can set that work too - it's something we should document once we're sure which is the most reliable. 

Best wishes, 

Neil

--

Neil C Smith

Artist & Technologist

www.neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org

[Ahmet Yakar]

I set the enviroment variables and also add it to my source code as you see below, and I also reinstalled the gstreamer by selecting all modules one by one in Custom installation menu. It is still not working for me. I have been working for couple of days and now I am about to go crazy. What could be the problem?

import com.sun.jna.platform.win32.Kernel32;

import org.freedesktop.gstreamer.Gst;
import org.freedesktop.gstreamer.elements.PlayBin;


import javax.swing.*;
import java.awt.*;
import java.net.URI;
import java.net.URISyntaxException;




public class Test2 {


    private static SimpleVideoComponent vc;
    private static PlayBin playBin;


    public static void main(String[] args) {
        System.setProperty("jna.library.path", "C:\\gstreamer\\1.0\\x86_64\\bin");

        Kernel32 k32 = Kernel32.INSTANCE;
        k32.SetEnvironmentVariable("path", "C:\\gstreamer\\1.0\\x86_64\\bin");

        Gst.init();
        EventQueue.invokeLater(() -> {


            vc = new SimpleVideoComponent();
            playBin = new PlayBin("VideoPlayer");


            try {
                playBin.setURI(new URI("rtsp://10.0.30.66/stream2"));
            } catch (URISyntaxException e) {
                e.printStackTrace();
            }

            vc.setPreferredSize(new Dimension(330, 240));

            vc.setSize(new Dimension(330, 240));
            playBin.setVideoSink(vc.getElement());
            playBin.play();


            JFrame fr = new JFrame("Camera Test");

            fr.add(vc);

            fr.pack();
            fr.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
            fr.setVisible(true);
        });


        Gst.main();
    }
}

[Neil C Smith]

Hi,

Note that my code does not set jna library path, and adds the GStreamer path to the existing system path. Other path items may be necessary.

Try not setting the playbin video sink and seeing if the default video sink does anything.

Check any log output - you should be getting some error messages from GStreamer? Check the GStreamer docs for the env variables for controlling log output if you need more information - you can set them the same way you set path.

Best wishes,

Neil

--
You received this message because you are subscribed to the Google Groups "gstreamer-java" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gstreamer-jav...@googlegroups.com.
To post to this group, send email to gstream...@googlegroups.com.
Visit this group at https://groups.google.com/group/gstreamer-java.
For more options, visit https://groups.google.com/d/optout.

[Ahmet Yakar]

Hi my friend,

I tried the things you said and eventually I achieved playing RTSP stream with playbin. I managed corresponding enviroment variables and build the java bindings. After that, I packed the build files to jar file. I did with with that jar file. Now I am trying to do it with Pipeline structure but I get an "Internal data stream error". I read some information about it and they say that it is about connecting the the source pad to the following element. I also add it to my source code but still it does not work. Do you have any idea to this issue?

Thanks in advance.

import org.freedesktop.gstreamer.*;
import org.freedesktop.gstreamer.elements.AppSink;


import javax.swing.*;
import java.awt.*;


public class Test1 {


    static Pipeline pipeline;
    static Element source, queue, decodeBin, windowsink;
    static AppSink appsink;
    static SimpleVideoComponent vc;
    static JFrame frame;

    public static void main(String[] args) {

        Gst.init();


        source = ElementFactory.make("rtspsrc", "source");
        source.set("location", "rtsp://10.0.30.66/stream2");
        source.set("latency", 0);
        decodeBin = ElementFactory.make("decodebin", "decodebin");
        appsink = (AppSink) ElementFactory.make("appsink", "sink");


        pipeline = new Pipeline("pipeline");
        pipeline.getBus().connect((Bus.ERROR) (source, code, message) -> {
            System.err.println(message);
            Gst.quit();
            System.exit(-1);
        });




        pipeline.getBus().connect(new Bus.STATE_CHANGED() {
            /**
             * Called when a {@link Pipeline} element executes a {@link State} change.
             *
             * @param source  the element which posted the message.
             * @param old     the old state that the element is changing from.
             * @param current the new (current) state the element is changing to.
             * @param pending the pending (target) state.
             */
            @Override
            public void stateChanged(GstObject source, State old, State current, State pending) {
                if (source == pipeline)
                    System.out.println("Pipeline state changed from " + old + " to " + current);
            }
        });


        SwingUtilities.invokeLater(() -> {


            vc = new SimpleVideoComponent();
            windowsink = vc.getElement();


            pipeline.addMany(source, decodeBin, appsink, windowsink);


            Element.linkMany(source, decodeBin, appsink, windowsink);


            frame = new JFrame("Pipeline RTSP Test");
            frame.add(vc);
            frame.setSize(new Dimension(1024, 768));
            frame.setLocationRelativeTo(null);
            frame.setVisible(true);
            pipeline.play();
        });




        source.connect((Element.PAD_ADDED) (element, pad) -> {
            source.link(decodeBin);
        });


        Gst.main();
    }
}

[Neil C Smith]

Hi,

On Mon, Dec 18, 2017 at 1:00 PM Ahmet Yakar <ahmety...@gmail.com> wrote:

I tried the things you said and eventually I achieved playing RTSP stream with playbin. I managed corresponding enviroment variables and build the java bindings. After that, I packed the build files to jar file. I did with with that jar file. Now I am trying to do it with Pipeline structure but I get an "Internal data stream error". I read some information about it and they say that it is about connecting the the source pad to the following element. I also add it to my source code but still it does not work. Do you have any idea to this issue?

The obvious issue in the code is that you're trying to make the wrong connections in the PAD_ADDED signal.  You need to connect the decodebin to the following element - the appsink.

Don't try and link decodeBin to appsink before playback.

What is windowsink?  It doesn't seem to ever be created, and you shouldn't have two sinks in that order anyway.

Don't use ElementFactory for DecodeBin or AppSink - create them with new DecodeBin() / new AppSink()

If you're putting some GStreamer code inside EventQueue.invokeLater(), put it all in there!  Except perhaps for Gst.init().  You've got some weird threading going on there.  And you don't need Gst.main() with a GUI application, although you might want EXIT_ON_CLOSE on the JFrame.

Consider using Pipeline.launch() and Bin.launch().  That way GStreamer itself handles the dynamic pad connections for you.

Best wishes,

Neil

[Ahmet Yakar]

Hi,

I tried to link decodebin and appsink as you said. It didn't work. I think linking problem is not between decodebin and appsink because I often get this error;

"GstMessageWarning, gerror=(GError)NULL, debug=(string)"./grammar.y\(510\):\ gst_parse_no_more_pads\ \(\):\ /GstPipeline:pipeline0/GstBin:bin0/GstRTSPSrc:source:\012failed\ delayed\ linking\ some\ pad\ of\ GstRTSPSrc\ named\ source\ to\ some\ pad\ of\ GstDecodeBin\ named\ decodebin";"

I am also not clear about whether linking the elements or linking the pads in the PAD_ADDED signal.

I added the console outputs of two different attempts. I also attacted the source code. Could you please check them out?

I tried to use Bin.launch() in one attempt and Pipeline.launch() in other one. I could not achieve it in both of them. Everytime I try, I see an annoying empty black screen.

What should I focus on for next? Any recommendation?

Thank you very much,

Attempt #1;

import org.freedesktop.gstreamer.Bus;
import org.freedesktop.gstreamer.Element;
import org.freedesktop.gstreamer.Gst;
import org.freedesktop.gstreamer.Pipeline;

import javax.swing.*;
import java.awt.*;


public class Test1 {

    private static Pipeline pipeline;
    private static SimpleVideoComponent vc;
    private static JFrame frame;

    public static void main(String[] args) {


        Gst.init();

        SwingUtilities.invokeLater(() -> {

            frame = new JFrame("Pipeline RTSP Test");

            frame.setSize(new Dimension(1024, 768));
            frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);


            vc = new SimpleVideoComponent();
            frame.add(vc);


            pipeline = Pipeline.launch("rtspsrc name=source latency=0 location=rtsp://10.0.30.66/stream2 ! decodebin name=decodebin ! appsink name=appsink");
            pipeline.add(vc.getElement());
            Pipeline.linkMany(pipeline, vc.getElement());


            pipeline.getBus().connect((Bus.ERROR) (source, code, message) -> System.err.println(message));
            pipeline.getBus().connect((Bus.MESSAGE) (bus, message) -> System.out.println(message.getStructure()));


            pipeline.getElementByName("source").connect((Element.PAD_ADDED) (element, pad) -> {
                pipeline.getElementByName("source").link(pipeline.getElementByName("decodebin"));
            });


            pipeline.getElementByName("decodebin").connect((Element.PAD_ADDED) (element, pad) -> {
                pipeline.getElementByName("decodebin").link(pipeline.getElementByName("appsink"));
            });


            frame.setLocationRelativeTo(null);
            frame.setVisible(true);
            pipeline.play();
        });
    }
}

Attempt #2;

import org.freedesktop.gstreamer.*;


import javax.swing.*;
import java.awt.*;


public class Test2 {


    private static Pipeline pipe;
    private static SimpleVideoComponent vc;
    private static JFrame f;

    public static void main(String[] args) {


        Gst.init();

        EventQueue.invokeLater(() -> {


            vc = new SimpleVideoComponent();
            vc.setPreferredSize(new Dimension(640, 380));


            f = new JFrame("Camera Test");
            f.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);


            Bin bin = Bin.launch("rtspsrc name=source latency=0 location=rtsp://10.0.30.66/stream1 ! decodebin name=decodebin", true);


            pipe = new Pipeline();
            pipe.addMany(bin, vc.getElement());
            Pipeline.linkMany(bin, vc.getElement());


            pipe.getBus().connect((Bus.ERROR) (source, code, message) -> System.err.println(message));
            pipe.getBus().connect((Bus.MESSAGE) (bus, message) -> System.out.println(message.getStructure()));
            pipe.getBus().connect((Bus.INFO) (source, code, message) -> System.out.println(message));


            pipe.getElementByName("source").connect((Element.PAD_ADDED) (element, pad) -> {
                /*
                 *
                 * Here I tried to link the rtspsrc source pad to decodebin sink pad because it says on the console output;
                 *
                 * "GstMessageWarning, gerror=(GError)NULL, debug=(string)"./grammar.y\(510\):\ gst_parse_no_more_pads\ \(\):\
                 * /GstPipeline:pipeline0/GstBin:bin0/GstRTSPSrc:source:\012failed\ delayed\ linking\ some\ pad\ of\ GstRTSPSrc\ named\ source\ to\ some\ pad\ of\ GstDecodeBin\ named\ decodebin";
                 *
                 *
                 */
                for (Pad p : pipe.getElementByName("decodebin").getPads())
                    pad.link(p);
            });


            f.add(vc);
            f.pack();
            f.setLocationRelativeTo(null);
            f.setVisible(true);


            pipe.play();
        });


    }


}

[Neil C Smith]

Hi,

On Thu, Dec 21, 2017 at 8:10 AM Ahmet Yakar <ahmety...@gmail.com> wrote:

I added the console outputs of two different attempts. I also attacted the source code. Could you please check them out?

I tried to use Bin.launch() in one attempt and Pipeline.launch() in other one. I could not achieve it in both of them. Everytime I try, I see an annoying empty black screen.

What should I focus on for next? Any recommendation?

OK, have a look at the attached.  First uses Pipeline.launch().  Note that you need a videoconvert element, and you also have to pass in the appsink to SimpleVideoComponent.  I've changed the source to something we should both be able to see, and removed the latency parameter - latency=0 causes it to fail most of the time for me!

The second example uses PlayBin - is there a reason you're building the pipeline yourself?  If it's just for playback, PlayBin should work fine.

I can't get Bin.launch() to work easily either - the "true" parameter creates ghost pads on the bin for any unconnected pads - I think it's creating one for the sink on the decodebin because rtspsrc isn't connected at that point, and by the time it can connect the pad on decodebin is no longer free.  You would have to pass "false" to Bin.launch(), but then handle all the ghost pad setting up yourself.

Best wishes,

Neil

[Ahmet Yakar]

First of all, thank you very much for your attention,

Actually, at the end of the day I should be able to process the frames for detecting motion, detecting human faces or recognizing them or kinda processes. As I read articles on the web, I can do this with gstreamer + openCV. That is why I need a low level design so that I can add that kind of filters without sacrificing the speed of the stream.

In conclusion, I need low latency and image processing. What can you recommend for that?

Sincerely,