Can't launch a transcode pipeline, get either "Your GStreamer installation is missing a plug-in. DecodeBin" or "Internal data stream error. BaseSrc"

[Bob F]

I am trying to work my way up to a more complicated workflow, but I'm starting with a pipeline that works in gst-launch:

gst-launch-1.0 filesrc location=$src ! \
                decodebin name=dec  ! \
                audioconvert ! faac ! queue ! mux. \
                dec. ! videoconvert ! x264enc pass=5 quantizer=21 ! queue ! mux. \
                mpegtsmux name=mux ! filesink location=/tmp/test.ts

And I am trying to launch it from java like so:

public static class HandsOff
    {
        public static void main(String[] argv)
        {
            Gst.init();

            EventQueue.invokeLater(HandsOff::mission1);
        }

        public static void mission1()
        {
            Pipeline pipe = new Pipeline();
            if (true) {
                buildPipeline1(pipe);
            } else {
                builPipeline2(pipe);
            }

            Bus bus = pipe.getBus();
            addDebuggingBusListeners(bus);

            if (true) {
                pipe.play();
            } else {
                new Thread(pipe::play).start();
            }

        }

        public static void buildPipeline1(Pipeline pipe)
        {
            Element fs = ElementFactory.make("filesrc", "src");
            fs.set("location", srcVideo);

            //Element dec = ElementFactory.make("decodebin", "dec");

            Bin bin = Bin.launch("decodebin name=dec ! " +
                "audioconvert ! faac ! queue ! mux. " +
                "dec. ! videoconvert ! x264enc pass=5 quantizer=21 ! queue ! mux. " +
                "mpegtsmux name=mux ! filesink location=/tmp/test.ts", true);

            pipe.addMany(fs,
//                    dec,
                bin);
            Element.linkMany(fs,
//                    dec,
                bin);
        }

        public static void builPipeline2(Pipeline pipe)
        {
            Bin bin = Bin.launch("filesrc name=src ! decodebin name=dec ! " +
                "audioconvert ! faac ! queue ! mux. " +
                "dec. ! videoconvert ! x264enc pass=5 quantizer=21 ! queue ! mux. " +
                "mpegtsmux name=mux ! filesink location=/tmp/test.ts", true);

            Element fs = bin.getElementByName("src");
            fs.set("location", srcVideo);

            pipe.add(bin);
            //pipe.link(bin);
        }

    } 

    public static void addDebuggingBusListeners(Bus bus)
    {
        bus.connect((Bus.EOS) gstObject -> System.out.println("EOS"));

        bus.connect((Bus.ERROR) (gstObject, i, s) -> System.out.println("ERROR "+i+" "+s+" "+gstObject));

        bus.connect((Bus.WARNING) (gstObject, i, s) -> System.out.println("WARN "+i+" "+s+" "+gstObject));
    }

If I use the buildPipeline2() which has a monolithic Bin.launch() then i get

ERROR 12 Your GStreamer installation is missing a plug-in. DecodeBin: [dec]
ERROR 1 Internal data stream error. Element: [typefind]

If I use buildPipeline1() which separates the filesrc from the rest of the pipeline I get

ERROR 1 Internal data stream error. BaseSrc: [src]

I am at a loss as to what I should do to make this work.  I suspect some of the EventQueue and Thread.start() stuff isn't necessary, but I was grasping at straws, trying to make my code look as much like the (very few) examples as possible. 

[Neil C Smith]

Hi,

Do the examples work on your machine?

What OS, Java and GStreamer versions are you using?

As a punt I'd guess Windows and you're using jna.library.path?  Anyway, that error looks like you're managing to load the library fine, but the plugin folder location isn't set up so GStreamer can't find its plugins.

I've set your account to receive all emails.

Best wishes,

Neil

--
You received this message because you are subscribed to the Google Groups "gstreamer-java" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gstreamer-jav...@googlegroups.com.
To post to this group, send email to gstream...@googlegroups.com.
Visit this group at https://groups.google.com/group/gstreamer-java.
For more options, visit https://groups.google.com/d/optout.

--

Neil C Smith

Artist & Technologist

www.neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org

[Bob F]

I was able to use the PlayBinVideoPlayer to watch the video in question.

I am running gentoo.  The 1.12.3 gst-plugins are installed.

 $ java -version
java version "1.8.0_152"
Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode)

[Neil C Smith]

On Mon, 5 Feb 2018 at 18:26 Bob F <thot...@gmail.com> wrote:

I was able to use the PlayBinVideoPlayer to watch the video in question.

I am running gentoo.  The 1.12.3 gst-plugins are installed.

Cool! Trying the common issues (Windows!) first! ;-)

I don't have an faac plugin, but switching to voaacenc and using the following code works for me (Ubuntu 16.04 with GStreamer 1.8) -

```
    Pipeline pipe = Pipeline.launch("filesrc name=src ! decodebin name=dec ! "
                + "audioconvert ! voaacenc ! queue ! mux. "
                + "dec. ! videoconvert ! x264enc ! queue ! mux. "
                + "mpegtsmux name=mux ! filesink location=/tmp/test.ts");

        Element fs = pipe.getElementByName("src");
        fs.set("location", srcVideo);

        Bus bus = pipe.getBus();
        addDebuggingBusListeners(bus);

```

Note that using Bin.launch is not the same as what you're doing on the CLI - use Pipeline.launch to create all the elements in the top level.  I don't know if that's the cause of the issue you're seeing though.

If you're not using a GUI then the EventQueue isn't really required.  Something in your setup meant I wasn't seeing any bus messages, possibly because the application was shutting down too soon?  Anyway, use

```

Gst.init();

mission1();

Gst.main();
```

You can call Gst,quit() in the EOS signal handler.

Best wishes,

Neil

[Bob F]

Switching from Bin.launch() to Pipeline.launch() did prevent the errors, and it appears the transcode is actually generating /tmp/test.ts .

I suspect that the final form of my app will not allow me to use Pipeline.launch() because I need to drop frames that are outside a target time range (is there a gstreamer element that does time trimming?) and that means I will probably be using java to pull samples and drop or forward them from pipeline to pipeline.

So, what extra steps are necessary when filling a Pipeline with Bin-s to make the pipeline work as well as Pipeline.launch() -created pipelines work.

If we can get this basic transcoding example working I am going to recommend it be added to the examples git repo so the next person can learn from it.  The existing examples seem to lack instances that are a finite process.

[Neil C Smith]

Hi,

On Mon, 5 Feb 2018 at 20:21 Bob F <thot...@gmail.com> wrote:

Switching from Bin.launch() to Pipeline.launch() did prevent the errors, and it appears the transcode is actually generating /tmp/test.ts .

Great!

 

I suspect that the final form of my app will not allow me to use Pipeline.launch() because I need to drop frames that are outside a target time range (is there a gstreamer element that does time trimming?) and that means I will probably be using java to pull samples and drop or forward them from pipeline to pipeline.

I'm not sure on the time trimming - maybe someone else can comment, or try upstream on the GStreamer lists.

For this sort of task you can probably also consider multiple Pipelines - the quick answer was mainly to make sure the bindings are working for you.

 

So, what extra steps are necessary when filling a Pipeline with Bin-s to make the pipeline work as well as Pipeline.launch() -created pipelines work.

Nothing!  But you're going to have to understand why the Bin approach fails and how to fix it, which means understanding sometimes and ghost pads.  The upstream documentation is worth a read - https://gstreamer.freedesktop.org/documentation/application-development/basics/pads.html

Putting a decodebin in a Bin using Bin.launch(..., true); is going to be a problem.  Passing true as the second parameter creates ghost pads on the bin for any unconnected pads.  The likely problem you have is that decodebin has a sometimes src pad (check gst-inspect decodebin), meaning that at that point decodebin and audioconvert / videoconvert are not connected, a ghosts pad gets created for one of the convert elements, which means decodebin can't connect later.

To put the decodebin in a bin you'll have to pass false and set up the ghost pads manually to the correct elements.  Or, keep the decodebin in the top level pipeline.

The good thing about sometimes connections in Pipeline.launch() or Bin.launch() is that GStreamer manages them for you.  If you're creating the connections yourself using Element::link or Element::linkMany then you'll also have to add something along the lines of -

decodeBin.connect((Element.PAD_ADDED)(element, pad) -> {
                        element.link(videoConvert);
                        element.link(audioConvert);
                });

 

If we can get this basic transcoding example working I am going to recommend it be added to the examples git repo so the next person can learn from it.  The existing examples seem to lack instances that are a finite process.

Pull requests for new examples would be very welcome.  There are a lot of examples in the 0.10 bindings that have not been ported over yet too.

Best wishes,

Neil

 

[Neil C Smith]

On Tue, 6 Feb 2018 at 09:47 Neil C Smith <ne...@neilcsmith.net> wrote:

Putting a decodebin in a Bin using Bin.launch(..., true); is going to be a problem.  Passing true as the second parameter creates ghost pads on the bin for any unconnected pads.

PS! ;-)

I just tried adding the following into your buildPipeline1() -

for (Pad pad : bin.getPads()) {
    System.out.println(((GhostPad)pad).getTarget().getParentElement().getName());
}

This outputs -

typefind
videoconvert0

The latter is as I expected, the former surprised me - the typefind is a child of decodebin - I didn't realise this recurses down inside children looking for a free pad.  That will also stop your pipeline from working.

Best wishes,

Neil

[Bob F]

On Tuesday, February 6, 2018 at 4:48:14 AM UTC-5, Neil C Smith wrote:

On Mon, 5 Feb 2018 at 20:21 Bob F <thot...@gmail.com> wrote:

 

So, what extra steps are necessary when filling a Pipeline with Bin-s to make the pipeline work as well as Pipeline.launch() -created pipelines work.

Nothing!  But you're going to have to understand why the Bin approach fails and how to fix it, which means understanding sometimes and ghost pads.  The upstream documentation is worth a read - https://gstreamer.freedesktop.org/documentation/application-development/basics/pads.html

Putting a decodebin in a Bin using Bin.launch(..., true); is going to be a problem.  Passing true as the second parameter creates ghost pads on the bin for any unconnected pads.  The likely problem you have is that decodebin has a sometimes src pad (check gst-inspect decodebin), meaning that at that point decodebin and audioconvert / videoconvert are not connected, a ghosts pad gets created for one of the convert elements, which means decodebin can't connect later.

I remembered the nature of sometimes pads yesterday and came up with a new app:

https://github.com/mutantbob/gst1-java-examples/blob/experiments/src/main/java/org/freedesktop/gstreamer/examples/TooMuchPlumbing.java

It took me a long time to figure out how to link the various pads (hierarchy is a major issue).  The fact that pad.link() returns error codes instead of throwing exceptions is actually quite a major pain in the butt.  Maybe I should create a new method ( linkOrThrow() ? ) and pull request that.

Even after I got the filter graph looking correct in graphviz, the pipeline does not actually run.

What steps are missing to get the pipeline moving again after the sometimes pads have been linked?

Pull requests for new examples would be very welcome.  There are a lot of examples in the 0.10 bindings that have not been ported over yet too.

 https://github.com/gstreamer-java/gst1-java-examples/pull/8

[Neil C Smith]

Hi,

On Tue, 6 Feb 2018 at 16:00 Bob F <thot...@gmail.com> wrote:

It took me a long time to figure out how to link the various pads (hierarchy is a major issue).

Also don't forget ghost pads (eg. bin.addPad(new GhostPad("name", targetPad));  That's how to deal with hierarchy, although check requirements for setting active too.

The fact that pad.link() returns error codes instead of throwing exceptions is actually quite a major pain in the butt.  Maybe I should create a new method ( linkOrThrow() ? ) and pull request that.

Open to suggestions, but change the existing method if you've got a better idea about it - now is the time for breaking API!

 

Even after I got the filter graph looking correct in graphviz, the pipeline does not actually run.

What steps are missing to get the pipeline moving again after the sometimes pads have been linked?

I just tried your code and couldn't get it to work.  But moving the venc bin into mission1 and doing this does -

pipe.addMany(venc, mux, fileSink);
Pipeline.linkMany(venc, mux, fileSink);

dec.connect((Element.PAD_ADDED) (d,p) -> {
    d.link(venc);
});

Connecting venc to mux in the callback works, but not adding it to pipe - I think it's something to do with the play state not being picked up - can't recall off the top of my head.

 

Pull requests for new examples would be very welcome.  There are a lot of examples in the 0.10 bindings that have not been ported over yet too.

 https://github.com/gstreamer-java/gst1-java-examples/pull/8

Thanks!  Looks good - I'll merge ASAP.

Best wishes,

Neil

[Bob F]

On Tuesday, February 6, 2018 at 12:27:43 PM UTC-5, Neil C Smith wrote:

I just tried your code and couldn't get it to work.  But moving the venc bin into mission1 and doing this does -

pipe.addMany(venc, mux, fileSink);
Pipeline.linkMany(venc, mux, fileSink);

dec.connect((Element.PAD_ADDED) (d,p) -> {
    d.link(venc);
});

Connecting venc to mux in the callback works, but not adding it to pipe - I think it's something to do with the play state not being picked up - can't recall off the top of my head.

That could be a problem.  It is reasonable (in my case, but not all cases) to expect there will be only one video stream that needs transcoding.  But when I add audio transcoding, I really need to be able to fabricate as many transcode bins as are necessary to get all the audios.

I think there needs to be a way to add bins to a pipeline dynamically.  I would assume that this gets accomplished inside complicated plugins like decodebin and playbin.  What are they doing that is missing from this code?  The tutorial at https://www.abc15.com/news/data/school-shootings-in-u-s-when-where-each-shooting-has-occurred-in-2018 does not address this topic either (it uses the same preallocation strategy you got working).

Maybe I'll experiment with setting the state on the newly-linked transcode pipeline.

[Neil C Smith]

On Tue, 6 Feb 2018 at 18:04 Bob F <thot...@gmail.com> wrote:

I think there needs to be a way to add bins to a pipeline dynamically. ...

There is, you've just got to figure out how! ;-)

Maybe I'll experiment with setting the state on the newly-linked transcode pipeline.

Actually, I forgot that bit.  You need to manage the state of dynamically added elements yourself.

So, by commenting out all the fakesink / element trash block (there's no need to link a pad you don't need), and calling venc.setState(State.PLAYING) at the end of connectVideoReencode() your code works here.

Best wishes,

Neil

[Bob F]

On Tuesday, February 6, 2018 at 1:28:09 PM UTC-5, Neil C Smith wrote:

Actually, I forgot that bit.  You need to manage the state of dynamically added elements yourself.

So, by commenting out all the fakesink / element trash block (there's no need to link a pad you don't need), and calling venc.setState(State.PLAYING) at the end of connectVideoReencode() your code works here.

Well, "works" is not exactly the verb I would use.  While it does cause the pipeline to actually run and generate a file, that transport stream ends up being a steaming pile of unplayable garbage.

The resulting transport stream seems to put the video payload into PID 0 which is supposed to be the PAT.  The PMT even says "I'm putting the video in PID 0" which is as wrong as a football bat.

I suspect that the MPEG TS mux wasn't coded well enough to support pads being added after its initialization phase.  I suspect that I'll have to hold off on creating the mux until the decodebin is done telling me about sometimes pads and I'm done linking transcoding pipelines to those sometimes pads.

I have no idea how to detect when a decodebin is done adding pads.

[Neil C Smith]

Hi,

On Tue, 6 Feb 2018, 19:24 Bob F, <thot...@gmail.com> wrote:

Well, "works" is not exactly the verb I would use.  While it does cause the pipeline to actually run and generate a file, that transport stream ends up being a steaming pile of unplayable garbage.

I only went as far as checking it created a file, and playing it back. 

Highly recommend taking this conversation on to the main upstream GStreamer mailing list and figuring out what's missing in how you're setting up the pipeline. 

[Bob F]

On Tuesday, February 6, 2018 at 2:24:21 PM UTC-5, Bob F wrote:

The resulting transport stream seems to put the video payload into PID 0 which is supposed to be the PAT.  The PMT even says "I'm putting the video in PID 0" which is as wrong as a football bat.

I suspect that the MPEG TS mux wasn't coded well enough to support pads being added after its initialization phase.  I suspect that I'll have to hold off on creating the mux until the decodebin is done telling me about sometimes pads and I'm done linking transcoding pipelines to those sometimes pads.

Nope.  It's actually the fact that the tail end of the sink_%d request pad is actually the PID.  Since I started counting at 0 and created sink_0 it obediently put the video in PID 0.  When I switched to counting up from 480, I got a playable mpeg transport stream.

So I can create the mpegtsmux early on (in the same phase as creating decodebin) and I can add transcoding pipelines as the decodebin's sometimes pads are passed to my callback, and those transcode pipelines can be linked to the mpegtsmux.  Although I must make sure I switch the freshly-created transcode pipelines to state PLAYING.