Gstreamer + Maven + JavaFX don't work

[Valentin Dubois-Pivot]

Hi,

I am developing a small test application with AdoptOpenJDK 11 and JavaFX 14.0.1.

My application works fine with three files:

- Main.java

- MainController.java

- MainPage.fxml

I wanted to switch this small application to use with Maven, for that I downloaded all the dependencies of JavaFX, GStreamer (core, fx and jna). But when I run my Main the app crashes and gives me this error:

(java:6533): GStreamer-CRITICAL **: 16:42:54.060: gst_element_factory_make: assertion 'gst_is_initialized ()' failed
Exception in Application start method
Exception in thread "main" java.lang.RuntimeException: Exception in Application start method
    at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
    at com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javafx.fxml.LoadException:
/home/ihm-rfn/Documents/EssaisIHMS40SA/EssaiVideoGstreamerMaven/target/classes/main/ui/MainPage.fxml

    at javafx.fxml.FXMLLoader.constructLoadException(FXMLLoader.java:2625)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2603)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2466)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:3237)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:3194)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:3163)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:3136)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:3113)
    at javafx.fxml.FXMLLoader.load(FXMLLoader.java:3106)
    at main.ui.MainUI.start(MainUI.java:15)
    at com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
    at com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
    at com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
    at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
    at com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:277)
    ... 1 more
Caused by: java.lang.IllegalArgumentException: Invalid native pointer
    at org.freedesktop.gstreamer.glib.Natives.initializer(Natives.java:83)
    at org.freedesktop.gstreamer.glib.Natives.initializer(Natives.java:67)
    at org.freedesktop.gstreamer.Pipeline.initializer(Pipeline.java:130)
    at org.freedesktop.gstreamer.Pipeline.<init>(Pipeline.java:125)
    at main.ui.MainPageController.buildPipelineCam1(MainPageController.java:132)
    at main.ui.MainPageController.initialize(MainPageController.java:111)
    at javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2573)
    ... 17 more

I don't understand where this can come from because on the same app without Maven it works fine ... The fxml file is located in the resource folder. Here is my code in detail so that I can explore it:

Package java/main/ui:

MainUI.java

package main.ui;

import javafx.application.Application;
import javafx.fxml.FXMLLoader;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;
import org.freedesktop.gstreamer.Gst;

public class MainUI extends Application {

    @Override
    public void start(Stage primaryStage) throws Exception {
        primaryStage.setTitle("SAMPLE APP");
        Pane mainPane = FXMLLoader.load(MainPageController.class.getResource("MainPage.fxml"));
        Scene mainScene = new Scene(mainPane);
        mainScene.getStylesheets().add("main/ui/application.css");
        primaryStage.setScene(mainScene);
        primaryStage.setResizable(false);
        primaryStage.show();
    }

    public static void main(String[] args)
    {
        // GStreamer Initialization
        String[] gstreamerArgs = new String[1];
        gstreamerArgs[0] = "-v";

        // GStreamer launch
        Gst.init("RPDFVideo",gstreamerArgs);

        launch(args);
    }
}

MainPageController.java

package main.ui;

import javafx.animation.AnimationTimer;
import javafx.fxml.FXML;
import javafx.fxml.Initializable;
import javafx.scene.control.Label;
import javafx.scene.image.ImageView;
import javafx.scene.layout.AnchorPane;
import org.freedesktop.gstreamer.*;
import org.freedesktop.gstreamer.elements.AppSink;
import org.freedesktop.gstreamer.fx.FXImageSink;
import org.freedesktop.gstreamer.message.Message;

import java.net.URL;
import java.nio.ByteOrder;
import java.util.ResourceBundle;

public class MainPageController implements Initializable {

    @FXML
    private AnchorPane anchorIHM;
    @FXML
    private ImageView imageViewCam1;

    Bus busCam1;

    private static final int BUFFER_SIZE                        = 4000000;

    private final int        portCam1                           = 5004;

    private final String     addressMulti                       = "239.192.5.1";

    protected Pipeline       pipelineCam1                       = null;

    private Element          elmt_udpsrcCam1                    = null;

    private Element             elmt_capsfilter_scaleCam1            = null;

    private Element             elmt_rtph264depayCam1                = null;

    private Element             elmt_vaapih264decCam1                = null;

    private Element             elmt_videoconvertCam1                = null;

    private Element             elmt_avdec_h264Cam1                = null;

    private Element          elmt_autovideosinkCam1             = null;

    private Element          elmt_queue0                        = null;
    private Element          elmt_queue1                        = null;
    private Element          elmt_queue2                        = null;
    private Element          elmt_queue3                        = null;

    private AppSink          videoSinkCam1                      = null;


    @Override
    public void initialize(URL location, ResourceBundle resources)
    {

        // Launch information
        System.out.println("Controller launched");

        // Set imageView visible
        imageViewCam1.setVisible(true);

        // PIPELINE CAM 1 CREATION
        buildPipelineCam1();

        // AFFECT PIPELINE 1 --> IMAGEVIEW 1
        var imageSinkCam1 = new FXImageSink(videoSinkCam1);
        imageViewCam1.setPreserveRatio(true);
        imageViewCam1.imageProperty().bind(imageSinkCam1.imageProperty());
    }

    /** PIPE 1 **/

    private void buildPipelineCam1(){

        // Create a pipeline from the text pipeline description
        pipelineCam1 = new Pipeline("PipelineCam1");

        // Create elements
        // Element udpsrc (source udp)
        elmt_udpsrcCam1 = ElementFactory.make("udpsrc", "udpsrc");
        elmt_udpsrcCam1.set("multicast-group", addressMulti);
        elmt_udpsrcCam1.set("auto-multicast", true);
        elmt_udpsrcCam1.set("port", portCam1);
        elmt_udpsrcCam1.set("retrieve-sender-address", "false");
        elmt_udpsrcCam1.set("buffer-size", BUFFER_SIZE);

        // Set Caps
        Caps caps = new Caps("application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, payload=(int)96, a-framerate=(string)25");
        elmt_udpsrcCam1.set("caps", caps);

        // Element rtph264depay
        elmt_rtph264depayCam1 = ElementFactory.make("rtph264depay","rtph264depay");

        // Element vaapih264dec
        elmt_vaapih264decCam1 = ElementFactory.make("vaapih264dec", "vaapih264dec");
        elmt_vaapih264decCam1.set("low-latency", true);

        // Element avdec_h264
        elmt_avdec_h264Cam1 = ElementFactory.make("avdec_h264", "avdec_h264");

        // Element videoconvert
        elmt_videoconvertCam1 = ElementFactory.make("videoconvert", "videoconvert");

        // Video size
        elmt_capsfilter_scaleCam1 = ElementFactory.make("capsfilter", "elmt_capsfilter_scale");
        Caps capsConvert = new Caps("video/x-raw,width=1920,height=1080");
        elmt_capsfilter_scaleCam1.set("caps", capsConvert);


        // Add queus
        elmt_queue0 = ElementFactory.make("queue", "queue0");
        elmt_queue0.set("max-size-buffers", 0);
        elmt_queue1 = ElementFactory.make("queue", "queue1");
        elmt_queue1.set("max-size-buffers", 0);
        elmt_queue2 = ElementFactory.make("queue", "queue2");
        elmt_queue2.set("max-size-buffers", 0);
        elmt_queue3 = ElementFactory.make("queue", "queue3");
        elmt_queue3.set("max-size-buffers", 0);

        this.createAppSinkCam1();
    }

    private void createAppSinkCam1(){

        videoSinkCam1 = new AppSink("GstVideoComponentCam1");
        videoSinkCam1.set("emit-signals", true);
        videoSinkCam1.set("drop", true);

        /** videoSink caps method 1 **/

        final StringBuilder caps = new StringBuilder("video/x-raw,pixel-aspect-ratio=1/1,");

        // JNA creates ByteBuffer using native byte order, set masks according to that.

        if (ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN) {
            caps.append("format=BGRx");
        } else {
            caps.append("format=xRGB");
        }

        final Caps theCapsToApply = new Caps(caps.toString());

        videoSinkCam1.setCaps(theCapsToApply);

        elmt_autovideosinkCam1 = videoSinkCam1;
        elmt_autovideosinkCam1.set("sync", "false");

        /** Build Pipeline **/

        // Add elements to pipe
        pipelineCam1.addMany(elmt_udpsrcCam1, elmt_rtph264depayCam1, elmt_vaapih264decCam1, elmt_videoconvertCam1, elmt_autovideosinkCam1);

        // Link elements together
        Element.linkMany(elmt_udpsrcCam1, elmt_rtph264depayCam1, elmt_vaapih264decCam1, elmt_videoconvertCam1, elmt_autovideosinkCam1);

        inspect(pipelineCam1);

        busCam1 = pipelineCam1.getBus();
        busCam1.connect(new Bus.MESSAGE() {

            @Override
            public void busMessage(Bus arg0, Message arg1) {

                System.out.println(arg1.getStructure());
            }
        });
        pipelineCam1.play();
    }
}

Package resources/main/ui

MainPage.fxml

<?xml version="1.0" encoding="UTF-8"?>

<?import javafx.scene.control.Label?>
<?import javafx.scene.image.ImageView?>
<?import javafx.scene.layout.AnchorPane?>

<AnchorPane fx:id="anchorIHM" prefHeight="1080.0" prefWidth="1920.0" xmlns="http://javafx.com/javafx/11.0.1" xmlns:fx="http://javafx.com/fxml/1" fx:controller="main.ui.MainPageController">
   <Label fx:id="countRate" layoutX="36.0" layoutY="28.0" text="Label" />
   <ImageView fx:id="imageViewCam1" fitHeight="512.0" fitWidth="910.0" layoutX="688.0" layoutY="149.0" pickOnBounds="true" preserveRatio="true" />
   <ImageView fx:id="imageViewCam2" fitHeight="310.0" fitWidth="390.0" layoutX="1096.0" layoutY="699.0" pickOnBounds="true" preserveRatio="true" />
</AnchorPane>

[Neil C Smith]

On Tue, 8 Sep 2020 at 16:14, Valentin Dubois-Pivot
<valentin.d...@gmail.com> wrote:
> I am developing a small test application with AdoptOpenJDK 11 and JavaFX 14.0.1.
>
> My application works fine with three files:
> - Main.java
> - MainController.java
> - MainPage.fxml
>
> I wanted to switch this small application to use with Maven, for that I downloaded all the dependencies of JavaFX, GStreamer (core, fx and jna). But when I run my Main the app crashes and gives me this error:

Curious. Are you saying the exact same code works when not using
Maven? If so, can you put your Maven project on GitHub or similar as
a whole?

I've set your account back to all email, as for anyone posting questions.

Best wishes,

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD

[Valentin Dubois-Pivot]

Hi,

Here is the link to my project on GitHub:

https://github.com/ValentinD45/FormationTestsUnitaires

Best Regards,

[Neil C Smith]

On Wed, 9 Sep 2020 at 14:49, Valentin Dubois-Pivot
<valentin.d...@gmail.com> wrote:
> Here is the link to my project on GitHub:
>
> https://github.com/ValentinD45/FormationTestsUnitaires

Are you using Main.main() or MainUI.main()? Try moving the GStreamer
initialization into Main.main() before Application.launch(). I'm not
sure it's being called where it is now?!

As an aside, you might want to look at the Maven JavaFX plugin too.

[Valentin Dubois-Pivot]

Hi,

Thanks it works now. It is true that at the beginning I left that in the MainUI of JavaFX but in itself the init is done before this file so it seems logical to move it in the Main.

Best Regards,

[Neil C Smith]

On Thu, 10 Sep 2020 at 08:05, Valentin Dubois-Pivot
<valentin.d...@gmail.com> wrote:
> Thanks it works now. It is true that at the beginning I left that in the MainUI of JavaFX but in itself the init is done before this file so it seems logical to move it in the Main.

It would work if MainUI.main() was called - I assume it never is.