The future of the GStreamer Java bindings?

[Neil C Smith]

Hi All,

I'd like to start a conversation on the future of the Java bindings
for GStreamer, gst1-java-core [1] and related projects. I've not been
able to put much time into the bindings recently. And despite seeing
an increasing number of commercial users of the library, none of
Codelerity's commercial work over the last 18 months has been
GStreamer related. That situation is not sustainable for me! At least
while I remain the sole maintainer, and primary developer and source
of support. Simply put, I cannot afford to invest for free, at least
alone, the level of time that is required to make necessary updates to
the library to meet changes in both Java and GStreamer.

I was one of the maintainers of the GStreamer 0.10 bindings for Java,
and started the GStreamer 1.x supporting fork in 2015 to meet some
personal project needs. I made the project openly available, but never
intended to stay its sole maintainer. Despite many valued
contributions from others over the years, no-one with the necessary
knowledge has volunteered to help on an ongoing basis with development
and maintenance - even with some heavy hinting at times!

Unfortunately, if we cannot find a sustainable way forward, then we
just might have to accept that it's time to call it a day on these
bindings. That would be a shame!

[1] https://github.com/gstreamer-java/gst1-java-core

Thanks for any input you have!

With best wishes,

Neil


--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD

[Scooter Willis]

Neil

I would be interested in getting more involved. 

That said it does appear that Java is working to replace JNI. 

Would be interesting to try Foreign Function and Memory API to see if it makes support l/maintenance easier for interfacing to c/c++ code. In particular to get access to memory related to images without the JNI overhead. 

With deep learning models becoming increasingly popular to run against live video and python being the primary programming interface I suspect their will be stronger interest in processing video in Java that is production worthy. 

Nvidia is all in on DeepStream to simplify/hide the camera/gpu layer. I see the gstreamer  pipeline  approach getting more traction moving forward. 

I would propose moving up a level and have  Java gstreamer for deep learning related gstreamer development.

Could be a way to get Nvidia or Google to bring in Java support. Amazon has put in significant resources for Java interfaces/abstraction to the various deep learning stacks. Yet they can't process video. Java Deep Learning library. 

Check out ridgerun.com gstreamer business model.

Scooter 

--
You received this message because you are subscribed to the Google Groups "gstreamer-java" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gstreamer-jav...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gstreamer-java/CAPxOS5FiEJdGDwSgtZ7iB3XJ8_0fJ_%2BpjXGRF3-iK%2BebERXOXQ%40mail.gmail.com.

[Neil C Smith]

On Wed, 22 Feb 2023 at 13:24, Scooter Willis <will...@gmail.com> wrote:
> I would be interested in getting more involved.

Great! But could you be more specific? Where do you think you could
contribute? Where do you think priorities should be?

> That said it does appear that Java is working to replace JNI.
>
> Would be interesting to try Foreign Function and Memory API to see if it makes support l/maintenance easier for interfacing to c/c++ code. In particular to get access to memory related to images without the JNI overhead.

Yes, definitely aware of Panama and related JDK projects. It remains
to be seen what benefit it would offer for GStreamer interaction in
practice. I've had a quite detailed conversation with one of the
developers around the foreign thread management model (eg. GStreamer's
threads attaching to the VM) - that's a key crunch point for the
bindings.

Various threads on the JNA list possibly worth a read, including
https://groups.google.com/g/jna-users/c/xnbkfLFraWY

To be clear, I'll still be using GStreamer and the JDK together in
future. Over the last few years I've developed a number of bespoke
bindings for clients who for one reason or another didn't want to use
gst1-java-core. For a number of Codelerity's projects I may switch
away from the bindings to a bespoke integration.

I've also looked again at automatic generation of lowlevel code using
Gir files again, partly with the thought of targeting both JNA and
Panama. That would be to replace the current lowlevel package mapping
code. The hard part would be switching the main API of the bindings
to this, particularly if the aim is to keep it (fairly) compatible.
There is also the current one-to-one mapping of Java and GObject
references, including memory tracking on both sides. That's one of
the useful aspects of the bindings, but also one of the more difficult
to maintain, and judging from some client work, difficult to
understand!

The key question for me is whether to invest much more of mine or my
company's own time into developing and maintaining the higher level
bindings for the future. Particularly when our current work is
focused elsewhere. The disconnect between the growing number of
commercial users I'm aware of, with their reliance on free support,
and their unwillingness to invest either time or money into the
maintenance of the project, make the answer to that, at least with a
business hat on, no.

> I would propose moving up a level and have Java gstreamer for deep learning related gstreamer development.
>
> Could be a way to get Nvidia or Google to bring in Java support. Amazon has put in significant resources for Java interfaces/abstraction to the various deep learning stacks. Yet they can't process video. Java Deep Learning library.

That's not an area of work we're currently focused on. I'd certainly
be happy to see development in this area though, and would possibly be
interested in assisting with.

> Check out ridgerun.com gstreamer business model.

Thanks. Am aware of RidgeRun, and looked at integration with elements
of theirs for a client a few years back.

Our business model is not that dissimilar, really - our scale and
areas of focus are.

Thanks and best wishes,