How to get raw bytes from playing playbin?

[Jimin Kim]

I want to get raw bytes from playbin because I want to print my video to lwjgl(Java binding for OpenGL).

I found code from SO:

cout<<"Creating appsink"<<endl; appsink = gst_element_factory_make("appsink", "asink"); gst_app_sink_set_emit_signals((GstAppSink*)appsink, true); gst_app_sink_set_drop((GstAppSink*)appsink, true); gst_app_sink_set_max_buffers((GstAppSink*)appsink, 1); //creating and initialising pipeline g_object_set(G_OBJECT(playbin), "video-sink", appsink, NULL); g_signal_connect(appsink, "new-buffer", G_CALLBACK(DisplayFrame), (gpointer) mark); //callback function looks like this gboolean Core::DisplayFrame(GstAppSink *fks, gpointer mark) { static bool init = false; static IplImage *frame; GstBuffer* buf; Mark* mk = (Mark*) mark; if(!init) { init = true; frame = cvCreateImage(cvSize(mk->frame_w, mk->frame_h), IPL_DEPTH_8U, 1); } buf = gst_app_sink_pull_buffer(fks); frame->imageData = (char*)GST_BUFFER_DATA(buf); ProcessFrame(frame); gst_buffer_unref(buf); return true; }
But I cannot found binding of gst_app_sink*

I found APP_API in lowlevel package but this doesn't contains gst_app_sink_set_emit_signal and etc.

Is there api which is able to use like

Long framePtr = playbin.getCurrentFrame();

or

ByteBuffer frameData = playbin.getCurrentFrame();

Thanks

Jimin Kim

[Neil C Smith]

Hi,

On Sun, 1 May 2022 at 13:37, Jimin Kim <xda...@gmail.com> wrote:
> I want to get raw bytes from playbin because I want to print my video to lwjgl(Java binding for OpenGL).

...

> I found APP_API in lowlevel package but this doesn't contains gst_app_sink_set_emit_signal and etc.

Don't touch lowlevel for this. You want the AppSink type in the
elements package. In general, gst_ prefixes are dropped and type /
function names changed to standard Java case.

You can find an example of using an AppSink with PlayBin to get the
image bytes in -

https://github.com/gstreamer-java/gst1-java-fx/blob/master/src/main/java/org/freedesktop/gstreamer/fx/FXImageSink.java
https://github.com/gstreamer-java/gst1-java-examples/blob/master/FXPlayer/src/main/java/org/freedesktop/gstreamer/examples/FXPlayer.java#L97

This is for passing in to JavaFX's PixelBuffer API, but the principle
is the same, including the need to be careful in passing Sample
between GStreamer callback and OpenGL threads. This code is adapted
from PraxisCORE anyway, which uses this with LWJGL.

Ideally we'd let GStreamer handle OpenGL textures directly rather than
uploading the texture data. I know there is some code at
https://github.com/Chatanga/gst1-java-plugins-base which does some of
this. We need to find time to fully implement GL texture sharing
directly in the bindings.

Hope that helps.

Best wishes,

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD