AppSink not receive new sample

[Noux Ha]

Hello,

I used  the SwingCamera sample (https://github.com/gstreamer-java/gst1-java-examples/tree/master/SwingCamera) by replacing Gst.parseBinFromDescription by (Pipeline) Gst.parseLaunch and it works normally.

Here is my code:

public class SwingCamera {

private static Pipeline pipeline;

public static void main(String[] args) {

Utils.configurePaths();

Gst.init(Version.BASELINE, "SwingCamera", args);

EventQueue.invokeLater(() -> {

pipeline = (Pipeline) Gst.parseLaunch("rtspsrc location=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! videoconvert ! capsfilter caps=video/x-raw,width=640,height=480 ! appsink name=sink");

AppSink appsink = (AppSink) pipeline.getElementByName("sink");

appsink.set("emit-signals", true);
appsink.set("max-buffers", 50);

GstVideoComponent vc = new GstVideoComponent(appsink);

JFrame f = new JFrame("Camera Test");
f.add(vc);
vc.setPreferredSize(new Dimension(640, 480));
f.pack();

f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

pipeline.getBus().connect((Bus.ERROR) (source, code, message) -> System.err.println(message));
//pipeline.getBus().connect((Bus.MESSAGE) (bus, message) -> System.out.println(message.getStructure()));

pipeline.play();
f.setVisible(true);
});
}
}

but when I added a listener at Appsink it receives the first sample only, also after that gui is suspended!

The added code:

appsink.connect((AppSink.NEW_SAMPLE) s -> {
System.out.println("Appsink NEW_SAMPLE");
final Sample sample = s.pullSample();
if (sample == null)
{
System.out.println("Sample is Null");
return FlowReturn.OK;
}

Caps caps = sample.getCaps();
Structure structure = caps.getStructure(0);
int width = structure.getInteger("width");
int height = structure.getInteger("height");

final Buffer buffer = sample.getBuffer();
try
{
final ByteBuffer buf = buffer.map(false);
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
int[] pixels = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
buf.asIntBuffer().get(pixels, 0, width * height);
ImageIO.write(image, "jpg", new File("images\\" + System.currentTimeMillis() + ".jpg"));
}catch (Exception ex){
ex.printStackTrace();
}
finally
{
sample.dispose();
buffer.unmap();
}
return FlowReturn.OK;
});

Any help?

Thanks in advance

[Neil C Smith]

Hi,

On Mon, 23 Jan 2023 at 18:09, Noux Ha <nou...@gmail.com> wrote:
> but when I added a listener at Appsink it receives the first sample only, also after that gui is suspended!

I'm not sure you can add two listeners to the AppSink. The Swing
component already uses one.

> The added code:
...
> sample.dispose();
> buffer.unmap();

Calling dispose before unmap is suspicious, and might be a cause of issue.

If all you really want to do is save the images to files as well as
display, I would look into doing that within the GStreamer pipeline
rather than with ImageIO.

Best wishes,

Neil


--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD

[Noux Ha]

I think that we can't use two  listeners to the AppSink as you mentioned.

I tried to put the unmap of the buffer before the dispose of the sample but the result was the same.

My goal is to analyze the frames of the stream received from the pullSample method. Any Idea to do that ??

Best regards...

[Neil C Smith]

On Mon, 23 Jan 2023 at 18:24, Noux Ha <nou...@gmail.com> wrote:
>
> I think that we can't use two listeners to the AppSink as you mentioned.
> I tried to put the unmap of the buffer before the dispose of the sample but the result was the same.
>
> My goal is to analyze the frames of the stream received from the pullSample method. Any Idea to do that ??

Mostly like you are doing.

Simplify down the code so you have no GUI and a simple listener - eg.
print the buffer size, or similar, and check you're getting multiple
frames.

Check how both the Swing and JavaFX components hand over the data to a
separate processing thread, in slightly different ways. Move your
analysis off of GStreamer's callback thread as much as you can.

If you still want to display the video in a GUI, you'll have to
consider rendering it yourself. You could use a pipeline with two
appsinks, but that's probably a bad idea.

You could also adapt the Swing component to support accessing the
image via a listener - a PR could be merged upstream to allow that.

Good luck!

[Noux Ha]

Sorry for disturbing you..

I used one app sink listener for gulling a sample and deleted the video component for swing, and that to render the frames in my own java code...

the pipeline works normally but it gives me a BufferUnderflowException..

the error handled at the line with stars:

appsink.connect((AppSink.NEW_SAMPLE) s -> {
System.out.println("Appsink NEW_SAMPLE");

Sample sample = s.pullSample();
System.out.println(sample);
if (sample == null)
{
System.out.println("Sample Null");

return FlowReturn.OK;
}

Caps caps = sample.getCaps();
Structure structure = caps.getStructure(0);
int width = structure.getInteger("width");
int height = structure.getInteger("height");

final Buffer buffer = sample.getBuffer();
try
{

ByteBuffer buf = buffer.map(false);
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);

int[] pixels = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();

**buf.asIntBuffer().get(pixels, 0, width * height);**

(do something with image) 

}catch (Exception ex){
ex.printStackTrace();
}
finally
{
sample.dispose();
buffer.unmap();
}
return FlowReturn.OK;
});

[Neil C Smith]

You're probably missing setting the format in the capsfilter or appsink that the Swing component does. Without it the bytes per pixel are probably different.

See  https://github.com/gstreamer-java/gst1-java-swing/blob/master/src/main/java/org/freedesktop/gstreamer/swing/GstVideoComponent.java#L86

Best wishes,

Neil

--
You received this message because you are subscribed to the Google Groups "gstreamer-java" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gstreamer-jav...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gstreamer-java/e790cb3b-526b-42c8-8a09-389a93a3ffc8n%40googlegroups.com.