v4l2h264enc 'extra-controls' doesn't seem to be working through gstreamer-java
1,003 views
Skip to first unread message
Mark Anthony Garcia
Jun 28, 2020, 3:50:46 AM6/28/20
to gstreamer-java
I can create a successful pipeline utilizing a v4l2h264enc element but when I attempt to set the "extra-controls" on the v4l2h264enc element I get java crashes or stderr of java.lang.NullPointerException. the crash seems to happen in native code and not in java itself. Maybe the gstreamer-java Element.set() is not utilizing the appropriate vernacular necessary for gstreamer itself.
the "extra-controls" string value appears different than other typical values of properties in elements. I wonder if the semicolon is causing the problem?
encoderElement = ElementFactory.make("v4l2h264enc", "encoder");
encoderElement.set("extra-controls","controls,video_bitrate=4000000;"); //this is the statement I use that presents the issue. commenting out this line allows the pipeline to work properly albeit in auto bitrate mode.
Sun Jun 28 02:54:11 UTC 2020 | # A fatal error has been detected by the Java Runtime Environment:
Sun Jun 28 02:54:11 UTC 2020 | #
Sun Jun 28 02:54:11 UTC 2020 | # SIGSEGV (0xb) at pc=0x91b92662, pid=5735, tid=5778
Sun Jun 28 02:54:11 UTC 2020 | #
Sun Jun 28 02:54:11 UTC 2020 | # JRE version: OpenJDK Runtime Environment (11.0.7+10) (build 11.0.7+10-post-Ubuntu-2ubuntu218.04)
Sun Jun 28 02:54:11 UTC 2020 | # Java VM: OpenJDK Server VM (11.0.7+10-post-Ubuntu-2ubuntu218.04, mixed mode, serial gc, linux-)
Sun Jun 28 02:54:11 UTC 2020 | # Problematic frame:
Sun Jun 28 02:54:11 UTC 2020 | # C [libgstreamer-1.0.so+0x79662] gst_structure_copy+0x29
Sun Jun 28 02:54:11 UTC 2020 | #
Sun Jun 28 02:54:11 UTC 2020 | # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
Sun Jun 28 02:54:11 UTC 2020 | #
Sun Jun 28 02:54:11 UTC 2020 | # An error report file with more information is saved as:
Sun Jun 28 02:54:11 UTC 2020 | # /usr/local/bin/hs_err_pid5735.log
Sun Jun 28 02:54:11 UTC 2020 | #
Sun Jun 28 02:54:11 UTC 2020 | # If you would like to submit a bug report, please visit:
Sun Jun 28 02:54:11 UTC 2020 | # https://bugs.launchpad.net/ubuntu/+source/openjdk-lts
Sun Jun 28 02:54:11 UTC 2020 | # The crash happened outside the Java Virtual Machine in native code.
Sun Jun 28 02:54:11 UTC 2020 | # See problematic frame for where to report the bug.
Sun Jun 28 02:54:11 UTC 2020 | #
I can successfully use the following gst-launch command. so the problem must not be within gstreamer itself.
gst-launch-1.0 videotestsrc pattern=0 horizontal-speed=2 is-live=true ! textoverlay text="Test1" valignment=center font-desc="Sans, 24" ! clockoverlay halignment=left valignment=top time-format="%Y/%m/%d %H:%M:%S" ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! v4l2h264enc extra-controls='controls,video_bitrate=4000000;' ! h264parse config-interval=1 ! mux. audiotestsrc volume=0.5 freq=400 ! 'audio/x-raw, channels=2' ! avenc_mp2 ! mpegaudioparse ! mux. mpegtsmux name=mux alignment=7 si-interval=1000 pat-interval=1000 pmt-interval=1000 ! udpsink host=192.168.1.101 port=1234
Anybody else see this problem using v4l2 elements in gstreamer-java?
br,
Mark Garcia
other info...
v4l2h264enc parameters
v4l2-ctl --device /dev/video9 --all
Driver Info (not using libv4l2):
Driver name : coda
Card type : CODA960
Bus info : platform:coda
Driver version: 4.20.7
Capabilities : 0x84208000
Video Memory-to-Memory
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04208000
Video Memory-to-Memory
Streaming
Extended Pix Format
Priority: 2
Format Video Capture:
Width/Height : 1920/1088
Pixel Format : 'H264'
Field : None
Bytes per Line : 0
Size Image : 3133440
Colorspace : Rec. 709
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to Rec. 709)
Quantization : Default (maps to Full Range)
Flags :
Format Video Output:
Width/Height : 1920/1088
Pixel Format : 'NV12'
Field : None
Bytes per Line : 1920
Size Image : 3133440
Colorspace : Rec. 709
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to Rec. 709)
Quantization : Default (maps to Limited Range)
Flags :
Selection: compose, Left 0, Top 0, Width 1920, Height 1088
Selection: compose_default, Left 0, Top 0, Width 1920, Height 1088
Selection: compose_bounds, Left 0, Top 0, Width 1920, Height 1088
Selection: compose_padded, Left 0, Top 0, Width 1920, Height 1088
Selection: crop, Left 0, Top 0, Width 1920, Height 1088
Selection: crop_default, Left 0, Top 0, Width 1920, Height 1088
Selection: crop_bounds, Left 0, Top 0, Width 1920, Height 1088
Streaming Parameters Video Output:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Write buffers : 0
User Controls
horizontal_flip 0x00980914 (bool) : default=0 value=0
vertical_flip 0x00980915 (bool) : default=0 value=0
Codec Controls
video_gop_size 0x009909cb (int) : min=0 max=99 step=1 default=16 value=16
video_bitrate 0x009909cf (int) : min=0 max=32767000 step=1000 default=0 value=0
number_of_intra_refresh_mbs 0x009909d6 (int) : min=0 max=8160 step=1 default=0 value=0
sequence_header_mode 0x009909d8 (menu) : min=0 max=1 default=1 value=1
maximum_bytes_in_a_slice 0x009909db (int) : min=1 max=1073741823 step=1 default=500 value=500
number_of_mbs_in_a_slice 0x009909dc (int) : min=1 max=1073741823 step=1 default=1 value=1
slice_partitioning_method 0x009909dd (menu) : min=0 max=2 default=0 value=0
vbv_buffer_size 0x009909de (int) : min=0 max=262144 step=1 default=0 value=0
initial_delay_for_vbv_control 0x009909e1 (int) : min=0 max=32767 step=1 default=0 value=0
h264_i_frame_qp_value 0x00990a5e (int) : min=0 max=51 step=1 default=25 value=25
h264_p_frame_qp_value 0x00990a5f (int) : min=0 max=51 step=1 default=25 value=25
h264_maximum_qp_value 0x00990a62 (int) : min=0 max=51 step=1 default=51 value=51
h264_level 0x00990a67 (menu) : min=0 max=11 default=11 value=11
h264_loop_filter_alpha_offset 0x00990a68 (int) : min=0 max=15 step=1 default=0 value=0
h264_loop_filter_beta_offset 0x00990a69 (int) : min=0 max=15 step=1 default=0 value=0
h264_loop_filter_mode 0x00990a6a (menu) : min=0 max=1 default=0 value=0
h264_profile 0x00990a6b (menu) : min=0 max=0 default=0 value=0
mpeg4_i_frame_qp_value 0x00990a90 (int) : min=1 max=31 step=1 default=2 value=2
mpeg4_p_frame_qp_value 0x00990a91 (int) : min=1 max=31 step=1 default=2 value=2
mpeg4_level 0x00990a95 (menu) : min=0 max=7 default=7 value=7
mpeg4_profile 0x00990a96 (menu) : min=0 max=0 default=0 value=0
gst-inspect
gst-inspect-1.0 v4l2h264enc
Factory Details:
Rank primary + 1 (257)
Long-name V4L2 H.264 Encoder
Klass Codec/Encoder/Video
Description Encode H.264 video streams via V4L2 API
Author ayaka <ay...@soulik.info>
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstvideo4linux2.so
Version 1.14.5
License LGPL
Source module gst-plugins-good
Source release date 2019-05-29
Binary package GStreamer Good Plugins (Ubuntu)
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----GstV4l2VideoEnc
+----GstV4l2H264Enc
+----v4l2h264enc
Implemented Interfaces:
GstPreset
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h264
stream-format: byte-stream
alignment: au
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "v4l2h264enc0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
device : Device location
flags: readable
String. Default: "/dev/video9"
device-name : Name of the device
flags: readable
String. Default: null
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
output-io-mode : Output side I/O mode (matches sink pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
capture-io-mode : Capture I/O mode (matches src pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
extra-controls : Extra v4l2 controls (CIDs) for the device
flags: readable, writable
Boxed pointer of type "GstStructure"
Neil C Smith
Jun 28, 2020, 2:21:01 PM6/28/20
to gstream...@googlegroups.com
On Sun, 28 Jun 2020 at 08:50, Mark Anthony Garcia
<excursi...@gmail.com> wrote:
> Maybe the gstreamer-java Element.set() is not utilizing the appropriate vernacular necessary for gstreamer itself.
Welcome! This statement is partly true - *you're* not using the
<excursi...@gmail.com> wrote:
> Maybe the gstreamer-java Element.set() is not utilizing the appropriate vernacular necessary for gstreamer itself.
appropriate vernacular necessary for GStreamer itself!
> the "extra-controls" string value appears different than other typical values of properties in elements.
> extra-controls : Extra v4l2 controls (CIDs) for the device
> flags: readable, writable
> Boxed pointer of type "GstStructure"
extra-controls is not a String - it's a Structure (whether that will
> flags: readable, writable
> Boxed pointer of type "GstStructure"
work or not is another question right now).
Your best bet is probably to set the value as a string.
Either use Gst::parseLaunch to create the individual element or the
whole pipeline.
Or use the recently added GstObject::setAsString which will use
GStreamer's support for deserializing values from a String for you.
https://javadoc.io/static/org.freedesktop.gstreamer/gst1-java-core/1.2.0/org/freedesktop/gstreamer/Gst.html#parseLaunch-java.lang.String-
https://javadoc.io/static/org.freedesktop.gstreamer/gst1-java-core/1.2.0/org/freedesktop/gstreamer/GstObject.html#setAsString-java.lang.String-java.lang.String-
Note the parseLaunch Javadoc on that one method incorrectly states
"Create a new pipeline" - you can use it for single elements too.
Hopefully that will work for you.
Best wishes,
Neil
--
Neil C Smith
Codelerity Ltd.
www.codelerity.com
Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD
Mark Anthony Garcia
Jun 29, 2020, 3:08:44 PM6/29/20
to gstreamer-java
Thanks Neil! It makes a lot of sense now that the type of the value that I was using was wrong. I'll be circling back to attempting to use a GstStructure in the future.
For now parseLaunch worked as you described to create the individual element from a string. I will proceed on my project utilizing the parseLaunch strategy.
Successful Example:
Element encoderElement = Gst.parseLaunch("v4l2h264enc extra-controls=controls,video_bitrate=2000000,video_gop_size=15");
I wanted to also provide some feed back on using setAsString(). setAsString() yielded a java.lang.NoSuchMethodError. It can't be the case though. I'm thinking my linking or order of linking must be the problem.
I owe you a pint next time I'm in the UK.
Cheers,
Mark A Garcia
Neil C Smith
Jun 30, 2020, 5:38:11 AM6/30/20
to gstream...@googlegroups.com
On Mon, 29 Jun 2020 at 20:08, Mark Anthony Garcia
<excursi...@gmail.com> wrote:
> I wanted to also provide some feed back on using setAsString(). setAsString() yielded a java.lang.NoSuchMethodError. It can't be the case though. I'm thinking my linking or order of linking must be the problem.
It was only added in gst1-java-core v1.2.0, so most likely thing is
<excursi...@gmail.com> wrote:
> I wanted to also provide some feed back on using setAsString(). setAsString() yielded a java.lang.NoSuchMethodError. It can't be the case though. I'm thinking my linking or order of linking must be the problem.
that you have an older version you're using / picking up somewhere.
> I owe you a pint next time I'm in the UK.
Reply all
Reply to author
Forward
0 new messages