Not getting h264 encoding decoding to work

[André]

I'm working on an Omap 3 platform, basically creating a SIP
application. I run into some issues when creating the H264 encoding
and decoding parts of the application.

I've created the following gst pipeline for debugging purposes.

gst-launch v4l2src ! 'video/x-raw-
yuv,width=352,height=288,framerate=(fraction)15/1' ! videoparse !
dsph264enc ! dspvdec! autovideosink

I get the following output of gst:

# gst-launch v4l2src ! 'video/x-raw-
yuv,width=352,height=288,framerate=(fraction)15/1' ! videoparse !
dsph264enc ! dspvdec! autovideosink
Setting pipeline to PAUSED ...
0:00:00.211328265 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dspvdec0> NULL -> READY
0:00:00.211999651 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dsph264enc0> NULL -> READY
0:00:00.360040423 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dspvdec0> READY -> PAUSED
0:00:00.360559222 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dsph264enc0> READY -> PAUSED
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
0:00:00.430200335 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dspvdec0> PAUSED -> PLAYING
0:00:00.430627581 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dsph264enc0> PAUSED -> PLAYING
New clock: GstSystemClock
0:00:00.707696671 2002 0xd9b00 INFO dsp
gstdspbase.c:1201:sink_event:<dsph264enc0> event: newsegment
0:00:00.709588762 2002 0xd9b00 INFO dsp
gstdspvenc.c:230:check_supported_levels:<dsph264enc0> level bitrate:
2000000
0:00:00.709832903 2002 0xd9b00 INFO dsp
gstdspvenc.c:231:check_supported_levels:<dsph264enc0> max bitrate:
2000000
0:00:00.710016008 2002 0xd9b00 INFO dsp
gstdspvenc.c:232:check_supported_levels:<dsph264enc0> level: 20
0:00:00.809167619 2002 0xd9b00 INFO dsp
gstdspvenc.c:152:create_node:<dsph264enc0> dsp node created
0:00:00.810846086 2002 0xd9b00 INFO dsp
gstdspbase.c:748:gstdsp_start:<dsph264enc0> dsp node running
0:00:00.811181779 2002 0xd9b00 INFO dsp
gstdspbase.c:777:gstdsp_start:<dsph264enc0> creating dsp thread
0:00:00.814660783 2002 0xdca88 INFO dsp
gstdspbase.c:598:dsp_thread:<dsph264enc0> begin
0:00:00.830316301 2002 0xe1550 INFO dsp
gstdspbase.c:1201:sink_event:<dspvdec0> event: newsegment
init_node: error while parsing
0:00:00.835260148 2002 0xe1550 ERROR dsp
gstdspbase.c:1046:init_node:<dspvdec0> error while parsing
0:00:00.835839982 2002 0xe1550 INFO dsp
gstdspvdec.c:143:create_node:<dspvdec0> algo=h264vdec_sn.dll64P
dsp_thread: failed waiting for events: 5
0:00:00.931268449 2002 0xdca88 ERROR dsp
gstdspbase.c:608:dsp_thread:<dsph264enc0> failed waiting for events: 5
gstdsp_got_error: unable to get event
0:00:00.931695695 2002 0xdca88 ERROR dsp
gstdspbase.c:584:gstdsp_got_error:<dsph264enc0> unable to get event
0:00:00.932122941 2002 0xdca88 INFO dsp
gstdspbase.c:637:dsp_thread:<dsph264enc0> end
ERROR: from element /GstPipeline:pipeline0/GstDspH264Enc:dsph264enc0:
unable to get event
Execution ended after 501922606 ns.
Setting pipeline to PAUSED ...
0:00:00.933862443 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dspvdec0> PLAYING -> PAUSED
0:00:00.934198136 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dsph264enc0> PLAYING -> PAUSED
Setting pipeline to READY ...
0:00:00.936578508 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dspvdec0> PAUSED -> READY
0:00:00.939416643 2002 0xd9b00 INFO dsp
gstdspbase.c:1157:pad_chain:<dsph264enc0> status: error
0:00:00.940271135 2002 0xd9b00 INFO dsp
gstdspbase.c:1201:sink_event:<dsph264enc0> event: eos
create_node: dsp node allocate failed
0:00:00.940698381 2002 0xe1550 ERROR dsp
gstdspvdec.c:169:create_node:<dspvdec0> dsp node allocate failed
init_node: dsp node creation failed
0:00:00.941339250 2002 0xe1550 ERROR dsp
gstdspbase.c:1066:init_node:<dspvdec0> dsp node creation failed
0:00:00.942132707 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dsph264enc0> PAUSED -> READY
0:00:00.943048235 2002 0xe1550 INFO dsp
gstdspbase.c:508:output_loop:<dsph264enc0> pad push failed: error
0:00:00.943353410 2002 0xe1550 INFO dsp
gstdspbase.c:313:pause_task:<dsph264enc0> pausing task; reason error
0:00:00.945092912 2002 0x15050 INFO dsp
gstdspbase.c:651:destroy_node:<dsph264enc0> dsp node deleted
0:00:00.945459123 2002 0x15050 INFO dsp
gstdspbase.c:870:_dsp_stop:<dsph264enc0> dsp node terminated
Setting pipeline to NULL ...
stop: could not unmap
0:00:00.953668352 2002 0x15050 INFO dsp
gstdspbase.c:982:change_state:<dspvdec0> READY -> NULL
dsp_deinit: dsp detach failed
0:00:00.954461809 2002 0x15050 ERROR dsp
gstdspbase.c:702:dsp_deinit:<dspvdec0> dsp detach failed
Freeing pipeline ...


A simplified version of the pipeline without h264 encoding/decoding is
working fine:

I've used the head of master branch to compile gst-dsp.

Any thought?

Regards,
André

[Miguel]

Hi, from initial inspection it looks a problem with the decoder.

Try a simpler pipeline with the encoder only and to filesink
check that the resulting file is ok.
You can do this by wrapping the file to an mp4 using MP4Box
then play it with the dsp decoder.

That should be fine. Then try to do the same pipeline that you did
orinally but use a sw decoder. My bet is that this is an issue with
both dsp nodes run at the same time. we have seen thi in the past.
If this is the case then there is little we can do at gst-dsp.
but I haven't been hacking lately in gst-dsp so i would let Felipe to
check it out
M

[Felipe Contreras]

On Thu, Sep 1, 2011 at 11:56 AM, André <andre.kw...@gmail.com> wrote:
> I'm working on an Omap 3 platform, basically creating a SIP
> application. I run into some issues when creating the H264 encoding
> and decoding parts of the application.
>
> I've created the following gst pipeline for debugging purposes.
>
> gst-launch v4l2src ! 'video/x-raw-
> yuv,width=352,height=288,framerate=(fraction)15/1' ! videoparse !
> dsph264enc ! dspvdec! autovideosink

What happens if you try only the encoder? BTW, I don't think you need
videoparse there.

--
Felipe Contreras

[Rudolf]

Thanks for taking the time to reply to Andre's question.

We've been able to narrow it down to the encoder; a simple pipeline
like:
GST_DEBUG=dsp*:5 gst-launch v4l2src num-buffers=150 ! 'video/x-raw-
yuv,width=352,height=288,framerate=(fraction)15/1' ! dsph264enc !
filesink location=capture.fail.h264

crashes with the error mentioned above or MMU_FAULTS. We're unable to
pinpoint where it goes wrong.
Changing the number of output buffers (from 4 to 2) makes the
pipeline succeed once. The next tries result in same errors as
mentioned above.

We've been able to set up encoding/decoding pipelines with the same
kernel/bridgedriver module and an older version of gst-dsp; felipec's
github c2531569d4e5996d0224f4f03c6ed9c78b2772e6
with the following patches: http://pastebin.com/J060JBQz . Probably
some code from the parses branch for h264 fixes and patches to make it
work with telepathy/sofiasip.

but this is very unstable and the pipeline crashes randomly with '**
ERROR **: wrong buffer size' errors. This also terminates the
application.
That's why we are looking into updating to a newer version of gst-dsp,
but to no avail.

Another error is 'dsp node creation failed'; http://pastebin.com/nipKSBsX
This keeps occuring until the bridgedriver is reloaded.

Any advice on this matter would be highly appreciated.

[Felipe Contreras]

What version of the tidspbridge driver are you using? And how about
the H.264 encoder socket-node?

--
Felipe Contreras

[Rudolf]

Thank you for your swift reply.

#modinfo bridgedriver
filename: /lib/modules/2.6.32/kernel/drivers/staging/tidspbridge/
bridgedriver.ko
version: 0.3
license: GPL
author: Texas Instruments
srcversion: 09AD920845499579E2467BC
depends:
staging: Y
vermagic: 2.6.32 mod_unload modversions ARMv7
parm: dsp_debug:Wait after loading DSP image. default =
false (int)
parm: dsp_test_sleepstate:DSP Sleep state = 0 (int)
parm: base_img:DSP base image, default = NULL (charp)
parm: shm_size:shm size, default = 4 MB, minimum = 64 KB
(int)
parm: tc_wordswapon:TC Word Swap Option. default = 0 (int)

Source of H.264 encoder socket node is unknown.
Bas van de Berg (http://groups.google.com/group/gst-dsp/browse_thread/
thread/b6273ff5c81904cc) found them some time ago, and committed the
files as 'added DSP files from Nokia N900'

#strings h264venc_sn.dll64P | grep vers*

@(#)Id:H264V_ENCODERSN_64X Rev:version 0.49 Tag:\tii_int_3.0, DSP-MM-
TII-VIDEO_RLS_3.00.49 Built:Jul 28 2009 19:22:28

I've been unable to match it to any other release of binary packages.
And most of them lack an H.264 encoder.
The only packages that does is this one:
http://dev.wlan-si.net/browser/meshapp/wificonfig/lib/dsp?rev=3611047e2ba8ab43ef687df7a877ce63a86e8508&order=name

@(#)Id:H264V_ENCODERSN_64X Rev:version 0.49 Tag:\tii_int_3.0, DSP-MM-
TII-VIDEO_RLS_3.00.49 Built:Sep 24 2009 19:03:03

On 13 sep, 15:35, Felipe Contreras <felipe.contre...@gmail.com> wrote:
> On Tue, Sep 13, 2011 at 4:28 PM, Rudolf
>
>
>
>
>
>
>
>
>
> <rudolfhetroodneuzigerend...@gmail.com> wrote:
> > Thanks for taking the time to reply to Andre's question.
>
> > We've been able to narrow it down to the encoder; a simple pipeline
> > like:
> > GST_DEBUG=dsp*:5 gst-launch v4l2src num-buffers=150 ! 'video/x-raw-
> > yuv,width=352,height=288,framerate=(fraction)15/1' ! dsph264enc !
> > filesink location=capture.fail.h264
>
> > crashes with the error mentioned above or MMU_FAULTS. We're unable to
> > pinpoint where it goes wrong.
> > Changing the number of output  buffers (from 4 to 2) makes the
> > pipeline succeed once. The next tries result in same errors as
> > mentioned above.
>
> > We've been able to set up encoding/decoding pipelines with the same
> > kernel/bridgedriver module and an older version of gst-dsp; felipec's
> > github c2531569d4e5996d0224f4f03c6ed9c78b2772e6

> > with the following patches:http://pastebin.com/J060JBQz. Probably

[Felipe Contreras]

Hi,

Sorry for the long delay.

On Wed, Sep 14, 2011 at 10:35 AM, Rudolf
<rudolfhetrood...@gmail.com> wrote:
> Source of H.264 encoder socket node is unknown.
> Bas van de Berg (http://groups.google.com/group/gst-dsp/browse_thread/
> thread/b6273ff5c81904cc)  found them some time ago, and committed the
> files as 'added DSP files from Nokia N900'
>
> #strings h264venc_sn.dll64P | grep vers*
>
> @(#)Id:H264V_ENCODERSN_64X Rev:version 0.49 Tag:\tii_int_3.0, DSP-MM-
> TII-VIDEO_RLS_3.00.49 Built:Jul 28 2009 19:22:28

> @(#)Id:H264V_ENCODERSN_64X Rev:version 0.49 Tag:\tii_int_3.0, DSP-MM-

> TII-VIDEO_RLS_3.00.49 Built:Sep 24 2009 19:03:03

I was asking because you can play with different SN_API values. These
seem to be old ones so probably SN_API=0 would work (for the N900).

This is what I see for N9:

@(#)Id:H264V_ENCODERSN_64X Rev:version 0.49 Tag:\tii_int_3.0,

DSP-MM-TII-VIDEO_RLS_3.00.49 Built:Apr 6 2011 18:48:48

And that uses SN_API=2.

I tried this pipeline: gst-launch videotestsrc !
'video/x-raw-yuv,width=352,height=288,framerate=15/1' ! dsph264enc !
fakesink. And works fine.

Did you manage to get this working?

Cheers.
--
Felipe Contreras