Re: NuPlayer and OMX.google.* decoders on JB

[Raghu naik]

I am also facing similar issue.Any update ?

On Thursday, 31 January 2013 01:17:49 UTC+5:30, Daniel Kim wrote:

Hi all,

I'm trying to get CTS verifier to pass RTSP streaming video quality test. Can someone tell me if NuPlayer (player for RTSP stream) works with OMX.google.* codec? I see codec is decoding video fine but (see the buffer going in and out.) I just see a black rectangle

when I look at the code,

ACodec.cpp:3100

    if (msg->findObject("native-window", &obj)                                                      

            && strncmp("OMX.google.", mCodec->mComponentName.c_str(), 11)) {                        

        sp<NativeWindowWrapper> nativeWindow(                                                         

                static_cast<NativeWindowWrapper *>(obj.get())); 

Seems if it uses SW codec, it doesn't register its native window. I tried removing the check for SW codec and it caused the player to segfault.

Is there an update for this? 

I also noticed HTTP test which uses AwesomePlayer with the same codec plays the video fine. (Because it uses different mechanism for rendering I think.)

Thanks,

Daniel

[Martin Storsjö]

Hi,

NuPlayer should work using the SW decoders in the latest version of AOSP
master now that https://android-review.googlesource.com/66262 has been
merged, but this isn't part of any existing release.

// Martin

> --
> You received this message because you are subscribed to the Google Groups
> "android-platform" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to android-platfo...@googlegroups.com.
> To post to this group, send email to android-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/android-platform.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Fabio Fumi]

Hi Martin,

I have an issue with a device where JB (4.2.1) has been ported to (project details at: http://groups.google.com/group/renesas-emev-osp/topics) for the playback of m3u8 URLs:

    V/MediaPlayerJNI( 1064): setDataSource: path http://rai-i.akamaihd.net/i/20140610/tg1notte-100620141.00.00_REPLAY_,400,600,800,.mp4.csmil/master.m3u8
    V/MediaPlayer( 1064): setDataSource(http://rai-i.akamaihd.net/i/20140610/tg1notte-100620141.00.00_REPLAY_,400,600,800,.mp4.csmil/master.m3u8)
    V/MediaPlayer( 1064): setVideoSurfaceTexture
    V/MediaPlayerJNI( 1064): setAudioStreamType: 3
    ...
    D/MediaPlayer( 1064): getMetadata
    E/MediaPlayerService(   74): getMetadata failed -38
    V/MediaPlayerJNI( 1064): start
    V/MediaPlayer( 1064): start
    ...
    V/MediaPlayer( 1064): start
    I/avc_utils(   74): found AVC codec config (700 x 394, Main-profile level 3.1)
    V/ACodec  (   74): Now uninitialized
    V/ACodec  (   74): onAllocateComponent
    I/ESQueue (   74): found AAC codec config (44100 Hz, 2 channels)
    I/EV2OMXPlugin(   74): ***run OMFPlugin ------- makeComponentInstance !!!!!!!!
    V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] Now Loaded
    V/ACodec  (   74): onConfigureComponent - msg: AMessage(what = 'setu', target = 14) = {
    V/ACodec  (   74):    string mime = "video/avc"
    V/ACodec  (   74):    int32_t width = 700
    V/ACodec  (   74):    int32_t height = 394
    V/ACodec  (   74):    ABuffer *csd-0 = 0x2a06af40
    V/ACodec  (   74):    ABuffer *csd-1 = 0x2a0722b0
    V/ACodec  (   74):    RefBase *native-window = 0x2a026e88
    V/ACodec  (   74):  }
    V/ACodec  (   74): initNativeWindow
    E/OMXNodeInstance(   74): OMX_GetExtensionIndex (index:'7fffffff') failed
    F/ACodec  (   74): frameworks/av/media/libstagefright/ACodec.cpp:3107 CHECK_EQ( (status_t)OK,mCodec->initNativeWindow()) failed: 0 vs. -2147483648
    F/libc    (   74): Fatal signal 11 (SIGSEGV) at 0xdeadbaad (code=1), thread 1490 (NuPlayerDecoder)
    ...

The OMX_GetExtensionIndex call fails, as the vendor OMX implementation doesn't include the NativeBuffer-related APIs.

In the OMXCodec (used by the AwesomePlayer to succesfully play mp4 files) initNativeWindow() isn't invoked at all (as per debug logs I've captured).

Our JB porting included this additional patch for the componentName, in the OMXCodec creation:

    OMXCodec::OMXCodec(
        const sp<IOMX> &omx, IOMX::node_id node,
        uint32_t quirks, uint32_t flags,
        bool isEncoder,
        const char *mime,
        const char *componentName,
        const sp<MediaSource> &source,
        const sp<ANativeWindow> &nativeWindow)
    : mOMX(omx),
    ...
        mNativeWindow(
            (!strncmp(componentName, "OMX.google.", 11)
            || !strncmp(componentName, "OMX.RENESAS.", 12)
            || !strcmp(componentName, "OMX.Nvidia.mpeg2v.decode"))
                    ? NULL : nativeWindow) {
    ...

So, looks like mNativeWindow is nullified in our case (OMX.RENESAS.) and that makes a bunch of logic being skipped, including initNativeWindow of course. That makes OMXCodec (AwesomePlayer) work as expected, on mp4 playback.

Adding this same check, not to call the InitNativeWindow logic, in ACodec::onConfgureComponent:

    if (msg->findObject("native-window", &obj)
            && strncmp("OMX.google.", mCodec->mComponentName.c_str(), 11)

        && strncmp("OMX.RENESAS.", mCodec->mComponentName.c_str(), 12)) {
            ...
        CHECK_EQ((status_t)OK, mCodec->initNativeWindow());
    }

NuPlayer isn't crashing anymore on the m3u8 playback, but screen remains black on m3u8 playback, while audio goes on!

I'm missing the patch you mentioned above in my code, but I'm wondering if that's the correct approach in my case as well...

Any hint greatly appreciated!

thanks
Fabio

[Fabio Fumi]

Martins's patch, that wasn't enough to make m3u8 video appear...

Comparing logs from OMXCodec (mp4 - OK) vs. ACodec (m3u8 - black video output), I can see similar calls repeating:

V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocating 2 buffers of size 1048576 on input port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a04b020 on input port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a065ca8 on input port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocating 5 buffers of size 3133440 on output port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a065dd0 on output port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a061de0 on output port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a061ec0 on output port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a061fc0 on output port
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] allocated buffer 0x2a0620a0 on output port
...
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] EMPTY_BUFFER_DONE(buffer: 0x2a04b020)
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] Calling emptyBuffer on buffer 0x2a04b020 (length 18339), timestamp 40000 us (0.04
...
V/OMXCodec(   74): [OMX.RENESAS.VIDEO.DECODER.H264] FILL_BUFFER_DONE(buffer: 0x2a065dd0, size: 449280, flags: 0x00000010, timestamp: 40000 us (0.04 secs))
...

V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] Now Loaded->Idle
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] Allocating 2 buffers of size 1048576 on input port
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] Allocating 5 buffers of size 3133440 on output port
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] Now Idle->Executing
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling fillBuffer 0x2a068970
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling fillBuffer 0x2a056fd0
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling fillBuffer 0x2a0570e8
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling fillBuffer 0x2a057228
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling fillBuffer 0x2a0572f8
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] Now Executing
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling emptyBuffer 0x2a054898 w/ codec specific data
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] onOMXEmptyBufferDone 0x2a054898
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling emptyBuffer 0x2a03cb88 w/ codec specific data
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling emptyBuffer 0x2a054898 w/ time 0 us
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] onOMXEmptyBufferDone 0x2a03cb88
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] onOMXEmptyBufferDone 0x2a054898
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling emptyBuffer 0x2a03cb88 w/ time 40000 us
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] onOMXEmptyBufferDone 0x2a03cb88
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling emptyBuffer 0x2a054898 w/ time 80000 us
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] calling emptyBuffer 0x2a03cb88 w/ time 120000 us
V/ACodec  (   74): [OMX.RENESAS.VIDEO.DECODER.H264] onOMXFillBufferDone 0x2a068970 time 0 us, flags = 0x00000010
...

So, out buffers seems being filled in with data, but it looks like something else is still missing ...

Any suggestion? Any place where to add additional debugging info?

[shivpratap chauhan]

Hi,

I don't know whether your codec implementation is in s/w or h/w. So, depending upon this, following are my suggestions:

1. s/w decoder  : You should look into SoftwareRenderer.cpp whether your NativeWindow buffer are updated correctly  with your decoded data? Also make sure to check color format.

2. h/w decoder: You should implement all required extensions and update native window buffers in codec (may

Thanks,
Shiv

--

You received this message because you are subscribed to the Google Groups "android-platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-platfo...@googlegroups.com.
To post to this group, send email to android-...@googlegroups.com.

Visit this group at http://groups.google.com/group/android-platform.

For more options, visit https://groups.google.com/d/optout.

[Fabio Fumi]

Hi Shiv, thanks for answering.

It's passed some time, but I'm still struggling with this...

This is a case of an H/W decoder. Could you be a bit more specific about "implement all required extensions"? What is the extension I'm missing? Seems to me exactly the same APIs are being called (emptyBuffer, EmptyBufferDone) for both the working MP4 playback, using OMXCodec, and the black-screen M3U8 stream, using ACodec. So I'm wondering if there's something's missing in the ACoded instead, like not being properly set up for my H/W, or something around these lines... But can't find what.

I'm also evaluating if a S/W codec could be forced in case of M3U8 stream, and I partially succeeded, but I later found another issue, better described here:

 http://stackoverflow.com/questions/26181554/stock-android-h264-decoder-failing-on-m3u8-url-files

Can you provide some help, in either case?

thanks in advance
Fabio

[shivpratap chauhan]

Hi,

S/W decoder:
I read this <http://stackoverflow.com/questions/26181554/stock-android-h264-decoder-failing-on-m3u8-url-files>. AFAIK stock Google's s/w decoder is only capable of decoding base line profile. So just make sure your stream stick to baseline profile.

H/W decoder:  You may want to follow these quick steps.

1. make sure mNativeWindow should not be null for these decoders.
2. OMX codec core, tries to get extension index for following features "OMX.google.android.index.enableAndroidNativeBuffers", "OMX.google.android.index.useAndroidNativeBuffer2", "OMX.google.android.index.storeMetaDataInBuffers" etc.
3. Once you successfully handle all these extension, from now onward, emptyBuffer, EmptyBufferDone buffers are basically of type nativeWindow handle (this handle let you update buffer on gpu also).
4. Queue dequeue on this handle to show your decoded frame.

Thanks,
Shiv