Possible fix for green video when using hardware acceleration

[lambdadroid]

Hi,

I noticed that you've disabled the hardware accelerated "OMX.ffmpeg.vp9.decoder" temporarily due to green video when using it with YouTube (in this commit). I recently tested the FFMPEG hardware acceleration for my own Android port and had a similar problem when using the "OMX.ffmpeg.h264.decoder" in Firefox or in the YouTube app, while it was working fine when playing a local media file.

As far as I could tell, the error only occurred when adaptive bitrates are used (e.g. when streaming from YouTube when it changes the resolution automatically). The decoder is created with a higher height/width than the original video (to avoid re-allocation when the resolution changes) and has to adapt the video to the bigger buffer size. Normally, the FFMPEG decoder does this by scaling the video.

Currently, there is a small mistake in the call to "sws_scale" that prevents scaling when the final height is different from the source height. Instead of the original height like required by the method, the final height is passed to it (which is only correct when the source height is equal to the final height, so only when playing the local media file). Correcting the method call fixed the green video for me.

Fix green video when scaling video
https://github.com/lambdadroid/external_stagefright-plugins/commit/34ff121ac239c235d880e73453fbd9edb5be6a43

I didn't test this with the VP9 decoder or directly on Android x86, but I could imagine that these issues are related.

Additional note (unrelated to the green video issue):

If you want to make the YouTube app use the FFMPEG H264 decoder you need to add the supported H264 profiles to the decoder. Otherwise, the YouTube app will always skip it and fall back to the Google software decoder.

Add profile levels for h264 decoder
https://github.com/lambdadroid/external_stagefright-plugins/commit/8ef706555e698db07151b25aac650030b58814c5

Maybe this is useful for you too. :)

[Chih-Wei Huang]

2017-08-23 20:22 GMT+08:00 lambdadroid <lambd...@gmail.com>:
>
> I noticed that you've disabled the hardware accelerated
> "OMX.ffmpeg.vp9.decoder" temporarily due to green video when using it with
> YouTube (in this commit). I recently tested the FFMPEG hardware acceleration
> for my own Android port and had a similar problem when using the
> "OMX.ffmpeg.h264.decoder" in Firefox or in the YouTube app, while it was
> working fine when playing a local media file.
>
> As far as I could tell, the error only occurred when adaptive bitrates are
> used (e.g. when streaming from YouTube when it changes the resolution
> automatically). The decoder is created with a higher height/width than the
> original video (to avoid re-allocation when the resolution changes) and has
> to adapt the video to the bigger buffer size. Normally, the FFMPEG decoder
> does this by scaling the video.
>
> Currently, there is a small mistake in the call to "sws_scale" that prevents
> scaling when the final height is different from the source height. Instead
> of the original height like required by the method, the final height is
> passed to it (which is only correct when the source height is equal to the
> final height, so only when playing the local media file). Correcting the
> method call fixed the green video for me.
>
> Fix green video when scaling video
> https://github.com/lambdadroid/external_stagefright-plugins/commit/34ff121ac239c235d880e73453fbd9edb5be6a43

Ah! Many thanks!
That solves one of my headache.

> I didn't test this with the VP9 decoder or directly on Android x86, but I
> could imagine that these issues are related.
>
> Additional note (unrelated to the green video issue):
> If you want to make the YouTube app use the FFMPEG H264 decoder you need to
> add the supported H264 profiles to the decoder. Otherwise, the YouTube app
> will always skip it and fall back to the Google software decoder.
>
> Add profile levels for h264 decoder
> https://github.com/lambdadroid/external_stagefright-plugins/commit/8ef706555e698db07151b25aac650030b58814c5

Fine.
Indeed I have a very similar patch in my local repo
but I didn't publish it because it is affected
by the green video issue.
With the profiles Youtube will select my
OMX.ffmpeg.h264.decoder however
it just shows green video as you know.
Now I can enable it and enjoy hw accelerated
decoding in Youtube.

> Maybe this is useful for you too. :)

It's very helpful!
Thank you again!


--
Chih-Wei
Android-x86 project
http://www.android-x86.org

[Chih-Wei Huang]

2017-08-24 1:18 GMT+08:00 Chih-Wei Huang <cwh...@android-x86.org>:
>
> Ah! Many thanks!
> That solves one of my headache.
>
>> I didn't test this with the VP9 decoder or directly on Android x86, but I
>> could imagine that these issues are related.
>>
>> Additional note (unrelated to the green video issue):
>> If you want to make the YouTube app use the FFMPEG H264 decoder you need to
>> add the supported H264 profiles to the decoder. Otherwise, the YouTube app
>> will always skip it and fall back to the Google software decoder.
>>
>> Add profile levels for h264 decoder
>> https://github.com/lambdadroid/external_stagefright-plugins/commit/8ef706555e698db07151b25aac650030b58814c5
>
> Fine.
> Indeed I have a very similar patch in my local repo
> but I didn't publish it because it is affected
> by the green video issue.
> With the profiles Youtube will select my
> OMX.ffmpeg.h264.decoder however
> it just shows green video as you know.
> Now I can enable it and enjoy hw accelerated
> decoding in Youtube.
>
>> Maybe this is useful for you too. :)
>
> It's very helpful!
> Thank you again!

If I re-enable ffmpeg vp9 decoder, youtube in Chrome
doesn't show green video now but the video still messed up
as shown:

http://i.imgur.com/29aCALV.png

Do you have any idea what the problem is?

[lambdadroid]

That's weird, I'm not sure. I remember a problem where the video wasn't scaled properly but it didn't look like that. Does it change if you switch the resolution manually? Usually, at least one of the resolutions displayed correctly for me.

You can also try removing the adaptive-bitrate feature from the FFMPEG codecs. That means Android will need to reset the codecs when switching resolution instead of just passing in new video data. You need to remove this line for the video codecs:

<Feature name="adaptive-playback" />

However, if I remember correctly it was actually even worse when I removed that....