How much work does it take to replace chromium's ffmpeg with Android MediaCodec (for aac codec)

[Lingchao]

Hi guys, chromium has a build flag proprietary_codecs which would tell chromium to build ffmpeg with support for things like H264, AAC, MP3 and MP4 codecs. According to ffpmeg's own website the usage of this flag is a grey area at best and against patent laws at worst.

So if we want to use chromium to build a marketable product, I am wondering whether it is possible to replace chromium's background ffmpeg with the OS's codecs like Android MediaCodec, at least only do it for AAC format. I am curious how much work we need to do to achieve this or it is a huge work ?

Thanks,

Lingchao

[PhistucK]

I think Chrome used that at some point and then switched to FFMPEG (I am not helpful :)).

☆PhistucK

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/33405d4c-988e-4cb5-a0b4-9197b4835824%40chromium.org.

[Dale Curtis]

If you flip proprietary_codecs=true and media_use_ffmpeg=false, you should get a mostly working build (maybe completely working, but untested recently) that attempts to only use OS provided codecs.

- dale

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABc02_J_C4Mfe4GwHdXoOKr8mktY5XGkyqqHgmRjPUY%3Dq5cFmA%40mail.gmail.com.

[Lingchao]

Thanks Dale!

I tried to build a release version per your suggestion, and the args.gn file is like

target_os = "android"

target_cpu = "arm"

is_debug = false

is_component_build = false

symbol_level = 1 

ffmpeg_branding = "Chrome"

proprietary_codecs = true

media_use_ffmpeg = false

But it cause build failure when in the last link phase, and the error message is like:

../../content/renderer/media/android/webmediaplayer_android.cc:1019: error: unde   fined reference to 'media::MakeTextFrameForCast(std::__ndk1::basic_string<char,    std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, gfx::Size   , gfx::Size, base::Callback<gpu::gles2::GLES2Interface* (), (base::internal::Cop   yMode)1, (base::internal::RepeatMode)1> const&)'

../../content/renderer/render_frame_impl.cc:2730: error: undefined reference to    'media::WebMediaPlayerImpl::WebMediaPlayerImpl(blink::WebLocalFrame*, blink::Web   MediaPlayerClient*, blink::WebMediaPlayerEncryptedMediaClient*, base::WeakPtr<me   dia::WebMediaPlayerDelegate>, std::__ndk1::unique_ptr<media::RendererFactory, st   d::__ndk1::default_delete<media::RendererFactory> >, linked_ptr<media::UrlIndex>   , media::WebMediaPlayerParams const&)'

../../content/renderer/render_frame_impl.cc:2733: error: undefined reference to    'media::WebMediaPlayerImpl::SetMediaPlayerManager(media::RendererMediaPlayerMana   gerInterface*)'

../../content/renderer/render_frame_impl.cc:2734: error: undefined reference to    'media::WebMediaPlayerImpl::SetDeviceScaleFactor(float)'

collect2: error: ld returned 1 exit status

ninja: build stopped: subcommand failed.

Dale, do you know what cause these failures?

Thanks,

Lingchao

在 2017年6月9日星期五 UTC+8上午1:25:41，Dale Curtis写道：

If you flip proprietary_codecs=true and media_use_ffmpeg=false, you should get a mostly working build (maybe completely working, but untested recently) that attempts to only use OS provided codecs.

- dale

On Thu, Jun 8, 2017 at 5:54 AM, PhistucK <phis...@gmail.com> wrote:

I think Chrome used that at some point and then switched to FFMPEG (I am not helpful :)).

☆PhistucK

On Thu, Jun 8, 2017 at 3:13 PM, Lingchao <lingch...@gmail.com> wrote:

Hi guys, chromium has a build flag proprietary_codecs which would tell chromium to build ffmpeg with support for things like H264, AAC, MP3 and MP4 codecs. According to ffpmeg's own website the usage of this flag is a grey area at best and against patent laws at worst.

So if we want to use chromium to build a marketable product, I am wondering whether it is possible to replace chromium's background ffmpeg with the OS's codecs like Android MediaCodec, at least only do it for AAC format. I am curious how much work we need to do to achieve this or it is a huge work ?

Thanks,

Lingchao

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/33405d4c-988e-4cb5-a0b4-9197b4835824%40chromium.org.

[Maksim Sisov]

Lingchao писал 09.06.2017 10:44:

>> ☆PHISTUCK

>> On Thu, Jun 8, 2017 at 3:13 PM, Lingchao <lingch...@gmail.com>
>> wrote:
>>
>> Hi guys, chromium has a build flag proprietary_codecs which would
>> tell chromium to build ffmpeg with support for things like H264,

>> AAC, MP3 and MP4 codecs. According to ffpmeg's own website [1] the

>> usage of this flag is a grey area at best and against patent laws at
>> worst.
>>
>> So if we want to use chromium to build a marketable product, I am
>> wondering whether it is possible to replace chromium's background
>> ffmpeg with the OS's codecs like Android MediaCodec, at least only
>> do it for AAC format. I am curious how much work we need to do to
>> achieve this or it is a huge work ?
>>
>> Thanks,
>>
>> Lingchao
>>
>> --
>> --
>> Chromium Developers mailing list: chromi...@chromium.org
>> View archives, change email options, or unsubscribe:

>> http://groups.google.com/a/chromium.org/group/chromium-dev [2]

>> ---
>> You received this message because you are subscribed to the Google
>> Groups "Chromium-dev" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to chromium-dev...@chromium.org.
>> To view this discussion on the web visit
>>
> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/33405d4c-988e-4cb5-a0b4-9197b4835824%40chromium.org

>> [3].

>>
>> --
>> --
>> Chromium Developers mailing list: chromi...@chromium.org
>> View archives, change email options, or unsubscribe:

>> http://groups.google.com/a/chromium.org/group/chromium-dev [2]

>> ---
>> You received this message because you are subscribed to the Google
>> Groups "Chromium-dev" group.
>> To view this discussion on the web visit
>>
> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABc02_J_C4Mfe4GwHdXoOKr8mktY5XGkyqqHgmRjPUY%3Dq5cFmA%40mail.gmail.com

>> [4].

>
> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev
> ---
> You received this message because you are subscribed to the Google
> Groups "Chromium-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to chromium-dev...@chromium.org.
> To view this discussion on the web visit

> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/2e09575b-42ac-43a5-a88f-dbb53bd6f504%40chromium.org
> [5].
>
>
> Links:
> ------
> [1] https://www.ffmpeg.org/legal.html
> [2] http://groups.google.com/a/chromium.org/group/chromium-dev
> [3]
> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/33405d4c-988e-4cb5-a0b4-9197b4835824%40chromium.org?utm_medium=email&amp;utm_source=footer
> [4]
> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABc02_J_C4Mfe4GwHdXoOKr8mktY5XGkyqqHgmRjPUY%3Dq5cFmA%40mail.gmail.com?utm_medium=email&amp;utm_source=footer
> [5]
> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/2e09575b-42ac-43a5-a88f-dbb53bd6f504%40chromium.org?utm_medium=email&utm_source=footer

Hi,

"Undefined reference" means that one is missing deps in his/her BUILD.gn
file of that specific directory, where the source code resides in.

Br,
Maksim

[Lingchao]

Thanks Maksim!

I found the code snippet as below in /media/blink/Build.gn. Because I try to build an android version and also set media_use_ffmpeg=false, so the webmediaplayer_impl and webmediaplayer_cast_android cannot be built into chromium, and then cause the build error. Does it mean I need to write my own WebMediaPlayer if I set media_use_ffmpeg=false??? 

if (media_use_ffmpeg || !is_android) {

    sources += [

      "webmediaplayer_impl.cc",

      "webmediaplayer_impl.h",

    ]

    if (is_android) {

      sources += [

        "webmediaplayer_cast_android.cc",

        "webmediaplayer_cast_android.h",

      ]

      deps += [ "//gpu/command_buffer/client:gles2_interface" ]

    }

  }

在 2017年6月9日星期五 UTC+8下午3:56:05，Maksim Sisov写道：

[Raymond Toy]

On Thu, Jun 8, 2017 at 10:23 AM, Dale Curtis <dalec...@chromium.org> wrote:

If you flip proprietary_codecs=true and media_use_ffmpeg=false, you should get a mostly working build (maybe completely working, but untested recently) that attempts to only use OS provided codecs.

Just a note that if you want to use WebAudio, then this won't work.  All the code that WebAudio used for decoding audio with MediaCodec was removed.  For WebAudio, this was a good thing because the MediaCodec support was really quite buggy in various ways.  I was very, Very, VERY happy to see that go away in Project Spitzer. 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAPUDrwee%3DxcXmcTDeq282YPJ-4OSZGR7hWUtuf00cM2qtRuNRQ%40mail.gmail.com.

[Dale Curtis]

Sorry Lingchao I don't know the cause; if it doesn't immediately compile I expect there may be quite a series of errors and missing features (as rtoy@ notes below) you have to work through since this isn't a supported build setup.

- dale

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/2e09575b-42ac-43a5-a88f-dbb53bd6f504%40chromium.org.

[Lingchao]

Thanks Dale and Raymond, now I know the whole story. Raymond, yes, I mean WebAudio, what we concerned is aac format web audio decoding with ffmpeg may cause some potential issue for us. So if we decide to use Android MediaCodec to decode aac audio and use ffmpeg to decode other audios and videos, is it practical to do something like this in current chromium implmentation for a small team(1~2 devs with less media knowledge previously)?

I find the Project Spitzer document here, and if we do something described above, does it mean we need to rearchitect the media playback pipeline a little bit? Thanks in advance for your advices!

在 2017年6月10日星期六 UTC+8上午12:15:23，Dale Curtis写道：

[Raymond Toy]

On Mon, Jun 12, 2017 at 2:06 AM, Lingchao <lingch...@gmail.com> wrote:

Thanks Dale and Raymond, now I know the whole story. Raymond, yes, I mean WebAudio, what we concerned is aac format web audio decoding with ffmpeg may cause some potential issue for us. So if we decide to use Android MediaCodec to decode aac audio and use ffmpeg to decode other audios and videos, is it practical to do something like this in current chromium implmentation for a small team(1~2 devs with less media knowledge previously)?

I no longer remember the details, but I think if you find and revert the CL that removed MediaCodec support for WebAudio, you'll get what you want, mostly.  If you only want to decode aac with MediaCodec, you'll have to do some work; WebAudio used MediaCodec to decode all audio.

I don't recall any issues with decoding aac with MediaCodec.  Well, except that decoded result almost always differs from what ffmpeg does and what other browsers might do.  FFmpeg doesn't remove the trailing frames and sometimes removes the pre-roll frames.  MediaCodec somehow knew to remove both.  This is an issue if you want to do gapless playback with WebAudio.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/790ad4b4-4e4e-4910-8d5f-bd0ee6de5806%40chromium.org.

[Lingchao]

Thanks Raymond, I have found the CL here, and could you please help double confirm it is really this commit? Seems Dale is the author~:)

在 2017年6月13日星期二 UTC+8上午3:40:39，Raymond Toy写道：

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/790ad4b4-4e4e-4910-8d5f-bd0ee6de5806%40chromium.org.

[lingch...@gmail.com]

Hi Raymond and Dale, I have successfully bring back the kDisableUnifiedMediaPipeline switch, and then switch all media formats to WebMediaPlayerAndroid.

bool UseWebMediaPlayerImpl(const GURL& url) {

// Always use WMPI for playing blob URLs since WMPA could never play them very

// well and no longer has support for MSE based playbacks.

// if (url.SchemeIsBlob())

// return true;

And for blob urls, I also switch to WMPA, but according to your comments above and my own test, WMPA seems really have serious problem play blob urls. It cannot play youtube videos, but can play facebook videos, which are all blob urls. Could you guys tell me more about the story why WMPA could never play blob urls very well? Is there any way to solve this issue instead of switching back to unified media pipeline? I have tried to play youtube videos in a chromium webview and it works well, in my opinion, webview use android MediaPlayer to play media, and WMPA also use android MediaPlayer to play media, why the difference for blob urls?

Thanks,

Lingchao

在 2017年6月13日星期二 UTC+8下午6:10:28，Lingchao写道：

[Dale Curtis]

WebView also uses the unified media pipeline.

Some blob URLs are just Media Source Extensions; for those we've never used MediaPlayer, and always used something equivalent to the Unified Media Pipeline. To support those types of blob URLs (the ones used by FB/YT) you must leave the unified pipeline intact.

- dale

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/e3558650-48ac-4e19-80c9-1417b3dd3818%40chromium.org.

[Dale Curtis]

Notably to your original message, that should work even if you build with media_use_ffmpeg=false.

- dale

[Raymond Toy]

On Tue, Jun 13, 2017 at 3:10 AM, Lingchao <lingch...@gmail.com> wrote:

Thanks Raymond, I have found the CL here, and could you please help double confirm it is really this commit? Seems Dale is the author~:)

Sorry for the delay!  Yes, that's the one that removed MediaCodec support for WebAudio.

 

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/2aaa984c-d415-4c65-accc-7634c727bda7%40chromium.org.