AAC sound and FFmpegDemuxer and ChunkDemuxer

[Igor Bukanov]

Hi,

As I understand the split between media demuxer and decoder implies that implementation of AudioDecoder should work with any Demuxer implementation.

Yet in case of AAC audio there is a difference between ChunkDemuxer and FFmpegDemuxer. As we discovered during implementation of AudiCodec that uses platform libraries to decode AAC sound, while the data coming from ChunkDemuxer include adts_sequence() headers, the data coming from FFmpegDemuxer are raw AAC with adts header stripped.

Apparently the native FFmpeg decoder can deal with both types of data, but platform decoder libraries needs explicit configuration. So the question is there a way to deduce in AudioDecoder::Initialize() method which format the data will use based on AudioDecoder?

Regards, Igor

[Dan Sanders]

My understanding is that Chrome's demuxers always prefer ADTS (they will add ADTS headers), but notably this isn't actually possible for xHE-AAC so there is room for confusion. You can see this logic being relied on by the Android MediaCodec decoder.

I have a recollection that there was also code to detect ADTS somewhere in Chrome, but I can't find it. Maybe it was a different format that was being detected.

The future goal is to handle this the way we have defined in in WebCodecs: if there is extra_data then it's AAC, if there is no extra_data then it's ADTS. There has been some work toward this goal but I would not consider it a reliable mechanism in Chrome at this time.

- Dan

--
You received this message because you are subscribed to the Google Groups "media-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to media-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/media-dev/c27a4d7b-ae83-4fa2-8dd0-b9e08b0e96ecn%40chromium.org.

[Dale Curtis]

If you implement AudioDecoder::NeedsBitstreamConversion() to return true you'll always get ADTS headers for AAC content.

- dale

[Igor Bukanov]

On 2021-09-08 20:13, Dale Curtis wrote:
> If you implement AudioDecoder::NeedsBitstreamConversion() to return
> true you'll always get ADTS headers for AAC content.

Unfortunately that only work with FFmpegDemuxer if
USE_PROPRIETARY_CODECS is defined. Without that
FFmpegDemuxerStream::EnableBitstreamConverter() reports an error. And we
implement AudioDecoder using system libraries precisely to avoid
decoding bits of AAC software decoder in FFmpeg that are sre stil under
patents.

But then it seems wrong that the code in FFmpegDemuxer requires
USE_PROPRIETARY_CODECS in that code. FFmpeg on Mac includes own
implementation of AAC decoder using AudioToolbox and that can be made to
work with Chromium. So insisting on USE_PROPRIETARY_CODECS is
unnecessary restrictive.

Regards, Igor

[Igor Bukanov]

On 2021-09-08 20:07, Dan Sanders wrote:
> The future goal is to handle this the way we have defined in in

> WebCodecs [3]: if there is extra_data then it's AAC, if there is no

> extra_data then it's ADTS. There has been some work toward this goal
> but I would not consider it a reliable mechanism in Chrome at this
> time.

Thanks for the link. We observed that it is at least seems to be true
for ChunkDemuxer/FFmpegDemuxer. But if this is not reliable then I guess
we need to patch demuxers to pass that info via an extra field in
AudioDecoderConfig.

Regards, Igor

[Dale Curtis]

It's very possible we're overly conservative with what we've included under USE_PROPRIETARY_CODECS -- but that's the safest choice. I'd suggest you talk with your lawyers if you want to move that block out from under proprietary codecs.

- dale

[Igor Bukanov]

On 2021-09-08 20:53, Dale Curtis wrote:
> It's very possible we're overly conservative with what we've included
> under USE_PROPRIETARY_CODECS -- but that's the safest choice. I'd
> suggest you talk with your lawyers if you want to move that block out
> from under proprietary codecs.

Fortunately we do not need to depend on the presence of that block (even
if it is very tempting to patch out USE_PROPRIETARY_CODECS checks there
as it will do simplify codec implementation - thanks ). AudioToolbox and
Windows Media Foundation supports decoding with and without ADTS
headers. It is just a matter of properly configuring API. But then, as
was pointed in other threads, presence of extra_data in
AudioDecoderConfig is a good indicator of the format. And if that turned
out to be not reliable, we can patch the config and demuxers with more
precise indicators even if that is not ideal.

Regards, Igor

[Xiaohan Wang (王消寒)]

ChunkDemuxer always adds ADTS. FFmpegDemxuer by default doesn't have ADTS unless NeedsBitstreamConversion() returns true.

IIUIC, ADTS in audio samples is independent of the extradata (ASC) in the decoder config... 

--
You received this message because you are subscribed to the Google Groups "media-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to media-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/media-dev/8bd9a1bf81464fe9c2e22be228337456%40vivaldi.com.