videos of NDK-related sessions at I/O 2016

[Glenn Kasten]

Android NDK performance in an ART world - Google I/O 2016

https://www.youtube.com/watch?v=Ok97X9Z4Si8

Android high-performance audio - Google I/O 2016

https://www.youtube.com/watch?v=F2ZDp-eNrh4

[Felix Homann]

Am Samstag, 21. Mai 2016 21:42:23 UTC+2 schrieb Glenn Kasten:

Android high-performance audio - Google I/O 2016

https://www.youtube.com/watch?v=F2ZDp-eNrh4

I love that SAPA will be available for non-Samsung devices! Now, will this turn out to be "the" professional audio API (I'd be very comfortable with that) or will you still strive for something else? And can you give any hint on when I could be able to try this on my Nexus 9?

[Felix Homann]

And, while we're at it: On Android N Preview still only 8 of the 18 channels of my USB sound card (Behrigner XR18) are shown via AudioDeviceInfo. I had reported this for M Preview but can't find the report right now. I'd very much appreciate this to be fixed in final N ;-)

[Steven Winston]

Can I request that we get a finished API before we get yet another one?  I have no problem with OpenSL ES, AudioTrack, OpenMAX AL, or even MediaCodec Player, but none of them could rightly be considered a one stop shop for audio development in the NDK.  OpenSL ES and OpenMAX AL include so much more in the specification and even have promises that someday will have the missing pieces supported by Google in the docs surrounding statements of binary compatibility.  Neither has received much of any update in NDK generations.

[Glenn Kasten]

Thanks Felix and Steven for watching, and for your feedback.

Felix - sorry I can't comment further on SAPA at this time.

Regarding USB audio, my personal top priorities right now that I think need to be addressed include:

expanding the compatibility between Android devices and various USB class-compliant audio interfaces,

making sure that simultaneous I/O works well at various sample rates, bit depths, and channel counts (up to 8),

and providing good routing options for developers and users. Channel counts > 8 is certainly on my longer-term wish list also.

Please note I am not expressing commitment or release schedule in what I've just said.

Steven - you make good points about a "one stop shop" for APIs.

In some cases the right thing to do is to expand an API implementation,

and there are other times when a drastically different API is needed to better solve a problem.

For example, I've heard developers ask for a single callback that combines input and output streams together.

That could be added to OpenSL ES as a new combined Player/Recorder object, or it could be done with a different API.

Most of the developers I've spoken with have preferred a different API for combined I/O,

but it is good to hear differing opinions such as yours.

[Steven Winston]

Thanks Glenn,

I've been a fan of your work for years.  I think I understand where other developers are coming from, and I do agree, however, I wanted to make one further point if I may.  Imagine being tasked to write a media player for Android with the following requirements: 1.) must support HLS for streams 2.) must support non mpeg-ts for local files 3.) Must use the hardware decoders when possible.  3.) Must be in C++ as the entire code base will be used over several platforms.

Now to implement something so basic, we're in the territory of multiple APIs in the current system.  Native Media Codec doesn't support mpeg-ts so HLS can't be used for that requirement, OpenMAX AL doesn't support anything but mpeg-ts so can't be used for requirement number 2 and Java Media Player doesn't support things like being able to have direct access to PCM for making new filters etc.

Also in order to get the number 2 requirement for HEVC one would need to write a separate OpenSL ES enabled thread to take the audio decoded by native media codec and provide a place to play it.

One of the biggest reasons I can think of for doing work in C++ is having access to existing libraries, and writing code that can work in multiple platforms.  As soon as one says multiple platforms the problem only increases in scope and non-standard APIs fail this test.  While I care less about that it is still an annoyance.

Don't get me wrong, I love working in Android, but there's a time for some tough love when some things can be so much better about developing with this platform.  Also, worth pointing out that on some platforms we've had the ability to hit 10ms by getting access to SCHED-FIFO, GearVR allows developers to relax that requirement.  Realtime audio is definitely easier with that ability.  Not that latency is my current biggest concern, but wanted to point that out if you ever needed an example of how it is being used in the wild.

Oh, also I love the direction of Android Studio.  It has really come a long way in terms of working with the NDK!  Good job all around!  It's great progress to see cmake and existing android.mk support in 2.2! (now if we could just get gtest support).

--
You received this message because you are subscribed to a topic in the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/android-ndk/12gHP7gMzdg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/1d872ae4-9172-443f-a1a8-418302287cd4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Felix Homann]

Hi Glenn,

Am Montag, 23. Mai 2016 21:07:29 UTC+2 schrieb Glenn Kasten:

Felix - sorry I can't comment further on SAPA at this time.

Yeah, I guessed so ;-)

 

Regarding USB audio, my personal top priorities right now that I think need to be addressed include:

expanding the compatibility between Android devices and various USB class-compliant audio interfaces,

making sure that simultaneous I/O works well at various sample rates, bit depths, and channel counts (up to 8),

and providing good routing options for developers and users. Channel counts > 8 is certainly on my longer-term wish list also.

Can you elaborate on why you chose only channel counts up to 8? Sure, less powerful hardware can easily choke on too many channels.

Latency is one aspect of professional audio.  And surely the aspect I've been urging most over the last couple of years whenever we met here ;-)

But there are other aspects as well. A Midas M32 for example has a 32x32 channel USB class-compliant audio interface. You can record a live set from this console with a Raspi2. Sure you need huge buffers and make sure you silence your ethernet port. But you can do it with such cheap hardware. By comparison, a super-capable Pixel C is still useless for this purpose as long as it does support only 8x8 channels. You see, 8x8 might have been a lot of channels in 2012 for affordable hardware. Nowadays, with all those digital mixers coming up it's a minimum requirement not a maximum one.  What's professional in professional audio if you can't use your professional hardware at all? (Yes, I know, I'm exaggerating...)

This leads me to another plea I have for "SAPA4Android". Please, make it available for all devices that can run it and not only on devices that can run it with very low latency. The SAPA and Jack APIs are great. There are many legitimate use cases for those APIs that don't rely on low latency, e.g. the recording scenario above. As an app developer you can still filter out devices by the FEATURE_AUDIO_PRO flag if you rely on low latency. But it would be troublesome to use two different APIs for devices with and without the FEATURE_AUDIO_PRO flag if you don't rely on low latency but just want to profit from the SAPA/Jack APIs.

Another side note that might not be an issue anymore with USB C:

Truely professional audio Android devices should be able to use a USB audio interface and get charged at the same time. That's an issue with all the existing Nexus tablets. Even though these tablets can work perfectly fine with my audio interface in 2x2 mode I can't use them on stage for playback over USB as I can't charge them simultaneously. This should be considered as a future requirement for the FEATURE_AUDIO_PRO flag.

Kind regards,

Felix

[Glenn Kasten]

As mentioned earlier, I agree that the limit of 8 needs to be increased. We just have some other work in front of that.

Regarding USB C and power, see https://en.wikipedia.org/wiki/USB#PD

Some of the first USB C devices did not support USB PD.

I anticipate that devices with PD will be released, but I don't know the schedule.