Audio Metronome with Codename One - A Few Questions:

[tekk...@gmail.com]

Hello,

I’m looking into making a cross platform audio metronome app using Codename One. I have made a few music related Java Applets using MIDI but this will be my first audio program and also my first mobile app. I’ve done a bit of research already into working with audio in Java, but Codename One seems to work with audio quite differently than Java. From what I can tell by going through the Codename One API, it appears that Media and MediaManager are the only options for working with audio...is that correct?

Media:

https://codenameone.googlecode.com/svn/trunk/CodenameOne/javadoc/com/codename1/media/Media.html

MediaManager:

https://codenameone.googlecode.com/svn/trunk/CodenameOne/javadoc/com/codename1/media/MediaManager.html

If so, I have a few questions just to make sure that my metronome app will be possible to create using Codename One:

1) I’ve seen from different sources that low latency audio is very difficult to do on mobile devices and it seems to be a reoccurring theme that many people making mobile metronomes end up with severe time drift. Accuracy is of top priority, so will Media/MediaManager provide a way to play sounds consistently in time? What kind of latency will this have?

2) Will the accuracy and timing of playing audio with Media/MediaManager depend on the device, or will it be consistent among devices? For example, would android have stable time while the iPhone has time shifts, or vice versa?

I know there are accurate metronomes made for all kinds of devices, but they are written using native code. I haven’t seen any cross platform metronomes like what I’m hoping to do, so I am wondering if it will even be possible to do without writing native code for each device. That would be extremely difficult for me as I’m not that experienced of a programmer and most of my experience is with Java.

3) Can Media/MediaManager play multiple sounds simultaneously? The features I want to implement in the metronome will require occasionally playing more than one sound at the same time (for subdivisions, polyrhythms, etc)

3b) If Media/MediaManager cannot play multiple sounds simultaneously, what would be the best method to do that with Codename One? It will also need to have the option of being able to combine sounds that do not start and end at the same time but still have the sounds overlap.

4) This is unrelated to the audio side, but in Java you can use the method System.getnanotime() to time things based on nanoseconds, is there a way to do this in Codename One? I didn’t see a nanoseconds option in the API. I want to base the timing off of nanoseconds instead of milliseconds, because at certain tempos it would either speed up or slow down way too much using milliseconds. 

Also, I’d prefer not to use thread.sleep() for timing, as I’ve heard that this is unreliable and it doesn’t always return exactly at the specified time so it’s not good to use for a metronome.

I’ll probably have more questions down the road as I get going on this, but this should be enough to know if Codename One is the right path for me to continue on. Thanks!

PS. I have to say Codename One looks VERY impressive having watched several demonstration videos of it on youtube. I strongly hope that it will work for my project!

[Shai Almog]

Hi,
we don't support low level audio API's like the standard JVM since portability among devices is harder than among desktops and even on desktops the JVM was pretty problematic with media API's.

1. This would be a problem. It will also be a problem with GC stalls in the current VM.
The new VM should be better in that regard http://www.codenameone.com/3/post/2014/05/beating-the-arc.html

2. It will depend on the device.

3. Yes.

4. No. We don't provide that API at this time.


I think at least portions of your low level implementation have to be implemented natively since this specific type of application requires some realtime semantics and mobile OS's are by definition not realtime OS's (with the exception of QNX). You might be able to use native interfaces http://www.codenameone.com/how-do-i---access-native-device-functionality-invoke-native-interfaces.html

[tekk...@gmail.com]

Hi Shai,

That’s interesting that Media/MediaManager can play multiple sounds simultaneously, I was kind of expecting to hear that it wouldn’t be able to. 

And just to clarify, you are saying that with Media/MediaManager the latency could change each time it plays a sound file? Because even if the latency was ridiculously high, I think it should be ok as long as it was consistent and always takes the same amount of time to start playing the sound, since every click would simply be delayed by the same amount. Once the metronome starts it just needs to stay in time with itself from whenever starts playing. But it would be a problem if it took 50ms to play the first click, then 40ms for the next, then 60ms the next, etc...would that end up being the case with Media/MediaManager?

If it is at all possible to use Media/MediaManager that would be extremely helpful for sure, but If I do have to code a native low level audio player for each device using Codename One’s native interface that you mentioned...then essentially I would have to program the native audio player in Objective-C for IOS, Java for Android, C# for Windows phone, etc. is that correct?

And here I thought that all of the advanced features I have in mind were going to be the difficult part of this project...looks like just getting a simple ‘click’ to stay in time may be just as difficult. lol

Thanks!

[Shai Almog]

Hi,
the difference in terms of latency could be way higher. Mobile OS's are not realtime OS's, not even soft realtime OS's.
Since we also add a GC cycle and are not a realtime Java implementation this gets even worse.

Yes, you would need to do this in the native code of each platform to get anything remotely close to correct timing. Just to be clear, it won't be 100% accurate in the native platform code too since the "problem" is rooted in the OS's.

[tekk...@gmail.com]

Thanks again Shai!

On the native interface video you linked to, you mention Objective-C code for iOS, Java Dalvik code for Android, C# code for Windows Phone, and Java SE code....will that cover all of the devices that Codename One supports?

[Shai Almog]

There is also J2ME/RIM but I'm assuming you don't want to go there. For now that's all.

[tekk...@gmail.com]

I would like to cover as many devices as possible...but.....I'm not getting good vibes about J2ME/RIM. lol

That's Blackberry right? Is it difficult to work with, or buggy, or something else? I'd like to include Blackberry, but if it's ridiculously more difficult than the others then I may not be able too.

Thanks.

[Shai Almog]

Blackberry is awful to work with and so is Windows Phone unfortunately.

[tekk...@gmail.com]

Well, that sucks! lol I'll try looking into it, but I may get the others working first and then worry about Blackberry at some other time.

One last question...I've been thinking that in theory I should be able to implement a way to get nanoseconds using the native interface. However, I want to do the bulk of my code in Codename One, which would include a thread that would continuously call my "get nanoseconds" method coded in the native interface and then performing checks to see which sound files to play at certain times. Would that work ok for the threading to be done in Codename One and then have it call my native "get nanoseconds" method, or would the threading also need to be done native in order to get accurate timing?

I certainly hope I don't have to code most of this project native...I don't want to defeat the purpose of using Codename One.

Thanks.

[Shai Almog]

Yes that should work well. We actually recommend doing it that way since it makes your code easier to trace.

[tekk...@gmail.com]

Great to hear! Threads are going to be a huge part of this project, so that's good to know I'll only have to write them once in Codename One and it won't negatively affect the timing of the metronome.

Thanks again for all your help!...and I'm sure you hear this all the time but you are a GENIUS! :) I've spent a little time with the GUI builder in Codename One and it is beyond awesome. I have some pretty ambitious ideas that I'd like to try to do with the GUI to make it highly customizable and with the GUI builder it doesn't seem like such a daunting task to complete.