iOS sounds (also built-in) not working properly (crashes on device)

[JJ]

Hi,

I'm trying to add sounds to my app but cannot get it to work on iOS devices.

The app keeps crashing after a few sounds have played. (Successive sounds are not played rapidly one after another.)

A crash means the app is moved to background and when moved to foreground it either restarts or hangs.

Playing the sounds works fine in the Simulator and on Android device.

CN1 version is 1.0.77. NetBeans 7.3 & Windows 7. 

Devices: iPad/iOS 7.1.2 and iPodTouch/iOS 6.1.6, Nexus7/Android 4.4.4

I've tried several methods, see the code below.

1. with an input stream from resource or from jar - works fine in simulator and on android device, crashes on ios devices

2. as 1, wrapped in a call serially - same result

3. copy wav to Storage and use a uri - could not get this to work anywhere; how does a uri work for Storage files?

4. (re-)use a MediaPlayer and setDataSource(is) - simulator:ok, ipad:crash after few, nexus:NullPointerException

5. built-in sounds and install new - nothing to hear on iPad/iPod (not tested on android)

The functions below are called for instance like this: playSound1("click2.wav");

Please help as I ran out of things to try.

    

    private void playSound1(final String soundFile) {

        try {

            //final InputStream is = soundres.getData(soundFile);

            final InputStream is = Display.getInstance().getResourceAsStream(getClass(), "/" + soundFile);

            Media media = MediaManager.createMedia(is, "audio/wave", null);

            media.setVolume(SOUND_VOLUME);

            media.play();

        } catch (IOException ex) {

            Dialog.show("Error playing " + soundFile, ex.getMessage(), STR_OK, null);

        }

    }

    

    private void playSound2(final String soundFile) {

        Display.getInstance().callSerially(new Runnable() {

            public void run() {

                playSound1(soundFile);

            }

        });

    }

    private void playSound3(final String soundFile) {

        try {

            Storage s = Storage.getInstance();

            if (!s.exists(soundFile)) {

                InputStream  is = Display.getInstance().getResourceAsStream(getClass(), "/" + soundFile);

                OutputStream os = s.createOutputStream(soundFile);

                Util.copy(is, os);

            }

            //String uri = soundFile;              // ERROR: resources [...] must start with a '/' character

            //String uri = "/" + soundFile;        // IllegalArgumentException: uri.getScheme() == null

            //String uri = "file://" + soundFile;  // IllegalArgumentException

            //String uri = "file:///" + soundFile; 

            // ^ Simulator: IllegalArgumentException

            // ^ iPad: no sound, no crash

            // ^ Nexus: Error playing click2.wav: open failed: ENOENT (No such file or directory)

            String uri = "http://www.villagegeek.com/downloads/webwavs/affirmative.wav";

            // ^ works but not flawless: about 1:10 you hear nothing, and still a crash after 50 times on ipad

            Media media = MediaManager.createMedia(uri, false);

            media.setVolume(SOUND_VOLUME);

            media.play();

        } catch (IOException ex) {

            Dialog.show("Error playing " + soundFile, ex.getMessage(), STR_OK, null);

        }

    }

    

    private void playSound4(final String soundFile) {

        if (mp == null) {

            mp = new MediaPlayer();

        }

        try {

            InputStream is = Display.getInstance().getResourceAsStream(getClass(), "/" + soundFile);

            if (is != null) {

                mp.setDataSource(is, "audio/wave", null);

                mp.getMedia().setVolume(SOUND_VOLUME);

                mp.getMedia().play();

            } else {

                Dialog.show("Error", soundFile + " not found", STR_OK, null);

            }

        } catch (IOException ex) {

            Dialog.show("Error playing " + soundFile, ex.getMessage(), STR_OK, null);

        }

    }

    

    private void playSound5(final String soundFile) {

        System.err.println("isBuiltinSoundsEnabled = " + Display.getInstance().isBuiltinSoundsEnabled()); // true

        // try standard built-in sound

        System.err.println("SOUND_TYPE_ERROR available = " + Display.getInstance().isBuiltinSoundAvailable(Display.SOUND_TYPE_ERROR)); // true

        //Display.getInstance().playBuiltinSound(Display.SOUND_TYPE_BUTTON_PRESS); -- nothing

        //Display.getInstance().playBuiltinSound(Display.SOUND_TYPE_ALARM);        -- nothing

        //Display.getInstance().playBuiltinSound(Display.SOUND_TYPE_CONFIRMATION); -- nothing

        //Display.getInstance().playBuiltinSound(Display.SOUND_TYPE_ERROR);        -- nothing

        //Display.getInstance().playBuiltinSound(Display.SOUND_TYPE_INFO);         -- nothing

        //Display.getInstance().playBuiltinSound(Display.SOUND_TYPE_WARNING);      -- nothing

        //Display.getInstance().vibrate(500); // doesn't work either

        // try to add a sound

        String sound = soundFile.substring(0, soundFile.length()-4);    // - ".wav"

        if (Display.getInstance().isBuiltinSoundAvailable(sound)) {

            System.err.println("sound available!");

        } else {

            System.err.println("install sound...");

            InputStream is = Display.getInstance().getResourceAsStream(getClass(), "/" + soundFile);  // (1)

            try {

                Display.getInstance().installBuiltinSound(sound, is);

                System.err.println("...ok.");

            } catch (IOException ex) {

                Log.p(ex.getMessage());

            }

        }

        Display.getInstance().playBuiltinSound(sound);        

    }

[Shai Almog]

Hi,
we made a fix for that recently but generally I'd recommend against using this API since we need to read the whole stream then play it in order to implement this functionality.
I suggest copying the sounds to the app home directory of the FileSystemStorage then playing using the URL of the file under the FileSystemStorage. This will work far better both for Android & iOS.

[JJ]

Thank you for your reply.

I tried this with Storage instead of FileSystemStorage (see playSound3) - does that matter? 

The thing is I don't understand which URI to use in this case; can you show how this is done?

[Shai Almog]

Storage has no URL so yes it matters. Only file system storage has URL's.

[JJ]

Apparently my second question was too stupid to answer. The uri for createMedia is the same as the full path+name of the FileSystemStorage file.

Thus I arrive at the following version:

    private void playSound3a(final String soundFileName) {

        try {

            FileSystemStorage fss = FileSystemStorage.getInstance();

            final char SEP = fss.getFileSystemSeparator();

            String mySoundDir = fss.getAppHomePath() + DIR_SOUNDS;         // must use a directory! ("sounds")

            //String mySoundDir = fss.getAppHomePath() + SEP + DIR_SOUNDS; // Simulator needs extra + SEP, devices don't!

            if (!fss.exists(mySoundDir)) {

                fss.mkdir(mySoundDir);

            }

            String soundPathFileName = mySoundDir + SEP + soundFileName;

            if (!fss.exists(soundPathFileName)) {

                InputStream  is = Display.getInstance().getResourceAsStream(getClass(), "/" + soundFileName);

                OutputStream os = fss.openOutputStream(soundPathFileName);

                Util.copy(is, os);

            }

            String uri = soundPathFileName;

            Media media = MediaManager.createMedia(uri, false);

            media.setVolume(SOUND_VOLUME);

            media.play();

            Log.p(uri + " played!");

        } catch (IOException ex) {

            Log.p("Error playing " + soundFileName + " " + ex.getMessage());

        }

    }

Remark: FileSystemStorage.getInstance().getAppHomePath() returns a path including trailing '/' on the devices, but in the Simulator without the trailing '/'. Can this be changed?

Q: Is this version as it should be?

Because it exhibits the same behaviour as all the other versions: it works fine in the Simulator and on an Android device, but on iOS devices the app keeps crashing. With sounds disabled it runs fine on iOS.

I did some more testing (using Log.p) and found that the crash/freeze doesn't appear inside the code above.

Rather it seems to de-stabilize the UI, which crashes later (after playing sounds) when changed.

I made a button that plays a sound and does nothing else. This can be pressed repeatedly without problems.

However, as soon as the UI changes after that because of a user action (button press and processing thereof)

the crash/freeze appears.

Furthermore, I was able to make a scenario that reproduces a crash:

step 1. click a button that plays a (short) sound

step 2. click a button that results in a UI (layout) change

--> the app crashes/freezes

If I omit step 1, step 2 works fine. Adding extra Log statements, this output should be produced for step 2:

0.

1. [EDT] 0:0:0,0   - pieceButton.actionPerformed()

2. [EDT] 0:0:0,40  - pieceClicked

3. [EDT] 0:0:0,42  - pieceClicked: x,y = 9,6

4. [EDT] 0:0:0,44  - piece at 35 = 1

5. [EDT] 0:0:0,47  - newMoveShowRespond

6. [EDT] 0:0:0,52  - animateMove

7. [EDT] 0:0:0,54  - resetSelectedSquares

8. [EDT] 0:0:0,55  - showMovePiece

9. [EDT] 0:0:0,400 - showMovePiece: animateHierarchyAndWait() done

10.[EDT] 0:0:0,403 - showIfPromotion

If step 1 is executed first, the app crashes and the log file ends somewhere after line 0..5, usually after line 1 or 2.

So it seems that the execution is killed from outside (another thread?), and not because something goes wrong in the processing.

Finally I did some testing without setVolume (no difference) and with a Runnable onCompletion (preventing a new sound to start playing if the previous sound is still playing), also no difference. There doesn't seems to be a possibility (or even necessity) to cleanup the media(player). Also tried media.prepare() with the same result.

I really need only a few short sounds to work, so I could probably live with playBuiltinSound(), but they don't work. 

Please help, as I am without a clue how to proceed now.

[Shai Almog]

The trailing slash is a bug in the simulator, we'll fix that.
Can you isolate a test case we can reproduce and file an issue with that. This should work without a problem.

[JJ]

OK, I isolated a test case and filed an issue for that:

https://code.google.com/p/codenameone/issues/detail?id=1166&q=sound

The test case consists of the code of an earlier post,

https://groups.google.com/forum/embed/?place=forum/codenameone-discussions&showsearch=true&showpopout=true&hl=en&parenturl=http%3A%2F%2Fwww.codenameone.com%2Fdiscussion-forum.html#!searchin/codenameone-discussions/board$20game/codenameone-discussions/Yntcaj4i1bU/YzbJJpHpOBgJ

to which I added the latest playSound method and a button to test it.

See the issue on how to reproduce the problem.

[Shai Almog]

Thanks, I'll follow up in the issue.