getSound() doesn't work but getMusic() does

[guille.r...@gmail.com]

Hello all,

I have a strange problem with sound on the Java target. Basically if I load a sound using getSound(), it doesn't work (no sound), however if I use getMusic() then it works fine.

I would normally suspect something wrong in the .wav file itself or in my Java setup, however since it does work when I use getMusic() I guess there must be something else...

More info:

 - My environment: OpenJDK 1.8.0_141 on Ubuntu 14.04.5. 

 - PlayN 2.1-SNAPSHOT

 - Sound file available here: http://freesound.org/people/junggle/sounds/30341/

 - Simple test program follows

public class HelloPlayN extends SceneGame {

  public HelloPlayN (Platform plat) {

    super(plat, 33); // update our "simulation" 33ms (30 times per second)

    // create and add background image layer

    Image bgImage = plat.assets().getImage("images/bg.png");

    ImageLayer bgLayer = new ImageLayer(bgImage);

    // scale the background to fill the screen

    bgLayer.setSize(plat.graphics().viewSize);

    rootLayer.add(bgLayer);

    

    Sound dropSound = plat.assets().getSound("sounds/drop");

    Sound dropMusic = plat.assets().getMusic("sounds/drop");

    // Doesn't work    

    dropSound.play();

    // Works

    //dropMusic.play();

  }

}

Any hints?

Guillermo

[Andres Martinez Quijano]

which files do you have in sounds/ folder ?

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "PlayN" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to playn+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[guille.r...@gmail.com]

Hi Andres,

The wav file from http://freesound.org/people/junggle/sounds/30341/, renamed to drop.wav

The funny thing is that loading it using getMusic("sounds/drop") works, while if I use getSound instead then it doesn't work.

Guillermo

[Andres Martinez Quijano]

It has to be .mp3 or .caf

http://playn.io/docs/api/core/playn/core/Assets.html#getSound-java.lang.String-

[guille.r...@gmail.com]

That shouldn't be the case:

1. http://playn.io/cookbook/sound.html says .wav and .mp3 are supported for the Java target

2. As I said, if I try getMusic() instead of getSound(), it does work (same .wav file)

3. Just for the sake of it, I tried converting it to .mp3. Same results. getSound() doesn't work, getMusic() does

Guillermo

[Brigt Vik]

Perhaps you need to wait until the sound is loaded before calling play() on it, while a sound loaded using getMusic() somehow can handle playing itself at a delayed time?

This is merely a guess. I have not tested your code.

[guille.r...@gmail.com]

No, I tried that already. The original code where I found this problem plays the sound interactively when a certain target is touched, although this is not shown in my simplified example...

Guillermo

[Brigt Vik]

Your code works for me in Java, using both the wav you linked, a small mp3 file I have and some wav file I found at Windows/Media/Alarm01.wav. I don't know what is different with your setup.

[guille.r...@gmail.com]

Hi,

Thank you for taking the time to test this.

In the meanwhile I found out that the only difference between getSound() and getMusic() is that getSound uses a standard Java AudioClip directly, while getMusic wraps this in a "BigClip" instance (BigClip is an internal PlayN class). For some reason, whatever BigClip is doing to play the audio works for me, while AudioClip does not.

I also learned that AudioClip is somewhat fragile and there are many issues including platform-specific problems, driver issues, and lack of support for audio formats / encodings (within .wav files).

Do you think you could try again with the same .wav file that I am using? Here's a direct link:

https://github.com/kevinlondon/java-libgdx-drop-example/blob/master/android/assets/drop.wav

Thanks,

Guillermo

[guille.r...@gmail.com]

OK I found the problem or at least part of it.

PlayN uses AudioSystem.getClip() to create a clip instance for the sound (see playn.java.JavaAudio). In my system, this results in the "pulse audio" mixer being selected, instead of the default system mixer. And for some reason the pulse audio mixer seems unable to play this .wav file. Replacing the AudioSystem.getClip() call with AudioSystem.getClip(null) makes it work (this selects the default system mixer instead).

I have posted a question at SO to try to get some more info on this. Here's the link: https://stackoverflow.com/questions/46194666/java-audio-fails-to-play-wav-file-in-linux

Will post back with any updates.

Guillermo

[Brigt Vik]

Good luck then! I have no idea about audio mixers, so I hope you figure it out before I'd run into the problem at some point :).

[Andres Martinez Quijano]

I had problems with audio also on linux, it didn't work with OpenJDK,
it did work with Oracle JDK, never figured out why...

2017-09-13 8:53 GMT-03:00 Brigt Vik <bov...@gmail.com>:
> Good luck then! I have no idea about audio mixers, so I hope you figure it
> out before I'd run into the problem at some point :).
>

[guille.r...@gmail.com]

OK, there are two issues at play here.

First, currently PlayN is using AudioSystem.getClip() to create a Clip. This is not a good idea as there's no guarantee that the clip will be able to handle the target audio format. Instead, we can explicitly specify the format when the clip is created (there are several ways to do that)

In addition there is a bug in the PulseAudio mixer which, depending on the method used to create the clip, will result in an invalid format exception even when the mixer can actually handle the format.

I have submitted a pull request that fixes these problems (or rather, fixes the first one, and works around the second)

Some links to relevant documentation:

 - Question in SO: https://stackoverflow.com/q/46194666/450398

 - Upstream bugs in PulseAudio:

    * https://icedtea.classpath.org/bugzilla/show_bug.cgi?id=3452

    * https://icedtea.classpath.org/bugzilla/show_bug.cgi?id=2915

 - Issue in PlayN github repo: https://github.com/playn/playn/pull/46

 - Pull request: https://github.com/playn/playn/pull/47

Side note: I am not sure that we actually need to convert formats when the encoding is not PCM_SIGNED. I can see why that is needed when BigClip is used (i.e. when music == true). But I think it is unnecessary when BigClip is not used (music == false). Anyway I was not completely sure about this so the code in the pull request still does the same as before.

Guillermo

[Michael Bayne]

Thanks for digging into this! I'd be inclined to disable the format conversion entirely (because BigClip does it internally), but it seems like it's useful to tell AudioSystem.getLine that we're going to use PCM_SIGNED which would need to happen before we created the BigClip.

--

[guille.r...@gmail.com]

Hi,

On Monday, October 9, 2017 at 3:04:38 AM UTC+2, Michael Bayne wrote:

Thanks for digging into this! I'd be inclined to disable the format conversion entirely (because BigClip does it internally), but it seems like it's useful to tell AudioSystem.getLine that we're going to use PCM_SIGNED which would need to happen before we created the BigClip.

Yes, I spent some time looking at that and wasn't sure why the format conversion was being done in createSound(). I assumed that the internal conversion done by BigClip could fail in some cases. However there are still some points which were not clear to me:

- Why do the conversion at all when BigClip is not used? This seemed unnecessary but I wasn't 100% sure and decided not to touch it (I see you have taken care of that already)

- There is still one case where BigClip will convert the format internally, which is when channels < 2 (see https://github.com/playn/playn/blob/afed05ab83ef22c355018c0ed5b3698ed8e4e73e/java-base/src/playn/java/BigClip.java#L367). Then, why not combine both checks in createSound(), i.e.:

    if (format.getEncoding() != AudioFormat.Encoding.PCM_SIGNED
            || format.getChannels() < 2) {
        ...
    }

so that the conversion, if needed, is always done at the same point?

- These two comments in createAudio:

    16, // we have to force sample size to 16

    [...]

    false // big endian

If we really have to force sample size to 16 and big endian format, then why isn't this checked in the previous 'if' ?

Moreover if we don't do any conversion in createSound() (that is, if the stream is already in PCM_SIGNED format) BigClip does not seem to worry about sample size or endianness.

BR,

Guillermo