Can't get audio to work

[Jason Boyd]

This code:

self.pluck = SoundLoader.load('classical/p80.ogg')

fails saying no loader could be found if I try to load an MP3 *or* OGG file. 

I've tried doing:

os.environ['KIVY_AUDIO'] = 'avplayer' 

os.environ['KIVY_AUDIO'] = 'gstreamer'

os.environ['KIVY_AUDIO'] = 'sdl2'

os.environ['KIVY_AUDIO'] = 'pygame' 

The last one actually crashes at runtime on trying to load the audio, giving this error:

Python(46001,0x7fff76588000) malloc: *** error for object 0x106ecce50: pointer being freed was not allocated

*** set a breakpoint in malloc_error_break to debug

Abort trap: 6

Also this bit of the log output is interesting:

[DEBUG  ] [Audio       ] option <avplayer> ignored by config

[DEBUG  ] [Audio       ] option <pygst> ignored by config

[DEBUG  ] [Audio       ] option <ffpyplayer> ignored by config

2016-10-09 09:31:41.620 Python[46001:4097154] 09:31:41.620 WARNING:  140: This application, or a library it uses, is using the deprecated Carbon Component Manager for hosting Audio Units. Support for this will be removed in a future release. Also, this makes the host incompatible with version 3 audio units. Please transition to the API's in AudioComponent.h.

I'm on OS X 10.11.6 (El Capitan). As far as I know everything is installed, but I'm suspicious that SDL2 maybe isn't installed right. When I try to set SDL2 as the window provider, it fails:

[INFO   ] [Logger      ] Record log in /Users/macbookpro/.kivy/logs/kivy_16-10-09_26.txt

[INFO   ] [Kivy        ] v1.9.1

[INFO   ] [Python      ] v2.7.12 (v2.7.12:d33e0cf91556, Jun 26 2016, 12:10:39) 

[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

[INFO   ] [Factory     ] 179 symbols loaded

[DEBUG  ] [Cache       ] register <kv.lang> with limit=None, timeout=None

[DEBUG  ] [Cache       ] register <kv.image> with limit=None, timeout=60

[DEBUG  ] [Cache       ] register <kv.atlas> with limit=None, timeout=None

[INFO   ] [Image       ] Providers: img_tex, img_imageio, img_dds, img_gif, img_pygame (img_pil, img_ffpyplayer ignored)

[DEBUG  ] [Cache       ] register <kv.texture> with limit=1000, timeout=60

[DEBUG  ] [Cache       ] register <kv.shader> with limit=1000, timeout=3600

[DEBUG  ] [Window      ] Provider <pygame> ignored by config

[CRITICAL] [Window      ] Unable to find any valuable Window provider at all!

[INFO   ] [Text        ] Provider: pygame

[DEBUG  ] [Cache       ] register <textinput.label> with limit=None, timeout=60.0

[DEBUG  ] [Cache       ] register <textinput.width> with limit=None, timeout=60.0

[CRITICAL] [App         ] Unable to get a Window, abort.

 Exception SystemExit: 1 in 'kivy.properties.dpi2px' ignored

[CRITICAL] [App         ] Unable to get a Window, abort.

 Exception SystemExit: 1 in 'kivy.properties.dpi2px' ignored

[('avplayer', 'audio_avplayer'), ('pygst', 'audio_pygst'), ('ffpyplayer', 'audio_ffpyplayer'), ('pygame', 'audio_pygame')]

[DEBUG  ] [Audio       ] option <avplayer> ignored by config

[DEBUG  ] [Audio       ] option <pygst> ignored by config

[DEBUG  ] [Audio       ] option <ffpyplayer> ignored by config

[DEBUG  ] [Audio       ] option <pygame> ignored by config

[INFO   ] [Audio       ] Providers:  (audio_avplayer, audio_pygst, audio_ffpyplayer, audio_pygame ignored)

Last note: in my efforts to get a working Android build, based on someone's suggestion, I ended up uninstalling the Kivy.app distro and doing a pip install kivy, so it's installed into my system Python library. 

Any help appreciated...

[Jason Boyd]

Still no luck, but updating to show work so far:

At suggestion of helpful folks on IRC, I tried uninstalling kivy and pygame, reinstalling using pip install kivy. This failed to compile *any* libraries, reasons unknown. Tried repeating, but installing from zip file of kivy 1.9.1. This failed at runtime. Lastly, tried uninstalling and installing Kivy.app (which I had uninstalled last week taking suggestions for getting buildozer to succeed). 

So using Kivy.app, with kivy 1.9.1dev0, I get runtime errors trying to play OGG, MP3, or WAV (the last taken out of kivy examples directory). In general I get the following on crash, regardless which file format:

** (<unknown>:48978): WARNING **: Invalid channel positions

[ERROR  ] [AudioGstplayer] Internal data flow error.

[ERROR  ] [AudioGstplayer] The stream is in the wrong format.

Interestingly, only when trying to play a WAV file, I also get a lot of these messages before the above error:

(<unknown>:48978): GStreamer-WARNING **: Failed to load plugin '/Applications/Kivy2.app/Contents/Resources/../Frameworks/GStreamer.framework/Versions/Current/lib/gstreamer-1.0/libgstlibav.so': dlopen(/Applications/Kivy2.app/Contents/Resources/../Frameworks/GStreamer.framework/Versions/Current/lib/gstreamer-1.0/libgstlibav.so, 2): Library not loaded: libbz2.1.0.dylib

  Referenced from: /Applications/Kivy2.app/Contents/Frameworks/GStreamer.framework/Versions/1.0/lib/gstreamer-1.0/libgstlibav.so

  Reason: image not found

Given this is the pre-bundled Kivy.app, supposedly containing all dependencies, I'm really stuck. So far nothing I've tried has allowed playing any audio file, in any format...

[Jason Boyd]

Well, one minor success: building the app for Android, am able to play OGG files. Just cannot play any audio on OS X. Which remains a big blocker.

[Jason Boyd]

So, finally was able to get audio working on both my Mac and Android. Notes below.

Firstly, I cannot get this to work with the latest Kivy.app. It seems Gstreamer just does not want to work. Also I am completely unclear how it is possible to override the default sound provider. For instance doing os.environ['KIVY_AUDIO'] = 'audio_sdl2' has no discernible effect, the logs still show GStreamer as the audio provider. 

Likewise installing into system Python using pip insists on using GStreamer, with various errors depending on file format, as described above. 

So I did brew uninstall gstreamer. 

I also needed to do brew install --with-vorbis --with-smpeg2 sdl2_mixer

With these changes, running my app as python main.py succeeds, am able to play OGG file. MP3 file does not load:

[WARNING] [AudioSDL2   ] Unable to load classical/mp3.ogg: Mix_LoadWAV_RW with NULL src

Building for android, the OGG file also works:

buildozer android_new debug deploy run logcat

-- 

For anyone curious about high-performance audio with Kivy -- I am able to finally answer a question I posted several weeks back which has received no replies, about audio latency. At least on Android, using SDL2 for audio, latency from tapping a button -> Sound.play() -> hearing the audio begin is roughly 100-200ms (by my ear), although the latency seems consistent, so not variable. This is about what I expected, which is good enough for games or apps that play music, but not good enough for the types of music apps I work with, where latency needs to be down to < 20ms to feel responsive to user input (e.g. playing an instrument with touch). 

If anyone knows any tricks to improve this, I'd be excited to hear. I wonder also if it's even feasible for Kivy itself to pass calls to the underlying audio provider more quickly. 

I've got an app using HTML5/JS/WebAudio that I managed to get latency down to 5ms from tap to audible sound on both iOS and Android, though the latter requires either Android 5.1+ or a build using Crosswalk, which I'm still investigating. Another advantage of the WebAudio approach is advanced audio processing -- e.g. DSP nodes, realtime envelopes, adding limiters, delays, etc, recording audio, etc. Plus OGG and MP3 just work, out of the box. 

None of this is dissing Kivy, just fyi. I know the history of the tech stacks is very different, and am hopeful Kivy/Python can catch up in the mobile space.

 

[Jason Boyd]

[WARNING] [AudioSDL2   ] Unable to load classical/mp3.ogg: Mix_LoadWAV_RW with NULL src

Oops! Caught my typo on filename there. Corrected this, and MP3 does load and play on OS X. Loading fails on Android, with error that no loader could be found, presumably due to legal restrictions on distributing the MP3 codec. Ah well, OGG is good enough.