trouble with kivy 1.9 and pyaudio

[Eran Egozy]

Hello,

I've successfully used kivy 1.8 and pyaudio for a lot of kivy projects - generating interactive graphics / audio at the same time. This on on MacOS.

I upgraded to kivy 1.9 and unfortunately, pyaudio is now working very poorly. There is a lot of crackling in the audio output, which indicates that the audio callback (which supplies audio data to pyaudio ) and/or the audio thread of pyaudio is taking way too long.

I have done a bunch of tests to narrow down the issue. When kivy displays normal, non-animating graphics, pyaudio is happy. However, when I animate anything in kivy at full framerate (using kivy.clock.schedule_interval(self._update, 0)), pyaudio degrades massively with crackling audio. For example, only updating Label.text every frame causes the problem. Interestingly, the kivy framerate (as returned by kivy.clock.get_fps()) remains great. So, the cpu can handle plenty of graphics animation, but for some reason, when it does, the audio thread of pyaudio starves.

If I switch back to kivy 1.8, everything works great.

Anyone know what might be going on? Did something change in the fundamental kivy grahpics core that might explain this behavior?

Thanks,

Eran

[Eran Egozy]

An update on this: I've narrowed down the problem. In pyaudio (which is a wrapper around portaudio), a C callback is called on a separate thread. This callback needs raw audio data form python so it calls into a registered python callback. In the c-wrapper, before making any Py* style calls, the GIL must be acquired. So in c code it looks like this

   PyGILState_STATE gstate = PyGILState_Ensure();

   // do stuff using Py* API, including calling the PyObject* callback

   PyGILState_Release(gstate);

So far so good. I put timers around PyGILState_Ensure(). The difference between Kivy 1.8 and Kivy 1.9 are dramatic.

In 1.8, acquiring the GIL takes maybe 10 to 100 microseconds.

In 1.0, it ALWAYS takes at least 2000 microseconds, and often takes 13,000 or more when the graphics on screen are updating.

Obviously, this is a huge problem for an audio callback that is supposed to get audio data quickly. 

So my question is: why does acquiring the GIL take so long in Kivy 1.9 vs 1.8 when updating the graphics?

[Eran Egozy]

Another update: I tried switching the core.window provider from SDL2 to pygame. And... that made things work. So SDL2 is the culprit here. Thinking about it some more and trying some experiments, I have the following theory:

SDL2 is operating on a separate thread from the main python thread. SDL2 must somewhere grab the GIL by calling PyGILState_Ensure() in c-code. Then, my guess is that it hangs on to the GIL until end of video frame, even though it might have hardly anything to do. So during the large gap of time (~16ms, assuming 60fps) between python code setting up a change in graphics and the next video frame, SDL2 is hogging the GIL, and thereby not letting any other python threads do any work.

Does this make sense? Anyone out there understand this stuff to know if I am onto something or have any suggestions?

I guess my fallback plan is to just run with pygame as the window provider, but it sure would be nice to have all the benefits of SDL2 without this one serious drawback.