baresip-ios, audiounit
Kostiantyn Herasimov
Hello.
I am using Baresip as a library in iOS.
I have already posted a PR that allows compiling baresip-ios project using modern iOS sdk. There were some problems. Thanks for accepting commits from that PR.
Currently, the audiounit module is the one that allows iOS support. But it is not up to date with modern iOS API. Some API, like AudioSessionInitialize, has been deprecated a long time ago. Also, I found cases where the logic is not written in a way iOS API expects. That may cause some other issues. For, example, in iOS API there is only a single IO Audio Unit, but the Baresip audiounit module initializes several instances of that. There are other places where the implementation should be changed too (from my point of view).
On top of that, usually, iOS devs use VoIP solutions with the iOS CallKit integration which may want to control when to configure iOS Audio Session, start, stop, reinit iOS Audio Units, etc. Even without CallKit, in most cases, iOS Audio Session should be under developer control. Currently, it is not possible since Baresip audiounit module does not provide that kind of API.
I would like to update the audiounit module but I am not professional in c language and Baresip architecture. Anyway, I have the first question:
- Is it possible to write some functions inside audiounit module so they will be accessible at the time audiounit module is already initialized? In this scenario there should be some functions to determine when audiounit module is accessible. Or is there some other Baresip API so I can use to access hidden functions inside audiounit module (from the Baresip architecture point of view)?
Thanks.
Alfred E. Heggestad
many thanks for trying to upgrade audiounit.so
regarding the API, it is defined here:
ausrc and auplay:
https://github.com/baresip/baresip/blob/master/include/baresip.h#L473
i.e.:
typedef void (ausrc_read_h)(struct auframe *af, void *arg);
typedef void (ausrc_error_h)(int err, const char *str, void *arg);
typedef int (ausrc_alloc_h)(struct ausrc_st **stp, const struct ausrc *ausrc,
struct media_ctx **ctx,
struct ausrc_prm *prm, const char *device,
ausrc_read_h *rh, ausrc_error_h *errh, void *arg);
baresip core can "allocate" a new instance. the driver can
provide audio frames via "read" handler.
apart from this it is not possible to communicate
between core and module, it has been designed this way
on purpose.
if you make a PR I am happy to help with review
/alfred
On 24/02/2021 11:34, Kostiantyn Herasimov wrote:
> Hello.
>
> I am using Baresip as a library in iOS.
>
> You received this message because you are subscribed to the Google Groups "baresip" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to baresip+u...@googlegroups.com
> <mailto:baresip+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/baresip/565665e8-4a03-4ffe-97ef-3c76e12ea38an%40googlegroups.com
> <https://groups.google.com/d/msgid/baresip/565665e8-4a03-4ffe-97ef-3c76e12ea38an%40googlegroups.com?utm_medium=email&utm_source=footer>.
Kostiantyn Herasimov
struct audiounit_controller;
int audiounit_controller_get_instance(struct audiounit_controller **controller_ref);
int audiounit_player_alloc(struct auplay_st **stp, const struct auplay *ap,
struct auplay_prm *prm, const char *device,
auplay_write_h *wh, void *arg);
int audiounit_recorder_alloc(struct ausrc_st **stp, const struct ausrc *as,
struct media_ctx **ctx,
struct ausrc_prm *prm, const char *device,
ausrc_read_h *rh, ausrc_error_h *errh, void *arg);
int audiounit_audio_session_deactivated(void);
int audiounit_media_services_were_reset(void);
The first two lines are optional for now. The last three provide an ability to send to the baresip events from iOS app audio session management logic.
Also, I did the next things:
- got rid of outdated iOS audio session API
- removed AU Converter from recorder because IO unit has its own sample rate converter under the hood (need to check this to make sure it works correctly)
- modernize the other AudioUnit iOS API function calls so they are called as expected by the Apple AudioUnit framework
For now, I have some issues with the playback audio quality. Sometimes it is not good and have some rattle distortions, especially while switching between device audio inputs. According to this, I have the next question.
The AudioUnit is designed in a way so the player's output_callback is called in a real-time high priority thread. Apple strongly does not recommend using any mutexes, locks, memory allocations, etc in this thread (see attached screeshot from Apple WWDC video). So I would like to know is there a way to get rid of that mutexes (see here https://github.com/baresip/baresip/blob/master/modules/audiounit/player.c#L61)
pthread_mutex_lock(&st->mutex);
?
Alfred E. Heggestad
actually I think that the mutex is not needed.
it is only used to protect the write-handler
just make sure that the audiounit is properly stopped
in auplay_destructor.
/alfred
>
> pthread_mutex_lock(&st->mutex);
> ?
>>>> https://groups.google.com/d/msgid/baresip/565665e8-4a03-4ffe-97ef-3c76e12ea38an%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/baresip/565665e8-4a03-4ffe-97ef-3c76e12ea38an%40googlegroups.com>
>>>> <https://groups.google.com/d/msgid/baresip/565665e8-4a03-4ffe-97ef-3c76e12ea38an%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>>
>