adding codecs to rtpengine

[Jeff Brower]

Per Richard's advice (below) I'm continuing this discussion here. We need to add AMR and EVS, and possibly other, codecs to rtpengine **   Fred (Posner) mentioned this link:

  https://rtpengine.readthedocs.io/en/latest/transcoding.html#transcoding

Using EVS as an example, this page gives this explanation:

"The EVS codec implementation can be provided as a shared object library (.so) which is loaded in during runtime (at startup). The supported implementations can be seen as subdirectories within the evs/ directory. Currently supported are version 17.0.0 of the ETSI/3GPP reference implementation, 126.442 for the fixed-point implementation and 126.443 for the floating-point implementation. (The floating-point implementation seems to be significantly faster, but is not bit-precise.)"

But that doesn't describe a lib (.so) API, and the 3GPP reference implementation reads in mime or G.192 files for decode.  Are there specific C/C++ files that supply, via an API, bitstream frames for decode (and raw audio frames for encode) to the 3GPP reference implementation ? Are there similar source files for AMR ?

If we can understand the existing interface, we can figure out how to adapt to the voplib interface (https://github.com/signalogic/SigSRF_SDK/blob/master/codecs_readme.md).

Thanks.

-Jeff

** we have a number of enterprise and F500 companies that use our implementations for various reasons (performance, error detection and logging, malware detection, improved audio classification, avoiding ITU STL GPL license infringement, etc) and some are asking about using these same codec libs with rtpengine

Forwarded message from Richard Fuchs via sr-users <sr-u...@lists.kamailio.org>
    Date: Tue, 3 Jun 2025 17:24:33 -0400
    From: Richard Fuchs via sr-users <sr-u...@lists.kamailio.org>
Reply-To: "Kamailio (SER) - Users Mailing List" <sr-u...@lists.kamailio.org>
 Subject: [SR-Users] Re: [EXTERNAL] adding codecs to rtpengine
      To: sr-u...@lists.kamailio.org
      Cc: Richard Fuchs <...>

On 03/06/2025 16.31, Jeff Brower via sr-users wrote:
>Is there a documented procedure for adding codecs to rtpengine ? As I understand it so far, we would adapt rtpengine encoder and decoder interfaces to the voplib API (documented at https://github.com/signalogic/SigSRF_SDK/blob/master/codecs_readme.md), allowing use of voplib shared object libs.
>
>I've been searching (e.g. adding codecs to rtpengine site:kamailio.org) but have not found a step-by-step procedure yet.
>
>Thanks for any advice / guidance.

This is better discussed on the rtpengine mailing list: https://rtpengine.com/mailing-list

But as a short and quick answer: no, nobody has yet taken the time to write up a document about how to integrate new codecs, and as such the best documentation is the code. 😁

Cheers
----- End forwarded message -----

[Richard Fuchs]

On 03/06/2025 21.31, Jeff Brower wrote:
> Per Richard's advice (below) I'm continuing this discussion here. We
> need to add AMR and EVS, and possibly other, codecs to rtpengine

We already support AMR and EVS, but it sounds like you want to supply
your own implementation?

>
> https://rtpengine.readthedocs.io/en/latest/transcoding.html#transcoding
>
> Using EVS as an example, this page gives this explanation:

...

>
> But that doesn't describe a lib (.so) API, and the 3GPP reference
> implementation reads in mime or G.192 files for decode.  Are there
> specific C/C++ files that supply, via an API, bitstream frames for
> decode (and raw audio frames for encode) to the 3GPP reference
> implementation ? Are there similar source files for AMR ?

These instructions are specific to the EVS implementation and are needed
because it's the only one available (publicly) and it doesn't provide a
proper API or library that can be linked or used as-is. It's not typical
to how other codecs are implemented.

There isn't a singular API that is used for codec support. Instead
rtpengine supports a few different ones. Most codecs are supported
through the ffmpeg (libavcodec) API. The remaining ones are
codec-specific: There is separate support for Opus (libopus), EVS
(through the patched 3GPP codec), and G.729 (libbcg729). A few other
codecs are still implemented through ffmpeg but require some special
supporting code (AMR, iLBC).

The internal API is defined in `lib/codeclib.[ch]`. The primary
definition is via a `codec_type_t` object, which defines methods to open
and close an encoder or decoder, and encode and decode frames and
packets. See
https://github.com/sipwise/rtpengine/blob/master/lib/codeclib.c#L237

These are then referenced in the individual codec definitions
`codec_def_t`:
https://github.com/sipwise/rtpengine/blob/master/lib/codeclib.c#L338

Finally to maintain codec contexts, there are `encoder_t` and
`decoder_t` objects, which contain a union, with each member generally
corresponding to one codec type:
https://github.com/sipwise/rtpengine/blob/master/lib/codeclib.h#L286

The ffmpeg/libavcodec implementation is quite complicated so I would
suggest to ignore it and instead look at one of the simpler ones as an
example (e.g. bcg729).

Cheers