Using rnn_vad in my project

[Corey Cole]

I'm attempting to extract out the functionality of the rnn_vad into my C application. I've added an extern "C" wrapper for the main VAD functionality:

libwebrtc/src/modules/audio_processing/agc2/rnn_vad/rnn.cc

// ... rest of file above, I added below

float ComputeProbabilityC(float* signal) {

  const int frame_size_10ms = (int)16000 / (100 * 1000);

  std::vector<float> samples_10ms;

  samples_10ms.resize(frame_size_10ms);

  std::array<float, kFrameSize10ms24kHz> samples_10ms_24kHz;

  PushSincResampler resampler(frame_size_10ms, kFrameSize10ms24kHz);

  const AvailableCpuFeatures cpu_features = GetAvailableCpuFeatures();

  FeaturesExtractor features_extractor(cpu_features);

  std::array<float, kFeatureVectorSize> feature_vector;

  resampler.Resample(signal, samples_10ms.size(), samples_10ms_24kHz.data(),

  samples_10ms_24kHz.size());

  bool is_silence = features_extractor.CheckSilenceComputeFeatures(

  samples_10ms_24kHz, feature_vector);

  RnnVad rnn(cpu_features);

  float prob = rnn.ComputeVadProbability(feature_vector, is_silence);

  return prob;

}

} // namespace rnn_vad

} // namespace webrtc

extern "C" float WebrtcRnnVadComputeProbabilityC(float* signal) {

  return webrtc::rnn_vad::ComputeProbabilityC(signal);

}

Then I compiled the rnn_vad lib using ninja:

ninja -C out/Default rnn_vad_tool

And I linked the resulting library into my C project (libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/librnn_vad.a)

But, when I try to compile it with it linked I'm getting errors about undefined references:

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::RnnVad::RnnVad(webrtc::AvailableCpuFeatures const&)':

./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:47: undefined reference to `rnnoise::kInputDenseBias'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:48: undefined reference to `rnnoise::kInputDenseWeights'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:45: undefined reference to `webrtc::rnn_vad::FullyConnectedLayer::FullyConnectedLayer(int, int, rtc::ArrayView<signed char const, -4711l>, rtc::ArrayView<signed char const, -4711l>, webrtc::rnn_vad::ActivationFunction, webrtc::AvailableCpuFeatures const&, absl::string_view)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:54: undefined reference to `rnnoise::kHiddenGruBias'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:55: undefined reference to `rnnoise::kHiddenGruWeights'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:56: undefined reference to `rnnoise::kHiddenGruRecurrentWeights'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:52: undefined reference to `webrtc::rnn_vad::GatedRecurrentLayer::GatedRecurrentLayer(int, int, rtc::ArrayView<signed char const, -4711l>, rtc::ArrayView<signed char const, -4711l>, rtc::ArrayView<signed char const, -4711l>, webrtc::AvailableCpuFeatures const&, absl::string_view)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:61: undefined reference to `rnnoise::kOutputDenseBias'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:62: undefined reference to `rnnoise::kOutputDenseWeights'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:65: undefined reference to `webrtc::NoAvailableCpuFeatures()'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:59: undefined reference to `webrtc::rnn_vad::FullyConnectedLayer::FullyConnectedLayer(int, int, rtc::ArrayView<signed char const, -4711l>, rtc::ArrayView<signed char const, -4711l>, webrtc::rnn_vad::ActivationFunction, webrtc::AvailableCpuFeatures const&, absl::string_view)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::RnnVad::RnnVad(webrtc::AvailableCpuFeatures const&)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::RnnVad::~RnnVad()':

./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:74: undefined reference to `webrtc::rnn_vad::FullyConnectedLayer::~FullyConnectedLayer()'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:74: undefined reference to `webrtc::rnn_vad::GatedRecurrentLayer::~GatedRecurrentLayer()'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:74: undefined reference to `webrtc::rnn_vad::FullyConnectedLayer::~FullyConnectedLayer()'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::RnnVad::Reset()':

./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:77: undefined reference to `webrtc::rnn_vad::GatedRecurrentLayer::Reset()'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::RnnVad::ComputeVadProbability(rtc::ArrayView<float const, 42l>, bool)':

./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:89: undefined reference to `webrtc::rnn_vad::FullyConnectedLayer::ComputeOutput(rtc::ArrayView<float const, -4711l>)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:90: undefined reference to `webrtc::rnn_vad::GatedRecurrentLayer::ComputeOutput(rtc::ArrayView<float const, -4711l>)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:91: undefined reference to `webrtc::rnn_vad::FullyConnectedLayer::ComputeOutput(rtc::ArrayView<float const, -4711l>)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::RnnVad::ComputeVadProbability(rtc::ArrayView<float const, 42l>, bool)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `webrtc::rnn_vad::ComputeProbabilityC(float*)':

./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:101: undefined reference to `webrtc::PushSincResampler::PushSincResampler(unsigned long, unsigned long)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:102: undefined reference to `webrtc::GetAvailableCpuFeatures()'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:106: undefined reference to `webrtc::PushSincResampler::Resample(float const*, unsigned long, float*, unsigned long)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/rnn.cc:113: undefined reference to `webrtc::PushSincResampler::~PushSincResampler()'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `rtc::ArrayView<signed char const, -4711l>::ArrayView<signed char const>(signed char const*, unsigned long)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o: in function `rtc::ArrayView<float const, -4711l>::ArrayView<float const>(float const*, unsigned long)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/rnn.o:./../../rtc_base/checks.h:250: more undefined references to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)' follow

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::FeaturesExtractor(webrtc::AvailableCpuFeatures const&)':

./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:35: undefined reference to `webrtc::rnn_vad::PitchEstimator::PitchEstimator(webrtc::AvailableCpuFeatures const&)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:29: undefined reference to `webrtc::rnn_vad::SpectralFeaturesExtractor::SpectralFeaturesExtractor()'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::FeaturesExtractor(webrtc::AvailableCpuFeatures const&)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::Reset()':

./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:46: undefined reference to `webrtc::rnn_vad::SpectralFeaturesExtractor::Reset()'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::~FeaturesExtractor()':

./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:42: undefined reference to `webrtc::rnn_vad::SpectralFeaturesExtractor::~SpectralFeaturesExtractor()'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:42: undefined reference to `webrtc::rnn_vad::PitchEstimator::~PitchEstimator()'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::CheckSilenceComputeFeatures(rtc::ArrayView<float const, 240l>, rtc::ArrayView<float, 42l>)':

./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:57: undefined reference to `webrtc::BiQuadFilter::Process(rtc::ArrayView<float const, -4711l>, rtc::ArrayView<float, -4711l>)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:66: undefined reference to `webrtc::rnn_vad::ComputeAndPostProcessLpcCoefficients(rtc::ArrayView<float const, -4711l>, rtc::ArrayView<float, 5l>)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:67: undefined reference to `webrtc::rnn_vad::ComputeLpResidual(rtc::ArrayView<float const, 5l>, rtc::ArrayView<float const, -4711l>, rtc::ArrayView<float, -4711l>)'

/usr/bin/ld: ./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:70: undefined reference to `webrtc::rnn_vad::PitchEstimator::Estimate(rtc::ArrayView<float const, 864l>)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::CheckSilenceComputeFeatures(rtc::ArrayView<float const, 240l>, rtc::ArrayView<float, 42l>)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::FeaturesExtractor::CheckSilenceComputeFeatures(rtc::ArrayView<float const, 240l>, rtc::ArrayView<float, 42l>)':

./../../modules/audio_processing/agc2/rnn_vad/features_extraction.cc:78: undefined reference to `webrtc::rnn_vad::SpectralFeaturesExtractor::CheckSilenceComputeFeatures(rtc::ArrayView<float const, 480l>, rtc::ArrayView<float const, 480l>, rtc::ArrayView<float, 16l>, rtc::ArrayView<float, 6l>, rtc::ArrayView<float, 6l>, rtc::ArrayView<float, 6l>, rtc::ArrayView<float, 6l>, float*)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `webrtc::rnn_vad::SequenceBuffer<float, 864, 240, 480>::SequenceBuffer()':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `rtc::ArrayView<float, 864l>::ArrayView<float>(float*, unsigned long)':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o: in function `rtc::ArrayView<float, 42l>::operator[](unsigned long) const':

./../../rtc_base/checks.h:250: undefined reference to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)'

/usr/bin/ld: ../libwebrtc/src/out/Default/obj/modules/audio_processing/agc2/rnn_vad/rnn_vad/features_extraction.o:./../../rtc_base/checks.h:240: more undefined references to `rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)' follow

collect2: error: ld returned 1 exit status

ninja: build stopped: subcommand failed.

Does anyone have any tips for me to build the rnn_vad lib with gn/ninja so that all the dependencies are accessible to my C project that I am building with meson/ninja?

Thank you,

Corey

[Vitaly Ivanov]

Seems like you also need to include rnnoise lib itself which resides in src/third_party/rnnoise (see ack output in the screenshot). I don't have much experience with Ninja, but I did the same thing (VAD-only processing) with the original JM Valin's RNN denoiser which webrtc guys pulled in. It uses GNU Autotools though:

https://github.com/WickrInc/rnnoise

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/32f4dfdd-b09d-4dae-bd0c-a8ac35c0c9a1n%40googlegroups.com.

[Vitaly Ivanov]

And here's the screenshot

[Corey Cole]

> Seems like you also need to include rnnoise lib itself which resides in src/third_party/rnnoise

Looking at the librnn_vad.a config in the BUILD.gn file

rtc_library("rnn_vad") {

  visibility = [ "../*" ]

  sources = [

    "features_extraction.cc",

        "features_extraction.h",

        "rnn.cc",

        "rnn.h",

  ]

  defines = []

  if (rtc_build_with_neon && current_cpu != "arm64") {

      suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ]

    cflags = [ "-mfpu=neon" ]

  }

  deps = [

        ":rnn_vad_common",

        ":rnn_vad_layers",

        ":rnn_vad_lp_residual",

        ":rnn_vad_pitch",

        ":rnn_vad_sequence_buffer",

        ":rnn_vad_spectral_features",

        "..:biquad_filter",

        "..:cpu_features",

         "../../../../common_audio/",

        "../../../../api:array_view",

        "../../../../rtc_base:checks",

        "../../../../rtc_base:safe_compare",

        "../../../../rtc_base:safe_conversions",

        "//third_party/rnnoise:rnn_vad",

  ]

}

It looks like "//third_party/rnnoise:rnn_vad" is already linked as a dependency. I'm not sure what extra steps I need for it to be defined when I link `librnn_vad.a` into my project. Could you provide more details on what you mean?

[Arun Raghavan]

On Fri, 5 Feb 2021, at 6:38 PM, Corey Cole wrote:
> I'm attempting to extract out the functionality of the rnn_vad into my
> C application. I've added an extern "C" wrapper for the main VAD
> functionality:

If there's wider interest, this might be something we could also expose in the webrtc-audio-processing library[1].

Cheers,
Arun

[1] https://gitlab.freedesktop.org/pulseaudio/webrtc-audio-processing/