Need help with ideas

310 views
Skip to first unread message

Mirko Brankovic

unread,
Jun 12, 2020, 11:32:53 AM6/12/20
to meetecho-janus
Hi,
Due to the lack of my codec/audio formats knowledge I need a bit help with ideas how to make noise-cancellation feature in Audiobridge plugin.

Rnnoise library when you follow the demo, and record some audio while making noise with bag of chips and process it you will get chips free audio file (a bit robotic flavor in it).
 
So I started work here:

Rnnoise expects  16-bit raw PCM  and Janus is decoding into raw frames mixing and encoding again to opus, or g711 as recently.

I got to the point where audio got a little bit robotic but no noise is removed.

Basically I did similar thing that their Demo does, reads from file on file system and process 480 frame size of it and puts it back to the new file.

Now I don't really know if, when decoded from, for example, Opus raw frame is the same or I think I need t re-sample it to 16k maybe to match requirements of the lib.
I see they also import libopus so maybe it can be done after encoding...

Any suggestion is welcome since i'm stuck anyway.

Thanks,
mirko

Antonio Tuzzi

unread,
Jun 12, 2020, 1:21:31 PM6/12/20
to Mirko Brankovic, meetecho-janus
Are you Creating the RRNOISE STATE at the start of the room ?

Are you applying the RRNOISE to the sum coming from mixer ?



--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/1ea4ca37-c0d4-4ae7-9a8e-5dd107db8dcao%40googlegroups.com.

Mirko Brankovic

unread,
Jun 12, 2020, 3:09:40 PM6/12/20
to meetecho-janus
Yes state is created and then i pass audio frame before it goes to opus encoder and out to participant

Mirko Brankovic

unread,
Jun 12, 2020, 3:11:11 PM6/12/20
to meetecho-janus
Actually i created state when it is in rtp relaying function, so maybe i should create it only once, and not on every frame?

Antonio Tuzzi

unread,
Jun 12, 2020, 6:52:45 PM6/12/20
to Mirko Brankovic, meetecho-janus
you should create a single STATE that is applied to the entire session

if you apply the RRNOISE to a decoded 16bit raw data coming from a mix, only one STATE is needed

if instead you apply the RRNOISE on every incoming DATA form each client connected to the ROOM you need one PERSISTENT STATE  for each connected and decoded stream for the session



Mirko Brankovic

unread,
Jun 13, 2020, 3:36:19 AM6/13/20
to meetecho-janus
Great thank you, i will give it a try

Mirko Brankovic

unread,
Jun 16, 2020, 5:32:35 AM6/16/20
to meetecho-janus
Thanks for suggestions, it helped a lot 
I got it to work now and compared it to videoroom plugin at the same time, with the same noises (constantly squashing plastic bag next to the microphone and playing drilling and airplane sounds on phone next to laptop)

Now it is time for next stage, getting voice quality to better state (removing metallic flavor)

Regards,
Mirko 
--
Regards,
Mirko

Antonio Tuzzi

unread,
Jun 16, 2020, 6:59:27 AM6/16/20
to meetecho-janus
Molto bene !

I'll wait for your variations !

Antonio
To unsubscribe from this group and stop receiving emails from it, send an email to meetech...@googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetech...@googlegroups.com.



--
Regards,
Mirko

Mirko Brankovic

unread,
Jul 1, 2020, 11:54:20 AM7/1/20
to meetecho-janus
Sorry it took so long, but I was fighting with re-sampling and understanding how it works, which ended up in allowing only 16000 sample rate :D


If anyone is interested in noise suppression in AudioBridge plugin, give it a try. I tested it by making a noise with a  plastic bag of chips next to my microphone and also some construction works next to my house, and I'm happy with the results but not with  overall quality.
I think that quality can be increased allowing stereo and other sampling rates input into rnnoise lib, but that is another project.

Thanks,
Mirko 

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/086102b8-3809-4916-954f-0f322aa3f2d9o%40googlegroups.com.


--
Regards,
Mirko

rahma...@gmail.com

unread,
Jul 16, 2020, 3:20:23 AM7/16/20
to meetecho-janus
Hi mirko

         I saw in one of your commits that you tried to resample whatever input sampling rate to 16k, then give it as input to rnn and again resampled it to input sampling rate. But in next commit, you removed all of  this and made 16 K as mandatory sampling rate for this work. Can i know why you took this decision? 
    
    Also, thanks for the great work for this commit.

Regards
Rahman


Mirko Brankovic

unread,
Jul 16, 2020, 3:30:35 AM7/16/20
to meetecho-janus
Yes, the idea was to re-sample to 16k whenever a different bitrate in the room is used, but I got robotic voice in most cases, when room bitrate was not 16000.
The Helper for resampling I used was the one Lorenzo added for g.711 (pcma/pcmu) resampling to 8k and commented as "Ugly helper" because it was coping payload over just like I did.

 Lets see after the multistream branch, when I have time to revisit this, it should definitely be possible to do resampling.

Thanks, I look forward for any feedback, is this useful or not :) 

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/5dd5e591-75bf-4b58-b123-461d3507de55n%40googlegroups.com.


--
Regards,
Mirko

Pondia

unread,
Jul 16, 2020, 10:39:59 AM7/16/20
to meetecho-janus
Hi ! Discord is using the feature too ! I think they are using : krisp (https://krisp.ai/) but it's not free at all :/

Mirko Brankovic

unread,
Jul 16, 2020, 11:03:53 AM7/16/20
to meetecho-janus
ofcourse, thus why we try to make it free here, together with JM Valim's lib :D 

On Thu, Jul 16, 2020 at 4:40 PM Pondia <mehdi.ve...@gmail.com> wrote:
Hi ! Discord is using the feature too ! I think they are using : krisp (https://krisp.ai/) but it's not free at all :/

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/b83d362d-ea69-47c2-8565-d4d2967e5a00o%40googlegroups.com.


--
Regards,
Mirko
Reply all
Reply to author
Forward
0 new messages