Voice Activity Detection Feature in Kurento

[trur...@gmail.com]

Hello everyone,

I wanted to have a voice activity detection feature in kurento for dominant speaker selection. Does anyone has the pointers or implemented the same?

Actually i got hold of one gstreamer plugin "level" to get the decibel level of various streams. i am able to run this plugin independently successfully. Now can someone tell me where to integrate this plugin in kurento media server code for voice activity detection feature implementation.i assume that this plugin need to be injected to be in pipeline but need to know the location in kurento media server code.

please help me with this. Appreciate a quick response here.

regards

Inder

[Micael Gallego]

You will have to create a Kurento Module with it. 

https://doc-kurento.readthedocs.io/en/stable/user/writing_modules.html

Regards.

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/bad8419d-60fd-4dc0-a55e-2efd38011d4f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[truring truring]

Hi,

i followed all the instructions specified in the link shared by you. i have successfully created a module and generated the client code for java.

Now i am trying to use the module in my java application but it does not work. it is giving a compilation error as it is not able to find the module in my local repository.

i have already installed the module in my local maven repository. do i need to perform anyother step as well beside the steps specified in link.

regards

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_DuOYC213NeFQj-bnwfTmHX2Hqh8eb8onqZZwmRenv5KBQ%40mail.gmail.com.

[Micael Gallego]

It seems the module is not correctly installed on your local repository. 

Or maybe it is not well generated. Can you see the Java code generated? Try to put this code "by hand" in your project to see if it compiles. If yes, you have a problem installed lib in your local repository. If not, there is a problem with the code generation.

Best regards

Micael Gallego

Kurento / OpenVidu Project Lead

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CALvshjMxbCzu_L5-hw-xMfwLKz%3Du6mJhHNp3V6oqjjzOAYq8kw%40mail.gmail.com.

[truring truring]

Hi,

Thanks for the quick response. Somehow i resolved that error by putting my generated client code directly in my project.

but now while creating the pipeline i am getting this error on kurento media server.

2018-10-17T19:06:46,629345 20070 0x00007f4c8085e700   debug KurentoWebSocketTransport WebSocketTransport.cpp:427 processMessage()  Message: {"id":6,"method":"create","params":{"type":"voice_det","constructorParams":{"mediaPipeline":"237c5190-c4b4-449e-bbe8-10abd161c248_kurento.MediaPipeline"},"properties":{},"sessionId":"6187d2b5-fa66-44a8-a7d0-98b8e9a6b91e"},"jsonrpc":"2.0"}

2018-10-17T19:06:46,629580 20070 0x00007f4c8085e700   error KurentoModuleManager      ModuleManager.cpp:213 getFactory()  Factory voice_det not found: map::at

2018-10-17T19:06:46,629736 20070 0x00007f4c8085e700     log KurentoServerMethods      ServerMethods.cpp:332 postProcess()  Caching: {

   "error" : {

      "code" : 40108,

      "data" : {

         "type" : "MEDIA_OBJECT_NOT_AVAILABLE"

      },

      "message" : "Factory 'voice_det' not found"

   },

   "id" : 6,

   "jsonrpc" : "2.0"

I am not able to understand where to add the module in factory. what to do for that thing..? i have kept my module(libvoice_det.so) at  path /usr/lib/x86_64-linux-gnu/gstreamer-1.5.

please help me with this. Appreciate quick response here.

regards

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_DuGKBnsDBvMrFYf_%2Bim2V7Yfm-uNGJ50M4dOZaud3LvzA%40mail.gmail.com.

[Micael Gallego]

You will have to install your custom plugin in the Kurento Media Server used by OpenVidu.

Best regards

Micael Gallego

Kurento / OpenVidu Project Lead

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CALvshjOUtDB4YdzSj91%3D-jvuhze5eS3vxh9WpDYWWLBsP9fjWw%40mail.gmail.com.

[truring truring]

Hi,

Won't keeping the .so file directly at path /usr/lib/x86_64-linux-gnu/gstreamer-1.5 or 

/usr/lib/x86_64-linux-gnu/kurento/modules/ work? 

Regards 

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_Du%3DXSb-HiXHLtt9nb6VQ%3Dp5%3D%3DMrKZq%2BjsbVvqam2QDXJQ%40mail.gmail.com.

[trur...@gmail.com]

Hi,

In continuation to the development of voice activity detection feature, I need to send an asynchronous message from Media server to Client application on detecting a high decibel voice. So need some help regarding this how to send subscribe and send the events between media server and application. Do i need to create an new event for same or any existing event would work..?

requesting you help me with this. appreciate a quick response here.

regards

Inder 

[Micael Gallego]

You have to create a new event to send this information..take a look to the implementation of any other module that send events to client.

Do you plan to publish your module with an open source license? If it is Apache, we could include it in oficial KMS distribution.

Regards

--

You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/67c36f35-6e64-4121-b3eb-5b1bfa74ae10%40googlegroups.com.

[truring truring]

Hi,

Thanks for your reply.

the event trigger in my module will be based upon the messages i received on bus. for that i need to put a watchpoint as below

bus = gst_element_get_bus (pipeline);

  watch_id = gst_bus_add_watch (bus, message_handler, NULL);

In message handler, i am thinking about triggering the event. But the problem here is that i don't have an access to pipeline in my moule file (gstvoicedetection.cpp). the one which gets generated in gst-plugin folder by code generator.

Please help me with this.

regards

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_DsGSarduNxmQT7z_sxp1f%2BGe6bLQBt-t6vAxoqu6DmQ%3Dg%40mail.gmail.com.

[Micael Gallego]

Take a look how is implemented in other built in Kurento Modules.

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CALvshjNn-chSNFdCX_uvzs954Xi%2BXkB5o75ZKr%2B%3D4GJ-juf_UA%40mail.gmail.com.

[truring truring]

Hi 

I don't see the pipeline pointer created initially getting used in other modules. I am not able to locate that.

Please point me to that location in code where it is using pipeline pointer.

regards

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_DsAsgXBRMmp31mrLfhFO2QMS6RpbGz1XZTq6x7tQUF23Q%40mail.gmail.com.

[Micael Gallego]

Please implement event signaling as it is implemented in PlateDetector

https://github.com/Kurento/kms-platedetector/blob/master/src/server/implementation/objects/PlateDetectorFilterImpl.cpp

Best regards

Micael Gallego

Kurento / OpenVidu Project Lead

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CALvshjP8_KK1FDxHL7UTeojexerA3EjaApvW3MT_PcLvfCeJpQ%40mail.gmail.com.

[truring truring]

Hi,

i am getting this error while running my custom module

 gstelement.c:904 gst_element_get_static_pad()  no such pad 'sink' in element "voicedetection0"

2018-10-24T12:17:54,994394 5652 0x00007f5944f5c700    info GST_ELEMENT_PADS          gstelement.c:904 gst_element_get_static_pad()  no such pad 'src' in element "voicedetection0"

2018-10-24T12:17:54,994439 5652 0x00007f5944f5c700   error filterelement             kmsfilterelement.c:148 kms_filter_element_set_filter() <kmsfilterelement1>  Invalid factory "voicedetection", unexpected pad templates

i have defined the PAD templates in class_init function as below

gst_element_class_add_pad_template (GST_ELEMENT_CLASS (klass),

                                      gst_pad_template_new ("src", GST_PAD_SRC,

                                          GST_PAD_ALWAYS,

                                          gst_caps_from_string (VIDEO_SRC_CAPS) ) );

  gst_element_class_add_pad_template (GST_ELEMENT_CLASS (klass),

                                      gst_pad_template_new ("sink", GST_PAD_SINK,

                                          GST_PAD_ALWAYS,

                                          gst_caps_from_string (VIDEO_SINK_CAPS) ) );

do i need to do something else as well and define somewhere else. Please help me for same.

regards

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_DuYMR2ZJF6p1tOZZZgvuS5BQBd_Gdam58WbNN0eUMUypg%40mail.gmail.com.

[Micael Gallego]

It is difficult to say, the best you can do is use PlateDetector module as initial  module and start adding your own code.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CALvshjNg9aAZfWi9xXH4OmZ9wRPOUwaUVEiBQm9k-DhxjB%3D-ew%40mail.gmail.com.

[truring truring]

Hi,

As per your advice, i took platedetector code and modified slightly/added my code to fetch the root mean square values of audio stream using level plugin of gstreamer.

i have registered a signal handler on the bus. but in process bus message, following condition is never setting to true (i.e message->type is never getting set to GST_MESSAGE_ELEMENT)

if (message->type == GST_MESSAGE_ELEMENT) {

i have attached the code for reference. Please help me with this as this has become crucial for us.

Regards

Inder

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/CAFXW_DsXbyCArtrTEiYZ6_NBQOiwEfHcCCEzJzsKQkFk2DdbSQ%40mail.gmail.com.

[Truring Team]

Hi

I am still waiting for response for my previous mail. requesting someone to help me with this.

I have one more query.

As per my usecase, i am trying to use a gstreamer plugin "level" to retrieve the audio decibel levels for voice activity detection. i have used an existing module platedetector and added my code in that.

now my question is that since the platedetector module has already been added to plugin at java client side, do i need to add this gstreamer plugin level in pipeline after creating it using gst_element_factory_make function. 

level plugin was created inside platedetector moule so won't this plugin be automatically part of pipeline whenever we will create the object of platedetector and call this statement as below

PlateDetectorFilter plateDetectorFilter = new PlateDetectorFilter.Builder(pipeline).build();

Please help me with this

regards

Inder