How to send event to JavaScript from custom module with Kurento and OpenCV?

[Michael Dominguez]

Hi I have a face detection module that I have created blur everything but the face that it detects. I am using JavaScript to interface with it. I would like for the module to send a signal back to the a JS listener whenever the face is not detected so that my frontend knows when the face is not detected. Then add a message saying "Your face is not detected!". I have been researching and found this documentation and code CodeFoundEvent, ZbarFilterImpl, and  filters.ZBarFilter.kmd.json.

These have been useful but I do not understand GstSrtucture and how information is getting passed from the pipeline to signal the event. For example in plateDetector.cpp in the busMessge function, in the try statement:

1. How does this issue an event and how does the frontend subscribe to it?

2. Is it possible to create my own FaceDetected event, similar to PlantDetected and then signal that event?

Any help would be appreciated.

Thank you

Michael

[Michael Dominguez]

I found out a way to send an event to a JS frontend. 

This is not going to be a full explanation but if you have question I would be happy to help.

first you need to define an event and its properties in the .kmd.json file. in my case it looks like this:    faedetectionmodule.FaceDetectionModule.kmd.json

{

  "remoteClasses": [

    {

      "name": "FaceDetectionModule",

      "extends": "OpenCVFilter",

      "doc": "FaceDetectionModule interface. Documentation about the module",

      "constructor": {

        "doc": "Create an element",

        "params": [

          {

            "name": "mediaPipeline",

            "doc": "the parent :rom:cls:`MediaPipeline`",

            "type": "MediaPipeline",

            "final": true

          }

        ]

      },

      "methods": [

. . .

      ],

      "events": [

        "FaceFound"

      ]

    }

  ],

  "events": [

    {

      "properties": [

        {

          "name": "detected",

          "doc": "Weather it the face is detected or not.",

          "type": "boolean"

        }

      ],

      "extends": "Media",

      "name": "FaceFound",

      "doc": "Event raised by a :rom:cls:`FaceDetectionModule` when a face is found in the data being streamed."

    }

  ]

}

This will create  an event class for you to use the next time you build your project. The parameters it takes in the constructor are (shared_ptr, string, {any_other_props_you_defined}). The generated code provides this constructor:
 FaceFound (std::shared_ptr<MediaObject> source, const std::string &type, bool detected) : Media (source, type)

Then you need to add a few things in the {module_name}Impl.hpp and  {module_name}Impl.cpp (NOT {module_name}OpenCVImpl.cpp). This gets into gstreamer and I have no experience with it at all. This is where I am going to omit what I did here and move to how to use the event that is generated by the kmd.json file.

You usually use the Message Bus to put a "face-found" message onto the bus and then emit the corresponding event when you find the message with the correct name, but I am not sure how to do this properly.

So I found a hack way of doing what I need. In my case there are a good amount of messages being passed during the video processing. I just have an if-statement for whenever I need to send my event. Here is the code for when a message comes onto the bus:    FaceDetectionModule.cpp

void FaceDetectionModuleImpl::busMessage (GstMessage *message)

{

  bool emitState = FaceDetectionModuleOpenCVImpl::getEmitState();

  if (emitState){

    try {

      FaceFound event (shared_from_this(), "face-found", FaceDetectionModuleOpenCVImpl::getFaceDetected());

      sigcSignalEmit(signalFaceFound, event);

      FaceDetectionModuleOpenCVImpl::setEmitState(!emitState);

    } catch (std::bad_weak_ptr &e) {

    }

  }

}

This is how you would use the FaceFound event auto generated from the kmd.json. I am using three functions in my {module_name}OpenCVImpl.cpp file. but you can ignore those. The main thing is that you should pass shared_from_this() and a string  and then whatever other properties you define here.

The signalFaceFound that I am using I defined in the Impl.hpp file:

public:

. . .

  sigc::signal<void, FaceFound> signalFaceFound;

Again this is not a full explanation. My hope is that is leads you in the right direction.
If you have any question please replay.

Michael