Questions about face recognition

342 views
Skip to first unread message

Alex Dattilio

unread,
May 8, 2018, 11:47:43 AM5/8/18
to InMoov
Im getting my inmoov partially assembled for a very important event im hosting in a few days and i wanted to use facial recognition to greet each of my guests by name. Is there currently a way in mrl to use photos of people instead of live video from the cameras to train mrl to identify certain humans? I'll be running mrl on a Nvidia TX2 which has its own machine learnin object recognition software that i could use for this purpose; however i am unsure how i could incorporate that software into mrl. I suppose i could possibly create a service to run the commands necessary, but i have no idea how to create a service in the first place. Any help and suggestions would be very much appreciated. Thanks.

Morten Enholm

unread,
May 8, 2018, 12:25:21 PM5/8/18
to InMoov
Hi Alex,
You should also aske the question over in the chat at http://myrobotlab.org/ :)

gael langevin

unread,
May 8, 2018, 5:16:02 PM5/8/18
to Morten Enholm, InMoov
Hello,
You need to add each pictures in the train directory under the person's name.
If the train directory doesn't show in the myrobotlab directory, you will need to first launch the face recogniser via voice command:
START FACE RECOGNISER
Then in the OpenCV window, add a name in the tab and clik the button "train".
You need to train it for at least two different faces for it to learn.
Once this is done the train directory is loaded and you can add the pictures of your friends and names.



Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



--
You received this message because you are subscribed to the Google Groups "InMoov" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inmoov+unsubscribe@googlegroups.com.
To post to this group, send email to inm...@googlegroups.com.
Visit this group at https://groups.google.com/group/inmoov.
For more options, visit https://groups.google.com/d/optout.

Roger

unread,
Aug 17, 2021, 2:14:11 AM8/17/21
to InMoov
Hi Gael, 

I followed your instructions above, 

I added a name and clicked on train, 

and I can see a folder (for example)  C:\mrl\myrobotlab.1.0.2693.16TEST\training\Roger   and inside this folder, are many images of my face...., 

and I got other folders with other people names I have trained too, (in Training mode). 

and my question is what is the next step in activating InMoov to recognize my face or other people's face ? (and also say a simple greeting to the recognized person)....

Also during in training mode, I find that when I untick the "Train" check box to finish training, it crashes MRL straight away, is there something I'm not doing right here? 

Looking forward to your advice, 
thank you 

On Wednesday, May 9, 2018 at 7:16:02 AM UTC+10 hair...@gmail.com wrote:
Hello,
You need to add each pictures in the train directory under the person's name.
If the train directory doesn't show in the myrobotlab directory, you will need to first launch the face recogniser via voice command:
START FACE RECOGNISER
Then in the OpenCV window, add a name in the tab and clik the button "train".
You need to train it for at least two different faces for it to learn.
Once this is done the train directory is loaded and you can add the pictures of your friends and names.



Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



2018-05-08 18:25 GMT+02:00 Morten Enholm <enho...@gmail.com>:
Hi Alex,
You should also aske the question over in the chat at http://myrobotlab.org/ :)


tirsdag 8. mai 2018 17.47.43 UTC+2 skrev Alex Dattilio følgende:
Im getting my inmoov partially assembled for a very important event im hosting in a few days and i wanted to use facial recognition to greet each of my guests by name. Is there currently a way in mrl  to use photos of people instead of live video from the cameras to train mrl to identify certain humans? I'll be running mrl on a Nvidia TX2 which has its own machine learnin object recognition software that i could use for this purpose; however i am unsure how i could incorporate  that software into mrl. I suppose i could possibly create a service to run the commands necessary, but i have no idea how to create a service in the first place. Any help and suggestions would be very much appreciated. Thanks.

--
You received this message because you are subscribed to the Google Groups "InMoov" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inmoov+un...@googlegroups.com.

gael langevin

unread,
Aug 17, 2021, 5:08:57 PM8/17/21
to Roger, InMoov
Hello,
The crash after training is a bug that has been annoying me in Manticore, hopefully Nixie will have that solved.
Once you have trained faces, you can say to the robot: "face recognizer" or "stop face recognizer".
This will launch the recognition, if a face is recognized, it shows on screen the name of the person.
The greeting is not implemented in Manticore.
You would need to modify the facerecognizer.py file and the inmoovGestures.aiml to do that.

Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



Roger

unread,
Aug 18, 2021, 4:18:33 AM8/18/21
to InMoov
Hi Gael, 

1.Its awesome! I tried your advice and it works thank you. So happy to see InMoov MRL actually detects faces and puts a name on it, 

on the other hand though,  I find that it recognizes still photos better than a real human face standing infront of the camera (with static photos, it picks up alot faster), is that normal? 

Also, when I tried MRL OpenCV to recognize say a real human face in front of the camera (eye) -  it takes a long time to recognize, and sometimes when it recognized the person, it crashed the whole MRL straight away........ and with static photos , it works a lot better...., please advice or is that a bug as well?

Also, what I found is that MRL actually greets the photo on the first time it recognizes it (it says Hi Rachel (one of my test photo) .......I was like wow, how did MRL do that? ) 
but then in the following photos or person it recognize, it does not greet....., but do you know how MRL has provided a greeting the first time? its awesome.., 

1. As said, I did check the aiml file you suggested and I found these lines below:
<category><pattern>START FACE RECOGNIZER</pattern>
<template>I'm starting face recognizer<oob><mrl><service>python</service><method>exec</method><param>facerecognizer()</param></mrl></oob></template>
</category>

And I think it is calling the facerecognizer function in the py but not very sure if it did....and I dont understand how I can add a condition here for a greeting here to a face that is recognized...,please advice?  

2. I also checked this file you adviced me: the facerecognizer.py  

I only can understand that it has applied the PyramidDown and FaceRecognizer filters, also applied tracking for OpenCV 
but I dont understand the   "fr.train() " function , can you please advice where I can look up for this function in Manticore MRL  ?  to see what it actually does

Please advice, 

Thank you and looking forward to your response, 

gael langevin

unread,
Aug 18, 2021, 4:08:31 PM8/18/21
to Roger, InMoov
Hello,
Now that you mention it, I remember that we did implement the greeting.
So I looked it up and you can find it in InMoov/service/D_OpenCV.py.
With the function: onRecognizedFace(name)

How it works:
Say to the robot: "face recognizer"
It will launch the function, if your face is recognized it will greet you, start a chatbot session with your name and at the same time it disables the face recognizer filter.
This is the reason if you present another face it doesn't greet again. For to greet the new person you would need to launch again "face recognizer".

You could also try to comment out this line in the D_OpenCV.py:
    ##i01.opencv.disableFilter("FaceRecognizer")
It should keep the filter actif, therefore greet every known faces and start the chatbot session according to their name.

I am not sure but these lines should also be commented out in faceRecognizer.py:
##fr.setMode(OpenCVFilterFaceRecognizer.Mode.TRAIN)
##fr.train()# it takes some time to train and be able to recognize face
Because this is supposed to start training to save new faces.

Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



Roger

unread,
Aug 19, 2021, 12:26:54 AM8/19/21
to InMoov
Hi Gael, 

Yea I thought so that you have implemented that, its really cool thank you, 

1. I looked in D_OpenCV.py from your advice and I couldnt find this line to comment out.. 

##i01.opencv.disableFilter("FaceRecognizer")

is it in another py file? 

Also I found these 3 lines, is this where the greeting is?  where did "System_SAY_HELL0:" come from ? because I'm trying to change the greeting, 

  if isChatbotActivated:
      chatBot.setUsername(unicode(name,'utf-8'))
      chatBot.getResponse("SYSTEM_SAY_HELLO")

2. Also since chatbot is set to true on startup ,  when I start FACE RECOGNIZER,  I think one of these line has turned chatbot or VOICE off,  when face recognizer recognizes 
me, straight away, InMoov has gone slience and no voice coming out......, something i'm not sure on what has disabled the voice/chatbot, please advice?

and if I comment the first two lines and just leave this line: chatBot.getResponse("SYSTEM_SAY_HELLO")    , voice will still be active, chatbot still work but InMoov will non 
stop talking ......, please advice on this behaviour..

3. And if I run OpenCV Face Recognizer for say more than 5mins, the below error comes up from time to time and MRL crashes...., please advice?

13:03:45.418 [i01.opencv_videoProcessor] ERROR c.myrobotlab.opencv.VideoProcessor - stopping capture

4. In facerecognizer.py - I followed your advice and commented out " fr.train()"   and it didnt recognize me anymore...., should I just leave this line uncommented? 



Thank you and looking forward to your response. 

gael langevin

unread,
Aug 19, 2021, 5:07:41 PM8/19/21
to Roger, InMoov
Hello,
1- no wonder you didn't find this line, I mixed up files between Manticore and Nixie:
##i01.opencv.disableFilter("FaceRecognizer")
So forget about uncommenting this line.
2-SYSTEM_SAY_HELLO is found in the InMoov/chatbot/bots/en/inmoov_chatbot.aiml
It uses a SRAI to access the pattern WAKE_UP line 120.
You can modify the answers there.
3-Don't run the filter faceRecognizer directly in OpenCV, because it is buggy.
InMoov service had been modified to apply the filter in order to avoid that bug.
 
Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



Roger

unread,
Aug 20, 2021, 3:51:03 AM8/20/21
to InMoov
Hi Gael,

1. Yea that line wasn't in Manticore, so I will leave it thank you. 

2.  SYSTEM_SAY_HELLO   are causing these REPITITIONS BELOW and once Manticore has recognized me, it starts talking non stop (I can like hear 4 to 5 voices at the same time overlapping each other..., one of the voices 
was a greeting but the other voices are talking something else which I dont understand), please advice? 


Matched: REPETITIONDETECTED <THAT> * <TOPIC> * personality.aiml
writeCertainIFCaegories learnf.aiml size= 78
input: REPETITIONDETECTED, that: Are you aware you just said that, topic: ruining the sounds of it raining, chatSession: org.alicebot.ab.Chat@19be3a6e, srCnt: 0
Matched: REPETITIONDETECTED <THAT> * <TOPIC> * personality.aiml
writeCertainIFCaegories learnf.aiml size= 78
input: REPETITIONDETECTED, that: Again, topic: ruining the sounds of it raining, chatSession: org.alicebot.ab.Chat@19be3a6e, srCnt: 0
Matched: REPETITIONDETECTED <THAT> * <TOPIC> * personality.aiml
writeCertainIFCaegories learnf.aiml size= 78
input: REPETITIONDETECTED, that: The definition of insanity is doing the same thing over and over and expecting different results, topic: ruining the sounds of it raining, chatSession: org.alicebot.ab.Chat@19be3a6e, srCnt: 0
Matched: REPETITIONDETECTED <THAT> * <TOPIC> * personality.aiml
writeCertainIFCaegories learnf.aiml size= 78
input: REPETITIONDETECTED, that: The definition of insanity is doing the same thing over and over and expecting different results, topic: ruining the sounds of it raining, chatSession: org.alicebot.ab.Chat@19be3a6e, srCnt: 0
Matched: REPETITIONDETECTED <THAT> * <TOPIC> * personality.aiml
writeCertainIFCaegories learnf.aiml size= 78
input: REPETITIONDETECTED, that: Are you aware you just said that, topic: ruining the sounds of it raining, chatSession: org.alicebot.ab.Chat@19be3a6e, srCnt: 0
Matched: REPETITIONDETECTED <THAT> * <TOPIC> * personality.aiml
writeCertainIFCaegories learnf.aiml size= 78


3.  I have found SYSTEM_SAY_HELLO i in the InMoov/chatbot/bots/en/inmoov_chatbot.aiml  and modified this line to something like this: 

<category><pattern>SYSTEM_SAY_HELLO</pattern>
<template><srai>HI name this is the change I have made</srai></template>

and some how it didnt pick up the name that matched the face encoding from OpenCV FaceRecognizer and it didn't say on what I typed above 
within the srai tags... 
any advice? on how I can put the name (matched) variable here returned back from the FaceRecognizer

Also, after recognizing me, it goes on introducing itself with what it can do etc... repeatedly with overlapping 3 or 4 of poppy voices... 
I'm not sure why it does that even browsing in the codes..., please advice? 

input: name this is the change I have made, that: I can do gestures, place a call, send a text, search for information, open applications, translate languages, play videos and music and more, topic: ruining the sounds of it raining, chatSession: org.alicebot.ab.Chat@4fe5a72c, srCnt: 2
Matched: NAME * <THAT> * <TOPIC> * reductions1.aiml
2. <srai>WHAT IS this is the change I have made</srai> from NAME * <THAT> * <TOPIC> * topic=ruining the sounds of it raining)
Matched: WHAT IS THIS * <THAT> * <TOPIC> * reductions_update.aiml
3. <srai>DESCRIBE YOURSELF</srai> from WHAT IS THIS * <THAT> * <TOPIC> * topic=ruining the sounds of it raining)
Matched: DESCRIBE YOURSELF <THAT> * <TOPIC> * reductions1.aiml
4. <srai>NAME</srai> from DESCRIBE YOURSELF <THAT> * <TOPIC> * topic=ruining the sounds of it raining)
Matched: NAME <THAT> * <TOPIC> * bot_profile.aiml

Thank you and looking forward to your response, 

gael langevin

unread,
Aug 20, 2021, 4:30:25 PM8/20/21
to Roger, InMoov
Hello,
1- you should not add directly in the SRAI but bellow, like this:
<category><pattern>WAKE_UP</pattern>
<template><li>
        <srai>HI</srai>
        <random>
          <li>Hello <get name="name"/>, I am ready and listening.</li>
          <li>What can I do <get name="name"/>.</li>
          <li>How can I help you, <get name="name"/>.</li>
        </random>
      </li></template>
</category>

Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



gael langevin

unread,
Aug 20, 2021, 4:40:48 PM8/20/21
to Roger, InMoov
If you wish to modify the HI pattern, you need to edit the personality.aiml, line 1430
<category><pattern>HI</pattern>
<template><random>
<li>Hi <get name="name"/>, nice to see you!</li>
<li>Hi it's great to see you <get name="name"/> !</li>
<li>Hi <get name="name"/>, how are you?</li>
<li>Hi! <get name="name"/>, I can really feel your smile today.</li>
<li>Hi! It's delightful to see you <get name="name"/>.</li>
</random></template>
</category>
Note that the HI pattern comes as a SRAI in  the WAKE_UP pattern
So the robot, when waking up will say something randomly from those two patterns, for example;
Hi Roger, how are you? What can I do for you?

Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



Roger

unread,
Aug 22, 2021, 12:49:20 AM8/22/21
to InMoov
Hi Gael, 

1. I made the modifications from your advice, it is awesome! thank you so much! 
It is greeting me in a much better way now, it still repeats itself though... I guess InMoov everytime 
recognizes me, it will say the same thing, repeating over and over again and also triggers and repeat the below as well (in personality.aiml), 
(when I have my face in front of its camera)


<category><pattern>REPETITIONDETECTED</pattern>
<template><random>
<li>You are repeating yourself.</li>
<li>Are you aware you just said that?</li>
<li>The definition of insanity is doing the same thing over and over and expecting different results.</li>
<li>Did you mean to repeat yourself?</li>
<li>You've already said that.</li>
<li>Again?</li>
</random></template>
</category>

and is it possible that I can take out the SRAI tags which is linked to the WAKE_UP pattern?


2. Last point you made in the previous email, you mentioned "Don't run the filter faceRecognizer directly in OpenCV, because it is buggy.""
Does that mean it's best not to run FaceDetector filter as well directly in OpenCV? 

3. and you also mentioned that "InMoov service had been modified to apply the filter in order to avoid that bug."", so does that mean only run OpenCV face Recognition from voice command "START FACE RECOGNIZER" only? 

Thank you and looking forward to your response, 


gael langevin

unread,
Aug 22, 2021, 3:12:08 AM8/22/21
to Roger, InMoov
Hello,
1-Try adding this red line in the D_OpenCV.py to see if it stops the repeating effect:
    chatBot.setUsername(unicode(name,'utf-8'))
    opencv.disableFilter("FaceRecognizer")
    chatBot.getResponse("SYSTEM_SAY_HELLO")

2-Removing the SRAI tag"HI" from the WAKE_UP pattern will diminish the random answering. 
But if you wish to modify it you can of course.

3- Only run OpenCV face Recognition from voice command "START FACE RECOGNIZER". Otherwise you might create twice the same filter, and it will crash OpenCV.

Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



Roger

unread,
Aug 22, 2021, 11:54:12 PM8/22/21
to InMoov
Hi Gael, 

1. I tried and added this line in  opencv.disableFilter("FaceRecognizer")  as your advice and it seems it didn't repeat ....but I notice that it shuts off face recognizer, 
so does that mean everytime if I want to activate FaceRecognizer, I need to voice command "START FACE RECOGNIZER"?

2,3.  Only run OpenCV face Recognition from voice command "START FACE RECOGNIZER". -  Ok, understood ths

Thank you and Looking forward to see your reply 

gael langevin

unread,
Aug 23, 2021, 4:07:29 AM8/23/21
to Roger, InMoov
Hello,
Good that it stopped repeating.
Having faceRecognizer "ON" at all time is really resource consuming anyway.
Maybe it should run with a timer, this way it would auto check every now and then if there is a new face.
Or maybe take pictures and do face recognition on the picture instead of continuous video analysing.
We have this setup for object recognition with yolo in Nixie, it really helps the CPU to not go overloaded.

Gael Langevin
Creator of InMoov
InMoov Robot
@inmoov



Roger

unread,
Aug 24, 2021, 8:10:14 AM8/24/21
to InMoov
Hi Gael, 

Yes right,  having faceRecognizer "ON" at all the times results in my CPU always hitting 100%, 
and great idea about a timer .....guess I will need to invent some code in there right? (scratching my head now....) 

Thank you for your help, I'm glad I can reach this stage with OpenCV. 
Reply all
Reply to author
Forward
0 new messages