New Humanoid Head (with silicone skin) for InMoov Robot

[Thomas Messerschmidt]

Alan,

Have you seen this: 

This new head is compatible with the three piston neck and the rest of the InMoov body. It has a silicon skin made in some 3D printable molds and can do face expressions. 

https://inmoov.fr/i2head-for-inmoov/

Thomas

[Chris Albertson]

Inmoov is the work of one person, a French sculptor, a professional artist.  And it really shows.  I’ve always thought Inmoov has a style that other robots lacked.  This is another step in that direction.  It is clear there was a lot of attention paid to how the expressions look.    

I think the thing for builders to do now is to connect it up with GPT or another LLM but software would be needed to “listen” to the generated text, interpret emotional content, and then feed that to the facial expressions.  An LLM can generate text but getting the delivery of that text to seem real will be a challenge.

Actually there is more, the face will need to react to what humans say in real time.  Doing a convincing response might actually be easier.

--
You received this message because you are subscribed to the Google Groups "RSSC-List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rssc-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rssc-list/CADyjTyaqWezyvwkKU51dv2OefA-a%3DGVLvzfZ-HGn4RXPr7naPQ%40mail.gmail.com.

[Sergei G]

Would be interesting if ChatGPT could generate not only text response, but also "emotional metadata" - the face could use it. Or there could be a post-processing AI filter which would analyze the text for emotional component. Whole new world for companion robots...

Best Regards,

-- Sergei

---

From: rssc...@googlegroups.com <rssc...@googlegroups.com> on behalf of Chris Albertson <alberts...@gmail.com>
Sent: Wednesday, June 26, 2024 10:53 PM
To: Gmail <thomas...@gmail.com>
Cc: RSSC-list <rssc...@googlegroups.com>
Subject: Re: [RSSC-List] New Humanoid Head (with silicone skin) for InMoov Robot

 

To view this discussion on the web visit https://groups.google.com/d/msgid/rssc-list/ECF49CF7-8CC4-472C-B3C0-7666D85539B9%40gmail.com.

[Chris Albertson]

To look real, the robot needs to react as the human speaks to it.    This has to be in real-time with no delay.

I can imagine a model trained to predict the emotional content of a sample of text.  This need not be an LLM.  This has been done with MUCH simpler models.     This model would have to process the text from both sides of the conversation

But, yes you could have CHatGPT produce metadata.   The API allows you to specify JSON as output format and one of thre things it can put in the JSON is funtion calls to some 3rd party API.  TRhe GPPT4 might call the funtions needed to move the face.     But this works only for GPT-generated text

[Gmail]

The newest GPT is supposed to VERBALLY chat WITH EMOTION, according to a recent video OpenAI posted. 

Thomas

-  

Need a great hardworking engineer? I am currently looking for a new job opportunity in robotics and/ or AI. Contact me directly or through LinkedIn:   

https://www.linkedin.com/in/ai-robotics/  

On Jun 27, 2024, at 6:40 AM, Sergei G <msg...@hotmail.com> wrote:



[Alan Timm]

Oh wow, I wasn't in until I checked out the video:

https://www.youtube.com/watch?v=Al08VSxt0tE

It looks really good despite the potato video quality.

The new face is actually able to convey emotions.  what sells it is the oversize anime eyes.  takes it just north of the uncanny valley.

It definitely has potential, I think it looks best with plain white silicone.

Alan

[Gmail]

Thanks for posting the video! Yes, it looks really good. Aside from the loud servos, it’s getting a whole lot closer to Amica! 

So is Grey going to get a “makeover” 😆?

Thomas

-  

Need a great hardworking engineer? I am currently looking for a new job opportunity in robotics and/ or AI. Contact me directly or through LinkedIn:   

https://www.linkedin.com/in/ai-robotics/  

On Jun 27, 2024, at 4:18 PM, Alan Timm <gest...@gmail.com> wrote:

Oh wow, I wasn't in until I checked out the video:

--

You received this message because you are subscribed to the Google Groups "RSSC-List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rssc-list+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rssc-list/c3e8b669-ebea-4010-b271-dc55bb046e29n%40googlegroups.com.

[Chris Albertson]

On Jun 27, 2024, at 4:18 PM, Alan Timm <gest...@gmail.com> wrote:

Oh wow, I wasn't in until I checked out the video:

https://www.youtube.com/watch?v=Al08VSxt0tE

It looks really good despite the potato video quality.

The new face is actually able to convey emotions.  what sells it is the oversize anime eyes.  takes it just north of the uncanny valley.

It definitely has potential, I think it looks best with plain white silicone.

Thanks, I noticed that this robot head was the work of a talented artist but I could not identify what he did.  Yes, it is the eyes.  He solved three problems at the same time.

1) the larger eyes make it look a little less like a severed human head and

2) The larger eyes are more expressive

3) Larger parts are easier to make and assemble.

Also, I think Inmoov is a scaled-down human, not quite full adult size.  Perhaps the eyes are not scaled down?   

As for the noise of the serves, I bet that can be addressed by using different servos and with some software changes.  Servos don’t usually sound like that unless they are being over-driven, software can address that.

Plain white Silicone?   I don’t know.  I was thinking of working with a makeup artist who works in ther TV industry.  She could maybe pain on reasistic color and a quality wig.   Again you’d need to be more of an artest then an engineer to do thios right.   You want the robot to still look like a robot while making it seem closer to a real person.   Here is So. Cal there are so many people who do this kind of work for a living, I would not attempt it myself.

Next question.   He made both a male and female version of the head.   Which would work best for which applications?    I’m looking at the parts.  All the plastic parts are identical for the male and female expect for thr silicon skin.

He is using a PCA9685.   So there is definitely room for improvements in how the servos are driven.  It looks like the servos are driven to predefined angles with not much though about angular velocity and acceleration (which is hard to do with a PCA9685.)

Alan

On Wednesday, June 26, 2024 at 8:30:05 PM UTC-7 Thomas Messerschmidt wrote:

Alan,

Have you seen this: 

This new head is compatible with the three piston neck and the rest of the InMoov body. It has a silicon skin made in some 3D printable molds and can do face expressions. 

https://inmoov.fr/i2head-for-inmoov/

Thomas

[Greg Perry]

The LLMs through Ollama are pretty quick.  Gael, Todd, myself and a few others have been experimenting with Ollama vs OpenAI and comparing different models & prompts.

There are a number of strategies to get  (meta-data/"function calling") info back.  How quick (which is typically the concern) depends
on which model and what kind of prompt.   e.g. llama2 + json == very quick well structured callbacks, llama3 8B is pretty slow when you ask for json format.
Recently, we've added the ability to /generate - which means it can use a prompt to create text about a submitted picture.
This will be pretty useful when we ask the robot, "What are you looking at" or some other visual inquiry.

I was honestly surprised how fast the responses are .. on the order of a couple seconds for llava and bakllava models.
I think this has a lot of potential, even with assisting in SLAM - since it could know the rotation angle of the viewport of the camera.
If you want to follow our progress we chat about it in discord - https://discord.gg/8NYKAUTxmZ 
Ollama is our first choice since its open source, free to use, and local if we desire.

[Chris Albertson]

You say “pretty quick”. I assume this is running locally. What hardware are you using? Can you quantify this? It is 10 seconds, 2 or zero lag?

I’m set up to run Llama3 8B on Apple Silicon hardware. I had no idea Llama3 would process images.

My use of this is just for robot commands in plain language. I’d like to capture speech and get JSON-formatted function calls out.

[Greg Perry]

GeForce RTX 2070 Mobile. The puter driving the bot doesn't have a nvidia card, but with a little config I can have it request the caption from a local puter that does.
To my knowledge LLama3 is not multi-modal, and does not process images, but LLava and bakllava do.
~3s .. you can see a quick gif I did here https://discord.com/channels/887362610360643664/1241261427319115807/1256974472771276830 ,
for some reason replying here I can't post images :/ 
For us 3s is acceptable, probably takes me longer to come up with a similar detailed description .. especially pre-coffee ;)
Llama3's chat completion and prompt processing is impressive, however, my personal experience is when you send the request with json:true it has abysmal performance.
There are models that are trained that handle structured json/function calling way better (Mystral, FireFunction, .. its a long list)
Although, there is a "special" param for json format in the request, I'm always wondering, would the prompt + prompt history be faster and more  reliable than that field .. 
dunno yet.

[Greg Perry]

Good idea on getting metrics of timing,  I'll add processing time to the ui

On Saturday, June 29, 2024 at 10:09:32 PM UTC-7 albertson.chris wrote:

[Chris Albertson]

I looked up the specs.  It seems the RTX2070 is the lowest model Nvidia card you would want to use for running Llama3 8B.    The performance of the Mac M2 compared to the RTX2070 is between half as fast and four times faster, depending on which Mac you have.   The lowest price RTX2070 is about $300 right now.   But of course, you’d need a computer to put the GPU into.

I doubt that anyone wants to drop an RTX2070 into their robot.  Most people might even prefer to use the cloud-based AI to spending the money and time to run locally.    I guess this is the end of the self-contained robot.  Robots will have to be tethered either to a nearby notebook computer or to OpenAI’s cloud-based servers.

On Jun 30, 2024, at 7:22 AM, Greg Perry <supe...@gmail.com> wrote:

Good idea on getting metrics of timing,  I'll add processing time to the ui

On Saturday, June 29, 2024 at 10:09:32 PM UTC-7 albertson.chris wrote:

You say “pretty quick”. I assume this is running locally. What hardware are you using? Can you quantify this? It is 10 seconds, 2 or zero lag?...

[Alan Timm]

There's a guy out there cyber_one that's documented his process of building the new head and face:

https://www.youtube.com/watch?v=mmI_BBNk1eM

So far he's posted 7 videos.