Using three.js to export to glTF?

[Andrew Lee]

Hi folks,

I've been using the mediapipe Javascript API with the face_mesh NPM package. I've got some example code that converts the face_mesh data from a single frame to a three.js using almost exactly the same process as demonstrated here. From there, I've used the three.js GLTFExporter class to export the mesh, which I can then import and interact with in Blender. The workflow is great.

However, up next, I would like to import the entire animation into Blender. I know this is, on some level, possible, because of tools like blendartrack that use the related ARTrack API to create a file that can be imported into Blender, albeit via different means.

However, this has been very difficult for me, in particular, because I am not sure how to set up and configure the armature that would move the face mesh. I need to create and bind three.js bones in such a way as to sensible contort the face mesh without the entire thing looking ridiculous. The documentation on this issue seems scant, so I wanted to ask the community if anyone has any experience using this, or any other workflow to ultimately export a face_mesh to another tool. Any file format is fine here-- my only goal is to get the face mesh to import into blender, which I hope is just a data conversion task. If I could export as COLLADA, or glTF, or any other related format, that would work. And the hard part of actually identifying the face is done.

I would appreciate any ideas here, or any tips from a person with related experience. Thanks so much to you all and I hope you have a great day.

Best,

Andrew