Mocap Face
Spencer Prather
No, but we recommend it. If your performances involve moving around, a mount is necessary to ensure continuous tracking. Traditional mounts and rigs for face capture are clunky, uncomfortable and expensive. We designed an affordable, comfortable and lightweight head mount, the Headrig, that places the iPhone in front of your face while also not blocking your field of view too much. The Headrig is also included in the Face Capture bundle.
Yes, the Rokoko Face Capture app connects to Rokoko Studio, our software, over WiFi. The app gives you real-time feedback of the tracking and allows you to trigger a recording in Rokoko Studio (or stream the data in real-time to your preferred 3D software, with one of our integrations). When the recording is over, the file is saved in Studio and ready for export.
After the 14-day free trial is over, you will need to subscribe to the Face Capture Add-on subscription (as well as the Studio Plus subscription, or higher, if you want to live stream the facial mocap data to our plugin integrations). We recommend buying the Face Capture bundle after your 14-day trial expires, as it includes 1 year of licenses as well as the Headrig as a substantial discount, you can buy it here.
At 240g, Headrig will go unnoticed while ensuring clear line of sight for accurate face tracking during the entire performance. The intuitive double tightening mechanism secures the performer with complete freedom of head movement in all directions, at any head motion speed.
We know first hand that helmet face capture solutions can be uncomfortable and also very expensive. We set out to design a solution that is affordable without compromising on both quality of capture (steady and secure phone placement) and comfort for the actor (lightweight, padded and snug fit with great breathability).
Yes! As long as your iPhone is in front of your face, the capture with Rokoko Face Capture app will work. We recommend the Headrig if your performances involve moving around, to ensure continuous tracking.
Headrig features two mechanisms for fitting it precisely and comfortably to any adult head: the circumference is secured by turning a fastening wheel on the back and a velcro strap that runs on the top of the head can be tightened for an even more secure and comfortable fit.
Designed by animators, for animators - Reallusion's latest facial mocap tools go further, addressing core issues in the facial mocap pipeline to produce better animation. Starting from all new ARKit expressions, to easily adjusting raw mocap data as well as retargeting, to multipass facial recording and essential mocap cleanup - we're covering all the bases to provide you with the most powerful, flexible and user friendly facial mocap approach yet.
Faceware Studio is a high-quality, realtime animation software that tracks facial movement from live or recorded video. It uses state-of-the-art machine learning and deep learning techniques to track your face and create animation in realtime.
Analyzer is our standalone facial tracking software. Analyzer converts video of facial performances into proprietary motion data that can be applied to any character via our Autodesk Retargeter Plug In.
Retargeter is our animation and solving plug-in to common Autodesk animation products such as Maya, 3DS Max, and MotionBuilder. Retargeter takes the data from Analyzer and applies the motion to your characters through an intuitive and pose-based workflow.
Shepherd combines all the integral features a face capture operator needs in one innovative and thoughtful app. Sync your entire performance capture workflow with timecode support and additional functionality to lync your Vicon, Xsens or Optitrack system to trigger your facial systems.
Explore our different licensing and product options to find the best solution for your facial motion capture needs. If you need a more tailored solution, talk to us about our Enterprise Program.
Xsens MVN Animate allows for the Manus Prime II glove data to be integrated into its software, through Manus Core and then connects to Unreal using the MVN Live Link plugin, that was just recently updated for 4.26.
For the body data, I am using the Link suit by Xsens. For the finger data, I am using the Manus Prime II gloves which are compatible with Xsens. And for the face data, I am using both my webcam as well as recording my face with my Canon 1DX mark II with a 50mm in order to have more control over lighting and recording at higher frame rates.
Ideally, I would like to test out the facial motion data with the Mark IV wireless camera system or the Indie Headcam built around the GoPro. The reason for this is that when looking at the video footage of the face captured with these tools, I notice even though the head moves, the face is perfectly centered in front of the lens.
I also want to mention this is my absolute first time recording facial data with Faceware Studio, having only tested and dabbled with the UE LiveLink Face app using my iPhone. The control I have in Faceware Studio is truly remarkable.
This laptop is actually a little beast! When looking for a laptop in order to start learning Unreal Engine, this laptop was suggested to me by Alex Bartz, who happened to write a blog about mobile workstations for VFX. Thank you, Alex!
When I went onto the Razer page and looked at the Studio model, I saw for the first time a filmmaker that created an entire film in Unreal Engine using this laptop. That filmmaker is Haz Dulull and he made the film Battlesuit with this laptop, in Unreal!! I never forgot that.
One of the challenges has been learning to retarget the data to these characters, since they have 5 spine bones and 2 neck bones, in the character remap asset chart. The Xsens MVN plugin prefers characters with 4 spine bones and one neck bone. I played around with the remap asset and got it to work, beautifully!
I have to thank Katie Jo, for not only helping me with this process but for believing in me and giving me the hand up that I needed when first starting out with virtual production a few months ago. Katie Jo, there is a special place in my heart for you. You have moved mountains for me and brought me closer to my dreams. Thank you, for being you!
Faceware Studio is quite easy to use. All you do is open up the software, calibrate your face and can then make adjustments to various facial expressions in the Animation Tuning. When you calibrate, the software then tracks your facial movements, which that alone is pretty crazy!
I discovered the video recording option thanks to a review by Solomon Jagwe. Thank you, Solomon! This review gave me the idea to record my face with my camera, and to be able to light my face better using softbox lights.
The other amazing option Faceware Studio has, is that it has a streaming panel, which is where you can turn on, Stream to Client. This option works with the Live Client plugin to Unreal by Glassbox Technologies.
This is how I am able to get the data from Faceware Studio, into the Metahumans project. You can press play in Unreal and see your data play out in real time, or as I prefer, to add a Live Link Skeletal component and see the data play out without having to be in play mode.
Besides being able to make adjustments to the facial motion in Faceware Studio, you are also able to make adjustments inside of the Faceware Live blueprint by adding multiply or addition floats. That is as far as I have gotten with regards to customizing the blueprint as it is the first facial motion blueprint I have ever set up.
One more thing I want to mention, is besides loving Faceware Studio, the team and people that make up this company have been so good to me. Helping me, teaching me how to use these tools, how to get them to work and the support I have received, has given me the encouragement to keep going. Thank you, Peter, Karen, Josh and Brian!
I wanted to experiment with the curves adjustments of the facial animations I recorded and see if I can not only make adjustments to the Faceware Studio Animation Tuning and the Faceware Live blueprints, but also if I can add more to the expressions I recorded by playing around with the tongue curves and trying to get enunciations that I am having a hard to capturing with the facial motion data.
I figured out how to use the backward solver which makes adjusting the curves much easier in sequencer. This, I want to spend more time with, in order to fine tune the animations I have recorded and to accentuate certain expressions, besides adding some that are not there.
I am also finding that I am getting lost in this project. You can spend days, weeks, months, possibly years, and still not discover everything going on with these characters and what they are truly capable of.
This week I watched and listened to the Pulse, The Rise of Real-Time Digital Humans. Some very interesting comments were made, but also some interesting questions. One being, can these characters elicit emotions from an audience? My question is, can I figure out how to do that with them?
When the release of the MetaHuman creator comes out, I know that my goal is going to be to use them for my ALH84001 project about the rock from Mars, that spawned the birth of Astrobiology. To recreate the ANSMET team that discovered the rock, but to also capture their faces, their conversations and bring those moments back from time. I get goosebumps thinking about the endless possibilities this release is bringing.
I find that when I spend so many hours inside of Unreal, that the line between real and unreal becomes blurry. Ever since I started working on this vlog, I have found that the time literally flies, and even when I sleep, I am troubleshooting and coming up with things I want to create with them.
I am hopeful that Dartan or someone who is in the know a bit more than me can give me a little clearer view on where to go for a project I am working on for someone. There is currently a workstation here for a bit with Iclone 8 and CC4 on it so I have gotten to play with it finally. It has the face live plug-in and my results on my Daz G8 characters is quite good thus far. I am trying to find a way to retarget them so I can bring them into Daz Studio to keep working with my current animations that I do in Daz.
c80f0f1006