Species to game engine? Suggestions?

[John Estes]

Have any of you our there had experience with using species generated characters in a gaming engine using mocap?  I have some experience in the mocap capture, but conversion  to a game engine acceptable rig and use within it utilizing softimage and species I am lacking.   Any examples out there?

 

Any suggestions significant issues or tutorials you can point me to that will shorten the learning curve? Advise? 

 

Sorry for a double post, accidently replied to an old message.

 

Thanks

 

John

[Chris Covelli]

Hi John,

Im no expert, but Id say the general workflow would be to animate/apply mocap to the Species AnimRig in Softimage.  Then plot the motion onto the rig's deformers, delete the control hierarchy, and finally export that to your game engine.

One of our star beta testers, Nicolas Esposito actually made a few videos detailing the conversion from Species in Softimage to both Unity and Unreal game engines.

Here are the links:
https://vimeo.com/groups/species/videos/36400436
https://vimeo.com/groups/species/videos/36400435

Hope this helps!

Chris Covelli
http://www.polygonpusherinc.com/
http://exocortex.com/products/species

TurboSquid Models

--
 
---
You received this message because you are subscribed to the Google Groups "Species Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to species_mailing_...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Robert Dowling]

Yeah I use the Species rig mostly for this. Plot onto the deformers, delete the rig and bones and selection export the deformers and mesh.

If your game engine has bone limits, you'll need to re-weight and remove the twist deformers to bring the count down (and likely delete the tongue bones too).

I put together a little script that does this but if you were to use it you would likely want to add or comment out a few things, look at it as a reference only. Use at your own risk!

http://snipt.org/Ahve2

[Chris Covelli]

Thanks so much for sharing that snippet Robert!

Chris Covelli
http://www.polygonpusherinc.com/
http://exocortex.com/products/species

TurboSquid Models

[Nicolas Esposito]

That is quite useful!

Thanks for sharing :)

[John Estes]

Nicolas ( or anyone willing to help),

I have been using your video for export to unity 3d and I can't get it to work.  Have you tried it recently with the species upgrades? 

If so, has any of your process changed that I should know about? 

I am just looking for the latter half of it, after the mocap has been applied to a rig, my steps to get the animation plotted to the rig without errors and then export to unity 3d

Thanks for any help you or anyone else can give me.  I am a novice at exporting into game engines so I appreciate your assistance as I have tried many things and I can get the mesh into the game engine, but the animation doesn't transfer.    I have tried just animating and importing a simple box, that works.  

Thanks,

John 

  

If I plot the bvh using the mocap rig, then remove constraints with all child nodes selected, it sperates the rig and the mesh and any 

[John Estes]

Sorry forgot to put that I was getting an error in some cases objecthomonid_animRig.kinIK_R_con.Kine.sciz is locked when plotting all transforms to an fcurve.  Not sure what that means.  If I continued,  would get the mesh center attached to the rig, but it wouldn't move any extremities it would just spin around as if the center of gravity was attached to the rig but nothing else. 

Thanks,

John

[Nicolas Esposito]

Hi John,

I haven't tried the new Species release with the same technique that I used in the tutorial, anyway I have the same error if you take a look at 5.39 of the tutorial

Based on what you're saying (remove constraints with child nodes selected ) its a strange error, I need to check it out

What I would suggest using Unity is to save a read-only version of your properly rigged character, then you'll have multiple files with different animations, with before and after the removing of child constraint and plotting, so you can modify/add anything you want...as far as I saw the removing of the constraints causes lot of troubles

Since both SI and Unity have been upgraded I suggest you to just plot the animation ( Plot all transforms ), DO NOT remove the constraints and just export it to Unity and see if it behave properly

Anyway I'll do some testing and see if I'll find a solution to this problem, but give it a try using the solution above

[John Estes]

Nicolas,

Thanks for the feedback, I really appreciate your help.  Not sure what you mean by a " read only" version.  You mean just save the "orginal model" then just save another version and plot the transformation?

 

I did get something to work, mocap to rig, plot animation, store all in transforms in F curve, then removed constrains as your video suggested.   I got it to work, but I will be doing this over and over again and I am hoping to get this more streamlined.   The result plays fine as in softimage now, but getting it in unity was ugly :)  

 

So definitely add to the list for species development a clean export by FBX to unity.  I too had to use the old mesh because the low res other versions were kept.

 

Looking forward to your attempt at import and learning from you.  

 

John

 

 

 

 

--

[Nicolas Esposito]

Hey John,

Lets start with the Unity export

I downloaded the latest Unity version ( 4.3 ) and I did some testing using as reference my tutorial, since has been a while I used Unity

The process is still the same, but there a couple of things you need to do:

- After you created the AnimRig just delete the Geo_low and Geo_mid meshes under the "Meshes" null, so you wont export multiple subdivision of the character....you can also save this as a base character, means that you'll save the SI scene, make it a read only, so each time you need to test or apply some mocap you'll save another version named by the animation or anything else, so you don't repeat the same retargeting process all over again.....alternatively you can just delete the animation on the rig via the dopesheet oe animation mixer

- Do not switch from IK to FK, it will mess up the rig inside SI ( at least using the backflip animation that I used in the video )

- Do not Remove Constraints, just select child nodes and plot the animation ( all transforms )

- From SI 2013 x64 exporting into FBX deselect Camera and lights, leave everything else as it is

In Unity drag&drop your character into one of the project folder, then go into the project folder and click once on the character

On the right you'll see its properties window, and if you click the preview button on the bottom right you'll see your animation playing in loop once you click the play button

Unfortunately, I think because of the new Mechanim animation pipeline, I'm not able to set loop animation on my character and just drag&drop into the scene, because if I do so and hit the play button ( the big one on the top ) my character doesn't move, while in the preview window on the bottom left ( as described above ) it works

Oddly enough I have some strange skin deformation on the character ( see attached picture ), but that could be because of some not frozen geometry in SI, or it might be that something else I don't get right now

To reply to your last post about mocap:

I've succesfully exported mocap data from SI using my custom FaceRig attached to the Species Rig, and it works just perfectly, you can see a very small test on my yuotube channel, check out the last video, is old but it still count! :-D

I have 2 kinect and I'm using iPi Desktop motion capture, that for static/idle/slightly moving around is perfectly fine, but for thing like runnig or complicated action is not a solution...although I already found a solution to the running and action scene in general, which requires time and large space, two things that currently I don't have :)

If you'll use ipi Mocap just export the BVH file as a 3ds max bvh file, then the steps for the retargeting are basically the same of my UDK/Unity tutorial with the Species plugin

If you're going to use any other mocap software, well, I guess it'll be a bit of trial and error before getting the proper result

By the end of the year I'm going to upgrade my mocap gear, first for the face and maybe later for the body ( since mocap body suits are bloody expensive ), and I'm also working on my tech demo related to game pipeline, which features lots of very cool stuff that I developed for both Unity and UDK

If you need some other help let me know

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote:

[MrClintypoo .]

Hi Nicolas

I am getting the same distortion in the upper arm within Unity, but not in Maya. However, in the Autodesk FBX reader it craps out as well. I thought it was down to some dodgy rotation thingy which affects both the shoulder and the upper arm. Could it be down to the extra twist controller from species which isn't recognised by Unity? Answers on a postcard please. :)

Thanks for your first vid on this, very helpful.

--

[John Estes]

Nicolas,

Awesome!!!  Thanks for taking the time to do all that.  I will try that and see how it goes.  I am getting ready to do a whole bunch of motion capture and it was getting really difficult and frustrating to work it through and I was considering trying something other than species which I didn't want to do.  

 

I have the same odd biceps deformation, not sure either why that is. 

 

Good to know that ipi soft works well for you for basic things. 

 

Re; Animation not playing

Try going to the Rig section and switch it to Legacy  and store in root node and see if the animation works for you.  I don't know that it will loop, but if you aren't getting the animation, that is what I tried and it worked. 

 

One problem I was having ( using old species and the system I got to work based off your video ( in xsi 2011 advantage pack 64 bit) was that when I would export say a long 1500 frame animation, xsi was giving me the grey screen of death.  I have successfully exported 500 frame sequences, but not 1000 frames, so I am wondering if the plot all transformations is too much information maybe ( I have a quad core 10 gig ram set up).  Is there any way to reduce the volume of information somehow?   I never got the new version of species to work so far.  So I will try out your steps and see how it goes. 

 

Thanks a ton Nicolas,

 

John

--

[Nicolas Esposito]

Hey John / Clinton,

Right now I can't check it because I'm at work, but I will soon do some other testing

Anyway, here are a couple of ideas:

- Regarding the upper arm in Unity if I remember correctly I don't have the same problem within UDK, so it might be a Unity problem, but I'm not 100% sure ( need to double check later )

- If its related to the mesh itself do a copy, freeze it and cut it from the hierarchy, then just replace the mesh following the Species tutorial on the Species vimeo channel...but if you already replaced the Species mesh with your own mesh and still you have the strange deformation on the upper arm I think that its a rig problem...alternatively try to export a .DAE ( collada ) file with the animation and see if the problem is still there ( since in SI there's no upper arm problem, and based on what Clinton said, nor in Maya )

- Regarding the long ( more than 1000 frames ) animation, maybe I'm saying BS, but try to reduce the animation steps while plotting ( set "Step value" to 2 instead of 1 when plotting "All transformations" )...if I'm correct if you do this you have a key every 2 frames, that means less ( half? ) data to be exported

Considering that the Species character with the rig without the animation is more or less 4mb and the plotted "version" is 14mb I guess all the data is within the animation itself, so if you reduce the Steps value you cut in half the data

I have a 4 years old i7 2.6 Ghz and 6gb of ram, while exporting it takes a while, but it does the job right, but I never exported very long animations, so I may have the same problem

I think that I'll do another updated version for the exporting into Unity once I figured out all the problems related to the FBX export, since in Unity is more or less the standard file format to import data

Hopefully by this week I'll sort it out and do another video :-)

Cheers

Nicolas

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote:

[John Estes]

Good stuff Nicolas. Looking forward to the video if you make it. Thanks for taking the time to make suggestions for me to try.  I would love to see the exocortex fellows create a export to fbx plugin to streamline the process of export as the issues are discovered. Maybe down the road. 

John

--

[Nicolas Esposito]

Problem solved ( kinda ) :-)

Once you import the character into Unity go to:

Edit > Project Settings > Other > Blend weights > change it from 2 to 4

There are still some weighting problems, but its a Unity problem, since in UDK ( see attached ) I don't have this problem, and in Unity you can't change more than 4 blendings, so I guess some tweaking on the envelope itslef is required

Works, kinda

Nicolas

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote:

[MrClintypoo .]

Coolio! Thanks for the info.
I'll give it a try in a few hours time.
Have you tried it on an animated rig? Only on the animation I had along with the twisted arm thingy, it also appeared that the shoulders were locked when the arms were moving.

--

[John Estes]

Hello all,

Anyone out there good at character design for unity 3d using species and willing to give me a quote on making a male/female fitness coach type character.  Mocap will be applied to him/her and he will be utilized in a fitness exercise performance environment.

You can send a message to bifil...@hotmail.com

Thanks,

John 

[Nicolas Esposito]

Hi Clinton,

The last post with the attached pictures is referred to a character with mocap applied ( the same as shown in the video ), so yes, the twisted-like arm is there when you import an animated character.

Since the problem is caused by bone blending on the skin ( as far as I understood ) the problem is only on animated character, with static meshes should be ok

I remember that I had the "shoulder locked" problem, but with my previous post with the detailed explanation on how to export from XSI to Unity I solved that problem :)

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote:

[Nicolas Esposito]

Quick update and possible solution:

I've been reading some threads about the skin deformation problem, and it's confirmed that is caused by the enveloping itself inside SI ( and it's a common problem from what I've seen )

In 3ds and Maya the trick is to change the bones influence, that is the same thing that you could do while enveloping a character inside SI ( as soon as you choose the Envelope operator )

Since with Species you can't choose the bone weights ( because the envelope its generated automatically ) I think that the solution would be to select the mesh when it's enveloped, expand the mesh properties inside the object browser, and open the Envelope tab which should be there, and try to tweak the bone influence from there ( which I assume is more than 4 since in Unity, if you change the bone influence to 4 the problem is still there? ), export, and check the results

I'll give it a try when I can, but in the meantime you can try tweaking the enveloping settings directly inside SI and see if the skin behaves properly inside Unity

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote:

[John Estes]

Nicolas,

Thanks again for all your help.  I am under the gun a bit and I am struggling to find the answers I need.  Liked what I saw on your youtube channel. 

So my current state of affairs is with respect to Unity import.

1.  I would like the lower resolution mesh to be my primary mesh in unity and to delete the other resolution meshes--how do I do that?  I saw how to keep the higher res Nicolas, but how do you only keep the low res.

2.  Is there a way without reenveloping the mesh and rerigging everything to get this on a more streamlined skeleton in unity ( kind of like a shadow rig for Mecanim)?  I could use the default SI skeleton, but I can only imagine the havoc that would ensue with the enveloping process and transfer.

3.  Can we get any exocortex guys to take a look at the mesh issue in SI and maybe provide a fix or weight in on how to simplify the bones of the skeleton?  Any chance guys?  Forces me to use older version of species which has other issues and increases the game performance overhead as I understand it. 

I realize today is Thanksgiving but it makes me thankful for you guys and your help when you get a chance :)  I am the Turkey here I know it, but I appreciate any gravy you can pass my way. 

All the best,

John

--

[Nicolas Esposito]

Hey John,

I've been away the entire week and I couldn't test anything, but based on what you've said

1. There could be two ways:

1.1: I imagine that all the versions of the meshes are rigged with the same envelope, so I suppose ( need to test it ) if you just keep the lower mesh and delete all the other meshes ( located under the Meshes null, "Geo" something ) it will export just the low res mesh

1.2: If I remember correctly the low res mesh is just separates cubes that resemble a body, so what you can do is: ( before do the AnimRig ) select all the cubes, merge them together into a single mesh, cut it from the hierarchy and freeze the mesh > use the Replace mesh function so you'll basically replace the "standard" mesh as soon as the PropRig is created > if you do this you'll have the low res mesh with the envelope inherited from the mesh you just replace

2. Honestly I haven't tried yet the shadowrig with Mechanim inside Unity, so I'm not sure....anyway I suppose that what you want to do is to use the Species rig combined with the Mechanim system in order to use the Mechanim features...well...I had a similar issue with UDK regarding the sharing of the animations, means that my Species rigged character wont work with the already available animations applied to the UDK characters, because of course of the different skeletal configuration

Question: how good is the Mechanim shadow rig? its works fine or its just for rapid prototyping and Species/SI rigs works better because of all the tweaking you can do?

3. If nor Chris or Eric answer I suggest you to contact them directly via their website/email, since its public ;-)

I'll do some other testing during the weekend, but I think that the problem is within SI itself and how the enveloping is done inside SI, since the bone influence is the main problem, everything else looks fine to me

If I'll come up with a solution I'll let you know and probably make an updated tutorial about the export to Unity

Cheers

Nicolas

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote:

[Nicolas Esposito]

There you go guys, a quick fix on how to export to Unity3D using Species 1.6 version

Update - Softimage to Unity3D using Exocortex Species plugin ( Quickfix for 1.6 version ) 

Cheers :-)

On Thursday, September 19, 2013 6:29:50 PM UTC+2, John Estes wrote: