Textures from colored vertices of mesh?

[vadasent...@gmail.com]

Hello! :) I'm new here and would like to know if the creators of Skanect can come up with a really quick plugin to turn colored vertices into textures, then textures into high quality textures :)

Thank You!

[Nicolas Tisserand]

Yes, we can! ©

:-)

Proper texture mapping is definitely on the radar, but it will take
some time. Computing seamless texture maps and coordinates from RGB+D
streams is not a quick and easy task, to say the least. We had some
prototype implementation last year, and will eventually get back to
it, but it won't happen overnight I'm afraid...

[Oliver]

I use MeshLab for this, takes around 1-2 minutes:

1. Go to 'File > Import Mesh...'.

2. Go to 'Filters > Texture > Parametrization: Trivial Per-Triangle' (if you get an error about the inter-triangle border being too much increase the texture dimension).

2. Go to 'Filters > Texture > Transfer Vertex Attributes to Texture (between 2 meshes)', the output .png will be saved in the same directory as your model.

4. Go to 'File > Export Mesh As' (we're resaving the .obj so it includes the new UVW coordinates).

5. You're done!

[Nicolas Burrus]

You're right Oliver, we could actually implement a simple workaround
like this to bake the vertex-colored mesh to a UV map. It is not
optimal for a quality point of view, but could solve the compatibility
issues with software that absolutely need UV textures.

Thanks for sharing this,
Nicolas

> --
> You received this message because you are subscribed to the Google Groups
> "Skanect" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to skanect+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Oliver]

Yeah, it's not ideal, but it does the job. I just noticed that the RGB images are also saved to the scan directory, which is great. I'll give something like this a try later in the week: Parameterization and Texturing from rasters http://www.youtube.com/watch?v=OJZRuIzHcVw

Couple of questions:

1. I assume either the sensor or Skanect accounts for barrel distortion etc in the image (I dunno how much of a problem it'd be anyway)? 

2. Can/does Skanect take control of auto-exposure settings on an XBox Kinect or Carmine sensor?

Cheers

[Nicolas Burrus]

Hi Oliver,

Thanks for sharing this tutorial. Meshlab is full of ressources, but
it's not always trivial to discover them..

To answer your questions:

1) The distortions are not taken into account in Skanect. This would
require an additional calibration step, involving the user, which we
try to avoid. So far we did not find distortion to be significant
enough to have a big impact.

2) Yes, we can control and disable the auto-exposure on some devices.
Kinect for Windows for sure, Carmine 09 maybe, we have to check. We
have this on our todo list, and we understand it's important to get
good colors.

[vadasent...@gmail.com]

Thank You, Oliver! That's AWESOME! :D

[vadasent...@gmail.com]

How long until we get decent color mapping to texture mapping plugin?

[Nicolas Burrus]

Not yet back to working on it, so unfortunately I can't tell yet. Any
success with the Meshlab method?

On Mon, May 6, 2013 at 7:46 PM, <vadasent...@gmail.com> wrote:
> How long until we get decent color mapping to texture mapping plugin?
>

[Oliver]

Hey Nicolas,

Unfortunately I've only had an hour or two play with Skanect since I last posted, so my only test in this area was with mapping a single RGB image taken with a DSLR. The results aren't bad considering it only takes a few minutes to take a shot/align it to the model, but not terribly exciting/useful for my intended purposes. I'll remember to update you if get a chance to test out how well multiple images actually works/using RGB images coming from the sensor.

Here's what I ended up with for anyone interested (using a creepy beautician training head thing)

Model: http://i.imgur.com/hAEv0lX.jpg

Model with per-vertex colours: http://i.imgur.com/cOdchlE.jpg

DSLR image: http://i.imgur.com/jDkFrUI.jpg

Mapped on to model:  http://i.imgur.com/Zed6N9f.jpg / http://i.imgur.com/cyCUVyU.gif

[Oliver]

Hah, I totally forgot to include the point of me posting. I don't know how many people would want this, but can I suggest looking into exporting MeshLab .MLP files inside Skanect, especially exporting per-frame camera positions, as aligning by hand is a total pain if that stuff is being solved on your end already. Maybe with some control over which frames are output.

I dunno if you've looked inside an .MLP file before, but its information structure should hopefully mean it's not horribly time consuming to do. For reference the .MLP below is of the head I posted and a couple of manually aligned cameras.

<!DOCTYPE MeshLabDocument>

<MeshLabProject>

 <MeshGroup>

  <MLMesh label="head.obj" filename="D:/Junk/head.obj">

   <MLMatrix44>

1 0 0 0 

0 1 0 0 

0 0 1 0 

0 0 0 1 

</MLMatrix44>

  </MLMesh>

 </MeshGroup>

 <RasterGroup>

  <MLRaster label="head-front.jpg">

   <VCGCamera TranslationVector="0.624941 -0.483159 -0.146029 1" LensDistortion="0 0" ViewportPx="5184 3456" PixelSizeMm="0.00990197 0.00990197" CenterPx="2592 1728" FocalMm="113.028" RotationMatrix="-0.632808 -0.771866 0.0614484 0 0.203508 -0.0892245 0.974999 0 -0.747086 0.629493 0.213544 0 0 0 0 1 "/>

   <Plane semantic="" fileName="D:/Junk/head-front.jpg"/>

  </MLRaster>

  <MLRaster label="head-back.jpg">

   <VCGCamera TranslationVector="-0.571542 0.486894 -0.00562644 1" LensDistortion="0 0" ViewportPx="5184 3456" PixelSizeMm="0.00990197 0.00990197" CenterPx="2592 1728" FocalMm="113.028" RotationMatrix="0.683624 0.729514 0.021612 0 -0.021276 -0.00967936 0.999727 0 0.729524 -0.683897 0.00890408 0 0 0 0 1 "/>

   <Plane semantic="" fileName="D:/Junk/head-back.jpg"/>

  </MLRaster>

 </RasterGroup>

</MeshLabProject>

On Wednesday, 8 May 2013 12:53:11 UTC+1, Nicolas Burrus wrote:

[Nicolas Burrus]

That's a good suggestion, but we only know the pose of the color
frames coming from our camera, not some the DSLR, so you will have
have to fine tune it manually, unless we add some magics (which we can
think of!).

Thanks for sharing your feedback!
Nicolas

[Oliver]

Hey, sorry I wasn't clear! I meant creating a MeshLab file that has information about the position of the PrimeSense device in 3D space from any particular frame and linking it to the RGB image also captured from the device, rather than a DSLR.

[Nicolas Burrus]

Yes, this is reasonable, could definitely make it as a "hidden"
feature to be used by advanced users.

[vadasent...@gmail.com]

Hi, Nikolas, how do I get a texture as good as yours in the images you posted up in May?

[vadasent...@gmail.com]

On Sunday, July 7, 2013 12:22:59 PM UTC-5, vadasent...@gmail.com wrote:

Hi, Nikolas, how do I get a texture as good as yours in the images you posted up in May?

Nikolas, if you could please get back to me, that would be GREAT! :) Thank You, and have a wonderful day! 

[Nicolas Burrus]

Hi,

Sorry for the delay, what images are you referring to? Note that UV
texture mapping has been included in Skanect 1.5.0 (only for Mac right
now), but will make it to the Windows version very soon.

Best,
Nicolas