I want to render real world objects that I see through kinect as 3D objects. As of right now, I can see the RGB data from the kinect on a 2D surface. I wish to render those objects as 3D using the depth map. Has anyone done anything of similar nature before? I am relatively new to Unity. As far as I could understand, I'll need to write shaders to do this. Basically create a mesh from the depth data and warp the rgb data as the texture on top of that mesh.
Am I heading in the right direction? Any tips, tutorials or advice to do this would be really appreciated.
Thanks