COLIBRI VR for Unreal projects?

[Irenee Liu]

Hi,

Thanks for sharing the tool which enables a good practice for light field capturing and rendering in VR. I'm wondering do you have any experience with integrating the tool to Unreal? It's troublesome to transplant the project to Unity.

Another way I can think of is to capture the dataset in Unreal and do the processing and rendering in Unity. Then the problem is how to encode the cameras.txt and images.txt in /sparse. I had a few tries but it seems it's not correctly translated.

Br,

Irenee

[Greg de Dinechin]

Hi,

Thanks for your interest, I'm very happy if the project generates appeal for doing the same in Unreal! I'm far more experienced with Unity than with Unreal, so unfortunately it's not in my current plans to do so myself, but hopefully the open-source project will encourage others to launch similar projects in other game engines.

In the meantime, I agree with your comment, the best way is probably to capture the data in Unreal and save it in a way that enables rendering it in Unity. Information on the file structure can notably be found here.

To do so, I recommend that you look to the COLMAPConnector.cs file, specifically the methods SaveCamerasInformation and SaveImagesInformation. For the images, you'll see that there is a coordinate conversion step, via ConvertCoordinatesUnityToCOLMAP. This is because the text files describe data in COLMAP's coordinate system (to ensure compatibility with the tool should it be used for 3D reconstruction), and therefore I had to convert the coordinates of the Unity cameras to fit with this coordinate system. A similar approach probably has to be done with Unreal, by transposing the ConvertCoordinatesUnityToCOLMAP method.

If this was not the issue, feel free to provide me details on where the translation failed, and I'll do my best to help.!

Best,

Greg