Running COLMAP + OpenMVS in Microsoft Azure blazing fast

[elit...@gmail.com]

Since I do not own a monster computer myself, I've figured out how to run COLMAP and OpenMVS pipeline in Microsoft Azure.

Based on price/performance (with timing on the actual executables) I found these machines best suited:

For COLMAP: NC6s_v2 Standard - 6 vCPU - 112 GB RAM - SSD - 1 nVidia P100 (PCIe) - $2.8 / hour

For OpenMVS: F32s_v2 Standard - 32 vCPU - 64 GB RAM - SSD - $2.6 / hour

The scene I ran consisted of 120 images 1920 x 1080, and it took around 45 minutes - $2.5 (including deploying, installing, copying files and deleting).

Here's how to do it:

Start by downloading the executables for COLMAP and OpenMVS:

https://demuc.de/colmap/3.4/COLMAP-3.4-windows.zip
https://github.com/cdcseacave/openMVS_sample


and place them in a folder structure like this:

Create folder [COLMAPOpenMVS]

Under COLMAPOpenMVS, create folder [colmap34]

Put these files under COLMAPOpenMVS\colmap34

colmap.exe
Qt5Core.dll
Qt5Widgets.dll
Qt5Gui.dll
cudart64_80.dll
libglog.dll
libblas.dll
libgcc_s_sjlj-1.dll
libgfortran-3.dll
liblapack.dll
libquadmath-0.dll
glew32.dll
FreeImage.dll
msvcp120.dll
msvcr120.dll
vcomp120.dll


Under COLMAPOpenMVS\colmap34, create folder [platforms]

Put this file under COLMAPOpenMVS\colmap34\platforms

qwindows.dll

Under COLMAPOpenMVS, create folder [openmvs]

Put these files under COLMAPOpenMVS\openmvs

DensifyPointCloud.exe
InterfaceVisualSFM.exe
ReconstructMesh.exe
RefineMesh.exe
TextureMesh.exe
vcomp140.dll

Under COLMAPOpenMVS, create empty folder [workspace]

Under COLMAPOpenMVS, create folder images and put the [images] you want to reconstruct in there

Under COLMAPOpenMVS, create RunOnNC6.bat and RunOnF32.bat

The RunOnNC6.bat file should contain this:

.\colmap34\colmap.exe feature_extractor --database_path .\workspace\database.db --image_path .\images\
.\colmap34\colmap.exe exhaustive_matcher --database_path .\workspace\database.db
.\colmap34\colmap.exe mapper --database_path .\workspace\database.db --image_path .\images\ --export_path .\workspace\
.\colmap34\colmap.exe model_converter --input_path .\workspace\0\ --output_path .\workspace\model.nvm --output_type nvm

The RunOnF32.bat file should contain this:

.\openmvs\InterfaceVisualSFM.exe -w .\images\ -i ..\workspace\model.nvm -o ..\workspace\model.mvs -v 0
.\openmvs\DensifyPointCloud.exe -w .\images\ -i ..\workspace\model.mvs -o ..\workspace\model_dense.mvs -v 0
.\openmvs\ReconstructMesh.exe -w .\images\ -i ..\workspace\model_dense.mvs -o ..\workspace\model_dense_mesh.mvs -v 0
.\openmvs\RefineMesh.exe -w .\images\ -i ..\workspace\model_dense_mesh.mvs -o ..\workspace\model_dense_mesh_refine.mvs -v 0
.\openmvs\TextureMesh.exe -w .\images\ -i ..\workspace\model_dense_mesh_refine.mvs -o ..\workspace\model_dense_mesh_refine_texture.mvs --empty-color 0 -v 0 --patch-packing-heuristic 0

Go to Azure and create the NC6s_v2 instance. If you don't have enough quota, create a support case and increase it to 38 cores. It'll take two days to get approved.

Connect to the machine via Remote Desktop.

Copy the COLMAPOpenMVS folder to the machine by select copy on folder on local machine and select paste on the desktop on the remote machine in remote desktop application.

While files are copying, Install the "Azure NC drivers for Windows 2016" by googling it on the remote machine.

Run the RunOnNC6.bat when all files are copied and the drivers installed successfully.

When the application has finished, create the F32s_v2 instance and deploy in the same Resource Group as the NC6s_v2.

Share the desktop folder on the NC6s_v2 to you as a user.

Mount the NC6 Desktop onto the F32, so that you can copy over the files.

Delete all resources belonging to the NC6 in Azure portal to save money.

Run the RunOnF32.bat on the F32.

When finished copy these files over to your local machine:

model_dense_mesh_refine_texture.ply
model_dense_mesh_refine_texture.png

When copied. Delete all remaining resources from the Resource Group to make sure Azure isn't charging you more than needed.

Download MeshLab, install and run. Drag and drop the model_dense_mesh_refine_texture.ply into MeshLab and watch your 3D model.

If you have any suggestions on how to improve this, please comment!

[cDc]

Cool tutorial, thank you!

First I am curious why you chose this strange combination? OpenMVG works great too and have direct export to OpenMVS (by the way, the export trough VisualSFM format can create problems, as that format is not playing nice with the general camera format used by most frameworks). Plus COMAP has integrated densification module, why not using it and complicate yourself with OpenMVS?

[elit...@gmail.com]

This is where it all started: https://www.google.se/amp/s/pfalkingham.wordpress.com/2017/05/26/photogrammetry-testing-11-visualsfm-openmvs/amp/

So I followed his recommendation to do COLMAP and OpenMVS.

COLMAP 3.4 is so lightning fast with the GPU support, but it lacks texturing of meshes and OpenMVS beats COLMAP on mesh refinement.

nVidia P100 runs COLMAP fine and a 32 core machine runs OpenMVS fine.

However, I still have issues with distance threshold for photos where the horizon is captured. The refinement seems to average out the small model when the world is 20000 times larger.

I also have refinement issues with surfaces that are flat, and could consist of two triangles, but consists of 100000.

So, a little help with the parameters are very welcome.

COLMAP is still progressing heavily, OpenMVG seems to have stopped development. But I can try...

[cDc]

The author of that link posted the update at my suggestion warning him of the VisualSFM problem, he probably chose COLMAP for the user interface, however the reconstruction that I shown him and convinced him was reconstructed with OpenMVG and OpenMVS: https://skfb.ly/6rxoR

As you can see the quality is even better, without any manual cleanup. Plus the last thing you need in a cloud system is user interface.

OpenMVG is already mature, this is one reason why the development is a bit slow, plus the developer is currently busier than usual and that does not help either.

Yes, a too broad area as the horizon passed to mesh refinement decreases a lot the quality of the region of interest. I suggest to add a step that automatically selects the region of interest of the scene and crop the mesh before refinement. One idea to do this is to use the camera positions for deducing the ROI.

[elit...@gmail.com]

You made me really interested in trying the OpenMVG + OpenMVS pipeline.

And... I’d like to try it in Linux. Even though I’m not really a Linux user.

However, running it in Linux seems to require compiling. It would be a pity if I had to spend dollars on Azure just to compile OpenMVG.

Would it be possible to precompile all the executables and just copy them to the remote machine?

I might have a try during the weekend.

If would be perfect to not have to move the work between a GPU optimized machine and a CPU optimized machine.

[cDc]

I'm not a Linux user either, but there must be some way to use precompiled binaries, but I do not know how.

If you are a windows guy, why do you want to use Linux? Both OpenMVS and OpenMVG compile on Windows quite easily. For example OpenMVS needs just 4 command lines to build on a clean machine. OpenMVG must be similar.

[elit...@gmail.com]

Just for one reason, Windows is quite expensive in Azure.

Are there OpenMVG executables ready for download just like for OpenMVS? Where if so?

Thanks

[elit...@gmail.com]

Sorry... OpenMVG is way too complicated to get running for Windows. I leave it to the hardcore experts.

Im fine with COLMAP at the moment, which runs out of the box after download.

Thanks anyway.

[Pierre Moulon]

Hi,

OpenMVG to use under windows is as hard as extracting an archive and run the existing binaries:

https://github.com/openMVG/openMVG/releases/download/v1.3/ReleaseV1.3.Yellowtail.WindowsBinaries_VS2015.zip

-> I think you are doing the same with colmap and openMVS (extracting precompiled binaries and run them).

BTW, every new software seems complicated before you start to learn how to use it.

OpenMVG have some part that are far more flexible than COLMAP.

Regards,

Pierre

[Peter Falkingham]

For some reason I thought you'd used your own proprietary matching portion of code - didn't realise you used OpenMVG - I've always had terrible results with OpenMVG for some reason.  Did you have to specify any specific parameters?  

I used COLMAP not for the GUI (I'm mostly running it from the command line these days), but because it was fast and robust without needing me to dig into reconstruction variables.

P

[cDc]

Actually you might be right, I do not remember if I used for sure OpenMVG for this case or my proprietary SfM software. However I often tested OpenMVG and the accuracy of the SfM results are very good (confirmed too by the public benchmark https://www.tanksandtemples.org/leaderboard with the results obtained by the authors of the benchmark). It is indeed not very fast though.

[Peter Falkingham]

Thanks for the info!  On the back of your reply I went and spend a couple of hours tinkering with openMVG and managed to get some really nice matching for my Styracosaurus model dataset.  I'll document it at some point with timings.

Cheers.

[elit...@gmail.com]

Hi,

I'm sorry. I haven't managed it to work, translating the python script to batch:

.\openMVG_main_SfMInit_ImageListing.exe -i .\images\ -o .\output\matches\ -d .\sensor_width_database\sensor_width_camera_database.txt -c 3
.\openMVG_main_ComputeFeatures.exe -i .\output\matches\sfm_data.json -o .\output\matches\ -m SIFT -f 1

.\openMVG_main_ComputeMatches.exe -i .\output\matches\sfm_data.json -o .\output\matches\ -r 0.8 -g e
.\openMVG_main_GlobalSfM.exe -i .\output\matches\sfm_data.json -m .\output\matches\ -o .\output\
.\openMVG_main_ComputeSfM_DataColor.exe -i .\output\sfm_data.bin -o .\output\colorized.ply
.\openMVG_main_ComputeStructureFromKnownPoses.exe -i .\output\sfm_data.bin -m .\output\matches\ -o .\output\robust.ply

.\openMVG_main_openMVG2openMVS -i .\output\sfm_data.json -d .\output\ -o .\output\model.mvs

Might it be:

'neato' is not recognized as an internal or external command,
operable program or batch file.


- Geometric filtering -
0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************
*
*
*
Task done in (s): 0

I've tried differens image-sets, so it can't be problem with the input data.

Any suggestions?

[cDc]

why don't u use directly the python script? it can be used both on windows and linux

[elit...@gmail.com]

I think I have to close this thread by giving my final word.

OpenMVG + OpenMVS works great in Azure F32s_v2 with 32 cores and 64 GB memory.

OpenMVG requires installation of VC++ 2015 Redist , but that takes only a few seconds.

The complete pipeline can be run on a single machine, minimizing time for deployment, copying files and deleting machines.

My new favorite is definitely OpenMVG + OpenMVS (until I can get hold on a GTX 1080 privately.)