Merge multiple models from separate reconstructions

[paul...@googlemail.com]

Hi,

first of all, great work, COLMAP is a really nice piece of software! :)

I'm using COLMAP for large scale reconstructions (11 cameras in a rig, 15 min of video each, sampled with 3 fps) to get precise camera locations for subsequent analysis of object tracks in the videos. With shorter videos, the incremental mapper worked fine, but now with the long videos (about 25000 images in total, OPENCV_FISHEYE model with shared intrinsics for the 11 cameras), the global bundle adjustments just takes too long. I'm running COLMAP on a HPC cluster with a wall time limit of 24 h, which is a problem for the bundle adjustments with >15000 images.

Since I'm running COLMAP on a cluster anyways, I thought of splitting the reconstructions into smaller parts (with enough overlap), which I can run in parallel on seperate nodes. This is actually much faster, and the separate reconstructions finish in less than a day. After the incremental mapper, I'd like to merge the models (20 models with about 1500 images each), using the COLMAP model merger functionality.

I saw that in the RunModelMerger function, and in Reconstruction::Merge, that models are compared for merging with the image_id and not the image name, which is a problem for me, since the different models don't come from the same reconstruction with same image_id name correspondences. So my question, is it possible to change image_ids in the already finalized models? Updating the database (images, inlier_matches) with shared common image_ids (and pair ids) for all separate reconstructions won't help, because the model is already written to the .bin files (and only those are imported), right?

Would it make more sense to update the image_ids in the database before starting the mapping part of the reconstruction? I guess they would be kept through the whole process?

I also saw that there is there is already a mapper that splits the reconstruction into clusters implemented, although I didn't manage to build/use this on the HPC cluster (and those clusters would then still be reconstructed sequentially, not in parallel). Same applies for the rig bundle adjustment (and I also don't know the precise relative camera locations prior to the reconstruction..). Would any of those two methods fasten things up for me, so that I don't need to split the reconstruction into parts?

Any help would be much appreciated :)

[Johannes Schönberger]

Hi Paul,

the ``model_merger`` command assumes that your different sub-models have consistent image identifies, as you already correctly found out. In your specific setting, you could first create a common database by extracting/matching features for all images in one process. Then, later you could run the reconstruction process only on a subset of images (note the ``--image_list_path`` argument for the ``mapper`` command, which allows you to restrict the reconstruction to a subset of images). Finally, you run the ``model_merger`` on the overlapping sub-models.

Does that make sense?

Alternatively, I have a hidden "hierarchical" mapper that is not publicly exposed yet. This clusters the scene automatically into overlapping sub-models, reconstructs them in parallel, and merges them in the end. I have not had time to fully test this feature yet but it worked in quite a few of my tests already.

Cheers,

Johannes

[paul...@googlemail.com]

Ok, I didn't know about the --image_list_path argument. I've rewritten the image_ids in all databases based on shared image names now directly after feature extraction, so that should also work then. I had to manipulate the databases anyways (with a small python script) to share cameras between images, or is there a way to share intrinsics based on image subsets (directory structure or shared prefix), without using the gui?

About the model merger, will I have to run it sequentially, adding one model at a time? If so, does it make sense to run a bundle adjustment after every merge, or just in the end? I'm not sure whether cameras will be shared based on the ids from the different reconstructions (which are are consistent), but the intrinsics should already be fine from the sub-model reconstructions.

Thanks a lot!

Paul

[Johannes Schönberger]

Running the bundle adjuster after each merge is probably the safe option, especially if your individual sub-models are not very accurate. Otherwise, a final bundle adjustment after merging all models should be sufficient.

> --
> You received this message because you are subscribed to the Google Groups "COLMAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to colmap+un...@googlegroups.com.
> To post to this group, send email to col...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/colmap/32a12318-7477-4f50-8b92-3abf9fb5065b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[ibd]

I have a similar use case as Paul.

I can see how to solve the problem from this thread, but from the rest of the workflow, it would be preferable to merge models based on the image name, instead of the (internally given) image ID.

Have there been any updates regarding this functionality since these posts in April?

Cheers

-- ibd