How to increase success rate with pano_modify?

[Juha Kallioinen]

Hello,

I'm trying to make hugin align quite a challenging set of pictures. I've taken handheld photos from a number of locations on most of the days during last year. The idea is to create a video of how the locations change over the course of the year.

I'm using hugin 2020.0.0 on Ubuntu.

I first tried using ffmpeg and vidstabdetect/transform to stabilise the video with not-that-great results. Then I turned to hugin and found this excellent example of a way to do it: https://photo.stackexchange.com/a/108146

I followed the steps there up to using nona, after which I again used ffmpeg to create the final video.

When it works, the results are excellent. Here's one example, unprocessed images on the left, hugin on the right:

https://www.youtube.com/watch?v=1M4SNpkFLCo

I have about 300 photos from five different locations taken around the same time of day with an iphone8.

Now to the problem:

With the other four locations the process most often halts (or slows down to a crawl) in the pano_modify step. I've added some progress output printouts to pano_modify to see what's taking it such a long time (eating 100% of CPU all the time).

I can now see that it's busy in src/hugin_base/algorithms/basic/CalculateOptimalROI.cpp

autocrop() method.  More specifically it keeps running the nonrecheck() method over and over again. I've stopped the run a number of times after testRects list has been well over 200k and the progress has been slowed significantly. I'm not sure if pto_modify will produce something passable or even finish at all if it goes that far.

Most often if pano_modify works it finishes in a few seconds.

Do you have any advice on what could I do to have a better chance of getting pano_modify to do its thing?

Is there a minimum number of matches found by cpfind that I should aim for? Sometimes it just can't find any matches between some images. In those cases I've excluded the problematic images from the series.

Is there a minium Average (rms) distance between Controlpoints that could lead to better results? In my current run the autooptimizer ended up with a value of 0.474933066833906. After that pano_modify has been running now for 16 hrs already and has 150k+ testRects in the list :-)

Any advise is welcome!

Best regards,

 Juha

[T. Modes]

Juha Kallioinen schrieb am Donnerstag, 14. Oktober 2021 um 17:26:15 UTC+2:

Now to the problem:

<snip>

Most often if pano_modify works it finishes in a few seconds.

Do you have any advice on what could I do to have a better chance of getting pano_modify to do its thing?

Can you provide the pto file so we can check for ourself?

 

[Juha Kallioinen]

Hi,

the pto file for pano_modify is quite large, about 2MB. I've gzipped it and it's available here: https://www.dropbox.com/s/5fquvlbfjcbhbgt/mod_input.pto.gz?dl=0

Here's a collage of four pictures from this series: https://imgur.com/a/dLVzp4S
To reiterate, I've made these steps:

1. pto_gen IMG*.png
2. <hugin IMG_0000-IMG_0318.pto, created a crop mask that covers the middle of the first image in the series, saved as default.pto>
3. cpfind --sieve1width=20 --sieve1height=20 --sieve1size=300 --kdtreesteps=400 --ransaciter=2000 --ransacdist=100 --sieve2width=10 --sieve2height=10 --sieve2size=2 --linearmatch -o cpfind.pto default.pto
4. celeste_standalone -s 1440 -i cpfind.pto -o celeste.pto
5. cpclean -o cpclean.pto celeste.pto
6. <hugin cpclean.pto, reset crop mask for the first image, save as cpclean.pto>
7. pto_var --opt="y, p, r, TrX, TrY, TrZ" -o ptovar.pto cpclean.pto
8. autooptimiser -n -o mod_input.pto ptovar.pto
9. pano_modify -o mod_out.pto --projection=0 --fov=AUTO --center --canvas=AUTO --crop=AUTOHDR --output-type=REMAPORIG mod_input.pto

Auto-optimizer ended up with this:

...

Strategy 2

Average (rms) distance between Controlpoints

after 15 iteration(s):        0.0216177135303645 units

Now pano_modify has been working for several hours and I suspect it will never finish.

Best regards,
 Juha

[T. Modes]

The main cause is there is only a single control point between image 2 and 3. Then the optimizer goes haywire. It sets the TrZ parameter of all images >=3 to -1.003 and then the output does not contain the images >=3. And now you try to calculate the HDR crop - the area which is contained in all images. But there are images with size 0 and therefore pano_modify/crop can't find a solution and runs infinitely. (The normal crop calculates without problems.)

This can be easily seen in the GUI: open pto file and check (fast) preview. You will see that preview contains only 3 images (all other images have "zero" size in the output).

Some more comments:

Juha Kallioinen schrieb am Montag, 18. Oktober 2021 um 17:37:47 UTC+2:

1. pto_gen IMG*.png
2. <hugin IMG_0000-IMG_0318.pto, created a crop mask that covers the middle of the first image in the series, saved as default.pto>

The crop can also set with pto_gen --crop=... or pto_var --crop

3. cpfind --sieve1width=20 --sieve1height=20 --sieve1size=300 --kdtreesteps=400 --ransaciter=2000 --ransacdist=100 --sieve2width=10 --sieve2height=10 --sieve2size=2 --linearmatch -o cpfind.pto default.pto
4. celeste_standalone -s 1440 -i cpfind.pto -o celeste.pto

You should combine these 2 steps. Simply add --celeste to cpfind parameters

5. cpclean -o cpclean.pto celeste.pto
6. <hugin cpclean.pto, reset crop mask for the first image, save as cpclean.pto>
7. pto_var --opt="y, p, r, TrX, TrY, TrZ" -o ptovar.pto cpclean.pto

If all images are shot with the same camera/lens (iPhone) you should not optimize TrZ. There are too less information to optimize the field of view (and therefore TrZ) from a single image stack.

[Juha Kallioinen]

Thomas, thanks a lot for the analysis!

If there's just a single/few control points between images, does it pay to manually create more points in th GUI, or will those be too inaccurate?

When looking at the control points between the images in the GUI, I see that hugin shows a red/yellow/green color bar between different images. Is it an indication of the quality of the control points? I've tried to manually add control points if the color is red, but it's never turned into yellow or green that way.

[T. Modes]

Hi Juha,

Juha Kallioinen schrieb am Dienstag, 19. Oktober 2021 um 20:28:39 UTC+2:

If there's just a single/few control points between images, does it pay to manually create more points in th GUI, or will those be too inaccurate?

No, in most cases manually picked cp will be more accurate (when set correctly) - because a human will select more appropriate places - e.g. prefer non-moving objects.

(And in the time before cp finder like cpfind or autopano-sift it was the only way to add control points.)

When looking at the control points between the images in the GUI, I see that hugin shows a red/yellow/green color bar between different images. Is it an indication of the quality of the control points? I've tried to manually add control points if the color is red, but it's never turned into yellow or green that way.

The color is an indicator for the current cp error distance. After adding cp you need to re-optimize the project so that the new cp are taken into consideration for calculation of the positions. Then the color will change. 

Thomas

[Stanley Green]

I am talking about a single row pano and I am definitely not an expert. My solution was to significantly over-sample. I went from 30% overlap to something approaching 50%.  I use an index rotator and decreased the rotation angle by about 50%.

Sent from my iPhone

On Oct 19, 2021, at 2:36 PM, Juha Kallioinen <kal...@gmail.com> wrote:



--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/d0ec02a3-251f-4174-aaba-34730d95905fn%40googlegroups.com.