What are the keypoints to create 360 panorama ?

[Wade Wang]

I am trying to use hugin to create 360 panoramic image whose leftmost side and rightmost side are continuous seamlessly.(Although my final goal is to create VR360 that no only has 360 HFOV,  but also has 180 VFOV, I need to first successfuly achieve 360 HFOV).

I tried several image-sets(share link) whose images cover more than 360 degree , but none of them I was able to create 360 panorama, so what are the keypoints to create  360 panorama ?

Following example comes from "Jie-Geng\14_only_ring1" in the image-sets, the saved .pto is the attachment.

Is it due to the image overlapping rate, control points or whatever ? Actually I have tried manually set and check control points, and they looks reasonable, but I don't know why hugin still can not stitch them correctly.

BTW, does anyone could share the imageset that can be used to create 360 panorama easily with hugin ? 

My huign version is 2023.0.0.d88dc56ded0e on Windows

[T. Modes]

iwtbask...@qq.com schrieb am Sonntag, 19. Mai 2024 um 00:31:33 UTC+2:

Following example comes from "Jie-Geng\14_only_ring1" in the image-sets, the saved .pto is the attachment.

Is it due to the image overlapping rate, control points or whatever ? Actually I have tried manually set and check control points, and they looks reasonable, but I don't know why hugin still can not stitch them correctly.

The problem is that you optimize the translation parameters for all images. This does not work. The translation parameters should only optimize for single images, but not for all (except you know exactly what you do)

I did the follow with this project:

1. Reset all image positions

2. Optimize Geometric: Positions (incremental, starting from anchor)

3. Optimize Geometric: Positions and view (y,pr,rv,)

The results looks reasonable then. The field of view of the images is reduced from 60 deg to 46 deg.

Not sure if this is related to your attempts or if the initial guess was wrong. I could not check this because your test images does not contain any EXIF data.

BTW, does anyone could share the imageset that can be used to create 360 panorama easily with hugin ? 

E.g. your example set SSARCandy/libray works fine with Hugin. (I started with a HFOV of 30 deg because the EXIF data are missing.)

[Wade Wang]

Great news ! Following T. Modes' instructions, I've successfully stitched these image-sets into panoramas using Hugin, and it truly feels magical ! It seems the keypoint is to optimize position first, and then optimize both position and view.

During the process of stitching these panoramas, I've encountered new thoughts and questions:

1. If my input images are in clockwise order and I only want to detect and match keypoints between adjacent images, and the last image also need to match keypoints with the first, how can I achieve this? I noticed a matching strategy called linear match, but it seems can not match the last image's keypoints with the first one.

2. What does 'prealigned' mean in 'CPFind prealigned' ? When should 'prealigned' be used, and is it suitable for the case of creating 360° panorama ? I tried it out, and its matching strategy seems similar to CPFind :

3.How can I automate the workflow so that I can one-click generate 360° panorama with new image sets in the future? Can this be achieved through a python script or command line? But I've found that my windows computer lacks command line tools of Hugin (e.g. cpfind, PToptimizer, nona etc) and hsi. Do these require extra installation steps?

Thanks in advance !

[T. Modes]

iwtbask...@qq.com schrieb am Montag, 20. Mai 2024 um 14:35:29 UTC+2:

During the process of stitching these panoramas, I've encountered new thoughts and questions:

1. If my input images are in clockwise order and I only want to detect and match keypoints between adjacent images, and the last image also need to match keypoints with the first, how can I achieve this? I noticed a matching strategy called linear match, but it seems can not match the last image's keypoints with the first one.

cpfind with --multirow switch does this. This should be the default strategy in the GUI.

 

1. What does 'prealigned' mean in 'CPFind prealigned' ? When should 'prealigned' be used, and is it suitable for the case of creating 360° panorama ? I tried it out, and its matching strategy seems similar to CPFind :

This works by comparing only overlapping images. But to make this to work you need to align rough positions to all images before (e.g. with a template or with pto_set or with a Papywizard XML file).

 

3.How can I automate the workflow so that I can one-click generate 360° panorama with new image sets in the future? Can this be achieved through a python script or command line? But I've found that my windows computer lacks command line tools of Hugin (e.g. cpfind, PToptimizer, nona etc) and hsi. Do these require extra installation steps?

There is the assistant which should work in most cases as one click solution. 

Before you try to automatize the workflow you need to understand what happens in each step.

The Hugin installer does contain all command line tools. There are no extra installation steps necessary.

[Wade Wang]

1.I find we can not use --multirow and --linearmatch simultaneously because they are mutually exclusive, then can I achieve my goal (matching only feature points between adjacent images and the last image matching the first image) ?

2.If my workflow is: loading images, with all images having the same initial position, lens type set to normal (rectilinear), HFOV set to 60, feature matching using CPFind + prealigned + celeste, geometric optimization using Position (incremental, start from anchor) first, then using Position and View (y, p, r, v), photometric optimization using Low dynamic range, variable white balance, then fit FOV

automatically, projection set to cylindrical, Center, Fit, Straighten, Autocrop, and exporting JPEG images with quality set to 100.

How can I convert this workflow into a series of commands and integrate it with Hugin?

As far as I know, it seems I can create a new .assistant file in the C:\Program Files\Hugin\share\hugin\data\assistant folder.

I already draft part of the commands:

```

[General]
Description=360 panorama assistant
Help=Assistant for create 360 degree panorama
StepCount=4

[Step0]
Description=Searching control points
Condition=not connected
Program=cpfind
Arguments=--prealigned --celeste -v --output=%project% %project%

[Step1]
Description=Check result
Condition=not connected
Program=checkpto
Arguments=%project%

[Step2]
Description=Optimizing
Program=autooptimiser
Arguments=-a -p -m -l -s --output=%project% %project%

[Step3]
Description=Searching for best crop
Program=pano_modify
Arguments=--canvas=AUTO --straighten --center --crop=AUTO --output=%project% %project%

```

But I don't know how to convert the rest workflow into commands and use it in hugin.

3.I found a difference in the way images are loaded between the two interfaces. If I use the 'Add images' in the expert interface, the initial positions of the images are all in the same place, with the first image covering all the subsequent images. However, if I use the 'Load images' in the simple interface, the images are arranged in two rows. I experimented with both arrangements (i.e., running CPFind and Position incremental optimize afterwards), and found that the first arrangement can produce a correct panorama, while the second one produces a distorted panorama. Is it possible for the 'Load images' button in the simple interface to also place all images' initial positions in the same place?

Thanks in advance !

[Bruno Postle]

On Tue, 21 May 2024, 10:10 Wade Wang wrote:

1.I find we can not use --multirow and --linearmatch simultaneously because they are mutually exclusive, then can I achieve my goal (matching only feature points between adjacent images and the last image matching the first image) ?

'multirow' is what you want, it tries to match all the ends of whatever chains are created after running a 'linearmatch'.

-- 

Bruno

[Wade Wang]

>  'multirow' is what you want, it tries to match all the ends of whatever chains are created after running a 'linearmatch'.

Inspired by this, I find I can achieve my goal (matching only feature points between adjacent images and the last image matching the first image) by two steps, the first step is run all images with linearmatch, the second step is select only the last image and the first image, then run linearmatch again or run multirow. But I don't know how to convert the second step into command line.

[Bruno Postle]

On Tue, 21 May 2024, 13:49 Wade Wang wrote:

>  'multirow' is what you want, it tries to match all the ends of whatever chains are created after running a 'linearmatch'.

Inspired by this, I find I can achieve my goal (matching only feature points between adjacent images and the last image matching the first image) by two steps, the first step is run all images with linearmatch, the second step is select only the last image and the first image, then run linearmatch again or run multirow. But I don't know how to convert the second step into command line.

'multirow' does all these things automatically, there is no need to run 'linearmatch' first.

-- 

Bruno

[Wade Wang]

Could anyone tell me how to convert following workflow into a series of commands?

The workflow is: loading images lens type set to normal (rectilinear), HFOV set to 46, feature matching using CPFind + linearmath + celeste, note that first step run cpfind on all iamges, second step run cpfind on only last image and first image, geometric optimization using Position (incremental, start from anchor) first, then using Position and View (y, p, r, v), photometric optimization using Low dynamic range, variable white balance, then fit FOV 

automatically, projection set to cylindrical, Center, Fit, Straighten, Autocrop, and exporting JPEG images with quality set to 100.

Currently I have tested following commands:

```

pto_gen -p 0 -f 46 -o project.pto *.jpg # -p 0 : rectlinear 

projection type

cpfind -o project.pto --linearmatch --celeste project.pto
cpclean -o project.pto project.pto
linefind -o project.pto project.pto
autooptimiser -a -p -l -s -o project.pto project.pto
pano_modify --projection=1 --center --straighten --canvas=AUTO --crop=AUTO -o project.pto project.pto # projection=1:cylindrical
nona -v -z LZW -r ldr -m TIFF_m -o project project.pto
enblend -o final-enblend.jpg project0000.tif project0001.tif project0002.tif project0003.tif project0004.tif project0005.tif project0006.tif project0007.tif project0008.tif project0009.tif project0010.tif project0011.tif project0012.tif project0013.tif project0014.tif project0015.tif

```

Take Xue-Mountain-Enterance image set as example, this is the result generated by mannual operation on hugin GUI:

this is the result generated by above command lines:

[Bruno Postle]

I would remove the '--linearmatch' parameter, as this prevents the last image being matched with the first image.

(You could try '--multirow' instead, but this shouldn't be necessary)

Possibly also removing the 'cpclean' command would help in this case, but I can't test from here.

-- 

Bruno

On Mon, 27 May 2024, 11:39 'Wade Wang' wrote:

Could anyone tell me how to convert following workflow into a series of commands?

[Wade Wang]

Hi, Bruno, I test replacing --linearmatch with --multirow on 5 image sets, for most image sets, the results look good. However, there are some artifacts in certain areas of some images. After checking, I found that the control points in these areas are incorrect, such as the control point between images 3 and 5 below:

If currently hugin does not support the feature I want, this is  okay   ~