Stitching errors despite numerous and precise manual control points

[Phanto B]

Dear All,

I'm working on a relatively high precision 360x180 panorama, and have an issue with the stitching.
(Note: I'm working with a 8mm lens, four pictures for the "horizon", one up and one down)

1) In the ground tiling, the stitching makes irregular cracks, even tough the control points that I added manually are relatively dense, and organized - following a grid :

Is there a way to improve this ?
Or do I have to add more control points ?


2) The previous is the output when I used "horizon" pictures only. When I added the "up" picture, the same cracks, strangely, changed (for the worse). IS this normal ? Is there a way to avoid this ?

Thanks in advance.

[Greg 'groggy' Lehey]

On Friday, 24 August 2018 at 17:26:06 -0700, Hugin developers list wrote:
> Dear All,
>
> I'm working on a relatively high precision 360x180 panorama, and have an
> issue with the stitching.

> 1) In the ground tiling, the stitching makes irregular cracks, even tough
> the control points that I added manually are relatively dense, and
> organized - following a grid :
>

> [image: 1.1-control points.jpg] <about:invalid#zClosurez>
> [image: 1.2-cracks-2.jpg] <about:invalid#zClosurez>

>
> Is there a way to improve this ?

In my experience, this is one of the most difficult things to stitch.
I'd suggest removing all the control points with an error larger than,
say, 4, and see what things look like. Then place control points
round the areas where the discontinuities occur. You might also have
success with masking. And of course meticulous attention to the
entrance pupil position will make life easier.

> 2) The previous is the output when I used "horizon" pictures
> only. When I added the "up" picture, the same cracks, strangely,
> changed (for the worse). IS this normal ? Is there a way to avoid
> this ?

Did you align before or after adding the additional images? I
wouldn't expect them to make any difference, but possibly masking
could help again. Run the cursor over the fast panorama preview while
holding down the control key (this works for me on X; not sure if it's
different on your platform), and it will show you the extent of each
image, and then you can get an idea of how to mask things.

Greg
--
Sent from my desktop computer.
Finger groo...@gmail.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed. If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA

[panostar]

Your stitching problems are almost certainly due to parallax. While you would normally set up the camera such that it rotated about the entrance pupil of the lens to avoid the effects of parallax, it's a sad fact that for a fisheye lens the apparent position of the entrance pupil varies according to the varying angles of the light rays entering the lens.  Light rays from the nadir area are entering at angles approaching 90 degrees to the lens axis. For these rays, the entrance pupil is close to the front surface of the lens, whereas for rays coming from the horizontal level, the entrance pupil is much further back into the lens.  The optimum position for the camera must therefore be a compromise. You might try shifting the camera back a little so as to give priority to the nadir area which is nearest to the camera and therefore more likely to be visibly affected by parallax errors.

Another possible cause of trouble with the zenith and nadir images is that they might be in different orientations to that of the horizontal row images. You should have all the images in the same orientation: all in landscape or all in portrait. But more than this, images can be in one of 4 possible orientations obtained by rotating the image in increments of 90 degrees. The zenith and/or the nadir images might be upside down relative to the other images owing to the orientation sensor in the camera giving arbitrary data when the camera points directly up or down due to gravity acting at right angles to the image sensor. It's important that all the images are in exactly the same orientation because of the way the global lens shift parameters are applied to correct for the lens axis not being exactly aligned with the centre of the image sensor.

If you can make available the set of images (half size jpegs of the ev0 set would be good enough), more specific help might be possible.

John

[Phanto B]

@ Greg, Panostar:thanks for the detailed responses.

Concerning the zenith/nadir: the two pictures in question are "horizon" ones (shot horizontally).

Concerning entrance pupil/parallax errors, the area that poses stitching problems spans from 1 to 3 meters approximately from the camera, and there is no closer object in-between. And it is a flat surface. Can even this condition pose parallax errors ?

May the parallax errors come from further objects in the scene ? The colosest "parallax object" is the standing man, he's about 10 meters from the problematic stitching zone, and there is no stitching problems between him and the zone.

@ Greg:

Concerning the exactitude of the control points, these were meticulously placed and verified : a few to begin with, then more and more numbered, and then organized following a kind of a grid. I've arrived at this level of density (more than the pictures I posted previously), and the algorithm simply overrides the control points, making seams.

I've also tried your suggestion to suppress the control points that has 5+ distance, even though they are very exactly placed (note: it appears that we don't have to always trust the distance value. Hugin/PT made high scores to some control points that were precisely placed, but maybe he has trouble with 8mm lens extreme distortions), to no avail.

Phanto.

[panostar]

There's no problem with parallax in the two images supplied. All the images for the full 360x180 are needed for a proper investigation.

John

On Monday, August 27, 2018 at 8:11:42 PM UTC+1, Phanto B wrote:

@ Greg, Panostar:thanks for the detailed responses.

Concerning the zenith/nadir: the two pictures in question are "horizon" ones (shot horizontally).

Concerning entrance pupil/parallax errors, the area that poses stitching problems spans from 1 to 3 meters approximately from the camera, and there is no closer object in-between. And it is a flat surface. Can even this condition pose parallax errors ?

May the parallax errors come from further objects in the scene ? The colosest "parallax object" is the standing man, he's about 10 meters from the problematic stitching zone, and there is no stitching problems between him and the zone.

@ Greg:

Concerning the exactitude of the control points, these were meticulously placed and verified : a few to begin with, then more and more numbered, and then organized following a kind of a grid. I've arrived at this level of density (more than the pictures I posted previously), and the algorithm simply overrides the control points, making seams.

[Greg 'groggy' Lehey]

On Monday, 27 August 2018 at 12:11:41 -0700, Hugin developers list wrote:
> @ Greg, Panostar:thanks for the detailed responses.
>
> Concerning the zenith/nadir: the two pictures in question are "horizon"
> ones (shot horizontally).
>
> Concerning entrance pupil/parallax errors, the area that poses
> stitching problems spans from 1 to 3 meters approximately from the
> camera, and there is no closer object in-between.

Yes, this is typical of parallax issues. They become more severe the
closer you are to the camera.

> And it is a flat surface. Can even this condition pose parallax
> errors ?

Absolutely. Each of these pavers appear in a slightly different
perspective in each image.

> May the parallax errors come from further objects in the scene ?

The parallax errors come from the position of the entrance pupil.
They affect the entire image, but they're less of an issue with
distant objects.

> Concerning the exactitude of the control points, these were
> meticulously placed and verified : a few to begin with, then more
> and more numbered, and then organized following a kind of a
> grid. I've arrived at this level of density (more than the pictures
> I posted previously), and the algorithm simply overrides the control
> points, making seams.

You don't need many control points. 10 are probably enough if they're
well positioned.

> [image: Nbr_1.JPG] <about:invalid#zClosurez>
> [image: Nbr_2.JPG] <about:invalid#zClosurez>

I tried to align these, but the control point detectors weren't very
happy with them. This may be due to the reduced images, which
sometimes cause problems. You say that you used an 8mm lens, but you
don't say what sensor format, and the images you supplied don't
include Exif data. This information is important. Looking at the
images, they're clearly full-frame fisheye, which suggests a Four
Thirds or possibly APS sensor. I've tried aligning them, but the
control point detectors don't like them much. Can you post all the
original files (raw if possible, but definitely with Exif data)
somewhere? Also your project file (the one with a name ending in
.pto) would be interesting.

> ... but maybe he has trouble with 8mm lens extreme distortions), to
> no avail.

I don't see any issue with distortion in your photos, though there's a
bit of CA. Fisheye lenses don't distort, they just have a different
projection, and Hugin handles that well. I do most of my panos with
an 8 mm fisheye lens, and though I have issues with the control point
detectors, I'm sure that they're not related to the lens, and they're
also not like the problems you're reporting.

It would be interesting to hear how you determined the position of the
entrance pupil for your lens.

[Phanto B]

@ Panostar: worked like a charm for you ! How did you achieve that ? Which software ? Manual contol points ?

[panostar]

On Tuesday, August 28, 2018 at 1:32:21 PM UTC+1, Phanto B wrote:

@ Panostar: worked like a charm for you ! How did you achieve that ? Which software ? Manual contol points ?

It shouldn't matter how you create the control points. You don't need a huge number, but they should be nicely distributed.  In fact, if you have a good set of lens parameters from a previous similar stitch that you can use, you can manage with only two control points per overlap.  However, since you did ask how I created the points for my stitch, I have to confess that Hugin and I don't get on very well.  My regular stitcher is PTGui and I created the points in that and transferred them over to Hugin as they have similar Panorama Tools format project files. The optimizer figures were: av cp dist 0; max 1.2. I have attached the project file.

John

[panostar]

On Tuesday, August 28, 2018 at 2:33:38 PM UTC+1, panostar wrote:

The optimizer figures were: av cp dist 0; max 1.2. I have attached the project file.

Oops,  Typo: the average cp dist was 0.6.

John

[Luís Henrique Camargo Quiroz]

   Hi!

   I tried today, using only Hugin and manual control points: 5 vertical lines in Nbr_1, 4 in Nbr_2, and 12 regular CPs between those images, as depicted below (with a brazilian Portuguese interface). My average error was 0.77, standard deviation 0.41 and maximum 1.75. The images appear to be without parallax errors, as John "panostar" has already said; perhaps little ones. But if one of the other images has issues, this could bring problems even for these two here.

  The result was indeed good:

   So, two questions for Phanto B now:

   1 - how have you done the optimizations? I did a first run with positions only, starting from anchor, then a second run for "v", then "b" (barrel distortion) and/or all parameters without translation;

   2 - Where is this place? Could you point us in Google Maps or any equivalent?

   Happy stitching,

   Luis Henrique

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/bcd94891-6a12-4d62-bbf0-5e681a037d61%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

--
Luis Henrique Camargo Quiroz
http://luishcq.br.tripod.com - http://www.christusrex.org/www2/cantgreg
http://panoramaslh.net/

[Phanto B]

Hello Luis,

Thanks for your support.

It's the Tokyo Metropolitan Government Building (simply googlemap it).

I've tried making manual control points roughly the same as yours, and the result is good:

It's not yet okay for the entire panorama, but I'm paying more attention to cp distance values, tweaking them a bit, and it seems improving.

What value did you choose for the focal length ?

@ panostar: you mentionned "good set of lens parameters from a previous similar stitch". Can you please explain ?

Thanks.

[panostar]

On Wednesday, August 29, 2018 at 10:02:37 PM UTC+1, Phanto B wrote:

@ panostar: you mentionned "good set of lens parameters from a previous similar stitch". Can you please explain ?

The lens distortion correction parameters (Hfov,a,b,c,d,e) are characteristic of the lens (at a particular focus/zoom setting) and shouldn't change significantly from run to run.  So if these have been evaluated by the optimizer in a similar (successful) previous run, you can use them in a similar subsequent run and so relieve the optimizer of this job.  You then need considerably fewer control points. All you need do is enter the parameters on the lens panel and then elect not to include these parameters in the optimization.  For a full 360 panorama, you can optimize hfov to fine tune it so that the images exactly fit around the spherical stitching surface.

John

[Luís Henrique Camargo Quiroz]

    Hello Phanto,

    I don't remember the v value I started with. Perhaps I used 90 degrees, or even more, like 120, as your lens is a fisheye lens the v value should the considerably high. The important thing is, even when we start with a wrong value, as the are vertical lines, or, if those lines are absent but there is a full 360 view around, Hugin can find the real HFOV (or v) value.

    This way the resulting v is 143.4 degrees.

   Just to be clear: my final optimization was done for "all geometric parameters without translation".

   I have found the place in Maps, thank you! I could not suspect that there is a 4 or 6 lane road crossing in front of those photos! 

  regards

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/7442b8fb-8665-48cc-9896-b4785af43e18%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Luís Henrique Camargo Quiroz]

  Sorry, you asked about focal length. It is somehow related to v, and Hugin as found 14.389 mm.

  In a similar way, when I shot with my wide angle Hugin starts with a nominal 10 mm focal length, reading this value from the EXIF data, and after optimizations the end value is 10.343 mm for one of my panoramas (shot hand held)

[Phanto B]

Hello everyone,

This is the progress so far. Any help is welcome.

Reminder/summary:
 360x180 pano
 8mm fullframe fisheye lens (samyang), canon eos 700d, tripod (1.5 meter from the ground), without a panohead
 4 horizon photos (landscape format), 1 zenith, 1 nadir (the nadir is shot handheld)
 Hugin

Symptoms:
 Seams on the ground (the closest element to the camera)


Track explored:
 Ground parallax errors caused by the absence of panohead, and the fact that the ground is not perfectly flat.



But maybe this is not the cause: the following optimisation made a perfect nadir hemisphere:

1) In the preview window, the nadir photo is moved to the center of the image (equirectangular projection, pitch to +90°)
2) In the custom optimiser tab, checked the following values
    for the horizon's first photo (reference): nothing
    for the nadir photo, everything: ypr, xyz, plane yaw, plane pitch, camera translation
    for the rest (horizon's three remaining photos + zenith): ypr xyz

This has the side effect of eliminating from the panorama the zenith hemisphere (maybe this is what this document is warning against http://hugin.sourceforge.net/docs/manual/Hugin_Optimiser_tab.html ), except for the reference photo. But the important thing is that the nadir hemisphere is absolutely seamless, so maybe there is no (or negligible) parallax errors for the ground.

[Phanto B]

And this is what the optimiser tab looks like:

[Bruno Postle]

Yes, the XYZ mosaic optimisation will resolve quite extreme parallax errors, but only for features on a single flat plane in the scene.

In this case your ground is close enough to being planar, so Hugin can fix parallax for all photos, not just the nadir.

By default this plane needs to be opposite the panorama view, which is why you had to tilt the scene by 90°. This is a good solution, just render the panorama like this and then load the result into Hugin and tilt it back. A more advanced usage is to use the 'plane yaw' and 'plane pitch' parameters to move this plane to the nadir, but I suggest you only try this if you are feeling confident.

--
Bruno

On 5 September 2018 19:47:44 BST, 'Phanto B' wrote:
>Hello everyone,
>
>This is the progress so far. Any help is welcome.
>
>Reminder/summary:
> 360x180 pano
>8mm fullframe fisheye lens (samyang), canon eos 700d, tripod (1.5 meter
>
>from the ground), without a panohead
>4 horizon photos (landscape format), 1 zenith, 1 nadir (the nadir is
>shot
>handheld)
> Hugin
>

>[image: six photos.jpg] <about:invalid#zClosurez>

>
>Symptoms:
> Seams on the ground (the closest element to the camera)
>
>Track explored:
>Ground parallax errors caused by the absence of panohead, and the fact
>that the ground is not perfectly flat.
>
>But maybe this is not the cause: the following optimisation made a
>perfect
>nadir hemisphere:
>
>1) In the preview window, the nadir photo is moved to the center of the
>
>image (equirectangular projection, pitch to +90°)
>2) In the custom optimiser tab, checked the following values
> for the horizon's first photo (reference): nothing
> for the nadir photo, everything: ypr, xyz, plane yaw, plane pitch,
>camera translation
> for the rest (horizon's three remaining photos + zenith): ypr xyz
>
>This has the side effect of eliminating from the panorama the zenith
>hemisphere (maybe this is what this document is warning against
>http://hugin.sourceforge.net/docs/manual/Hugin_Optimiser_tab.html ),
>except
>for the reference photo. But the important thing is that the nadir
>hemisphere is absolutely seamless, so maybe there is no (or negligible)
>
>parallax errors for the ground.
>

>[image: seamless-nadir-hemisphere.jpg] <about:invalid#zClosurez>

[Phanto B]

Le jeudi 6 septembre 2018 10:51:25 UTC+1, Bruno Postle a écrit :

By default this plane needs to be opposite the panorama view, which is why you had to tilt the scene by 90°. This is a good solution, just render the panorama like this and then load the result into Hugin and tilt it back.

It's what I've done (I think). But is it possible to do it (or tweaking it) without exploding the zenith hemisphere ?

Here are the steps:

Tilting +90°:

Applying the optimisation (the last photo on the list is the nadir):

Result: the zenith hemisphere (minus the portion of the reference shot) already disappears.

Le jeudi 6 septembre 2018 10:51:25 UTC+1, Bruno Postle a écrit :

A more advanced usage is to use the 'plane yaw' and 'plane pitch' parameters to move this plane to the nadir, but I suggest you only try this if you are feeling confident.

 

Can you please explain ?

[Bruno Postle]

On 6 September 2018 12:26:12 BST, 'Phanto B' wrote:
>Le jeudi 6 septembre 2018 10:51:25 UTC+1, Bruno Postle a écrit :
>>
>> A more advanced usage is to use the
>> 'plane yaw' and 'plane pitch'
>> parameters to move this plane to the
>> nadir, but I suggest you only
>> try this
>> if you are feeling confident.
>
>Can you please explain ?

The XYZ mosaic mode maps your photos onto a plane, imagine this plane is a painting hanging vertically at front in the middle of the scene.

This arrangement is ideal for stitching images of murals etc... because these are normally on vertical surfaces.

You stitched the floor of your scene by pretending that it was a wall in front of you.

The 'plane yaw' and 'plane pitch' parameters allow you to rotate this plane around, to the left and right and up or down - this is a whole new level of complexity that you probably don't need until you have everything else working.

--
Bruno