Can PTGui auto-match photos with barrel distortion?

[interstellar]

I'm wondering if PTGui can help me with this case. I have 2 large satellite photos of the same terrain. I don't have access to lens info for either photo. In the center of the photos, the ground can be matched manually. Near the edges, the photos differ in barrel-distortion. Can PTGui automatically match these 2 photos to one another, as best as possible? I don't need an "ortho" correction (proper rectangular unwrapping), I just want the 2 photos to be distorted near the edges so that they match one another. If needed, I can cut up the photos into e.g. 5x5 array of 25 smaller photos, so that the software has an easier task matching features.

[Erik Krause]

Am 23.08.2022 um 20:47 schrieb interstellar:

> I don't have access to lens info for either
> photo. In the center of the photos, the ground can be matched manually.
> Near the edges, the photos differ in barrel-distortion. Can PTGui
> automatically match these 2 photos to one another, as best as possible? I
> don't need an "ortho" correction (proper rectangular unwrapping), I just
> want the 2 photos to be distorted near the edges so that they match one
> another.

Yes, that should be possible, if the distortion is real barrel
distortion from a lens (that is, radial symmetric). You would need to
assign different lenses to the photos, to account for different
distortion. PTGui doesn't need the exact focal length, an estimate for
one image would do, the other will be optimized.

> If needed, I can cut up the photos into e.g. 5x5 array of 25
> smaller photos, so that the software has an easier task matching features.

No, that would be counterproductive.

--
Erik Krause
http://www.erik-krause.de

[interstellar]

Thanks. I'm new to PTGui, my impression is that the way to do this would be to first let PTGui find some matching points (it found 20), then manually add further control points for accuracy, then re-run the optimizer. Is there a tutorial on adding manual control points? Also, once I'm done, how would I save the distorted image? I don't want a panorama, I just want to save one image distorted to match the other.

[Erik Krause]

Am 23.08.2022 um 21:27 schrieb interstellar:

> Is there a tutorial on adding manual control points?

There is part 3 of the video tutorials:
https://ptgui.com/videotutorials.html

If you want only one image modified, you must switch to advanced
interface, Lens Settings and add a second lens profile. In the profile
for the unmodified image reset all lens correction parameters and enter
an estimated focal length (doesn't need to be near). Use the profiles
for the respective images (lower table)

Then on Optimizer tab (advanced interface) disable all optimizations for
the unmodified image's lens profile and disable all checkboxes for this
image. Enable FoV, a, b, c and Shift as well as Yaw, Roll and Pitch for
the other image. Press F5 to optimize.

> Also, once I'm done, how would I save the distorted image?

You can either disable the unmodified image (uncheck under "Include
images") on Create Panorama tab or output individual layers. If you save
to .psd or .psb you get both layers in one file.

[interstellar]

Thanks. I added 80 manual CPs. Under best settings, they're reported as "good", max distance 12 px on 3K images. But the panorama comes out black. For some reason, YPR (yaw, pitch, roll) for the 2 images are strangely different. For two North-oriented satellite images, showing the same area, I think YPRs should be very close. I tried 3 things: 1) For the master image, I set YPR manually: yaw 0 (N on top), pitch -90 (we're looking down), roll is 0 (no twist around the axis). After optimization (no YPR optimization for the master image) YPR for the second image is 150, -31, -150. These values sound wrong. 2) if I set YPR to 0,-90,0 for both images and I do not optimize YPR on either, then I get very bad CP distances ~1K. 3) if I allow YPR optimization on both images, I get very different angles: -26,-62,31 and 156,-59, -152. Like I said for two properly oriented satellite images the values should be much closer. If I'm making a mistake somewhere, please let me know.

[John Houghton]

It's difficult to advise without seeing exactly the state of the project.  It would help greatly if you would run File -> Save Project and supply supply copies of the images (jpg) and your saved project file (xxx.pts), preferably zipped together and made available for download from somewhere convenient (eg wetransfer.com, Dropbox etc).

John

[Erik Krause]

Am 23.08.22 um 23:20 schrieb interstellar:

> For the master image, I set YPR manually: yaw 0 (N on top), pitch -90
> (we're looking down)

That's certainly wrong. You are not looking down, you are looking
straight at the master image. Setting Pitch to -90° would place it at
the nadir in the spherical stitching canvas, which means it is outside
the result image, if that doesn't span 180° vertically. Set the master
image to YPR 0, and all distortion parameters 0.

--
Erik Krause

[interstellar]

Thanks. I'm new to PTGui so I don't know which axis is considered "down", there are different axis systems. I was thinking of an airplane, where down is a pitch of -90. In any case, I managed to export a meaningful panorama using "equirectangular" lenses with a field of view of 1 deg. But, looking at the layers in Photoshop, they're outer regions are still misaligned. I think the distortion is more complex than just radial or linar transform (scale, rotation, skew). I think the satellite-type of the picture can be the issue. As they're taken from orbit, it's probably an extreme telephoto lens (500mm), plus the image may have been processed to create a fake "ortho" projection. E.g. the satellite may have been really looking down at an angle of 60 deg w/r to Earth, but the image was then warped to remove any perspective distortion to make it look as if it was taken top-down. Tall objects still some odd distortions baked into them, thoguh I don't care about it. 

Anyway, I realized that a better tool for my purpose would be just a plain image warper (like the 1990s morphing programs), since all I care about is distorting 2 photos to match, not accuracy w/r to reality or creating a panorma. I found a decent image warper that can use existing matching points set. I'd like to re-use the points I created in PTGui, because I don't want to waste the time I put into creating 80 matching points, plus PTGui's way of creating matching points is superior to all other software. Is there a way to export the coords of the matching points (in 2 images), so that I can use it in the outside warper image? I looked at the PTGui files, but they're binary, not ASCII. In the interface itself, I don't see coordinates of the matching points, just distances. Thanks.

[Erik Krause]

Am 24.08.22 um 13:45 schrieb interstellar:

> As they're taken from orbit, it's probably an extreme telephoto
> lens (500mm), plus the image may have been processed to create a fake
> "ortho" projection. E.g. the satellite may have been really looking down at
> an angle of 60 deg w/r to Earth, but the image was then warped to remove
> any perspective distortion to make it look as if it was taken top-down.
> Tall objects still some odd distortions baked into them, thoguh I don't
> care about it.

If you make the images available for download (use wetransfer, dropbox
or similar) and post the link here, I can have a look. PTGui also
supports viewpoint correction, which might be need in such a case.

> Is there a way to export the coords of the matching points (in 2 images),
> so that I can use it in the outside warper image? I looked at the PTGui
> files, but they're binary, not ASCII. In the interface itself, I don't see
> coordinates of the matching points, just distances. Thanks.

Apparently you use the trial Version of PTGui, where the project files
are encrypted. The Full version writes JSON based files, where the
control points are stored with image number and pixel coordinates. So if
you are not afraid of programming you can very well extract the control
points.

--
Erik Krause

[interstellar]

Thanks. I had to pay for the satellite images, I can't post them publicly. It would have to be done privately. To get access to the points in JSON, does it matter if I get the regular or Pro version?

[John Houghton]

PTGui Pro can cope with perspective distortion using the viewpoint correction feature. (Tutorial available on the web site).  Care has to be taken to avoid creating control points on three dimensional objects poking up from the general plane surface.  You mention (laboriously?) creating 80 control points points.  You can target the automatic creation of points in a rectangular selected area using the Generate Control Points Here feature.  Drag a selection box with the shift key held down and then right click within the box to bring up that option.  However, the images need to be already roughly aligned for this option to work.  Again, we can be more help if you can supply a couple of images for investigation.

John

[Erik Krause]

Am 24.08.22 um 14:14 schrieb interstellar:

> Thanks. I had to pay for the satellite images, I can't post them publicly.

Then post the project file here. Just the .pts without the images. And
tell us

> It would have to be done privately.

Make them available to erik....@gmx.de

> To get access to the points in JSON,
> does it matter if I get the regular or Pro version?

No. But if you need viewpoint correction, that would make a difference:
https://ptgui.com/features.html

--
Erik Krause

[interstellar]

Here's the project without images, this was my most successful attempt, where I set the lenses to "equirectangular", 1 deg FOV. I it output a panorama, which looks decent, just rotated 30 deg around the viewing axis. I'll email a link to the 2 images. As I said, I don't care about the distortions in the tall objects, just the general layout of the terrain. BTW, it occurred to me there may be another type of distortion. The ground is sort-of-flat but still has some slight hills and depressions. If two satellites took the images from 2 somewhat different angles, and then the images were perspective-corrected for an "ortho" satellite look, then there are slight non-linear distortions baked-in: the 2 satellites saw the slight hills and depressions from 2 different angles, so some corresponding features may be shifted around (locally) a bit.

[interstellar]

Sent the images via WeTransfer, please let me know if you don't get them

[Erik Krause]

Am 24.08.22 um 16:01 schrieb interstellar:

> BTW, it occurred to me there may be another type of distortion. The ground
> is sort-of-flat but still has some slight hills and depressions. If two
> satellites took the images from 2 somewhat different angles, and then the
> images were perspective-corrected for an "ortho" satellite look, then there
> are slight non-linear distortions baked-in: the 2 satellites saw the slight
> hills and depressions from 2 different angles, so some corresponding
> features may be shifted around (locally) a bit.

Yes, that would be parallax which is always a problem when aligning
images. Perhaps it would be beneficial to set control points only on
features with same altitude.

I attached the version like I would have done it. Hard to decide without
the images.

--
Erik Krause

[Erik Krause]

Am 24.08.22 um 16:42 schrieb interstellar:

> Sent the images via WeTransfer, please let me know

Got them. Will have a look this evening...

--
Erik Krause

[John Houghton]

Erik, I got exactly the same result.

John

[interstellar]

Thanks. I exported the panorama from your file, first without optimizing, then with optimizing, but the 2 images still don't match. Like I said, the centers can be aligned, but the the 2nd image (color) still has odd distortions further from the center. I'll wait until you've had time to analyze the problem.

[interstellar]

Yes, I tried to place the CPs close to the ground, not on any tall objects. The distortion is more pronounced in the upper left and lower right, probably that's where the terrain differs the most from the average.

[Erik Krause]

Am 24.08.22 um 16:42 schrieb interstellar:

> Sent the images via WeTransfer

Well I had a quick try, and the problem is definitely parallax. While
image B was shot from almost directly above, image A was from further
top left. So a perfect fit can not be obtained. Nevertheless I added a
lot more control points (using "Generate control points here" as John
suggested) and perhaps could somehow improve the result.

Perhaps it would be worth to have a look at some photogrammetry
software, since parallax can be used to create a depth map. Also if you
rotate the images such that the displacement is left-right you can use
them as a stereo pair, which can be viewed cross-eye or anaglyph or even
as a wiggle-3D-image like on facebook. I tried that and it works sort
of, although I couldn't fuse the high buildings. Now my eyes hurt ;-)

--
Erik Krause

[interstellar]

Your file did improved the alignment. I realized that the color image should be used as a "master", since it was shot top-down, so, presumably, it has less distortion. Like I said, I'll see if I can use warping software to distort the B&W image to match the remaining differences (just on the ground level). Thanks for your help.

[interstellar]

An idea occurred to me: maybe the 2nd image can be warped to closely match the master by iterating the process of generating control points / optimizing a few times (iter #2 uses the result of iter #1 etc) This will reduce quality in the warped image, but after, say, 3 iterations, it may still look decent. I'll try it later.

[Erik Krause]

Am 24.08.2022 um 17:35 schrieb interstellar:

> I realized that the color image should be used as a "master", since
> it was shot top-down, so, presumably, it has less distortion.

That wouldn't make any difference. Distortion in this case is nothing
absolute but relative to the other image. Which one is master shouldn't
matter.

> An idea occurred to me: maybe the 2nd image can be warped to closely match
> the master by iterating the process of generating control points /
> optimizing a few times (iter #2 uses the result of iter #1 etc) This will
> reduce quality in the warped image, but after, say, 3 iterations, it may
> still look decent. I'll try it later.

I don't think this would work either. There are only a limited number of
types of warping PTGui can do, and none of them will move arbitrary
areas in the same image in opposite directions, while leaving other
areas stationary.

The old panotools had a mode called morph-to-fit, but this was never
implemented in either of the panotools GUIs, with probably PTAssembler
as an exception. However, you would need thousands of control points, to
morph all this tiny features in place. And PTGui doesn't export old
panotools scripts any more.

[John Houghton]

On Wednesday, August 24, 2022 at 8:58:24 PM UTC+1 Erik Krause wrote:

The old panotools had a mode called morph-to-fit, but this was never
implemented in either of the panotools GUIs, with probably PTAssembler
as an exception.

It was possible to use morph-to-fit with PTGui but it required some manual intervention.   I wrote  a "how to" tutorial in 2002:

http://www.johnhpanos.com/morphtut.htm

John