Re: Help optimizing images

[Andreas Metzler]

Caleb <robot...@gmail.com> wrote:
> I have 6 1280x960 images taken from 6 corresponding videos. They're all
> from within an extremely short time span of each other. The cameras they
> were all configured for a 170 hfov. The cameras were attached to the inside
> of a 19cm cube and it was suspended by one corner. However, they're all
> tilted so horizontal is almost diagonal. See the attached image for a
> better understanding.
[...]

Having played a little bit with this without success, I am really interested
in *your* .pto. ;-)
cu andreas

--
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

[Dr. Kurt]

On Friday, July 20, 2012 2:08:22 AM UTC-4, Caleb wrote:

I have 6 1280x960 images taken from 6 corresponding videos. They're all from within an extremely short time span of each other. The cameras they were all configured for a 170 hfov. The cameras were attached to the inside of a 19cm cube and it was suspended by one corner. However, they're all tilted so horizontal is almost diagonal. See the attached image for a better understanding.

The zip file can be found at http://res0l.net/images/PanoTest.zip

Also included in the zip file is one blended-fused output which represents the best exposure adjustments I was able to coax out of hugin.

I'm wondering if someone can do a better job of stitching and exposure adjusting than I was able to get. If so, I'd love to see your .pto/hear your process. I'm about to setup automation of thousands of such image sets and would love to have them looking the best possible. For my intended purpose, close up parallax issues won't exist except in the very rare case.

Cheers

Caleb

I've poked around with this one a bit too.  I couldn't get cpfind to come up with any control points at all; for that matter I'm having a hard time finding enough overlap between the images to even manually assign control points.  So I'd also like to see your .pto file and hear how you managed to create it.

I don't suppose you might consider adding more cameras and going to an octahedral - or even dodecahedral - geometry?  More overlap - giving more control points, and not just in the corners of each image - will make it much easier to calculate the appropriate lens distortion parameters, which in turn will make it much easier to get a good result...

Kurt "Computers were invented to help people waste more time faster" Hillig

[Jan Martin]

Please post a picture of your rig too.

[Caleb]

On Friday, July 20, 2012 11:54:12 AM UTC-5, Caleb wrote:

I setup the control points by hand. I've _never_ had automated control point finding work. I can get a .pto up this evening.

 

I don't have the exact .pto I built that image from, but here's one that's pretty close. I didn't back that version up before modifying. You'll want to fix the paths to the images in the .pto (attached)

Also, you can take a look at some stitched video from the frames I'm testing with at http://www.youtube.com/watch?v=ep-d0vIm3sA 

[Caleb]

On Friday, July 20, 2012 3:19:14 PM UTC-5, Jan Martin wrote:

Please post a picture of your rig too.

Hi Jan.

Nice to hear from you. I sent several emails to you back in February of 2011 regarding my project. I never heard from you, though.

Please see my first post for a pic of the rig (such as it is) in its current state.

Cheers

[Jim Watters]

Caleb,

Your multi-camera setup does not look too rigid. Small changes in the
positioning of a camera would cause havoc on the final result.

With a multi-camera rig there will always be some parallax error. When shooting
video with the rig moving around the sometimes the subject will be close
sometimes at infinity. If the camera is always held in the same way the ground
will most likely be closer than the rest of the pano.

The best batch script to stitch one scene might be quite different from another
if the main distance to subject changes or if the rig is held differently. If is
also possible to create a general batch script that can be used for most scenarios.

To minimize change in the pano video I would try to keep the orientation and
distance from the ground of the rig constant.

My process to create a batch stitching script is to run a control point finding
utility on many groups of images. Maybe 50 groups spread across the entire
video. So in your example every 40th set of images. Combine all the control
points from all the sets into one project. then optimize this. This gives a good
average of what the entire video needs to stitch.

The horizon is stitched with a subject distance that is maybe 10 meters away.
The ground at 1 or 2 meters away. They sky at the subject distance of 5 meters
away. But it really depends of what you are shooting.

Your basket ball hoop is too close to ever give a good stitch along with the
background. If possible in situations like this ensure a single camera can
capture close up images like this. Do not put them on a seam.

For blending the seams together I would not use a "smart" blender. They will
choose the best path for the seam for each frame and it will cause a wiggle in
the final video, especially if the camera is stationary. Use a simple feather
instead. or very narrow blending using masks to force the actual seam location.
If the camera is moving then this is less of an issue.

Jim Watters

On 2012-07-20 3:08 AM, Caleb wrote:
> I have 6 1280x960 images taken from 6 corresponding videos. They're all from
> within an extremely short time span of each other. The cameras they were all
> configured for a 170 hfov. The cameras were attached to the inside of a 19cm
> cube and it was suspended by one corner. However, they're all tilted so
> horizontal is almost diagonal. See the attached image for a better understanding.
>
> The zip file can be found at http://res0l.net/images/PanoTest.zip
>
> Also included in the zip file is one blended-fused output which represents the
> best exposure adjustments I was able to coax out of hugin.
>
> I'm wondering if someone can do a better job of stitching and exposure
> adjusting than I was able to get. If so, I'd love to see your .pto/hear your
> process. I'm about to setup automation of thousands of such image sets and
> would love to have them looking the best possible. For my intended purpose,
> close up parallax issues won't exist except in the very rare case.
>
> Cheers
>
> Caleb

> --

--
Jim Watters
http://photocreations.ca

[Bruno Postle]

On Sat 21-Jul-2012 at 10:29 -0300, Jim Watters wrote:
>
>With a multi-camera rig there will always be some parallax error.
>When shooting video with the rig moving around the sometimes the
>subject will be close sometimes at infinity.

I think Caleb's intention is to use this rig in a place where
parallax isn't a problem.

>For blending the seams together I would not use a "smart" blender.
>They will choose the best path for the seam for each frame and it
>will cause a wiggle in the final video, especially if the camera is
>stationary. Use a simple feather instead. or very narrow blending
>using masks to force the actual seam location.

There is an enblend --no-optimize option which should result in
identically placed seams regardless of photo content.

--
Bruno

[Jan Martin]

I got this result last year:
http://www.youtube.com/watch?v=u6DMSjtrTP0

You need to get the seams right first.
Be aware that a template will work for a single distance only.
So you need to know what kind of situation you will shoot later, and make a template for that exact situation.
E.g. ALL control points at 10 meters from the camera.
Yours are at 5 meters on the sand, and at the horizon in the opposite direction.
Also there are absolutely no cps on all the the seams of sky and asphalt.
That will never work.

From my experience using a tripod and setting up in a small clearing in the woods with 10 meter diameter to high trees really works well.
You need lots of features at ALL the seams.
Then place the cps manually because you do not have the overlap nor the resolution for any automatic cp finder to ever work.

Like Bruno suggested use "enblend --no-optimize" to avoid flickering of the seams.

Also you need decent images for making the template. Shoot at around noon for that.
Not very late in the afternoon with the sun close to the horizon, screwing up the images.

What I did when trying your images was to close hugin, make a copy of 2-2.png, change brightness and saturation in Gimp until I could actually see something, then use that image for manual cps. Then close hugin, rename images back.
Open hugin and optimize for exposure with the original image.

This is the pano I got:
http://bit.ly/NH6Fdb

Temp image for control points only:
http://bit.ly/QbYjNL

For shooting later:
Avoid shooting at dusk and dawn, the cameras you use are known to be bad with low-light situations.
Also do not optimize for exposure at all or the resulting video will flicker. Like yours does.

While we are at it:
It would be nice to have a way to adjust brightness and saturation for the images at the hugin control points tab.
Just to help place cps, nothing else.

And It would be even better to have a way to rotate the images on the control points tab.
Maybe even have an extra button to take over the rotation from the preview window.

Any developer reading this?

Jan

--
You received this message because you are subscribed to the Google Groups "Hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugi...@googlegroups.com
To unsubscribe from this group, send email to hugin-ptx+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/hugin-ptx

[Caleb]

So many good pieces of info from everybody. Thanks.

I'm uploading a new video that uses some of the input you've provided. The no-optimize helped a lot with the walking blending issue. I even generate the blend masks on the first input file and use it on all subsequent files. Presumably that saves a bit of time.

The new video will be live at http://www.youtube.com/watch?v=Q2nkUhQ-RvY

Parallax is not expected to present too much of a problem as the majority of the frames will be with the control points at infinity. For the cases where they're not I'll decide how to handle them at that time.

Jan, you mentioned not adjusting the exposure so as to prevent flicker. That makes sense (and I do see some flicker across the whole scene in this new video), however, how am I supposed to fix the over/under exposed images when they're no longer pointing at the bright sun/dark shadows? This is definitely not going to be a static platform and I won't have the luxury of making sure the rig never rotates.

I won't be able to configure the template beforehand. Each run will be unique and will have to be tailored to that run.

I 100% agree with Jan on the control-point selection GUI improvements. I'm not in the ideal use case and I really feel the impact of an inflexible control point selection process. I can even see the case for allowing control point selection to be done with two overlapping images that you "pin" a control point to. Allow me to skew and rotate the images and pan them around to get a feature lined up as best I can simply for control point selection. It wouldn't be locked to that pin and, in fact, the pin shouldn't be visible once established in this skew mode. My problem is that the only overlap I have is in the far reaches of the images where they're most distorted. Picking out features in two images that aren't rotated similarly and are up to 12 inches apart on my screen makes it pretty difficult to determine if that bright blob is the same rock in both images.

[Caleb]

My camera rotates. I'd like to try to remove the rotation. I've got a couple systems setup to help me retrieve the rotation data from frame to frame. I am wondering how to apply that data pre-stitching, rather than my current method of post-stitching. I noticed when I move the center of the stitched image in the GL Preview that it modifies all of the yaw/pitch/roll components for each image. Is there an easy way to apply a, for example, yaw of 4 degrees before creating the panorama? I can't very well go in and align each one by hand.

[Bruno Postle]

On Sun 29-Jul-2012 at 10:50 -0700, Caleb wrote:
>My camera rotates. I'd like to try to remove the rotation. I've got a
>couple systems setup to help me retrieve the rotation data from frame to
>frame. I am wondering how to apply that data pre-stitching, rather than my
>current method of post-stitching. I noticed when I move the center of the
>stitched image in the GL Preview that it modifies all of the yaw/pitch/roll
>components for each image. Is there an easy way to apply a, for example,
>yaw of 4 degrees before creating the panorama?

Yaw is simple, you just increment the y values for each photo in the
project file. For any combination of roll, pitch and yaw, there is
a transform-pano tool in the Panotools::Script perl module that can
do the calculation for you:
http://search.cpan.org/dist/Panotools-Script/bin/transform-pano

--
Bruno

[Caleb Anderson]

I was hoping I'd have better results than I am. I think I require more understanding.

I have several 360x180 images stitched from six cameras. The cameras were stable relatively, but rotated globally.

I want to correct the rotation in all images so as to match the first in camera orientation.

I took my first image in the sequence and aligned the second to it using hugin. After removing the roll, pitch, yaw values it added for the first image I asked it to optimize positions, which gave me a rpy for the first image of 0,0,0 and a rpy for the second image (1/30th of a second in time later) of -0.508, -1.08, 0.

I applied this to the original pto that created the second image and used nona and enblend to stitch it again. The resulting image is not aligned with the first image.

Am I misunderstanding something or making an incorrect assumption?

Further, if this should be expected to work with the right incantation, I'd like to further modify image 1 so it is aligned how I'd prefer, then align the rest to match it. This seems to be troublesome as hugin only seems to look at the 2d image as given, rather than a translated image as defined by the lens projection and hfov. That is, it can't tell the something at one of the poles of my equirectangular image matches something in the center of a different image, as the control points only seem to look at the pre-projection image. This is probably the least of my issues, though.

At the following URL you can find a file containing the first three frames. The folder "ideal" contains how I'd like the images in "compares" to look. The folder "stock" contains my next best (but totally acceptable) alignment target.

http://res0l.net/StratoSphere/stratoalign.tar.gz

Any sage words would be appreciated. Thanks

Caleb

On Wed, Nov 21, 2012 at 2:28 PM, Caleb Anderson <robot...@gmail.com> wrote:

On Sunday, July 29, 2012 3:55:26 PM UTC-5, Bruno Postle wrote:

Yaw is simple, you just increment the y values for each photo in the
project file.  For any combination of roll, pitch and yaw, there is
a transform-pano tool in the Panotools::Script perl module that can
do the calculation for you:
http://search.cpan.org/dist/Panotools-Script/bin/transform-pano

--
Bruno

Thanks Bruno. I'm about ready to start applying this information. Here's a sample of what I'm going for:  https://plus.google.com/107235276879849058634/posts/GYACShuG7KX

And I once again find myself butting up against the manual CP placing process.

--
You received this message because you are subscribed to the Google Groups "Hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugi...@googlegroups.com

To unsubscribe from this group, send email to hugin-ptx+...@googlegroups.com

[Carlos Eduardo G. Carvalho (Cartola)]

Hi Caleb,

I don't know if I understood half of what you said, as English is also not my native language, but I'll try to say some things to help.

1. Pay attention to your control points. With those images I would set up only manual control points, as the automation can get confused with so many equivalent points. It is like when trying to setup control points in clouds, grass, trees and so on.

2. Setup an horizontal line in your horizon. Your scene, as I understood, has an horizon line, so use it to level your panorama. Even not using it you can also level the panorama on the "Move/Drag" tab of the "Fast Preview" window. You can see this tutorial: http://hugin.sourceforge.net/tutorials/perspective/en.shtml to see how to set horizontal or vertical lines. They are set in only one image, differently of common control points, that are set between two images.

3. If I understood correctly, you are using different cameras. If they are different and/or have different lenses you must correctly set it in the Camera/Lens tab by adding a new different lens for each one. This tutorial talks about it: http://hugin.sourceforge.net/tutorials/multi-lens/en.shtml

Also, in those cases where there is difficulty of communication it is sometimes more effective to see your original images and try to stitch them to then tell you the possibilities of how to deal with them.

Good luck!

Carlos E G Carvalho (Cartola)
http://cartola.org/360
http://www.panoforum.com.br/

2012/11/26 Caleb Anderson <robot...@gmail.com>

Did I get caught in a spam filter? I replied to this thread last night and it still hasn't shown up.

--
You received this message because you are subscribed to the Google Groups "Hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugi...@googlegroups.com

To unsubscribe from this group, send email to hugin-ptx+...@googlegroups.com

[Bruno Postle]

On Sun 25-Nov-2012 at 17:43 -0600, Caleb Anderson wrote:
>I was hoping I'd have better results than I am. I think I require more
>understanding.
>
>I have several 360x180 images stitched from six cameras. The cameras were
>stable relatively, but rotated globally.
>
>I want to correct the rotation in all images so as to match the first in
>camera orientation.
>
>I took my first image in the sequence and aligned the second to it using
>hugin. After removing the roll, pitch, yaw values it added for the first
>image I asked it to optimize positions, which gave me a rpy for the first
>image of 0,0,0 and a rpy for the second image (1/30th of a second in time
>later) of -0.508, -1.08, 0.
>
>I applied this to the original pto that created the second image and used
>nona and enblend to stitch it again. The resulting image is not aligned
>with the first image.

..but it is otherwise ok?

I suggest you stitch all these equirectangular panoramas and then
align the output images as a separate stage of your workflow.

i.e. in Hugin you can load several equirectangular images into a
single project and align them into a stack. When you stitch in the
Stitcher tab select Remapped Images and you will get one remapped
file for each input file.

--
Bruno

[Caleb Anderson]

On Mon, Nov 26, 2012 at 3:41 PM, Bruno Postle <br...@postle.net> wrote:

..but it is otherwise ok?

Well, no. I'm not sure what you mean by otherwise ok, but the whole goal with that was to get the image aligned to the first image, so it's not really ok. Unless you mean that it's still a properly stitched image, for which the answer is yes.

 

 

I suggest you stitch all these equirectangular panoramas and then align the output images as a separate stage of your workflow.

i.e. in Hugin you can load several equirectangular images into a single project and align them into a stack.  When you stitch in the Stitcher tab select Remapped Images and you will get one remapped file for each input file.

That sounds like a reasonable idea. Will I run into issues drastically remapping an image? That is, how well does a projection shift preserve information stored at the poles and shifted to the center?

Additionally, I tried previously to align some of these images in a stack, but it seemed to prefer them to be identical framing. The tutorials I read on stacking all seem to involve a static scene with different depth of field settings for each shot. My data isn't limited to one camera angle, or even small changes in camera angle. The entire scene could be upside down and shifted 180 degrees at one point.

Cheers

[Caleb Anderson]

I gave that a quick shot with some quickly placed control points. It gives some really weird results when using 360x180 source images. It definitely did not want me to keep the images in one stack.

http://res0l.net/StratoSphere/stacked.tar.gz contains my pto, inputs, and sample outputs. Did I do something wrong?

[Caleb Anderson]

On Mon, Nov 26, 2012 at 4:59 PM, Caleb Anderson <robot...@gmail.com> wrote:

I gave that a quick shot with some quickly placed control points. It gives some really weird results when using 360x180 source images. It definitely did not want me to keep the images in one stack.

http://res0l.net/StratoSphere/stacked.tar.gz contains my pto, inputs, and sample outputs. Did I do something wrong?

I do get better results if I only link the first image to all subsequent images, instead of each image with each other. That actually makes sense when I think about it.

However, I still get the strange lens/projection issue, even when just optimizing for position. This video consists of 28 frames. 27 of them are aligned to the first frame.

http://res0l.net/StratoSphere/Viewer/images/28-optimize.webm

(And the webgl viewer)

http://res0l.net/StratoSphere/Viewer/final1.html?video=images/28-optimize.webm

What's going on here? I don't have the source data handy for that, but it's the same result seen in the previously linked archive.

[Bruno Postle]

On Wed 28-Nov-2012 at 11:25 -0600, Caleb Anderson wrote:
>
>I do get better results if I only link the first image to all subsequent
>images, instead of each image with each other. That actually makes sense
>when I think about it.
>
>However, I still get the strange lens/projection issue, even when just
>optimizing for position. This video consists of 28 frames. 27 of them are
>aligned to the first frame.
>http://res0l.net/StratoSphere/Viewer/images/28-optimize.webm

This does look like a problem with your lens parameters. Did you
calibrate your lens distortion parameters separately and import them
into your Hugin project? This isn't ideal data to use for lens
calibration.

--
Bruno

[Caleb Anderson]

I added my images and, in the popup on add, set to equirectangular and hfov of 360. Then I added control points and used the optimize tab to optimize positions only.

Nona with gpu refused to output remapped images. Turning off gpu output 'properly'. (that is, the gl preview did show the same poorly projected output)

--
You received this message because you are subscribed to the Google Groups "Hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugi...@googlegroups.com

To unsubscribe from this group, send email to hugin-ptx+unsubscribe@googlegroups.com

[Caleb Anderson]

On Wed, Nov 28, 2012 at 3:12 PM, Caleb Anderson <robot...@gmail.com> wrote:

Nona with gpu refused to output remapped images. Turning off gpu output 'properly'. (that is, the gl preview did show the same poorly projected output)

Err, forgot to finish my thought. Turning off gpu output spit out "properly" remapped images. (the broken images you see in the video)

If I understand correctly, I don't have much as far as lens distortion parameters in regards to a 360x180 image as we're just talking about a sphere, right?

[Bruno Postle]

On Wed 28-Nov-2012 at 19:09 -0600, Caleb Anderson wrote:
>
> Err, forgot to finish my thought. Turning off gpu output spit out
> "properly" remapped images. (the broken images you see in the
> video)
>
> If I understand correctly, I don't have much as far as lens
> distortion parameters in regards to a 360x180 image as we're just
> talking about a sphere, right?

Yes if you have equirectangular images then there are no lens
distortion parameters you can apply. The video doesn't work here (I
know it should just don't have time to figure out why), remapping an
equirectangular to another equirectangular should be straighforward,
I've never had a problem with this.

--
Bruno