Some curious blending results

[Terry Duell]

Hello All,
I have been a bit curious about how our new internal blender (verdandi,
which will be available in hugin-2015.0.0) compares with enblend and
multiblend.
Using the images in the scanned images tutorial
<http://hugin.sourceforge.net/tutorials/scans/en.shtml>, I have run
through the tutorial using each of the blenders, to compare run time and
any visible differences in the stitch.
To compare run time I have used a bash script and the 'time' command.
For this simple test, verdandi is the quickest, and enblend the slowest;
the results being verdandi: 6.08s, multiblend: 6.70s, enblend: 8.42s.
The curious part of my results is the comparison of the stitched images.
Here I am getting some blurring in the enblend result. A screenshot of all
3 stitches is attached. Left to right we have enblend, verdandi,
multiblend.
Note the blurring near the centre of the large leaf group to the right of
the image in the enblend stitch.
Note the same script (the pto_var version in the tutorial, modified to use
hugin_executor) has been used in each case. I have also run through the
process using the gui and enblend gives me this same result.
I used a local build of enblend (default branch snapshot
4.2.0-1158:d25158b50519) built 02/02/2015; a local build of
multiblend-0.6.2, and a local build of hugin-2014.1.0 (default branch
snapshot 6887:a8201bc48fa7).
It would helpful if someone could run through the tutorial, using either
the gui or a script, and report whether my blurred result using enblend
can be reproduced, or not. Please report the version of enblend used.
Any insights into the above welcomed.

Cheers,
--
Regards,
Terry Duell

[Monkey]

I think it's just down to the blender's differing methods of determining seams. I don't know how verdandi works, but Enblend and multiblend use different methods.

Enblend's initial seaming method is more mathematically "precise," but it's a bit of a false precision because it won't practically lead to better stitches. Enblend then also refines seams by following the contours in image overlaps, which is probably what's making the biggest difference. The blurred patch is not down to the blending as such, but is already present in scan-1.jpg. multiblend's seam has not run so close to the edge of the image, but Enblend has probably refined itself in that direction.

The best like-for-like test between Enblend would be with --no-optimize and --fine-mask added to the Enblend command line. However, even with those switches, once you start blending more than two images you'll find Enblend's seams will differ more and more from multiblend's.

Finally, if you're after timing comparisons, you're going to need a much bigger panorama. Most of the 6 seconds taken by all the stitchers will be loading and saving of images. multiblend's speed advantage over Enblend rises exponentially with panorama size. 172x faster was the best I got, and that was with Enblend crashing part way through.

[Roger Broadie]

Monkey <davidh...@gmail.com> commented on Wed, 22 Apr 2015 at
04:53:56 -0700 (PDT):

"The blurred patch [in Terry Duell's screenshot of Tue, 21 Apr 2015 at
17:37:32 +1000] is not down to the blending as such, but is already
present in scan-1.jpg."

I don't see that. True, there is blurring by the right-hand edge of the
image, presumably due to a lifting of the original at the edge of the
scanning area, and similar blurring, no doubt for a corresponding
reason, by the left-hand edge of tscan-2.jpg. But in neither is there
significant blurring in the region of the most noticeable blurring in
Terry's enblend example, which is around the central vein of the nearly
vertical leaf on the right of the stitched image, which is rotated with
respect to the original scans in order to orient it correctly. Whatever
is going on looks to me to be more than just a result of the positioning
of the seam.

I am not sure if the following is the sort of information Terry is
seeking, but I stitched the two scanned images using the Windows 64-bit
version of Hugin, 2014.0.0.5da69bc383dd built by Matthew Petroff. As
far as I can see, it uses Enblend version 4.1.3, 19 March 2014. I
followed the tutorial by optimising X, Y Z and roll for Image 2, with
the fov set at 10 deg (I persist in using only a single lens, since
there is no need to vary the parameters per image). That yielded a
stitch with no noticeable blurring, as in the other blender examples in
Terry's screenshot. However, when I superimposed the two remapped
images there proved to be a slight discrepancy in the area of the
blurred central vein, amongst other places, of roughly the width of the
lines defining the vein. Even if the seam had included a very broad
band of feathering it could not, I should have thought, have produced
the blurring seen. And there was a slight jink noticeable in the
left-hand border of the image area in the stitched output. Of course
both these imperfections might be the result of a poor choice of control
points.

Roger Broadie

[Terry Duell]

Hello David,

On Wed, 22 Apr 2015 21:53:56 +1000, Monkey <davidh...@gmail.com> wrote:

> I think it's just down to the blender's differing methods of determining
> seams. I don't know how verdandi works, but Enblend and multiblend use
> different methods.

Yes, that's understood, but the curious behaviour, which I should have
explained a bit more clearly, is that enblend has, in the past, produced a
result without the blurring, and using the images in the same order, as
shown in the tutorial.

[snip]

>
> Finally, if you're after timing comparisons, you're going to need a

> *much* bigger panorama.

I did say it was a simple test, and only aimed at providing an indication
of the differences in speed.
I agree, that most of the time for the project discussed, would be down to
other processing, which would be the same for each case, and an accurate
measure of blending time would require a more clever test.

[Terry Duell]

Hello Roger,

On Thu, 23 Apr 2015 07:53:09 +1000, Roger Broadie
<roger....@ogea.freeserve.co.uk> wrote:

> Monkey <davidh...@gmail.com> commented on Wed, 22 Apr 2015 at
> 04:53:56 -0700 (PDT):
>
> "The blurred patch [in Terry Duell's screenshot of Tue, 21 Apr 2015 at
> 17:37:32 +1000] is not down to the blending as such, but is already
> present in scan-1.jpg."
>
> I don't see that.

Thanks, neither do I... well not to the extent that is visible in the
final stitch. What really prompted my post on the matter was that the
enblend result was noticeably different to that achieved in the tutorial.

Thanks for your feedback on this Roger.
I'll try some earlier versions on enblend to see if I can learn a bit more.

[Terry Duell]

On Thu, 23 Apr 2015 08:20:54 +1000, Terry Duell <tdu...@iinet.net.au>
wrote:

[snip]

>
> Thanks for your feedback on this Roger.
> I'll try some earlier versions on enblend to see if I can learn a bit
> more.

Enblend-4.1.3 does the job much better, I don't see any blurring
anywhere...but I have been told I'm blind in one eye and can't see out the
other :-)
Attached screenshot show a stitch using enblend-4.1.3, produced using the
same method as described in my original post.
The result obtained using enblend-4.2.0 may be related to code changes
that affect the location of the seam, or there may be more going on
here...I don't know.
In the meantime I'll stick to enblend-4.1.3 for real work and when time
permits try a few more tests with enblend-4.2.0.

[Monkey]

On Wednesday, 22 April 2015 22:53:11 UTC+1, Roger Broadie wrote:

Monkey <davidh...@gmail.com> commented on Wed, 22 Apr 2015 at
04:53:56 -0700 (PDT):

"The blurred patch [in Terry Duell's screenshot of Tue, 21 Apr 2015 at
17:37:32 +1000] is not down to the blending as such, but is already
present in scan-1.jpg."

I don't see that.  True, there is blurring by the right-hand edge of the
image, presumably due to a lifting of the original at the edge of the
scanning area, and similar blurring, no doubt for a corresponding
reason, by the left-hand edge of tscan-2.jpg. But in neither is there
significant blurring in the region of the most noticeable blurring in
Terry's enblend example, which is around the central vein of the nearly
vertical leaf on the right of the stitched image, which is rotated with
respect to the original scans in order to orient it correctly.

I don't see that anything's been rotated.

Here's Terry's blend with tscan-1.jpg next to it, with the matching blurred areas highlighted:

http://s9.postimg.org/mdpmz69an/screenshot_2.jpg

[Roger Broadie]

Thanks to Monkey <davidh...@gmail.com> for pointing out on Thu, 23
Apr 2015 at 01:09:46 -0700 (PDT) that nothing in the three screen grabs
from Terry's original message had been rotated.

He's right, of course, in the sense that they all point the same way.
What I had failed to realise was that the selected segments were all at
right-angles to the intended orientation of the final stitch, which is
in portrait mode.

So the problem is that the new version of Enblender is selecting a
blurred band from the very edge of scan-1 for inclusion in the final
stitch, whereas the other blenders exclude it by putting the seam
roughly in the middle of the overlap area. I wonder if masking out the
blurred areas would help.

Roger Broadie

[Monkey]

I think those sample images were not the best that could have been chosen to illustrate flat stitching, with their abstract design and lack of visual clues as to orientation. I had to double check to make sure they hadn't been rotated as it's not immediately obvious, nor is it easy to gauge how the two images overlap just by looking at them.

I wonder if masking out the blurred areas would help.

Undoubtedly. The down side is that you rob yourself of image data that could still make a useful contribution to blending (even in this case - the fine lines might be blurred but the lower frequency information could still be useful).

I'd love to see some sort of seam hinting - perhaps some way of passing on to the blender that you don't want seams to cross a certain area, or to force them to follow a certain path. Blending water waves, for example, can be hard to do seamlessly, and one useful technique is to draw a meandering seam line. I did once write a (very basic) GUI that let you manipulate blending seams, but only between two images, and these days I use multiblend's --save-seams and --load-seams. I think the same thing is possible with Enblend, but is much more convoluted.

[John Muccigrosso]

I think what's important here is that enblend used to produce better results and on something like this with pretty crisp lines, that's crucial. Having to worry about putting in masks is troubling.