viewing PTO files without stitching

[kfj]

Hi group!

I've recently started this thread to point you to my image and panorama viewer:

https://groups.google.com/forum/#!topic/hugin-ptx/bj1g86oe6Wk

In my initial posting, I've already mentioned the topic of 'live stitching' - displaying a synoptic view of several registered source images *without stitching*. I've made good progress along these lines, and the latest version of pv can *directly read PTO files*. The file select dialog will show PTO files, and it's as simple as opening them like any other image file. On the command line, just pass a pto file like you'd pass an image file, like

pv pano.pto

pv will only process the source image information ('i-lines'). pv supports PTO geometry well: it will handle source images in rectilinear, cylindric, spherical, fisheye and stereographic format and honour the source images' orientation, horizontal field of view and all lens correction parameters. Photometric parameters are ignored except for Eev - pv expects sRGB or linear RGB images and does not handle colour profiles, nor does it do vignetting correction or camera response curve processing. Nevertheless, the display of a well-registered image set, especially with images taken with fixed aperture and exposure time, will be pretty close to the stitched result, especially with applied feathering.

'live stitches' have advantages and disadvantages, I'll start out with some advantages:

• Access to the output is much more immediate, because the lengthy stitching process is avoided. As long as your image set remains the same, you can quickly reflect changes in the PTO file by simply pressing 'F1' in pv, which will reload the PTO file and reuse the interpolators, which otherwise take some time to set up. You can use pv as an 'external preview' while working on the PTO in a stitcher like hugin, save work in the stitcher and refresh pv to see the output straight away.
• Interpolation is done directly on the source images. This preserves the full source image quality and resolution - you can, e.g., zoom into the view like into the source images, without being limited to the resolution you chose for the output of stitching, and without the images having been geometrically transformed and then stored before being read back in again, losing some quality in the process. Same goes for exposure.
• pv uses a geometric approach to select from which source image(s) a target pixel should be taken. Without feathering or alpha processing, the simple rule is: 'take from the source image whose center is closest to the target pixel'. Seams are not explicitly defined but occur as an emergent phenomenon. the 'closest-center' rule automatically chooses those parts of the source images which are usually the best - technically speaking - because they are least encumbered with lens flaws.
  
• You can experiment easily with several target projections (use --target_projection=...), again without having to stitch an output image. If you set up pv's viewer window with the right aspect ratio and use a snapshot magnification appropriate to your desired output size, making a snapshot of what you see is equivalent to stitching with a stitcher. The process is WYSIWYG, and output quality is defined by output size  and the 'quality interpolator' which is currently in use (per default, a cubic b-spline). It's much faster than 'true' stitching. The rendition of a snapshot is delegated to a separate thread 'on the back burner', so you can keep on viewing while the snapshot(s) are completed in the background.
  

Now for some disadvantages:

• 'live stitching' is memory-hungry. Every source image is read from disk and converted to an internal representation (typically a set of two float-based image pyramids). You can save on memory by passing --build_pyramids=no, which will impose a few limits - for simply viewing the PTO it's okay, though. But even then, more memory is needed than for viewing a stitched panorama.
• There is - as of yet - no seam optimization. The seams occur where two source images 'meet', and there is nothing you can do about it - registration errors can be masked to a degree by using feathering, but that's it.
• Animation can be slow. When viewing single images/panoramas, pans, zooms, etc. should not stutter on a reasonably powerful system. But with 'live stitching', frame rates drop. Especially when viewing views where many source images contribute, and with target projections other than rectilinear, frame rates may go down to a few per second.

This is a wide topic, so there are many more factors to consider, but I won't go on here. What I'd like to mention is a second 'blending mode' which pv offers. If you pass --blending=hdr, you can view registered exposure series as if they had already been blended into an HDR output. While pv has only rudimentary tonemapping (you might say it's only range compression), this type of view still can give you a good idea about how well-suited an image set is for HDR blending. There is no deghosting, though. Making snapshots of hdr-blended views can preserve the full dynamic range, provided the output format is capable of doing so - typically you'd use openEXR output. So you can use pv to HDR-blend sets of images as well.

Kay

[Luís Henrique Camargo Quiroz]

   Great Kay! I will try to compile it this time, the features of your pv are really compelling and would easy a lot to build great panoramas!

   thank you!

   Luis Henrique

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/b478196a-a336-4743-998e-92ea72769820%40googlegroups.com.

--

--
Luis Henrique Camargo Quiroz
http://luishcq.br.tripod.com - http://www.christusrex.org/www2/cantgreg
http://panoramaslh.net/

[David W. Jones]

Thanks, sounds fun. Tried to build it on Debian 10, got this:

~/pv$ make
clang++ -c -Ofast -std=c++11 -c pv_no_rendering.cc -o pv_no_rendering.o
clang++ -c -Ofast -std=c++11 -c pv_initialize.cc -o pv_initialize.o
clang++ -c -D PV_EXTERN=extern -mavx2 -D PV_ARCH=PV_AVX2 -Ofast
-std=c++11 -c pv_rendering.cc -o pv_avx2.o
In file included from pv_rendering.cc:108:
In file included from ./pv_common.h:59:
In file included from /usr/include/Vc/Vc:30:
In file included from /usr/include/Vc/vector.h:35:
In file included from /usr/include/Vc/avx/vector.h:32:
In file included from /usr/include/Vc/scalar/../common/../avx/casts.h:33:
In file included from
/usr/include/Vc/scalar/../common/../avx/../sse/casts.h:31:
/usr/include/Vc/scalar/../common/../sse/intrinsics.h:601:13: error:
argument to
'__builtin_ia32_vec_ext_v4sf' must be a constant integer
_MM_EXTRACT_FLOAT(f, v, i);
^~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/clang/7.0.1/include/smmintrin.h:890:11: note: expanded from
macro
'_MM_EXTRACT_FLOAT'
{ (D) = __builtin_ia32_vec_ext_v4sf((__v4sf)(__m128)(X), (int)(N)); }
^ ~~~~~~~~
1 error generated.
make: *** [makefile:38: pv_avx2.o] Error 1

Ideas?

> * Access to the output is much more immediate, because the lengthy

> stitching process is avoided. As long as your image set remains the
> same, you can quickly reflect changes in the PTO file by simply
> pressing 'F1' in pv, which will reload the PTO file and reuse the
> interpolators, which otherwise take some time to set up. You can use
> pv as an 'external preview' while working on the PTO in a stitcher
> like hugin, save work in the stitcher and refresh pv to see the
> output straight away.

> * Interpolation is done directly on the source images. This preserves

> the full source image quality and resolution - you can, e.g., zoom
> into the view like into the source images, without being limited to
> the resolution you chose for the output of stitching, and without
> the images having been geometrically transformed and then stored
> before being read back in again, losing some quality in the process.
> Same goes for exposure.

> * pv uses a geometric approach to select from which source image(s) a

> target pixel should be taken. Without feathering or alpha
> processing, the simple rule is: 'take from the source image whose
> center is closest to the target pixel'. Seams are not explicitly
> defined but occur as an emergent phenomenon. the 'closest-center'
> rule automatically chooses those parts of the source images which
> are usually the best - technically speaking - because they are least
> encumbered with lens flaws.

> * You can experiment easily with several target projections (use

> --target_projection=...), again without having to stitch an output
> image. If you set up pv's viewer window with the right aspect ratio
> and use a snapshot magnification appropriate to your desired output
> size, making a snapshot of what you see is equivalent to stitching
> with a stitcher. The process is WYSIWYG, and output quality is
> defined by output size  and the 'quality interpolator' which is
> currently in use (per default, a cubic b-spline). It's much faster
> than 'true' stitching. The rendition of a snapshot is delegated to a
> separate thread 'on the back burner', so you can keep on viewing
> while the snapshot(s) are completed in the background.
>
>
> Now for some disadvantages:
>

> * 'live stitching' is memory-hungry. Every source image is read from

> disk and converted to an internal representation (typically a set of
> two float-based image pyramids). You can save on memory by passing
> --build_pyramids=no, which will impose a few limits - for simply
> viewing the PTO it's okay, though. But even then, more memory is
> needed than for viewing a stitched panorama.

> * There is - as of yet - no seam optimization. The seams occur where

> two source images 'meet', and there is nothing you can do about it -
> registration errors can be masked to a degree by using feathering,
> but that's it.

> * Animation can be slow. When viewing single images/panoramas, pans,

> zooms, etc. should not stutter on a reasonably powerful system. But
> with 'live stitching', frame rates drop. Especially when viewing
> views where many source images contribute, and with target
> projections other than rectilinear, frame rates may go down to a few
> per second.
>
>
> This is a wide topic, so there are many more factors to consider, but I
> won't go on here. What I'd like to mention is a second 'blending mode'
> which pv offers. If you pass --blending=hdr, you can view registered
> exposure series as if they had already been blended into an HDR output.
> While pv has only rudimentary tonemapping (you might say it's only range
> compression), this type of view still can give you a good idea about how
> well-suited an image set is for HDR blending. There is no deghosting,
> though. Making snapshots of hdr-blended views can preserve the full
> dynamic range, provided the output format is capable of doing so -
> typically you'd use openEXR output. So you can use pv to HDR-blend sets
> of images as well.
>
> Kay

--
David W. Jones
gnome...@gmail.com
wandering the landscape of god
http://dancingtreefrog.com
My password is the last 8 digits of π.

[Kay F. Jahnke]

On 10.05.20 11:26, David W. Jones wrote:
> On 5/9/20 9:44 PM, Kay F. Jahnke wrote:

I think I found the problem. I use clang++ v.6 here, which works fine
with the Vc version from package management: my ubuntu also provides
1.3.3. But clang++-7, which you use, does not seem to work with this Vc
version.

So either you can try to downgrade to clang++ v.6 and stick with Vc
1.3.3 from Synaptic, or you build the Vc 1.4 branch from source. That
would be the better option, because 1.4 has some new functionality.
Building Vc is not hard, you can use the procedure I have given in pv's
documentation. Navigate to the place where you keep your cloned git
repos, then do this:

git clone https://github.com/VcDevel/Vc.git
cd Vc
git checkout 1.4
mkdir build
cd build
cmake -DCMAKE_CXX_COMPILER=clang++ ..
make
make install

This will install the Vc headers to /usr/local/include and libVc.a to
/usr/local/lib. My system gives preference to stuff in /usr/local, but
to be on the safe side, remove the Vc which Synaptic gave you to make
sure you include and link Vc 1.4.

After that, the compile should run through with either clang++-6 or
clang++-7. So after you've installed the Vc 1.4 branch, just go to the
pv root directory again and issue 'make'.

Again, let me know what happens.

Kay

[Kay F. Jahnke]

On 20.05.20 07:44, David W. Jones wrote:

> I removed the 1.3.3 that Synaptic had installed before I did this.

good.

>
> Running make reported this error:
> Scanning dependencies of target simdarray_avx
> [ 11%] Building CXX object
> tests/CMakeFiles/simdarray_avx.dir/simdarray.cpp.o
> In file included from /home/david/gits/Vc/tests/simdarray.cpp:28:
> /home/david/gits/Vc/tests/unittest.h:30:10: fatal error:
> 'virtest/vir/test.h' file not found
> #include "virtest/vir/test.h"
> ^~~~~~~~~~~~~~~~~~~~
> 1 error generated.
> make[2]: *** [tests/CMakeFiles/simdarray_avx.dir/build.make:383:
> tests/CMakeFiles/simdarray_avx.dir/simdarray.cpp.o] Error 1
> make[1]: *** [CMakeFiles/Makefile2:12646:
> tests/CMakeFiles/simdarray_avx.dir/all] Error 2
> make: *** [Makefile:141: all] Error 2
>
> For what it's worth, when I tried the "git checkout 1.4" line, it
> reported I already had 1.4 checked out.

That's how it should be, then.

>
> Ideas?
>

Vc has trouble building it's test suite. But to build the library and
headers, you can do without the tests. Building the tests can be
switched off by telling cmake not to build them.

To be on the safe side, please start afresh: remove the botched build
directory (rm -r build), create a new one (mkdir build), cd into it.
modify the call to cmake to read:

cmake -DCMAKE_CXX_COMPILER=clang++ -DBUILD_TESTING=0 ..

Then issue 'make' and 'sudo make install'. This should run through
without errors and install Vc 1.4 to /usr/local. You should now -
finally - be able to run pv's makefile in pv's root directory. Please do
a 'git pull' on my repo before building pv, there are some recent additions.

Let me know how it goes.
Kay

[David W. Jones]

Hey, thanks for the guidance. Vc 1.4 compiled and installed fine. PV
compiled file but offers no install option, apparently.

What should I see when I open a PTO file with PV? All I see is a bottom
frame of the pano, not the entire pano.

[David W. Jones]

Sorry, OK, I see that the mousewheel changes zoom into the panorama. And
dragging the mousepointer around kind of counter-intuitively moves the
image the opposite direction (vertically or horizontally).

But interesting and useful, this PV tool. Thanks for making it and for
the assistance in getting it to compile on Debian 10.

[Kay F. Jahnke]

On 21.05.20 10:14, David W. Jones wrote:

>> Hey, thanks for the guidance. Vc 1.4 compiled and installed fine. PV
>> compiled file but offers no install option, apparently.

It is meant to be run from it's root folder. This way, it finds stuff
like it's default font easily. pv is multiplatform software, and that's
the easiest way to handle assets consistently on several platforms
without forming a notion of the underlying file system. On Windows, I
produce it as stickware. The idea is that it can be added to a bunch of
images so that people have a tool to view the images without having to
install anything. If you like installing stuff, simply use a soft link:

sudo ln -s /your/path/to/pv /usr/local/bin/pv

>> What should I see when I open a PTO file with PV? All I see is a
>> bottom frame of the pano, not the entire pano.

you should see the images 'in place', as if the panorama had been
stitched already.

> Sorry, OK, I see that the mousewheel changes zoom into the panorama.

This is the way to zoom which most people discover first. It's much
smoother to do a secondary-mouse-button vertical click-drag - down zooms
out, up zooms in. The numpad plus and minus keys also produce smooth but
fixed-speed zooms. The click-drag mouse gestures are
displacement-sensitive: the further you move away from the click point,
the stronger the effect.

> And
> dragging the mousepointer around kind of counter-intuitively moves the
> image the opposite direction (vertically or horizontally).

The standard notion in pv is that you move the virtual camera, be it
with mouse gestures, keyboard or GUI. pv's interpretation of mouse
gestures is inspired by QTVR (see
https://en.wikipedia.org/wiki/QuickTime_VR), implemented for example by
360cities (see https://www.360cities.net), where it's optional. If you
find the direction of the moves counterintuitive, you can use these options:

-R, --reverse_drag=<yes/no>

inverts the effect of click+drag with the primary mouse button

-Z, --reverse_secondary_drag=<yes/no>

inverts the effect of click+drag with the secondary mouse button

Please note that pv is a complex program with many options, which are
documented in the documentation (scroll down on the bitbucket repo's
front page or consult README.rst). There is much more to it than just
displaying PTO files, but I think that's such a major achievement that
it deserved a dedicated post.

> But interesting and useful, this PV tool. Thanks for making it and for
> the assistance in getting it to compile on Debian 10.

Thanks for bearing with me. I'd like to see more people trying it :D

Helping you compile on your system helps me, too, because I learn how to
tweak my procedures to run on more systems. Without our exchange, I
might not have found the problems with Vc 1.3.x and clang++-7, nor the
problems with compiling Vc 1.4.

Kay