Next GUI - take 2-1/2
Yuval Levy
I finally got around to read all the interesting opinions expressed in
the previous thread [0]. Those who know me well know how hard it was to
let the discussion go without answering :-) but it was well worth it. I
wanted to hear *your* opinions.
My understanding is that V+W seems to be broadly acceptable to most; and
that the main point of contention is single output / limited set of
projections vs. multiple outputs / broad set of projections.
Let's first look at what most of us agree on; and then continue on the
main point of contention.
V is OpenGL. It's blazing fast. It is responsive right away (with place
holders while a background task loads the images and creates the
textures). Could it be a game engine (as precognized by Bart)? How
difficult would it be to make the current fast preview a spherical
viewer (while retaining all the functionality)? could be one of the
preview modes.
W needs more specification.
Thank you Bruno for the overview of the current tabbed interface.
Knowing where we come from is important when discussing where to go next.
To me personally, the major limitation of the current tabbed interface
is the rigidity of the single workflow, represented by the individual
tabs. It's broken.
In term of workflow I see it split in two: input and output, with the
input converging to a single image and the output diverging again into
different view / projections / formats.
The main point of contention is the point where input ends and output
starts, with some proponents of a single output / limited set of
projections; while others prefer a broad set of projections.
I tend to side with Seb, Lukas, James on this one. There is enough of a
use case to leave the output of the stitcher as-is, even if the choice
may look daunting complex to a newbie. I rather reduce the complexity
through interface work than through limiting the choices to equirect,
cube, and maybe cylinder. Where is mosaic in all of this?
I do understand the point raised by Bruno, Tom, Nicolas. There is a
point of inflection in the workflow, where input ends and output begins.
Input converges toward the "single output" equirect proposed by Tom
(although in the meantime, thanks to the discussion, we know that
equirect is not enough for all use cases). It's about generating control
points, pruning them, aligning the images.
Output diverges again. Starting from the "single input", it creates
different views from the same "equirect". Tom has analyzed this very
thoroughly. His Panini tool is an actual implementation of something
very similar.
Like Tom I also would like to see a workflow / graphic representation of
this. I'm still not sure about it's positioning, though. I think it does
not deserve to be at the top of the hierarchy as it is now (the tabs
represent a workflow). I'd rather have it as a tab, with blocks that can
be added/removed/connected.
In my present draft idea, those blocks could be input or output blocks,
to represent the input and output workflow.
Output is relatively easy: projection, view(hfov/vfov/yaw/pitch/roll),
canvas(width/height/crop); and we can line up as many of them as we want.
Input is slightly more complex, as it needs to keep status of whether
images have been added / moved since the last optimization, etc.
Although it looks very much like all of today's tabs with the exception
of the stitcher tab (that is, after all, Output).
Using "make" as the underlying "keeper of status" makes sense to me; and
I agree with Bruno's view that it would be very nice if the batch
processor could piggy-back on "make" rather than going through the whole
project from scratch every time.
Ideally, as I work through the GUI, individual steps are added / updated
to the makefile and the batch stitcher runs in the background, using
make to determine if something has changed and triggering the necessary
implementation actions.
Sometimes very far down the line it would be nice to have feedback about
make's completition status, e.g. by a changing color in the blocks on
the workflow representation (e.g.: gray = waiting in the pipeline; green
= done; the volume being progressively filled from gray to green to
signal the degree of completition of the block; and red = error/problem).
More thinking to do. More opinions welcome.
Last but not least, a word about wxPython. Binding C++ with Python is
doable [1]. Many projects, including commercial ones (Google) do that
often. Python for rapid development and UI work; C/C++ for the bottlenecks.
wxPython is equally supported as wxWidgets and can do the same. Adding
Python bindings to the functionality inside Hugin would make it
accessible to a wxPython GUI; and everything we can do in wxWidgets/C++
we can do in wxPython.
I think it is worth thinking about using Python for the UI.
Yuv
[0]
http://groups.google.com/group/hugin-ptx/browse_thread/thread/43d15deec2251b87
[1] http://www.swig.org/
Rogier Wolff
On Tue, Oct 20, 2009 at 09:31:14PM -0400, Yuval Levy wrote:
> Input converges toward the "single output" equirect proposed by Tom
> (although in the meantime, thanks to the discussion, we know that
> equirect is not enough for all use cases). It's about generating control
> points, pruning them, aligning the images.
In my mind, in hugin the single "output" is the abstraction of a
sphere with certain spherical angles painted with a certain color.
This is represented internally by the source images, their roll, pitch
and yaw parameters and all the other parameters. That's when output
starts.
(If you think of it this way, we do enblend at the wrong point in
time. :-) )
Roger.
--
** R.E....@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
Does it sit on the couch all day? Is it unemployed? Please be specific!
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
Yuval Levy
> (If you think of it this way, we do enblend at the wrong point in
> time. :-) )
we surely do. enblend is part of my post-processing after I've manually
edited the masks on the re-projected layers in Photoshop.
Yuv
Rogier Wolff
This is an input-processing step. Logically you should edit the
original JPGS to make the unwanted parts "transparent".
There is a technical problem with that: the jpeg format doesn't
support that.
Tom Sharpless
wrote:
> original JPGS to make the unwanted parts "transparent".
>
placement that control it) to the "input" side would make it a lot
easier to make panos of lively scenes. The fast preview would be the
right UI for this.
Yuv,
I think "doing make right" should be a priority for the very next
release, let's not wait for GUI 2. That means intelligent re-use of
intermediate files, status reporting, and a foolproof "clean"
operation.
It would certainly be feasible to make fast preview act as a spherical
pano viewer, it is just a matter of putting in a mesh sphere, mapping
the images to that, and viewing it. I think (with Roger) that the
panosphere is really the "canonical panorama" and should be the frame
of reference for all editing (too bad you can't file it). So I would
really like to see this idea pursued.
Best, Tom
Bruno Postle
>It would certainly be feasible to make fast preview act as a spherical
>pano viewer, it is just a matter of putting in a mesh sphere, mapping
>the images to that, and viewing it.
We already have this, set output to: 'orthographic', aspect ratio to
1:1 and angle of view to 180. Use drag mode to spin the sphere.
However, I really don't think looking at the panorama from the
outside like this is a very useful GUI interface - Scenes need to be
seen from inside.
--
Bruno
Yuval Levy
> On Wed, Oct 21, 2009 at 04:49:20AM -0400, Yuval Levy wrote:
>> Rogier Wolff wrote:
>>> (If you think of it this way, we do enblend at the wrong point in
>>> time. :-) )
>> we surely do. enblend is part of my post-processing after I've
>> manually edited the masks on the re-projected layers in Photoshop.
>
> This is an input-processing step.
no, for me it is not. I use masking to show and hide elements. It's an
artistic choice and it's part of my *output* processing. Sometimes I
want to show as many people as possible in a scene. Other times I want
to mask all of them away. From the same scene.
> Logically you should edit the
> original JPGS to make the unwanted parts "transparent".
this means to me that I would have to run input twice. not very efficient.
> There is a technical problem with that: the jpeg format doesn't
> support that.
my input is mostly TIFF, so I'm not conerned by this technical problem.
The major problems I see with manual masking and for which I want a
GUI's support are, in order of priority:
1. the ability to see through the layers when deciding what and where to
mask
2. masking difficult areas - this is projection related (try masking at
the nadir of an equirect)
both can be solved technically,
1. with a display of overlapping images prior to blending
2. by making this display dynamic and movable
I think we have an underlying misunderstanding about what makes the
connection between input and output. For you it is a stitched and
blended panorama. For me it is a bunch of aligned layers.
Yuv
Yuval Levy
> Roger is right on. Moving blending (or rather, the masking and seam
> placement that control it) to the "input" side would make it a lot
> easier to make panos of lively scenes. The fast preview would be the
> right UI for this.
masking is not only about seam placement. masking is first and foremost
about artistic choice. The mistake you are making is to think of the
scene as a static one, with no change during the time between frames.
> I think "doing make right" should be a priority for the very next
> release, let's not wait for GUI 2. That means intelligent re-use of
> intermediate files, status reporting, and a foolproof "clean"
> operation.
It should be a priority, no doubt. It will be one for the next release
after the feature is ready. Things move forward in parallel and what is
ready gets in to the next release. Patches are always welcome.
> panosphere is really the "canonical panorama" and should be the frame
> of reference for all editing
Disagree strongly. Editing the zenith or nadir of a panosphere is a
nightmare; For linear panoramas the sphere is too limiting as well.
As an artist, I don't want to be framed in any reference other than 3D
space. Everything else is an artificial limitation that will fall,
sooner or later.
To me, reducing the number of projections at the end of the input
process would be a regression.
Yuv
awbrody
Both zenith and nadir are problematic to align properly. It's not all
that hard to get the zenith within hugin by manually placing zenith
shots without control points. The nadir, however, is especially
difficult not only to align, but also to photograph since the tripod
and head are in the way. My work-around has been to produce an
equirectangular panorama that does not include the nadir, resize the
canvas in photoshop to the 2:1 aspect ratio and then remap into cubic
faces. I then use photoshop to "fake" the nadir as part of the bottom
cubic face. Then I convert back to equirectangular. The conversions
can be a problem for some codes ( i.e. not using good memory
management as per VIPS) if the panorama is in the gigapixel range. It
would be good to have a part of the workflow in hugin devoted to
nadir. At the very least there should be a projection output for cubic
faces and from cubic faces to equirectangular.
Bill Brody
Yuval Levy
awbrody wrote:
> Re: > Editing the zenith or nadir of a panosphere is a nightmare;
>
> Both zenith and nadir are problematic to align properly.
I was writing of editing, as in masking, paiting, etc. In my use I don't
see any difference in aligning images at Zenith/Nadir or elsewhere on
the screen. Can you elaborate why those images do not align for you?
> It's not all
> that hard to get the zenith within hugin by manually placing zenith
> shots without control points. The nadir, however, is especially
> difficult not only to align, but also to photograph since the tripod
> and head are in the way.
Why do you need to place the Zenith manually? Yes, the tripod is in the
way of photographing the Nadir, but there are ways around this such as
removing the tripod and shooting hand held (and soon, using XYZ
correction). From a stitching workflow perspective (as opposed to a
shooting workflow perspective) Nadir and Zenith are exactly the same
thing. It is one of the unsolved problem of stitching the sphere, to
determine computationally what is up and what is down. This is why
sometimes panoramas stitch upside down.
Yuv
Carl von Einem
adding two more buttons:
- Anchor this image as nadir shot
- Anchor this image as zenith shot
This is a simple step even for newbies and both new anchors would tell
hugin (i.e. the optimiser) that a certain image contains one pixel with
pitch=-90 or pitch=90 respectively.
I think these two new anchors together with at least one pair of
vertical CPs would work great to avoid images flipping over during
optimisation.
Maybe this could even be a replacement for the 'Anchor this image for
position' button.
Yuval Levy wrote:
> It is one of the unsolved problem of stitching the sphere, to
> determine computationally what is up and what is down. This is why
> sometimes panoramas stitch upside down.
Just a crazy thought: how about a special CP for the sun? If I see the
sun in my panorama and simply mark the sun's center with a "directional
CP" this could probably help to find the real North in case the time
(UTC+-...) (and date?) are correct. Maybe a cool addition when several
panoramas should later be linked within a tour -> automated Hotspot
addition...
Carl
awbrody
I take my panoramas in wilderness environments, often in the mountains
where the weather and the light is dynamic. Often the position is
somewhat precarious such as on a glacier near a crevasse. In general I
like to photograph with the camera at my eye level so the final
panorama is my view immersed in the scene. The camera setting is f/13
- f/16 for depth of field at 35mm so I need to take 12 shots for each
row. In practice this takes a few minutes per row since I often need
to use manual focus and take more than one shot to accommodate near
and far imagery. I use a Sony DSC-R1 on manual setting. The final
panorama is 24,000 pixels wide.
It is hard to get good control points for the nadir for a number of
reasons. I sometimes shoot the nadir handheld, in which case the
images are often motion blurred and of course the camera position is
not exactly that of all the other images. I even made a surpension
system for my tripod head, to cantilever my camera and hold it
pointing down. I move the tripod and then approximate the camera
position. The height is close to the original. Even using the
suspension system I find it hard to get good control points unless I
artificially introduce recognizable objects into the scene because
the nadir is often filled with a lot of fine detail like blades of
grass or a field of like-sized pebbles.
As for the zenith, the difficulty lies in that either there are
interesting clouds which means that there is also generally wind and
hence rapid changes in the clouds with corresponding issues in
alignment or the sky is clear and taking a series of featureless blue
gradients seems a waste of time. In that case a multipoint gradient
generator, say a gradient generated between the midpoints of the upper
part of the top row toward a darker blue might be a useful tool.
Photoshop's gradient tool just does not do the job well. Shake has a 4
point gradient that is better, but still inadequate. A better
generator would have a series of perimeter colors plus a central color
Yuval Levy
transform, roll 180°.
http://wiki.panotools.org/Hugin_FAQ#Why_is_my_panorama_upside_down_.3F
Yuv
Yuval Levy
thanks for sharing the detailed information about your workflow.
to my understanding, the issues at Zenith and Nadir are shooting issues,
not stitching issues.
The upcoming new Layout with XYZ parameters may be of help for you with
off-center nadir shots.
To ease manual placement of the nadir / zenith you can numerically
transform the panorama (pitch -90 or + 90), make the placement, and then
numerically transform in the opposite direction (pitch +90 or -90).
What might also be helpful for you is a zoom in the panorama viewer. You
can simulate it currently by shrinking the FOV after pitching the
panorama and before dragging the nadir or zenith shot (which with the
fast preview is really smooth). Once everything is in position, you can
revert to your overall FOV.
But I'm still skeptical about the benefit of manually placing the shots.
Other than that I don't see how the software could help you to a better
/ easier / comfortabler stitch.
Where are can I see your wilderness panos?
Yuv