"Skybox" output projection?

137 views
Skip to first unread message

Johan Kiviniemi

unread,
Feb 26, 2008, 10:51:19 AM2/26/08
to hugi...@googlegroups.com
Hi,

Would it be feasible to implement an alternative “projection” for full
sphere panoramas that would actually consist of six 90°×90° projections,
equivalent to the faces of a cube with the center of the cube as the
point of view?

http://heh.fi/tmp/skybox_faces
http://heh.fi/tmp/skybox_faces_alternative

If enblend and enfuse were modified to understand the proposed format
(each face’s border wraps to another face in the image), the
zenith/nadir problem could be avoided completely.

Additionally, if the format happens to be suitable for whatever is used
to view the resulting panorama, the image’s pixel estate would be more
evenly spent compared to an equirectangular projection.

Sincerely,
--
Jοhan Kiviniemi http://johan.kiviniemi.name/

signature.asc

Carl von Einem

unread,
Feb 26, 2008, 11:11:28 AM2/26/08
to hugi...@googlegroups.com
The "alternative" format has a width/hight ratio of 3/2, the same ratio
of files with equirectangular projection. I'm pretty sure this wold
confuse a lot of applications.

Carl

Seb Perez-D

unread,
Feb 26, 2008, 11:18:27 AM2/26/08
to hugi...@googlegroups.com
On Tue, Feb 26, 2008 at 5:11 PM, Carl von Einem <ca...@einem.net> wrote:
>
> The "alternative" format has a width/hight ratio of 3/2, the same ratio
> of files with equirectangular projection. I'm pretty sure this wold
> confuse a lot of applications.

Equirectangular is 2:1, so no, this would not confuse the programs


> Johan Kiviniemi wrote:
> > Would it be feasible to implement an alternative "projection" for full
> > sphere panoramas that would actually consist of six 90°×90° projections,
> > equivalent to the faces of a cube with the center of the cube as the
> > point of view?

There is already such a projection, except it is 6:1. Spi-V treats any
image with such aspect ratio as cubic.
http://wiki.panotools.org/Panorama_formats#Cubic
http://www.fieldofview.com/spv-dev/docs/howto/preparing

I think that making enfuse/enblend cope with the border problems in
the cubic projection is not simpler than making them understand
spherical projections directly.

Cheers,

Seb

Yuval Levy

unread,
Feb 26, 2008, 11:18:18 AM2/26/08
to hugi...@googlegroups.com
the ratio of equirect is 2/1 ...

and "skybox_faces" is already very well known and discussed as
"cubefaces" ...

now let's see if I add to the fun or to the confusion:

<humor>
IMO this thread deserves the nomination for best thread in the category
"I wish I had a coffee before processing e-mail" =8^D
</humor>

Yuv

Seb Perez-D

unread,
Feb 26, 2008, 11:38:54 AM2/26/08
to hugi...@googlegroups.com
On Tue, Feb 26, 2008 at 5:18 PM, Yuval Levy <goo...@levy.ch> wrote:
> <humor>
> IMO this thread deserves the nomination for best thread in the category
> "I wish I had a coffee before processing e-mail" =8^D
> </humor>

Come on Yuv, have a coffee !

Obligatory smiley to add even further to the confusion ;-)

Carl von Einem

unread,
Feb 26, 2008, 11:54:25 AM2/26/08
to hugi...@googlegroups.com
You're right. In fact I was already on my way to the coffee machine...

blush...

Tom Sharpless

unread,
Feb 26, 2008, 5:58:09 PM2/26/08
to hugin and other free panoramic software
Come on, guys...

Johan's proposal is actually very sensible. There are several good
reasons why it makes better sense to stitch spherical panos direct to
the "skybox" aka "panocube" faces than to an equirectangular
projection.


1) 99% of spherical images are going to end up as QTVR's. And IMHO
QTVR is the best 'preview' format for spherical or other very wide
angle panos. And a QTVR is just a decorated jpeg panocube.

2) It _does_ solve the zenith/nadir blending problem. Each face is a
simple rectilinear stitching job, piece of cake for enblend or
anything else. Don't worry about the cube edges spoiling things, just
stitch a little extra all around and saw it off after blending.

3) Remapping between circularly symmetric formats (e.g. nearly all
camera images and the cube faces) is both faster and less subject to
aliasing problems than mapping the same images to equirectangular. So
this path can lead to quicker and/or more accurate results.

I say it should be a hugin development goal.

Cheers, Tom

Bruno Postle

unread,
Feb 26, 2008, 6:37:35 PM2/26/08
to Hugin ptx
On Tue 26-Feb-2008 at 14:58 -0800, Tom Sharpless wrote:
>
>1) 99% of spherical images are going to end up as QTVR's. And IMHO
>QTVR is the best 'preview' format for spherical or other very wide
>angle panos. And a QTVR is just a decorated jpeg panocube.

Creating spherical panoramas is fairly extreme behaviour, I imagine
that most hugin users are creating single wide-angle images intended
to be seen as a complete composition.

>2) It _does_ solve the zenith/nadir blending problem. Each face is a
>simple rectilinear stitching job, piece of cake for enblend or
>anything else. Don't worry about the cube edges spoiling things, just
>stitch a little extra all around and saw it off after blending.

I'd like to see some tests, yes the cube edges should blend
similarly to the left/right edges of a 360 panorama, but I'm not
sure what will happen around the corners.

>3) Remapping between circularly symmetric formats (e.g. nearly all
>camera images and the cube faces) is both faster and less subject to
>aliasing problems than mapping the same images to equirectangular. So
>this path can lead to quicker and/or more accurate results.

What problem is being solved here? I don't see aliasing issues
mapping to and from equirectangular. Scaling equirectangular images
can be problematic, but so can scaling cube-faces (see any QTVR
preview track for evidence - and the extra work erect2qtvr does to
work around the problem).

With a cubic format, you still won't be able to open a single file
in photoshop and apply a large radius unsharp mask.

--
Bruno

Yuval Levy

unread,
Feb 26, 2008, 6:40:28 PM2/26/08
to hugi...@googlegroups.com
Maybe the :-) where not prominent enough...

Tom Sharpless wrote:
> Johan's proposal is actually very sensible.

nobody said it's not. AFAIK it has been discussed a few weeks ago by
different people under a different name.


> 1) 99% of spherical images are going to end up as QTVR's

as cubefaces / or stripes. Without starting a format war, it is not only
Quicktime. Cubefaces are the optimal format for interactive display,
used by SPi-V/Shockwave, Flashpanoramas, and many others.

Note that QTVR is not exactly cubefaces. IIRC, there is a one pixel
overlap on each side of each cubeface in the QTVR specs.


> 2) It _does_ solve the zenith/nadir blending problem.

you're right again. Andrew Mihal commented in that sense:
<http://groups.google.com/group/hugin-ptx/browse_thread/thread/a90169a794bb69d6/b63898bc952a02b3?lnk=gst&q=andrew+mihal#b63898bc952a02b3>


> 3) Remapping between circularly symmetric formats (e.g. nearly all
> camera images and the cube faces) is both faster and less subject to
> aliasing problems than mapping the same images to equirectangular. So
> this path can lead to quicker and/or more accurate results.

still, equirectangular "looks" more natural. It is easier to spot
stitching errors without being confused by the edges of the cube. I
personally need both formats.

I edit zenith and nadir in cubeface and I edit the 360° around me in
equirect.

And ah, not in QTVR - because QTVR comes after post-production / editing.


> I say it should be a hugin development goal.

what should be a hugin development goal and how would you spell it out
into specific functionalities?

I currently output the cubefaces straight out of hugin. I could see
slight user interface improvements, but I don't see it as such a radical
change. Am I missing something?

What I do is set the parameters right in the Stitcher tab, for the
Panorama type (rectilinear), the FOV (90°x90°) and the Canvas Size. Then
I use the numerical transform in the panorama preview window to point
the view up and down for zenith and nadir. Could also do the four side
faces if needed.

I'd love an automatism in the stitcher tab to create zenith and nadir
cubefaces without tweaking the numerical transform and the stitcher tab
every time, but even more pressing for me would be to have the layered
output go straight into a PSD file for post-processing in Photoshop.

Anyway, if you would like to give it a try, you have more C++ skills
than me, add a table to the stitcher tab. Call it cubefaces. headings +
six lines.

The six lines are called Zenith/Nadir/Front/Left/Right/Back.

On each line, a radio button to decide if that face should be output or
not. And the resolution for the faces (only one number, since
width=height, in pixes). Then a few columns for the output - eight if we
take the current output section of the stitcher tab). Then my dream
radio button: merge into a single multi-layered PSD.

Then hugin can programmatically extract the views at +90 pitch, -90
pitch, 0pitch {0,90,180,-90yaw}.

Yuv

Yuv

Yuval Levy

unread,
Feb 26, 2008, 6:42:37 PM2/26/08
to hugi...@googlegroups.com
Bruno Postle wrote:
> With a cubic format, you still won't be able to open a single file
> in photoshop and apply a large radius unsharp mask.

which brings a related question: does it make sense to apply such a
transform on an equirectangular? I mean, with the extrem stretches at
zenith and nadir, I would intuitively not apply noise filters to the
equirect. Same for sharpening?

Yuv

Erik Krause

unread,
Feb 26, 2008, 7:10:52 PM2/26/08
to hugin-ptx

I large radius USM does not sharpen, it enhances local contrast.
However, although I know that there should be problems I actually
didn't ever see some at zenith and nadir. There is a problem with
large radius USM at the 360° seam, if there are structures with very
different brightness just behind the seam.

Anyway: It's best to apply a basic sharpening before stitching. This
removes lens and bayer interpolation blur. Then probably only an anti
alaising slightly sharpening interpolator like Lanczos3 or
Blackman/Sinc is necessary for the final output, which (if I
understood that correctly) takes care of the source mapping.

BTW.: Does hugin/nona/PTMender support the anti alaising filters
meanwhile? My current snapshot (0.7.0.2825) lacks of a possibility to
choose an interpolator...

best regards

Erik Krause
http://www.erik-krause.de

Yuval Levy

unread,
Feb 26, 2008, 7:24:51 PM2/26/08
to hugi...@googlegroups.com
Erik Krause wrote:
> My current snapshot (0.7.0.2825) lacks of a possibility to
> choose an interpolator...

I am afraid this has been disabled in the user interface which needs
some rewriting that is likely to happen after 0.7.0 release.

Yuv

Tom Sharpless

unread,
Feb 27, 2008, 8:20:39 PM2/27/08
to hugin and other free panoramic software


On Feb 26, 6:40 pm, Yuval Levy <goo...@levy.ch> wrote:
> Maybe the :-) where not prominent enough...
>
> Tom Sharpless wrote:
> > Johan's proposal is actually very sensible.
>
> nobody said it's not. AFAIK it has been discussed a few weeks ago by
> different people under a different name.
>
> > 1) 99% of spherical images are going to end up as QTVR's
>
> as cubefaces / or stripes. Without starting a format war, it is not only
> Quicktime. Cubefaces are the optimal format for interactive display,
> used by SPi-V/Shockwave, Flashpanoramas, and many others.
>
OK, 99% of spherical images are going to ende up as cubes.

> Note that QTVR is not exactly cubefaces. IIRC, there is a one pixel
> overlap on each side of each cubeface in the QTVR specs.

For a cubic interpolation at the edge, I guess. More overlap would
allow bigger interpolation kernels.
>
> > 2) It _does_ solve the zenith/nadir blending problem.
>
> you're right again. Andrew Mihal commented in that sense:
> <http://groups.google.com/group/hugin-ptx/browse_thread/thread/a90169a...>
>
> > 3) Remapping between circularly symmetric formats (e.g. nearly all
> > camera images and the cube faces) is both faster and less subject to
> > aliasing problems than mapping the same images to equirectangular. So
> > this path can lead to quicker and/or more accurate results.
>
> still, equirectangular "looks" more natural. It is easier to spot
> stitching errors without being confused by the edges of the cube. I
> personally need both formats.
>
> I edit zenith and nadir in cubeface and I edit the 360° around me in
> equirect.

Or cylindrical. Perfectly sensible. However I seem to spot problems
better while spinning a live spherical view (not to say "QTVR" :>)
>
> And ah, not in QTVR - because QTVR comes after post-production / editing.
>
PTGui puts out QTVR previews. Hugin could too. In fact hugin needs a
more convenient way to make previews of all kinds.

> > I say it should be a hugin development goal.
>
> what should be a hugin development goal and how would you spell it out
> into specific functionalities?

1) One or more cubic VR formats as ldr output options (incl. low res
previews)
2) cube-face set as an output option, including all the layered,
merged, blended, etc varieties. These should be a little larger than
the strict cube faces to allow filtering across the cube edges
without creating artefacts.
3) convert (externally edited) cube face sets to cubic VR format(s),
with clipping, merging and tone mapping as needed.
4) convert cube face set to cube face set at a different rotational
orientation.
>
> I currently output the cubefaces straight out of hugin. I could see
> slight user interface improvements, but I don't see it as such a radical
> change. Am I missing something?
>
> What I do is set the parameters right in the Stitcher tab, for the
> Panorama type (rectilinear), the FOV (90°x90°) and the Canvas Size. Then
> I use the numerical transform in the panorama preview window to point
> the view up and down for zenith and nadir. Could also do the four side
> faces if needed.
>
> I'd love an automatism in the stitcher tab to create zenith and nadir
> cubefaces without tweaking the numerical transform and the stitcher tab
> every time, but even more pressing for me would be to have the layered
> output go straight into a PSD file for post-processing in Photoshop.
>
> Anyway, if you would like to give it a try, you have more C++ skills
> than me, add a table to the stitcher tab. Call it cubefaces. headings +
> six lines.
>
> The six lines are called Zenith/Nadir/Front/Left/Right/Back.
>
> On each line, a radio button to decide if that face should be output or
> not. And the resolution for the faces (only one number, since
> width=height, in pixes). Then a few columns for the output - eight if we
> take the current output section of the stitcher tab). Then my dream
> radio button: merge into a single multi-layered PSD.
>
> Then hugin can programmatically extract the views at +90 pitch, -90
> pitch, 0pitch {0,90,180,-90yaw}.
>
Yes. I think it needs to be easier to to "extract views" of all
kinds. That was one of the old PanoTools virtues that seems to have
been de-emphasized lately. I often reproject my rotating camera panos
to make nicer prints, and I find the PT plugins a bit more convenient
than hugin for that

Best regards, Tom

Tom Sharpless

unread,
Feb 27, 2008, 8:33:15 PM2/27/08
to hugin and other free panoramic software
Good point. The equirectangular and cylindrical formats are not
locally isotropic aka conformal, so the usual separable convolution
filters are guaranteed to produce artefacts. You may not notice them
much except near the edges, but they are measurable everywhere. It is
also very hard to do clean antialiasing on such grids (I used to write
medical image processing S/W for a living, and had to worry about such
things).The best bet is to stick to conformal mappings (rectilinear,
stereographic) and avoid extreme stretches by processing wide angle
images in pieces (such as cube faces).

Cheers, Tom

Ken Warner

unread,
Feb 27, 2008, 8:39:53 PM2/27/08
to hugi...@googlegroups.com
When I wrote my cylindrical and spherical viewers, I
had to combine interpolation with projection. Interpolating
before or after projection just didn't work. I'm talking
about projecting cylindrical and equirectangular images.

I had to interpolate each pixel as I projected it. A real
challenge to get any kind of performance out of the code.

Seb Perez-D

unread,
Feb 28, 2008, 1:33:28 AM2/28/08
to hugi...@googlegroups.com
On Thu, Feb 28, 2008 at 2:33 AM, Tom Sharpless <TKSha...@gmail.com> wrote:
> The best bet is to stick to conformal mappings (rectilinear,
> stereographic) and avoid extreme stretches by processing wide angle
> images in pieces (such as cube faces).

Just for the record, rectilinear is not conformal. Stereographic, yes.
Mercator, yes too.

Cheers,

Seb

Tom Sharpless

unread,
Feb 29, 2008, 11:59:55 AM2/29/08
to hugin and other free panoramic software


On Feb 28, 1:33 am, "Seb Perez-D" <sbprzd+...@gmail.com> wrote:
> On Thu, Feb 28, 2008 at 2:33 AM, Tom Sharpless <TKSharpl...@gmail.com> wrote:
> > The best bet is to stick to conformal mappings (rectilinear,
> > stereographic) and avoid extreme stretches by processing wide angle
> > images in pieces (such as cube faces).
>
> Just for the record, rectilinear is not conformal. Stereographic, yes.
> Mercator, yes too.
>
Thanks, Seb, I retract that claim. Most rectlinear lenses are narrow
angle and show little distortion of line intersection angles, but
really wide one do. And of course you can generate really, really
wide rectilinear images on a computer and see it clearly.

So stereographic and Mercator are where it's at. You can map a sphere
conformally with just two stereographic images of hemispheres or -- as
Bruno has recently pointed out to me -- two Mercator images wrapped
around at right angles like the halves of a tennis ball. In either
case the maximum "stretch factors" (I really dislike the term
"distortions") are tolerable; and of course can be reduced as much as
you like by using more images.

-- Tom

Tom Sharpless

unread,
Feb 29, 2008, 12:17:07 PM2/29/08
to hugin and other free panoramic software
PS the stereographic projection, being azimuthal (radially symmetric),
offers a considerable speed advantage for photographic work. Mapping
coordinates between azimuthal projections only involves stretching the
radius, and it is feasible to tabulate the function relating the two
radii. If fully optimized this reduces projecting a point to nine
multiplications, ten additions and a square root.

> -- Tom
Reply all
Reply to author
Forward
0 new messages