Tilt transformation...

[D M German]

(this code was brought to you by Google :)

As you might know, Dev Gosh was one of Google Summer of Code
students. Unfortunately he only completed part of his project.

His idea was to allow for the rotation of the camera plane in two axis:
around x, and around y. He did a good job with the math and, except for
some very minor bugs, his code worked as expected.

For example, for the tilt on X, imagine you take a photo, then you apply
a "pitch" rotation (rotation around x), then a translation to recenter
the image on (0,0). the same applies for rotation around Y.

When I integrated his code I discovered that it would be good to add the
ability to rotate around Z, and implemented it accordingly.

Here is in example of the input:

http://turingmachine.org/~dmg/temp/floorInput.jpg

and the output:

http://turingmachine.org/~dmg/temp/floorOutput.jpg

and this is a photo of the same place taken:

http://turingmachine.org/~dmg/temp/floorInput.jpg

In this case, there is a tiltX of 32 degrees, and a tiltZ of -11 and a
shear of -125 pixels.

I have also added 3 parameters to the parser: Tx, Ty, and Tz, which are
angles to rotate in degrees; and Ts, which is a scaling factor.

I have the inverse transformations done. they are described here:

http://turingmachine.org/~dmg/temp/tiltEq.pdf

I am struggling with the inverse ones. If anybody wants to help, I'll be
grateful. This is the only roadblock to have an alpha version of it.

Now, the real question for hugin developers: can somebody create a
branch of hugin that supports these new parameters during the
optimization? shearX, shearY, tiltX, tiltY, tiltZ and tiltScale?

I haven't committed my code but will do soon. in its current state the
optimizer nor the cropping works (we need the forward transform for that).

--
Daniel M. German
http://turingmachine.org/
http://silvernegative.com/
dmg (at) uvic (dot) ca
replace (at) with @ and (dot) with .

[D M German]

>
> and this is a photo of the same place taken:
>
> http://turingmachine.org/~dmg/temp/floorInput.jpg

sorry:

http://turingmachine.org/~dmg/temp/floor.jpg

And we should thank Dev and Google for this feature. I think it will go
a long way into making the nadir photos easier.

[Dale Beams]

Thank you sooo much Dev and Daniel.  I've been waiting for this for soooo long.

[Yuval Levy]

D M German wrote:
> Now, the real question for hugin developers: can somebody create a
> branch of hugin that supports these new parameters during the
> optimization? shearX, shearY, tiltX, tiltY, tiltZ and tiltScale?

how does this interact with James' new layout (the "j" parameters in the
.pto files) ?

Yuv

[dmg]

Is this read by libpano tools? I doubt it. In that case, not at all.

>
> Yuv
>
> >
>



--
--dmg

---
Daniel M. German
http://turingmachine.org

[Yuval Levy]

dmg wrote:
>> how does this interact with James' new layout (the "j" parameters in the
>> .pto files) ?
>
> Is this read by libpano tools? I doubt it. In that case, not at all.

should it be read by libpano tools?

Yuv

[dmg]

Remember that the PTO files are not compatible with libpano. One needs
to use the export feature in
hugin.

[Bruno Postle]

The 'j' parameter is really a hint for future tools that don't exist
yet. Currently in hugin we define a 'stack' purely by looking at
positions - Images that overlap by 90% are a 'stack', the 'j'
parameter would allow this relationship to be explicit rather thn
implicit.

--
Bruno

[Gerry Patterson]

On Sep 9, 2009, at 4:03 PM, D M German <d...@uvic.ca> wrote:

>
>
>
> Now, the real question for hugin developers: can somebody create a
> branch of hugin that supports these new parameters during the
> optimization? shearX, shearY, tiltX, tiltY, tiltZ and tiltScale?
>
>
> I haven't committed my code but will do soon. in its current state the
> optimizer nor the cropping works (we need the forward transform for
> that).
>

Hello,

I am confused. Would one not be able to continue development in the
same branch that Dev made the initial changes in? Why is another
branch needed? I apologize if I am missing something obvious.

Gerry

[dmg]

> Hello,
>
> I am confused.  Would one not be able to continue development in the
> same branch that Dev made the initial changes in? Why is another
> branch needed? I apologize if I am missing something obvious.
>
> Gerry

are you talking libpano or hugin?

-dmg

[Yuval Levy]

Dev never committed anything to
http://hugin.svn.sourceforge.net/viewvc/hugin/hugin/branches/gsoc2009_mosaic/

:-(
Yuv

[Gerry Patterson]

On Sep 9, 2009, at 10:56 PM, dmg <d...@uvic.ca> wrote:

>
>> Hello,
>>
>> I am confused. Would one not be able to continue development in the
>> same branch that Dev made the initial changes in? Why is another
>> branch needed? I apologize if I am missing something obvious.
>>
>> Gerry
>
> are you talking libpano or hugin?
>

> -dmg
>
> -~--~---
>

Hello,

And that would be the obvious something that I was missing. Again, my
apologies. It is my understanding that this code currently lives in
libpano. Correct?

So you are looking for changes to be made to hugin to allow for
selecting these new parameters in the images tab for optimization.
The optimization itself would be continued to be handled by a call to
libpano. Correct?

Best Regards,

Gerry

[Oskar Sander]

Nice work guys!

2009/9/9 D M German <d...@uvic.ca>

Now, the real question for hugin developers: can somebody create a
branch of hugin that supports these new parameters during the
optimization? shearX, shearY, tiltX, tiltY, tiltZ and tiltScale?

So trying to mentally picture these parameters in my head to understand how optimization should be done. 

 When assembling a "mosaic" camera position imagine subject beeing a plane Xs-Ys  with  Zs constant,  and camera moved across a paralell plane  X-Y   (with Z=0).   What would these parameter represent?

*    shearX,shearY   =  camera position in the camera plane, this would be different for each picture taken
*  tiltX,Y,Z.  rotation around the camera node (shearX,shearY), also different for each picture
* tiltScale,  I guess this afect scaling of the picture... (  how does this relate to current FOV or an imaginary shearZ parameter?)


Am I on the right track?     

Cheers
/O

[dmg]

>  When assembling a "mosaic" camera position imagine subject beeing a plane
> Xs-Ys  with  Zs constant,  and camera moved across a paralell plane  X-Y
> (with Z=0).   What would these parameter represent?
>
> *    shearX,shearY   =  camera position in the camera plane, this would be
> different for each picture taken
> *  tiltX,Y,Z.  rotation around the camera node (shearX,shearY), also
> different for each picture
> * tiltScale,  I guess this afect scaling of the picture... (  how does this
> relate to current FOV or an imaginary shearZ parameter?)
>

to be honest, I am not sure how much they will help for a mosaic. The
model for panos in libpano
is still spherical. But it is a matter of trying.

scale scales the tilt operations only. It actually scales the view
point from which the photo was originally taken.

I actually think that libpano needs a plain, simple, scaling option.
If you look at the PDF I sent, the scale
is a factor by which the field of view of the lens is multiplied in
those computations.

As I said, I am not sure what use it might have. But the feature is
there now. Perhaps for mosaic mode we
need to redesign the libpano internal model. we'll see.

--dmg

[Seb Perez-D]

On Wed, Sep 9, 2009 at 23:03, D M German <d...@uvic.ca> wrote:
> I have the inverse transformations done. they are described here:
>
> http://turingmachine.org/~dmg/temp/tiltEq.pdf
>
> I am struggling with the inverse ones. If anybody wants to help, I'll be
> grateful. This is the only roadblock to have an alpha version of it.

Looking at the formulas, I have the impression that the inverse
transformation should not be complicated - but this probably means I
missed something.

The inverse operation of V' = Mx My Mz V is
V = Mz- My- Mx- V' (1)
with Mx- defined as Mx but with a -theta instead of theta (rotation
around an axis can be undone by rotating by the opposite amount). You
just have to be careful with the order of the unrotations.

V' is known up to a multiplicative constant, and is proportional to
(x'/d ; y'/d; 1). So we find V up to a multiplicative constant by the
inverse operation (1) above, and x0 and y0 are obtained by:
x0 = V0 * d / V2
y0 = V1 * d / V2

Does this make sense?

S

[dmg]

Here are examples of the use:

This photo was not remapped:

But this one was:

http://turingmachine.org/~dmg/temp/floorInput.jpg

and this is the output, merged (difference mode):

http://turingmachine.org/~dmg/temp/merged.jpg

It was optimized using:

# specify variables that should be optimized
v Tx1 Ty1 Ts1 g1 t1 p1 r1 y1

using 17 control points in intersections of lines of floor.

And remapped using:

o f0 r-99.5957 p-0.992759 y-3.69248 v94.6652 a0.000000 b0.000000
c0.000000 g-726.252355 t558.903060 Tx54.770184 Ty-45.457790
Tz18.993815 Ts2.536055 d0.000000 e0.000000 u10 +buf

As you can see, the transformation works.

>
> and the output:
>
> http://turingmachine.org/~dmg/temp/floorOutput.jpg
>
> and this is a photo of the same place taken:
>

> http://turingmachine.org/~dmg/temp/floor.jpg

>
> In this case, there is a tiltX of 32 degrees, and a tiltZ of -11 and a
> shear of -125 pixels.
>
> I have also added 3 parameters to the parser: Tx, Ty, and Tz, which are
> angles to rotate in degrees; and Ts, which is a scaling factor.
>
> I have the inverse transformations done. they are described here:
>
> http://turingmachine.org/~dmg/temp/tiltEq.pdf
>
> I am struggling with the inverse ones. If anybody wants to help, I'll be
> grateful. This is the only roadblock to have an alpha version of it.
>
> Now, the real question for hugin developers: can somebody create a
> branch of hugin that supports these new parameters during the
> optimization? shearX, shearY, tiltX, tiltY, tiltZ and tiltScale?
>
>
> I haven't committed my code but will do soon. in its current state the
> optimizer nor the cropping works (we need the forward transform for that).
>
>
> --
> Daniel M. German
> http://turingmachine.org/
> http://silvernegative.com/
> dmg (at) uvic (dot) ca
> replace (at) with @ and (dot) with .
>
>

--

[Bruno Postle]

On Thu 10-Sep-2009 at 02:11 -0700, Daniel M. German wrote:
>
>But this one was:
>
> http://turingmachine.org/~dmg/temp/floorInput.jpg
>
>and this is the output, merged (difference mode):
>
>http://turingmachine.org/~dmg/temp/merged.jpg

That is a 404 error, this looks like the right link:

http://turingmachine.org/~dmg/temp/floorMerged.jpg

Very impressive.

Will it be possible to take the Tx, Ty, Tz & Ts parameters and
calculate a cartesian x, y & z vector between the two camera points?

--
Bruno

[Nicolas Pelletier]

Awesome. This will be quite useful (specially for some nadir shot that I badly took and invested quite some time to redo).

I'm always impressed by the power of this software, and where it is going.

nick

[dev g]

I committed my changes here:
http://panotools.svn.sourceforge.net/viewvc/panotools/branches/libpano_gsoc2009_mosaic/

[dmg]

Thanks Dev for your work.

BTW, this is now integrated into libpano HEAD.

[Oskar Sander]

2009/9/10 dmg <d...@uvic.ca>

to be honest, I am not sure how much they will help for a mosaic. The
model for panos in libpano
is still spherical. But it is a matter of trying.

scale scales the tilt operations only. It actually scales the view
point from which the photo was originally taken.

--dmg

Well, one major problem when "fudging" mosaics currently in Hugin without Dev's addition. Is that yaw, pitch and x,y are not independent of the lens distortion model and the panoramic node(what do you call it?).  Which means that the mosaic quickly gets FU  because: 
       1 Slightest camera tilt causes distortion that cannot be handled by yaw or pitch, as these are applied in the sphere center, not the translated camera position (x,y) and
       2. any x, y translation will move the image from the center within the distortion model and may give horrific results as obviously distortion center in the case of the mosaic application is in the center of each camera viewpoint

In my mind, your addition solves both of these issues, by optimizing your new parameters instead of y,p,x,y. 


Then the panosphere issue would be interesting to discuss. It could maybe overcome if you see each camera position as being closer to the sphere's surface and the final panoramic viewpoint further back...


Cheers
/O

[Oskar Sander]

Dmg,

If I am running an optimization using Autooptimize using the new Tilt parameters in the new panotools version, what is the parameter syntax in the pto-file?

Cheers
/O

2009/9/11 Oskar Sander <oskar....@gmail.com>

--
/O