Request for help on a perspective projection problem.

[step7]

I have split a photograph into 3 layers which represent the foreground, midground and background planes of the image. I'd like to reconstruct the image by shifting the layers left and right to obtain a representation of how our left and right eyes would see the same view. I know just shifting the layers is over simplistic, as view angles should change too, allowing me to see the front of a building in one eye and possibly some part of the side of the building in the other - but ignoring this for now to keep things simple how do I calculate the horizontal shift required in each plane?

I think I'm correct in making the following assumptions:

The two views are 70mm apart (the distance between my eyes).

If I set the midground as the focal plane(?), then that plane doesn't move at all.

Foreground and background planes will move in proportion to their distance (which I can guess at) from the focal plane..

..but by how much?

Any help would be much appreciated.

[step7]

Think I've figured it out by rereading my assumptions - just hadn't completed the mental picture.

The offset to each eye is 0.035m, this is the opposite side of a right angled triangle, with the distance (in this case 500m) to the focal plane being the adjacent. The shift required on any plane in each view is the opposite on a similar triangle, so will be = 0.035/500 * distance of that plane from the focal plane. So 0.0175m shift if it's is 250m from the focal plane. Question is now, can I scale off the focal plane to convert the shift required to a number of pixels?

[Edgar Bonet]

Hi!

On 02 Jul 2012, step7 wrote:
> I have split a photograph into 3 layers which represent the foreground,
> midground and background planes of the image. I'd like to reconstruct the image
> by shifting the layers left and right to obtain a representation of how our

> left and right eyes would see the same view. [...] how do I calculate

> the horizontal shift required in each plane?
>
> I think I'm correct in making the following assumptions:
> The two views are 70mm apart (the distance between my eyes).
> If I set the midground as the focal plane(?), then that plane doesn't move at all.
> Foreground and background planes will move in proportion to their distance
> (which I can guess at) from the focal plane..
>
> ..but by how much?

First you have to figure out the scale of your midground plane, i.e. how
to translate pixels to meter in that plane. This should be easy if you
have a face. Let's say

35 mm = N pixels (in the midground plane).

then, shift each plane by

shift_in_pixels = N * (d_m/d - 1)

where d is the distance from the camera to that particular plane and d_m
the distance from the camera to the midground plane.

Regards,

Edgar.

[step7]

Thanks for taking the time to reply Edgar, really appreciate it!

Your formula looks straight forward enough for me to apply and write some code to.   I'm struggling to make sense of how it works though and my brain starts to ache when I try and visualise what's going on. First surprise is that the relationship between the distance from the camera to a plane and the dimensions represented in that plane isn't linear. Also struggling with why I'd get infinities if I had a plane 0m from the camera.

Another question is what effects are depth of field* and foreshortening* having here? Is FOV automatically accommodated when I calculate the scale of the midground plane? Would it be possible to simulate the effect of changing the FOV by scaling and shifting each plane?

*I must admit I don't fully understand these terms but have a feeling I may real soon!

I'll go try and read up to better understand some basic principles here but would appreciate any light you can shed.

Thanks again :)

[Edgar Bonet]

Hi!

On 2 Feb 2012, step7 wrote:
> First surprise is that the relationship between the distance from the camera
> to a plane and the dimensions represented in that plane isn't linear.

I am not sure to understand.

The magnification is defined as the ratio of the dimensions on the image
to the dimensions on the object plane. It is roughly

M = f / d

where d is the distance from the camera to the object plane and f the
focal length of the lens. Beware that this formula is not valid for
close-ups, i.e. when d is of the same order as f.

If by "the dimensions represented in that plane" you mean the inverse
magnification, then yes, it *is* a linear function of object distance.

The shift you have to apply to the plane at distance d would be
proportional to (d_m - d) if computed as a dimension in that particular
plane. But you then have to multiply by the magnification to get the
shift in image space. That's how you get something proportional to
(d_m/d - 1).

> Also struggling with why I'd get infinities if I had a plane 0m from the
> camera.

Look at the upper edge of your nose. No, not in a mirror, that is
cheating. ;-) You see that it's very far to the left, at the edge of
your field of view... when viewed through the right eye. And very far to
the right, when viewed through your left eye. And yet it is not on the
same plane as your eyes, it's still in front of it.

> Another question is what effects are depth of field* and foreshortening*
> having here?

Depth of field has the effect of blurring things that are not in the
plane of best focus. If you want to simulate the depth of field of the
human eye, you should make sure that your lens is opened at the same
aperture as your eye: same diameter of the entrance pupil. This diameter
depends on the light level. If the picture was taken with too-big an
aperture, you cannot fix it. It it was taken with a too small aperture,
you may want to add some extra blurring.

But I do not think that simulating the depth of field of the eye is a
good idea. When you look at a scene, you actually scan it with your
eyes, and automatically refocus along the way. Then you hardly see
anything blurry. This is in contrast to a camera that takes a whole
picture with a single focus setting.

As for foreshortening, this is already in your picture. Unless you are
talking about the difference between the foreshortenings of the left and
right eye, which would be very difficult to reproduce from a single 2D
picture.

> Is FOV automatically accommodated when I calculate the scale of the
> midground plane?

Err... is there something in the FoV that has to be "accommodated"?

> Would it be possible to simulate the effect of changing the FOV by scaling
> and shifting each plane?

You can reduce the field of view by cropping. You cannot increase it
once the picture is taken.

Edgar.

[step7]

>The shift you have to apply to the plane at distance d would be 

>proportional to (d_m - d) if computed as a dimension in that particular 
>plane. But you then have to multiply by the magnification to get the 
>shift in image space. That's how you get something proportional to 

?(d_m/d - 1)

This really clears a lot of confusion up for me and answers my original questions :)

>Look at the upper edge of your nose..

Yes sat here playing with lots of pencils, fingers etc. close to my eyes trying to understand this earlier - hope my neighbours haven't seen me they'll think I'm mad. Also been contemplating, "How far away is the sky?" - if you can see the relevance of this to my problem.

>When you look at a scene, you actually scan it with your 
>eyes, and automatically refocus along the way. 

That's interesting, I have been compositing planes from different photographs, with each plane in good focus - but the results don't look hugely odd - probably because this is the way we (think) see the world anyway.

>> Is FOV automatically accommodated when I calculate the scale of the 
>> midground plane? 

>Err... is there something in the FoV that has to be "accommodated"?  

I meant the effects of changing the FoV, but what I really should have said was effects like foreshortening. If I have understood what you have said correctly, I don't need to be concerned with these effects in the photo and can just follow the method and formula you gave.

Time to do some mathmapping.

Thanks again Edgar - you must feel like you've been writing a dummies guide for me!