Managing stitching for stereo

[GregoryA]

Hi,

What will be the right way to stitch the stereo panoramas in PTGui ?

I will use two cameras for L and R view. 

I understand that when I will stitch the R view panorama, I can use it like template to process the L- View  panorama,

but both panoramas are shifted from the center in the opposite directions?

Thank you,

Gregory

[Jim Watters]

Add all images into PTGui.

Two Cameras and Two lenses will have two although close but different lens
settings. It is best to precalculate these and use them again.

Add normal control points between center of the images making up each pano.
Add horizontal control points between L & R pano and out at the edges.

I first add control points to all images. Mark those as all horizontal.
Use Mask to hide the L and leave a band down the center of right. This will add
normal control point to only the right. Do the same for the left.

Add a couple normal control point on the distant horizon between L & R to keep
them aligned.

Optimize | Hor / Vert Control Points = Include in single pass.

Disable all the left and stitch the right.
Disable all the right and stitch the left.

Jim

--
Jim Watters
http://photocreations.ca

[Grigoriy360 .]

Hi Jim,

I don't have the right words to express my feelings, just WOW!

This is so interesting! I will try it ASAP.

For the first test I don't have the second camera, I just offset the vertical bar (5cm).

What do you mean " leave a band down the center of right"?

Thank you,

Gregory

--
--
You received this message because you are subscribed to the Google Groups "PTGui" group.
To post to this group, send email to pt...@googlegroups.com
To unsubscribe from this group, send email to ptgui+unsubscribe@googlegroups.com
Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at http://groups.google.com/group/ptgui

--- You received this message because you are subscribed to a topic in the Google Groups "PTGui Support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ptgui/miTMKr7lw88/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ptgui+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Roger D Williams]

Jim,

I am delighted to have detailed explanations of how to stitch stereo panoramas from twin cameras but disappointed that I can't follow them. Duh! Can you think how to explain to a bear of very little brain? If that's too vague, I can spell out the bits I don't understand...

Roger W

Sent from my iPad

> --
> --
> You received this message because you are subscribed to the Google Groups "PTGui" group.
> To post to this group, send email to pt...@googlegroups.com

> To unsubscribe from this group, send email to ptgui+un...@googlegroups.com

> Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
> For more options, visit this group at http://groups.google.com/group/ptgui
>

> --- You received this message because you are subscribed to the Google Groups "PTGui Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+un...@googlegroups.com.

[Jim Watters]

Roger,

Here is a video showing the process. The quality is really bad. The screen
capture program I tried made it low res and heavily compressed. I'll need to do
this again. But if you are familiar with PTGui I hope you can follow along.
https://youtu.be/5dsqJLjisRs

Jim

Jim Watters
http://photocreations.ca

[Roger D Williams]

Jim,

I'm most grateful, and will do my best to follow the video. At the moment I don't have two cameras, so it's a bit academic. I did have two Pentax cameras but traded them in for my shiny new Sony A6000. Convincing my wife I need a second one, with nothing left to trade in for it, will be a bit of a stretch!

Roger W.

Sent from my iPad

[Roger D Williams]

Wow! First reaction: on reading your original note I thought I understood the general idea but didn't follow some of the details. Having seen the video, I now see I had almost no understanding of how you were going about it, and that there are LOTS of details I will need to master. Some of the ways you are using PTgui are unfamiliar to me, and as you say, the screen resolution is not good, so it's hard to find the same screen and pull-down menus on my version. But at least I have somewhere to start, now! Thanks again!! After wanting to do it, and talking about it for years, I am really enjoying the challenge of producing stereo panoramas.

One question. What do you mean by "horizontal control points"?

Roger W

Sent from my iPad

> On Mar 15, 2015, at 11:08 PM, Jim Watters <jwat...@photocreations.ca> wrote:
>

[Jim Watters]

Roger,

Panotools and PTGui can use normal, horizontal and vertical control points.
Vertical control points are often used to level a pano by putting them on
objects that are vertical in real life. Horizontal control points allow changes
in the horizontal movement but produce an error if any vertical movement is
detected.

Normal control points use both the error in horizontal and vertical.

When shooting stereo you need to keep your cameras separated horizontal from
each other. Both L&R can not be shot rotated around the NPP. The parallax error
of this will cause a shift between near and far objects between the two views on
the horizontal axis. By using horizontal control points it allows the left and
right pano to maintain a good stereo alignment. Makes the two views to stitch
well even though they were not shot rotated around the NPP.

I don't have two cameras and two lenses. I have also shot a stereo pano by
shooting my normal process with my Nikon D7000 and 10.5mm fisheye of 6 around
+Z+N. I them shift the camera on the rail and shoot another pano of 18 around
off the NPP. I use the N&Z of first pano in the second. Most people are right
eye dominate so I make the perfect pano for the right eye and shift the cameras
to the left for the other stereo pair. The nice thing about it is you get a good
mono version at the same time.

Here is the anaglyph version. I had to replace the distant smoke so they were
the same in L&R.
http://photocreations.ca/home2/Spruce_Ave_Snow.html#node3

The disadvantage of not using two cameras is the inability to freeze motion.

Jim

Jim Watters
http://photocreations.ca

[Roger Williams]

Aha! BIG AHA!!

I do know about horizontal and vertical line control points but I also knew from experience that the horizontal line normally has to be the true horizon, as this is the only line that is straight in an equirectangular image. But I now see that they are perfectly capable of identifying points that DO lie along straight lines corresponding to the linear separation of the cameras. Nice miscommunication!

Now I think I've got it. I imagined you firing away with twinned cameras. Now I know you are like me, faking it with a single camera, I am greatly encouraged. I have been using (or rather trying to use) Wim's technique with a single camera masking out all but a wedge on the LH and RH sides of the fisheye images. Still not there yet. It requires rotation in a true horizontal plane as near perfectly as possible. I am not nearly there with my shaky camera-perched on-monopod-on-top-of-rotator-on-a-mini-tripod.

I do have a slide bar that will allow me to move one camera the 6.5 or 7cm needed.

I am tempted to use a larger, firmer tripod than my normal setup to achieve more reproducible imaging, although I'm too old to want to lug lots of heavy stuff around. Wife does not take kindly to being my Sherpa. I am thinking of using P2VR to limit the angles of view so that no-one gets to see the messes I am leaving at the zenith. There really is very little a most zeniths, and my feeling is that the "wow" of the stereo effect should be enough to please people even if they can't look straight down... How do you feel about that? In the coming era of immersive VR it may be important to allow people to look all around, I suppose. But maybe by then Joost will have developed 3D image field techniques that can solve the parallax problems of 3D stitching? (hint, hint!)

Roger W.

PS Apologies to people for this extensive exchange on a minority interest. It's not exactly Off Topic, but maybe we should take it to Personal Mail?

-----Original Message-----
From: pt...@googlegroups.com [mailto:pt...@googlegroups.com] On Behalf Of Jim Watters
Sent: Monday, March 16, 2015 10:49 AM
To: pt...@googlegroups.com
Subject: Re: [PTGui] Managing stitching for stereo

Roger,

Panotools and PTGui can use normal, horizontal and vertical control points.
Vertical control points are often used to level a pano by putting them on objects that are vertical in real life. Horizontal control points allow changes in the horizontal movement but produce an error if any vertical movement is detected.

Normal control points use both the error in horizontal and vertical.

When shooting stereo you need to keep your cameras separated horizontal from each other. Both L&R can not be shot rotated around the NPP. The parallax error of this will cause a shift between near and far objects between the two views on the horizontal axis. By using horizontal control points it allows the left and right pano to maintain a good stereo alignment. Makes the two views to stitch well even though they were not shot rotated around the NPP.

I don't have two cameras and two lenses. I have also shot a stereo pano by shooting my normal process with my Nikon D7000 and 10.5mm fisheye of 6 around
+Z+N. I them shift the camera on the rail and shoot another pano of 18

+Z+around

[Grigoriy360 .]

Hi Jim,

Thank you for the video. It helps allot, but it will take some time to practice this method. For the fast test, I just export the L and  R panorama after aliment and had this result.
​

What to do if there is not the obvious horizontal line?  

Best,

Gregory

--
--
You received this message because you are subscribed to the Google Groups "PTGui" group.
To post to this group, send email to pt...@googlegroups.com

To unsubscribe from this group, send email to ptgui+unsubscribe@googlegroups.com

Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at http://groups.google.com/group/ptgui

--- You received this message because you are subscribed to a topic in the Google Groups "PTGui Support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ptgui/miTMKr7lw88/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ptgui+unsubscribe@googlegroups.com.

[zakato]

Roger, Jim, Gregory,

 Do please keep this discussion open (it is actually a very trending topic with the appearance of all these head mounted display and cardboard googles) and your ideas and info are definitely helping others a lot (i.e. me :)

Thanks for the great info Jim, all your generous sharings are really helpfull.

Thanks,

Antonio @zakato360  

[Grigoriy360 .]

I agree with Antonio, let keep this discussion open.

You received this message because you are subscribed to a topic in the Google Groups "PTGui Support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ptgui/miTMKr7lw88/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ptgui+un...@googlegroups.com.

[Jim Watters]

Roger,
I agree with the others, lets keep the discussion here.
I have asked to give a talk on stereo stitching at the IVRPA meeting in Prague
http://ivrpa.org/event/prague-2015/

On 2015-03-16 2:34 AM, Roger Williams wrote:
> Aha! BIG AHA!!
>
> I do know about horizontal and vertical line control points but I also knew from experience that the horizontal line normally has to be the true horizon, as this is the only line that is straight in an equirectangular image. But I now see that they are perfectly capable of identifying points that DO lie along straight lines corresponding to the linear separation of the cameras. Nice miscommunication!
>
> Now I think I've got it. I imagined you firing away with twinned cameras. Now I know you are like me, faking it with a single camera, I am greatly encouraged. I have been using (or rather trying to use) Wim's technique with a single camera masking out all but a wedge on the LH and RH sides of the fisheye images. Still not there yet. It requires rotation in a true horizontal plane as near perfectly as possible. I am not nearly there with my shaky camera-perched on-monopod-on-top-of-rotator-on-a-mini-tripod.

I think a sturdy tripod is a must.

> I do have a slide bar that will allow me to move one camera the 6.5 or 7cm needed.

My example of the back deck only had about 25mm separation. I hope to experiment
with different separations.

> I am tempted to use a larger, firmer tripod than my normal setup to achieve more reproducible imaging, although I'm too old to want to lug lots of heavy stuff around. Wife does not take kindly to being my Sherpa. I am thinking of using P2VR to limit the angles of view so that no-one gets to see the messes I am leaving at the zenith. There really is very little a most zeniths, and my feeling is that the "wow" of the stereo effect should be enough to please people even if they can't look straight down... How do you feel about that? In the coming era of immersive VR it may be important to allow people to look all around, I suppose. But maybe by then Joost will have developed 3D image field techniques that can solve the parallax problems of 3D stitching? (hint, hint!)

I think you want to still look down. Having edges and boundaries takes away from
the immersion. When looking on a HMD I had to extend the tilt so I could look
past the zenith and nadir. That can be disorienting on the monitor with a mouse
but with a HMD your head know where it is looking and having a limit is very
distracting.
If I had a wider fisheye lens them I would extend the stereo further. But I
limit it to what my 10.5 on cropped sensor can see. I could shoot two rows but
that is starting to get complicated. Anyone serious about doing stereo I think
should use two cameras to freeze motion.

>
> Roger W.
>
> PS Apologies to people for this extensive exchange on a minority interest. It's not exactly Off Topic, but maybe we should take it to Personal Mail?
>

[Karel Gillissen]

Ah, that is great, Jim.

Looking forward to your talk.

I am currently busy with a dual-camera setup, and trying to find out the minimum number of shots needed for an acceptable pano. 

From the top of a series 2 pole in the open field I think I can manage with the same number of shot as with a normal non-stereo pano. But more testing is needed here... 

Karel

360vr.nl

On Tuesday, March 17, 2015 at 11:07:01 PM UTC+1, Jim wrote:

Roger,
I agree with the others, lets keep the discussion here.
I have asked to give a talk on stereo stitching at the IVRPA meeting in Prague
http://ivrpa.org/event/prague-2015/

[Jim Watters]

Gregory,

You don't need any horizontal lines in the pano. Horizontal line control points
were originals called vertical control points because they only take in account
the vertical error. They allow the points to be a horizontal distance from each
other and allow to create a horizontal line. They optimize to have the least
amount of vertical error. That is all we are interested in keeping correct when
aligning the L&R pano together.

Because of the parallax error between the L&R camera when the pictures were
taken and because the images were not taken rotated around the NPP there will be
parallax error. But because the lenses will be perfectly horizontal to each
other there wont be any vertical error. The optimizer does not care about any
horizontal error. That is why I add a couple normal control points on the
horizon to lock the yaw of the two panos together.

Jim

On 2015-03-16 4:33 AM, Grigoriy360 . wrote:
> Hi Jim,

> What to do if there is not the obvious horizontal line?
>
> Best,
> Gregory
>

[Roger D Williams]

I WILL be there!

Roger

Sent from my iPad

[PTGui Support]

I'll be there as well! Looking forward to seeing your talk Jim.

Joost

[Mike Spence]

This event sounds great. Will the talks be in English?

thanks

Mike

-----Original Message-----
From: pt...@googlegroups.com [mailto:pt...@googlegroups.com] On Behalf Of PTGui Support
Sent: 18 March 2015 08:59
To: pt...@googlegroups.com
Subject: Re: [PTGui] Managing stitching for stereo

[PTGui Support]

Yes they should all be in english.

Joost

[Grigoriy360 .]

Jim,

I did some tests stereo panoramas shooting, using your technique. I have Canon 5D_MIII+Canon 15mm lens+NN5.

I did 6+2 in NN position and moved the vertical bar to the right and did 6+2 for the right view.

It works, but some areas have noticeable vertical shift. How could I fix it?

Thank you,

Gregory
​ 

ptgui+unsubscribe@googlegroups.com

Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at
http://groups.google.com/group/ptgui

--- You received this message because you are subscribed to the Google Groups "PTGui Support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
--
You received this message because you are subscribed to the Google Groups "PTGui" group.

To post to this group, send email to pt...@googlegroups.com To unsubscribe from this group, send email to ptgui+unsubscribe@googlegroups.com

Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at http://groups.google.com/group/ptgui

---
You received this message because you are subscribed to the Google Groups "PTGui Support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
--
You received this message because you are subscribed to the Google Groups "PTGui" group.
To post to this group, send email to pt...@googlegroups.com

To unsubscribe from this group, send email to ptgui+unsubscribe@googlegroups.com

Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at http://groups.google.com/group/ptgui

--- You received this message because you are subscribed to a topic in the Google Groups "PTGui Support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ptgui/miTMKr7lw88/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ptgui+unsubscribe@googlegroups.com.

[Jim Watters]

Gregory,

I would use the Z&N from the first pano.
When taking the the second set of images off the NPP I woud shoot about 3 times
as many to minimize the parallax error from one image to the next.

The noticeable vertical shift should be minimized by adding horizontal control
points between the L & Right pano. Then set the Optimization to optimize H & V
in a single pass.

Jim

On 2015-04-10 6:15 PM, Grigoriy360 . wrote:
> Jim,
>
> I did some tests stereo panoramas shooting, using your technique. I have Canon
> 5D_MIII+Canon 15mm lens+NN5.
> I did 6+2 in NN position and moved the vertical bar to the right and did 6+2
> for the right view.
> It works, but some areas have noticeable vertical shift. How could I fix it?
>
>
> Thank you,
> Gregory
> ​

[Grigoriy360 .]

Jim,

I did today next test, I was more careful with leveling and as you can see on attached print screen the combined pano looks good.

My next step will be to align images for L and R panoramas. 

Thank you,

Gregory

--
--
You received this message because you are subscribed to the Google Groups "PTGui" group.
To post to this group, send email to pt...@googlegroups.com
To unsubscribe from this group, send email to ptgui+unsubscribe@googlegroups.com
Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at http://groups.google.com/group/ptgui

[Grigoriy360 .]

Hi Jim,

This is how the final stereo panorama looks:

http://arbitphoto.com/Malibu_Park_stereopano/anaglyph_01.html

Thank you,

Gregory

[Roger D Williams]

Two comments. I thought the colour was very good for an anaglyph. I have given up on colour, sticking to B&W only. And I also thought the vertical alignment might not be quite right on the dappled sunlight/shadows on the ground under the trees. Something seems to prevent me from seeing a true stereo image there. Not talking about the nadir--we all know how difficult these are in stereo panos...

Roger W

Sent from my iPad

To unsubscribe from this group, send email to ptgui+un...@googlegroups.com

Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
For more options, visit this group at http://groups.google.com/group/ptgui

---

You received this message because you are subscribed to the Google Groups "PTGui Support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+un...@googlegroups.com.

[Jim Watters]

Gregory,
I think you have the technique down pretty good. I would tend to stay away from
zenith that have a lot of depth. The reason being as you look up the stereo
looks correct but as you look past the zenith the stereo is reversed. To help
with this I make the zenith & nadir mono, using the same zero parallax image in
both.

Shooting stereo with one camera by shooting for one eye then the other is
difficult to keep everything from moving. There may be some changes in the
shadow of the trees. There are a couple groups of people one down the road and
another next to the river. They only show up in one set of images. It would be
better to have these people either in or out of both. Either by cloning or
including some of the images from the other set when stitching the pano.
They are far enough away that the stereo info is not huge. Loosing a little bit
of stereo is better than not having the same subject in left and right.

The sense of immersion into to this pano is great.

I now have two GoPro and two 180° lens that I am going to make a stereo rig
with. I am hoping to avoid these little discrepancies and be able to shoot where
there is movement.

Jim

On 2015-04-20 2:50 AM, Grigoriy360 . wrote:
> This is how the final stereo panorama looks:
> http://arbitphoto.com/Malibu_Park_stereopano/anaglyph_01.html
>
> Thank you,
> Gregory
>

[John Goodman]

Gregory,

A very nice effort IMHO but there is disconcerting retinal rivalry here and there. (RR = noticeable differences between the L & R views that aren't related to parallax, such as something present in one side but not the other.)

Roger W wrote, "I also thought the vertical alignment might not be quite right on the dappled sunlight/shadows on the ground under the trees. Something seems to prevent me from seeing a true stereo image there." I think the problem is severe retinal rivalry, especially at the initial view. If you alternately view the pano with just your left or right eye, you'll see the shadows dance back and forth because they are so different between the shots. Elsewhere, there are people in one view but not the other. I'm not up on the techniques & tradeoffs involved when making stereo panos, but if you used a twin rig, at least they could be synched in time. I think one's eyes can handle some degree of vertical disparity without spoiling the stereo effect.

There's another aspect of this stereo pano (and perhaps all fixed stereo-base 3D panos) that feels odd when I explore it in a browser window, and that is that the perceived distance between an object in the view and the plane of the stereo window remains the same no matter what the degree of zoom. OTOH, you've done a great job of making sure there are no window violations.

Mr. Katsuhiko Inoue has created some amazing immersive 3D videos with a quad-GOPRO rig he hacked to use round fisheye lenses. The cameras are mounted on faces of a small cube and stereo info is extracted from adjacent pairs. I saw a live demo last October, where he panned and zoomed in real-time while the 3D video was playing.

http://www.3dstreaming.org/3d-media/videos/7142-3d-vr-walkthrough-mp4.html

I imagine this technique would work for still panos using two cameras mounted at 90° but can't imagine how to do it. Has anyone done 8 or 12 shots in quick succession with a round fisheye setup and extracted stereo info from orthogonal pairs?

 John Goodman

[GregoryA]

John,

Thank you for the comments, what can I say, it was just a test. In order to build some panorama  stereo rig, I do tests to found the optimal configuration.

All the best,

Gregory 

[Jason Reinhardt]

Hi Jim! I've been reading your material for weeks now, I've just finished building an 18-cam stereo 360 rig (using Xiaomi Yis) and your documentation on shooting panoramas, stereo stitching and your Mobius rig have been absolutely invaluable. This thread in particular was a lifesaver when jumping in to stereo stitching. Thanks for all of that!

The lateral cameras on my rig are arranged in 6 parallel stereo pairs. At the corners, where pairs overlap, of course there's a good bit of parallax error between near objects and background (PTGui can properly stitch one or the other at a time, no surprise here). My current process is to stitch once for near objects, stitch again for distant objects, and then mask the two together in post. Is this about what you would recommend, or is there a simpler way?

Best regards,

Jason

[Jim Watters]

Jason,

While at IVRPA a couple weeks ago we had several stereo pano shooters in the same room. We each had slightly different process to stitching stereo panoramic images. But we all managed to make something that worked. I am hoping to figure out a simpler way, but because of the fact that parallax is involved and needs to be hidden but still maintain good stereo all around it becomes difficult. It comes less of a science and more of an art form. You will need to find what works best for you. Make sure you check the results in a HMD to see how well the stereo is.

Going from still images to video while still shooting stereo is even harder. It is much easier if the camera stays stationary.

When shooting with parallax error it is best to keep the subject away from the camera, but when shooting stereo the point is to have the subject close to show depth.

With the adjacent pairs of cameras, the parallax error will be bigger and harder to hide.

My recommendations:
- Precalculate all distortion values for each camera and lens by shooting a pano with each on the NPP.
- Bring all the images into PTGui and apply the individual settings for each lens.
- Joint the left and right panos only with horizontal control points except for a couple points on the distant horizon to lock the yaw.

because of the parallax error it is necessary to introduce corrections to force the images together. View Point Correction is the best way to make the images come together.

Jim