Panorama Fisheye HDR - Bendy Horizon

[Alex Meddick]

Hi All,

We currently PTGui to stitch HDRs and are looking for a Linux based alternative.

Using PTGui, and without any particular intervention, the horizon line has always come out perfectly horizontal and straight. As in you could draw a straight line from left to right on the resulting HDR and the horizon line would follow it.

I'm stitching the same HDR frames (I've tried more than one example) using the Panorama Fisheye HDR pipeline in Meshroom and the horizon line is wavy, curving up and down, as it moves left to right.

The only examples I've seen online from Meshroom seem to be wavy as well. So firstly I wanted to ask if this intentional? If not, can anyone point me at how to improve my results.

Thanks,

Alex

[Fabien Castan]

Hi Alex,

In Meshroom, the panorama is aligned on one of the input cameras, so your shot needs to be aligned with the horizon to avoid a wavy output.

I'm not sure what is done in ptgui regarding that without manual intervention.

Could you share a dataset and the result you get in ptgui (without manual intervention)?

Best,

--
You received this message because you are subscribed to the Google Groups "AliceVision" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alicevision...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/alicevision/ebe45977-1695-4ba5-8dff-ede730f5f1dbn%40googlegroups.com.

Virus-free. www.avast.com

[David Cattermole]

Hi Alex,

I tried out the HDRI Fisheye workflow in Meshroom when v2020 was released and I didn't find this, but I did test with CR2 files and JPEG (generated from the same camera) to see the difference. In my test I found the HDRI to be flipped with the JPEG, but not with the CR2s.

The source images were shot with a nodal ninja set up, which unfortunately is quite significant in the resulting images, but does provide equal angles to stitch with.

I can provide a test data scene for that if you'd like?

In fact, if it helps Fabian, I think I can provide 2 different data sets as an example for people to use? I'd prefer not to host the files however. All the images together are about 4GB.

Thanks,

David Cattermole

To view this discussion on the web visit https://groups.google.com/d/msgid/alicevision/CAJQWia6GWfVw85%3DadVW6GoekH2DbN4ziC-%2BN4ifcmvU4y7RyRw%40mail.gmail.com.

--

David Cattermole

[Alex Meddick]

Thanks for the quick responses Fabien and David. 

Unfortunately all of the HDRs I have immediate access to are not ones that I am allowed to share. The test images were shot in a Canon 5D Mark IV using an 8mm lens.

I will see if our data team has anything that I'm able to share, though if it was possible to get a test data set, I will run it through and compare result between Meshroom and PTGui.

The only other thing I can think of is that "Add missing Camera Models to the Sensor Database to improve your results" did pop up the first time I launched, though has not launched since. The Canon 5D Mark IV is in the list when I open cameraSensor.db. An internet connection is not required it is?

Thanks,

Alex

[David Cattermole]

Hi Alex,

I don't believe an internet connection is required, it's just a text file. Fabien could correct me on that.

I'll see what I can do about getting you the data set Alex.

David

To view this discussion on the web visit https://groups.google.com/d/msgid/alicevision/4624dd19-0fbf-427e-9549-659706d0edc8n%40googlegroups.com.

--

David Cattermole
Mobile: +447852584464

[Fabien Castan]

The Canon 5D Mark IV is in the list when I open cameraSensor.db. An internet connection is not required it is?

No, Meshroom uses the file "cameraSensor.db" embedded in the release.

Virus-free. www.avast.com

[Fabien Castan]

Hi David,

In my test I found the HDRI to be flipped with the JPEG, but not with the CR2s.

That's strange, the orientation flag is supposed to be recognized in the same way.

How did you convert the RAW to JPG? Maybe the rotation is backed?

I can provide a test data scene for that if you'd like?

In fact, if it helps Fabian, I think I can provide 2 different data sets as an example for people to use? I'd prefer not to host the files however. All the images together are about 4GB.

Yes, it would be great to have a sample dataset online, but I'm not sure what platform to use for 4GB.

Virus-free. www.avast.com

[Alex Meddick]

I've got a test data set now. Give me a little bit of time to check it out and update the thread. Thanks again for the quick replies.

Alex

[David Cattermole]

Hello Fabien,

That's strange, the orientation flag is supposed to be recognized in the same way.

How did you convert the RAW to JPG? Maybe the rotation is backed?

We shot with the "RAW + JPEG" option on a Canon 5D Mk3. The JPEGs were straight from the camera.

I created the HDRI panorama (using the "fisheye" preset) using the CR2s and JPEGs otherwise they were exactly the same.

The only difference was the JPEG version was flipped (vertically), the CR2 version was not. I flipped the resulting panorama vertically (using Natron) and it looked correct.

Yes, it would be great to have a sample dataset online, but I'm not sure what platform to use for 4GB.

To be honest the images I have are overkill, there are lots and lots of angles. We can probably cut the number of angles down for a demo.

I have two different image sets, I will send only one of them and it should be 3GB, (with CR2 RAW + JPEG). For a demo, we could cut the CR2s out, and/or reduce the number of angles. It was shot with a Sigma 8mm fisheye lens, I cannot remember exactly which nodal head we used.

You should have received a link to download the files - if not, I can send you the link directly.

David

[Fabien Castan]

Hi David,

I do not see the download link.

Virus-free. www.avast.com

[Paris Downes]

Hi David. Good to see you around! I had a go at stitching your example images and, like Alex, I get a sinus shaped horizon line. Comparing your .mg setup with mine, the only things I noticed that were different are a) you used Windows and I used Linux, b) you were using 2020.1.1 and we were using 2020.1.0, and c) the camera metadata was being read in slightly differently: CameraMount and LensMount values were different, and CanonFocusUnits are called FocusUnits in mine. Don't know if that would affect anything?

Anyway, here is the result I got:

And here is the same thing stitched using PTGui, for comparison:

[David Cattermole]

Hello Paris!

It's great to hear from you, too.

I'm glad I can help you reproduce the problem :)

Just some quick thing the

e difference between version 2020.1.0 on Linux and 2020.1.1 on Windows is because of a Windows only bug that occurred (reported somewhere on this mailing list) and was fixed.

I don't think it's likely this bug fix causes or fixes the problem.

Fabian mentioned that the EXIF metadata is used to orient the cameras. Perhaps there's a different library or library version used on the different platforms - just throwing out initial ideas.

Maybe if I get time this weekend I can run my sample images on my CentOS install and I'll see if I can reproduce the problem.

Fabian, can you think of anything?

Thanks,

David

--

You received this message because you are subscribed to the Google Groups "AliceVision" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alicevision...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/alicevision/243fcbf7-bba1-4617-9060-dc6c700ffe57n%40googlegroups.com.

[Paris Downes]

Thanks David! By the way, I still have your Mother and Son DVDs. Do you need them back?

[Alex Meddick]

Hi Fabian and David,

Just following up to see if there is any more information required from us on this one to help move forward.

Thanks,

Alex

[David Cattermole]

Hey Alex and Paris,

I'm sorry I haven't had much time recently to test on Linux, or in general.

It's been pretty hectic for the last few months.

I saw that the Meshroom team has released 2021.1.0 using AliceVision 2.4.0, and the new AliceVision version's release notes contains this:

"[panorama] New options to init with known poses PR".

I'm not sure if this is exposed in Meshroom's UI, but maybe this new version is worth installing and testing?

Meshroom release notes:

https://github.com/alicevision/meshroom/releases/tag/v2021.1.0

AliceVision release notes:

https://github.com/alicevision/AliceVision/releases/tag/v2.4.0

David

To view this discussion on the web visit https://groups.google.com/d/msgid/alicevision/4a8fcfb8-764e-4d69-8150-5bb3adfb6225n%40googlegroups.com.

[David Cattermole]

After looking a bit closer at the changes (see below), I can see the latest v2021.1.0 does expose a few new features.

Screenshot of v2021.1.0 "ParameterInit2" node:

Screenshot of v2020.1.1 "ParameterInit" node:

Looking at the documentation, it's also possible to use an .xml file from "Roundshot VR Drive", however the new features, such as "Horizontal" might help.

The exact file that does the init is here:

https://github.com/alicevision/AliceVision/blob/cdb20e9db7aadf80dfa43649442e9670dcab3936/src/software/pipeline/main_panoramaInit.cpp#L802

If I'm reading the code correctly the "Horizontal" option will work very nicely for a simple panorama with the camera only.panning.

Hopefully this helps.

Later this week I'll try running the same "car park" images with v2021.1.0 (on Windows) and see what I get.

I've been super busy recently so I'm not sure I can promise, but testing on WIndows is much easier for me right now, Linux is more difficult.

David

P.S. I'm not part of the AliceVision or Meshroom teams, I just keep an eye on the forum :)

[Alex Meddick]

Hey David,

Thanks for digging into this for us more. I do appreciate the help even though I know you're not required to give it. I am glad you're busy, you deserve to be :D.

We're pretty busy too, and as much as I hate to leave this thread unfinished for the next person that comes along looking for an answer, we've had to stick with our current workflow for now. So no pressure if you don't have time to look into the older version on Windows.

Take care.

Alex