Dewarping

[Yuv]

Hi all,

I am strapped for time and need help dewarping a two hours movie
footage. The frames will be extracted with avconv. Sample: [1]

The extracted frames need to be dewarped, ideally into cylindrical
projection (although there is some leeway with regard to the vertical
proportions). Any idea?

Thanks
Yuv

[1] <https://www.dropbox.com/s/rojw0oh2g9wmnk7/frame00003.png>

[Terry Duell]

Hello Yuv,

On Tue, 02 Oct 2012 13:13:02 +1000, Yuv <goo...@levy.ch> wrote:

> Hi all,
>
> I am strapped for time and need help dewarping a two hours movie
> footage. The frames will be extracted with avconv. Sample: [1]
>
> The extracted frames need to be dewarped, ideally into cylindrical
> projection (although there is some leeway with regard to the vertical
> proportions). Any idea?

I suspect that you have probably already thought more about this than I
have, so my thoughts may not be a lot of help, and I'm guessing a bit/lot
here.

I had a bit of play and loading the sample into hugin as a circ-fisheye
with fov=360, then selecting the preview window, setting proj=cylindrical,
and then move/drag gets the image looking OK, then manually set a crop to
end up with a reasonable view then stitch.

If you need every frame dewarped, I would be thinking about a script that
passed each image file to pto_gen, then run pto_modify to set the output
projection, But I don't now how one can script the effects of my
move/drag...that's the missing bit for me.

If there are only some important frames that are needed, then I guess you
could set up a slide show to run through them all and stop it when it you
get to the interesting bit, then do a manual dewarp of those few frames.


As I said, I'm guessing.
Good luck.


Cheers,
--
Regards,
Terry Duell

[kfj]

Am Dienstag, 2. Oktober 2012 09:01:56 UTC+2 schrieb Tduell:

Hello Yuv,

On Tue, 02 Oct 2012 13:13:02 +1000, Yuv <goo...@levy.ch> wrote:

> The extracted frames need to be dewarped, ideally into cylindrical
> projection (although there is some leeway with regard to the vertical
> proportions).  Any idea?

If you need every frame dewarped, I would be thinking about a script that  
passed each image file to pto_gen, then run pto_modify to set the output  
projection, But I don't now how one can script the effects of my  
move/drag...that's the missing bit for me.

How about using transform-pano from the panotools-script collection?
As long as the transformation needed is the same for all frames, this should do the trick.

Kay

[Lukas Jirkovsky]

I think this would be a good job for OpenCV rather than nona or
panotools because of its performance. However, I'm afraid this would
require writing an application for it.

I'll take a look (though I won't have the time till tomorrow) at one
of apps that I wrote that's using OpenCV for transformations, it might
be possible to reuse some of its code.

Lukas

[Jim Watters]

Yuv,

Use mpremap. That is what it was made for. Batch dewarp processing a stream of
images and putting back into a video.
The project has been added to Panotools for a while. It has been a couple years
since I used it.

http://webuser.fh-furtwangen.de/~dersch/mp/MotionPanoramas.html

Jim

--
Jim Watters
http://photocreations.ca

[Lukas Jirkovsky]

On 2 October 2012 13:37, Lukas Jirkovsky <l.jir...@gmail.com> wrote:
> I'll take a look (though I won't have the time till tomorrow) at one
> of apps that I wrote that's using OpenCV for transformations, it might
> be possible to reuse some of its code.

It turned out that the code I already wouldn't be useful (it's using
only affine transformations). I think mpremap that Jim mentioned is
probably a good way to go.

Lukas

[Yuv]

Thanks all for the hints. I should have given more background: this
is not about speed / performance. I used a catadioptric lens as a
poor man solution to the lack of equipment. I recorded the
interaction between barristers and the bench in a moot court room. If
I had the means, I would have pointed one fixed video recorder on the
bench and one on the barrister.

I ended up dewarping one frame with the software that is delivered
with the lens, and then reconstructing the same in ImageMagick to
create an equirectangular image. From the equirectangular, Hugin
helped me to visually frame the two streams I needed. Next I will
script nona to mass-process the frames, and use ImageMagick and avconv
again to compose and put the movie back together. The process is
almost finished scripting and will run sometimes in the next days.

The relevant part for the dewarping (and for making 0-360 catadioptric
lenses usable with Open Source software) is to first square the circle
-- determine the square side and the coordinate offset from the top of
the image, then transform with ImageMagick.

One obstacle I encountered was that the TIFF output of avconv is not
processed well by Hugin (weird lines and coloring) nor by ImageMagick
(weird coloring), so I ended up extracting the frames as PNG (JPG
quality was horrible). On my machine avconv PNG extraction is about
six times slower than its TIFF extraction, but it is a batch process
so time is relative.

Then I tried to reconstruct with ImageMagick the same output that the
0-360 proprietary software produces. The bash script is below. I am
still trying to figure out why ImageMagick's image size is 70% of the
proprietary software image size and whether that matters (i.e.: am I
losing resolution with my current ImageMagick process, or is the
proprietary software product pixel-inflated)?

Next step would be cool to do this with panotools directly, and maybe
add functionality to Hugin (or another kind of dedicated front-end
GUI) to automatically determine the center of the donut and use it to
unwrap this kind of photo/video.

For those with better skill and more time than me, here are:
- original image [1]
- unwrapped by 0-360 software [2]
- unwrapped by ImageMagick with the command below [3]

I am happy with my process as it stands now (and I don't have time to
fiddle with it more than that anyway). I do not expect to have any
more footage like this to process in the next three months. I can
provide access to original footage; to higher resolution images taken
with the lens (I have an adapter to use it with my dSLR); and to the
lens itself if somebody wants to take this to the next level.

Thanks again for all the help
Yuv

----- BASH SCRIPT -----

#!/bin/bash
# project from the catadioptric lens to equirectangular
square_side=901
offset_x=451
offset_y=41
crop="${square_side}x${square_side}+${offset_x}+${offset_y}"
# center of the circle x and y coordinates, as well as radius of the
circle, are the same
c=$(echo "scale=1;$square_side/2" | bc)
# for the final equirect image height it must be integer
h=$(echo "scale=0;$square_side/2" | bc)

infile=000001.png
outfile=000001.tif

# reconstruct the output from the 0-360 unwrapper software. It was
1288x389, black space cropped.
# the lens according to 0-360.com specs: (52.5° above horizon, 62.5°
below horizon)
# must add 37.5 deegrees of black above, then target to equirect size
# from the same input footage ImageMagick yields 901 horizontal
resolution

# and here is the magick line:
convert $infile -crop $crop -distort DePolar "$c 0 $c $c -180 180" -
flip -resize 100%x39%! +repage -gravity south -background black -
extent ${square_side}x${h} 01$outfile


[1] <https://www.dropbox.com/s/uq7rqzvy7fmlx9u/Frame00001.png>
[2] <https://www.dropbox.com/s/5d27ceqq35i4ksy/Proprietary0-360.jpg>
[3] <https://www.dropbox.com/s/hyn9vgzu4nxkmts/ImageMagick901.tif>

[Jim Watters]

Yuv,

I managed to get pretty good results using panotools.

On 2012-10-06 3:42 PM, Yuv wrote:
> Next step would be cool to do this with panotools directly, and maybe
> add functionality to Hugin (or another kind of dedicated front-end
> GUI) to automatically determine the center of the donut and use it to
> unwrap this kind of photo/video.
>
> For those with better skill and more time than me, here are:
> - original image [1]
> - unwrapped by 0-360 software [2]
> - unwrapped by ImageMagick with the command below [3]

If I set the lens as circular fisheye with 310 deg it looks pretty good to me.
I just eye balled the setting.
To optimize to even better setting. Set straight line control points on the
lights in the ceiling.

p w2000 h1000 f2 v360 u20 n"TIFF"
m g1 i0 f0
i n"Frame00001.png" w1920 h1080 f2 y0 r0 p-90 v310 a0 b0 c0 d0 e0 g0 t0
C431,1390,7,966
o n"Frame00001.png" w1920 h1080 f2 y0 r0 p-90 v310 a0 b0 c0 d0 e0 g0 t0
C431,1390,7,966

This simple script could easily be fed into mpremap.

> Thanks again for all the help
> Yuv
>
>
>
>

> [1] <https://www.dropbox.com/s/uq7rqzvy7fmlx9u/Frame00001.png>
> [2] <https://www.dropbox.com/s/5d27ceqq35i4ksy/Proprietary0-360.jpg>
> [3] <https://www.dropbox.com/s/hyn9vgzu4nxkmts/ImageMagick901.tif>
>

[Jim Watters]

Except the image is mirrored Left to Right. ;)
https://www.dropbox.com/s/sc6xz3gve4xzovp/Frame00001%20warped.jpg

Ceiling showing it is a pretty good guess.
https://www.dropbox.com/s/mu73fuv60pxg5ph/Frame00001%20zenith.jpg

Jim

On 2012-10-06 5:45 PM, Jim Watters wrote:
> Yuv,
>
> I managed to get pretty good results using panotools.
>
> On 2012-10-06 3:42 PM, Yuv wrote:

>> Next step would be cool to do this with panotools directly, and maybe
>> add functionality to Hugin (or another kind of dedicated front-end
>> GUI) to automatically determine the center of the donut and use it to
>> unwrap this kind of photo/video.
>>
>> For those with better skill and more time than me, here are:
>> - original image [1]
>> - unwrapped by 0-360 software [2]
>> - unwrapped by ImageMagick with the command below [3]

> If I set the lens as circular fisheye with 310 deg it looks pretty good to me.
> I just eye balled the setting.
> To optimize to even better setting. Set straight line control points on the
> lights in the ceiling.
>
> p w2000 h1000 f2 v360 u20 n"TIFF"
> m g1 i0 f0
> i n"Frame00001.png" w1920 h1080 f2 y0 r0 p-90 v310 a0 b0 c0 d0 e0 g0 t0
> C431,1390,7,966
> o n"Frame00001.png" w1920 h1080 f2 y0 r0 p-90 v310 a0 b0 c0 d0 e0 g0 t0
> C431,1390,7,966
>
> This simple script could easily be fed into mpremap.
>
>

>> Thanks again for all the help
>> Yuv
>>
>>
>>
>>

>> [1] <https://www.dropbox.com/s/uq7rqzvy7fmlx9u/Frame00001.png>
>> [2] <https://www.dropbox.com/s/5d27ceqq35i4ksy/Proprietary0-360.jpg>
>> [3] <https://www.dropbox.com/s/hyn9vgzu4nxkmts/ImageMagick901.tif>
>>
>
>