open source lens distortion correction - camera models

[Jeffrey Warren]

Grant - sorry it took me a while, but the LensFun database format covers various types, like vignetting and transversal chromatic aberration, but for simple lens distortion it looks like this:

<lens>
    <maker>Pentax</maker>

    <model>SMC PENTAX DA 12-24mm F/4 ED AL IF</model>
    <mount>Pentax KAF2</mount>
    <cropfactor>1.5</cropfactor>
    <calibration>
        <distortion model="poly3" focal="12" k1="-0.01919" />

        <distortion model="poly3" focal="15" k1="-0.00774" />
        <distortion model="poly3" focal="18" k1="-0.00345" />
        <distortion model="poly3" focal="21" k1="-0.00199" />

        <distortion model="poly3" focal="24" k1="0.00061" />
    </calibration>
</lens>

There's a lot more detail here: http://lensfun.berlios.de/lens-calibration/lens-calibration.html

I've posted some of these notes at http://publiclaboratory.org/wiki/mapknitter-correcting-lens-distortion

If you post or attach a checkerboard pattern I'd be happy to take some photos of it for calibration. Could I simply display them on my computer monitor and use a tripod for the camera?

Jeff

On Mon, Oct 17, 2011 at 11:51 PM, Grant W Fraley <g2b...@gmail.com> wrote:

Jeff et al,

Here are the values for resolution and areal coverage as promised for the two Canons.

Canon SD1400 IS: https://docs.google.com/spreadsheet/ccc?key=0AjYMqsueyFWmdGNsTWcwczZqVVRZUXVaSHhYd0JpOHc&hl=en_US

Canon A490: https://docs.google.com/spreadsheet/ccc?key=0AjYMqsueyFWmdE5yU1N5dVZOa0dJQ3d6QUJYT0huc2c&hl=en_US

If you have a more preferred way that I share this data, let me know. Feel free to distribute.

These values are all in meters, and I assumed a fixed focal length and the widest angle for each camera as well as the largest size image. Of course the assumption is also that the camera is viewing at nadir (straight down). I can re-run the calculations for a range of focal lengths if needed.

Hope this helps in estimating spatial resolution and flying height needed before you go out and map.

Throwing up a XBee + GPS with telemetry down to another XBee on a laptop might be a good way of monitoring your altitude and position during mapping campaigns. Maybe that idea was visited at some point by the group already?

I'll take a look at the calibration / lens distortion info you sent and see if I can be of any help. I can definitely produce a camera matrix and distortion vector for your Canons if you take some images of the checkerboard pattern I have. I haven't yet looked to see what kind of format or camera model LensFun uses though.

Best regards,

Grant

On Mon, Oct 17, 2011 at 2:33 PM, Jeffrey Warren <je...@unterbahn.com> wrote:

Interesting thoughts. Personally, I'd be more inclined to just do this empirically -- put a GPS up, and send a balloon or kite up, recording the time when i've sent up 500', 1000', 2000' of string, and build a chart from that.

On a related note, if you generate a camera model for the A490, please share it!! We're thinking of building automatic lens distortion corrections into MapKnitter, but the open source lens database LensFun doesn't have models listed for cheaper cameras, since usually only people with nice SLRs are interested in lens distortion. You can follow some of the notes for this feature here: https://github.com/jywarren/mapknitter/issues/65; it would really help getting those oblique images to stitch together better on a map like this one: http://mapknitter.org/tms/saugus-incinerator-landfill/openlayers.html

Jeff

On Mon, Oct 17, 2011 at 2:29 PM, Grant W Fraley <g2b...@gmail.com> wrote:

Adam,

Thanks, I'll take a look at the cameras and get back to you if I need anything. 

You can calculate the estimated ground resolution and footprint using fairly simple equations usually found in remote sensing texts. The two common methods use representative fraction (scale calculated as f / altitude) or using view angles. Typically, both ways require you to know the number of pixels or sensor dimensions.

We use projective geometry to model cameras in our software. Projective geometry produces the exact same result as the common equations if you have no information on camera distortions or miscalibrations. 

But, projective geometry also allows us to calculate footprints in geographic coordinates and take into account pitch, roll, and yaw of an aircraft as well. Further, if we do have a calibrated model for a camera, we can take into account the distortions to get better estimates of the footprint or to project imagery into geographic coordinates once we have conducted a flight.

Actually, your method of imaging a meter stick isn't too far off. For calibration we use a checkerboard pattern with known square size and repeatedly image it from different angles.

I'll look up the two Canon cameras and get back to you sometime this afternoon / evening with some estimates of footprint and spatial resolution for a range of altitudes.

Grant

On Mon, Oct 17, 2011 at 10:48 AM, Adam Griffith <adamdg...@gmail.com> wrote:

Canon A490
Canon SD1400 IS

Not to be difficult but..... ok, to be difficult, I have calculated the ratio using a meter stick taped to a wall.  How is your way better?  I am sure it is, but I am curious.  How does the CCD influence the camera's inherent ratio?  Or do you know all the lens distortions too?  That is something I am curious about as well.

Awesome!

Adam

On Mon, Oct 17, 2011 at 1:42 PM, Grant W Fraley <g2b...@gmail.com> wrote:

Jeff and Adam,

If you send me info on the camera(s) you will be using over the Thatchmore farm I can calculate the image footprint and ground resolution for you. Just the camera make + model should do so that I can lookup the CCD and focal length characteristics. It would also help to know the range of altitudes you typically "fly" with. Although, determining alt is probably most of the reason for this thread ;-).

We have the ability to calculate these values in our F-Planar software (http://www.terrapanlabs.com/products/f-planar), but it's really only setup for fixed wing aircraft currently. And, it is not free and/or open source (sorry). That being said, I'd be happy to use our code base to calculate a few values for you as a starting point.

A better estimate of the footprint / field of view can be obtained through a quick calibration procedure to take into account distortions and camera intrinsics. We will probably put this up as an online service soon, but let me know if this is something you're interested in during the short-term. Given you're altitude precision and platform instability during flight with kites / balloons you'll probably need a large margin of error (i.e. lots of potentially overlapping coverage) anyway so I'm not sure how much calibration to remove lens distortions, etc., would really buy you.

Best regards,

Grant

On Mon, Oct 17, 2011 at 10:19 AM, Adam Griffith <adamdg...@gmail.com> wrote:

Yes, that is great, Dave.

I updated the perimeter of Thatchmore Farm just now.  (I visited the farm a few hours ago and Tom, the owner, gave me a copy of the plat and even found the pins marking the property corners!)

Jeff has hit the nail on the head though.  How high should I fly to capture x acres?  4x acres?  16x acres?  etc...

Great to see this issue getting some press and ultimately leading to a simple chart and/or tool.  Inclinometers or clinometers are about $100, so a plumb bob and protractor are a good start.

Regarding your question, Dave, apparently three tethers were used recently over the Occupy Wall Street demonstrations with high levels of success.  I have used more expensive laser sighting devices that do the trig for you that gave me the straight line distance and the height.  That was targeting a kite being towed behind a boat with the laser.  It worked pretty well, as I recall.

Adam

On Mon, Oct 17, 2011 at 12:38 PM, Jeffrey Warren <je...@unterbahn.com> wrote:

Wow, cool Dave! Here's a typical height application: 

you have a 1/2 mile x 1/2 mile square plot of a farm (keeping it simple here) and you want to know how high you'd have to get the balloon to photograph the whole farm (or even "most of" the farm) in a single photograph. 

Alternatively, you want to know how many flights from different launching points you'll need to get the whole quarter of a square mile mapped if you only have 1000' of string -- see the very crude diagram I've attached.

This week some of us will be mapping Thatchmore Farms in North Carolina -- Adam or anyone else, can you help finalize the extent of the farm area in this editable map so we could apply this to a real-life upcoming mapping attempt? 

I'm still wondering if most people would confuse your graph for a literal diagram of kite strings -- i.e. "but the string sags *down*, not up!" and whether although it's graphically very elegant, actually showing a series of balloon strings with people and balloons illustrated in might be a more broadly legible way to present the information. Like this image: http://www.flickr.com/photos/jeffreywarren/6254526526/in/photostream

It could even show the expected resolution for, say, a 10 megapixel camera at given heights.

Jeff

On Mon, Oct 17, 2011 at 11:03 AM, Dave Haffner <daveh...@gmail.com> wrote:

Hey Everyone,

Here's a simple plot that should be easy to use to measure height
given tether length and angle. Angle here is defined w/r/t the zenith
(straight up into the sky). I'm going to give it a shot once I set up
a good way to measure the line length.

For angle I'm rigging a protractor and a plumb bob on the line to
record the angle at regular time intervals. To measure length I was
thinking of either using Stoney's idea of measuring distances off on
the line with marking tape, or using a bike computer to count the
rotations of the tether spool. I think that approach could do the job
accurately and cheaply. Most inexpensive computers use a Reed switch

http://en.wikipedia.org/wiki/Reed_switch

The idea is to count the passes of the rotating spool under the
spindle handle.  If the spool diameter isn't big enough to assume the
line is unwinding off a circle of constant radius you'd need to
calibrate the readout of the spool computer by measuring the length of
line that unwinds from the spool at intervals and interpolate between
them to get continuous length as a function of the # of rotations. On
a bigger spool with thinner line I think constant radius could be
assumed, so you could just tell the computer the radius of the spool
and it would read out the length of line directly.

It seems like a cheap and elegant way to get height. My only concern
with accuracy is with measuring angles with adequate precision as the
angles get larger. A small error in angle with the balloon overhead is
not a big deal, but the plot I made says that it gets more important
to measure the angle precisely as the angle increases, though I guess
the height accuracy really depends on the application.

What kind of accuracy have folks needed/wanted?

Dave

On Sat, Oct 15, 2011 at 4:36 AM, Stoney <sto...@gmail.com> wrote:
> I would rather estimate the height using the amount of line that has
> been unwound.  You could put marks or colored tape on your line.  I
> used a Leica laser measurement tool to make this type of measurement
> and I was never able to get very good results.  I would check the
> height using another method and the laser measurement tool would be
> way off.
>
> On Sep 29, 4:12 pm, Patrick Coyle <p...@coyles.com> wrote:
>> Dave,
>>
>> This is great chart. Looks like it would be handy in the field if you have
>> the estimated length of line.
>>
>> Probably similar to the android apps Stewart suggested, I also looked for
>> iPhone apps and see a couple that might be useful:http://taptapsoft.com/our-apps/pointmeasurehttp://itunes.apple.com/us/app/easymeasure/id349530105?mt=8
>>
>> I have a hand-me-down iPhone with no data plan, so I'll try these and report
>> back.
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Sep 28, 2011 at 2:18 PM, Dave Haffner <davehaff...@gmail.com> wrote:
>> > I was curious how this would look so I made a graph. If something like
>> > this would be useful in the field I can tweak it. Zenith angle is
>> > defined as angle between the vertical (straight up) and the tether.
>>
>> > Dave
>>
>> > On Wed, Sep 28, 2011 at 4:19 PM, Dave Haffner <davehaff...@gmail.com>
>> > wrote:
>> > > Apologies if this is a dumb question, but is the balloon tethered? If
>> > > so, and there a way to measure the length of the tether extended, one
>> > > could log the angle and length of tether and use an old-school table
>> > > for quick rough determination in the field. Back at home, a calculator
>> > > would get you a more precise number.  And your errors would be a
>> > > simple combination of those made in measuring length and angle.
>>
>> > > Dave
>>
>> > > On Wed, Sep 28, 2011 at 2:57 PM, Patrick Coyle <p...@coyles.com> wrote:
>> > >> Any suggestions for smartphone developer who could implement the
>> > application
>> > >> to measure the height of a balloon or kite as described below? In
>> > summary,
>> > >> first use the camera to take a picture of the balloon, then capture the
>> > tilt
>> > >> angle using the built in protractor tool, and then do the math to
>> > calculate
>> > >> the angular diameter and then the height. I think it would interest many
>> > of
>> > >> us who are trying to answer the question of how high we are flying the
>> > >> platform. Thanks.
>>
>> > >> ---------- Forwarded message ----------
>> > >> From: Patrick Coyle <p...@coyles.com>
>> > >> Date: Wed, Sep 28, 2011 at 6:18 AM
>> > >> Subject: Thanks for update on height measurement approaches, re:
>> > >> photomapping session at Sunol AgPark CA on 9/24
>> > >> To: Mike Atkinson <mlatkin...@comcast.net>
>> > >> Cc: Roger Kubalek <ro...@sagecenter.org>, Stewart Long
>> > >> <stew...@publiclaboratory.org>, toep...@comcast.net
>>
>> > >> Mike,
>>
>> > >> Thanks for update. Has anybody tried to use the online task sites? In
>> > this
>> > >> case, it would be to seek a smartphone developer who could do what Mike
>> > has
>> > >> described.
>>
>> > >> Pat
>>
>> > >> On Tue, Sep 27, 2011 at 11:54 PM, Mike Atkinson <mlatkin...@comcast.net
>>
>> > >> wrote:
>>
>> > >>> Pat:
>>
>> > >>> I also looked at the golf and hunting range finders and discovered that
>> > a
>> > >>> unit with sufficient distance capabilities is pretty expensive--around
>> > $400.
>> > >>>  The hypotenuse of the triangle is about 1450 ft so a cheap unit would
>> > not
>> > >>> work.  It is possible that the math solution is the easiest.
>>
>> > >>> Finding the angle of elevation is easy and can be done with a straw
>> > glued
>> > >>> to a protractor and string with a weight.Also the weakness of the way I
>> > did
>> > >>> it is the apparent width of the balloon.  Since it was small when I
>> > measured
>> > >>> it any error in width really affects the calculation.  I am thinking
>> > about
>> > >>> how to use an inexpensive rifle scope with mil dot markers to improve
>> > the
>> > >>> process of estimating the apparent width of the balloon.  I think that
>> > it
>> > >>> may be premature to offer the details because they are primitive at
>> > best and
>> > >>> may not be of much use.  A really elegant solution would first use the
>> > >>> iPhone camera to take a picture of the balloon, then capture the tilt
>> > angle
>> > >>> using the built in protractor tool, and then do the math to calculate
>> > the
>> > >>> angular diameter and then the height.  However, I do not know how to do
>> > >>> that.
>>
>> > >>> So, I will think about the inexpensive solution for a while.
>>
>> > >>> Mike
>>
>> > >>> On 9/27/2011 11:09 PM, Pat Coyle wrote:
>>
>> > >>>> Thanks Mike,
>>
>> > >>>> All right, 700 feet +/- a bit! Our error bars are always open to
>> > >>>> discussion.
>>
>> > >>>> I really liked your approach, your other suggestions, this result and
>> > >>>> your idea for an iPhone app.
>>
>> > >>>> I'm tempted to post a description of the iPhone/other smart phone app
>> > on
>> > >>>> one of the on-line freelance task sites and request bids to do it.
>> > Maybe one
>> > >>>> of the brains on the rest of the planet can do it for under $50, if so
>> > I'd
>> > >>>> buy it. If $500, not me personally, but maybe a consortia, like the
>> > BAP/KAP
>> > >>>> community or a vendor to them (It could be us. Have you filed your
>> > record of
>> > >>>> invention yet? :->). I did do a quick look at golf laser range
>> > finders, but
>> > >>>> what I saw started at $200 and went on up to $400 + for one with
>> > better
>> > >>>> reviews.
>>
>> > >>>> Can you just detail a little more on the technique so we can post it
>> > as a
>> > >>>> tool others can use? Your description would just need a tiny bit more
>> > to
>> > >>>> provide a "how-to". Maybe a link to the URL, a sketch showing our
>> > traced
>> > >>>> angle, etc.
>>
>> > >>>> Alternatively, I could draft a strawman for your review. Let me know
>> > if
>> > >>>> that would help. I was really glad you made it. Good fun.
>>
>> > >>>> Thanks again,
>> > >>>> Pat
>>
>> > >>>> 9/27/11 6:56 PM, Mike Atkinson wrote:
>>
>> > >>>>> Hello Pat:
>>
>> > >>>>> I did a rough calculation on the height of the balloon.  Just to
>> > recap
>> > >>>>> the method, I used my multi-tool as crude sextant and marked the
>> > apparent
>> > >>>>> diameter of the balloon on the index card you gave me.  Using an
>> > online
>> > >>>>> calculator with the crude readings gives an approximate height of 700
>> > feet.
>> > >>>>>  I know that it is possible to get a much better result with better
>> > >>>>> measuring tools.
>>
>> > >>>>> Mike
>>
>> > >>>>> On 9/25/2011 10:32 AM, Patrick Coyle wrote:
>>
>> > >>>>>> Roger, Stewart, John and Mike,
>>
>> > >>>>>> I thought the photomapping session at Sunol AgPark yesterday was a
>> > lot
>> > >>>>>> of fun and I know we have some great images for mapping both visible
>> > and IR.
>> > >>>>>> We shot for over an hour and took over 3000 images on each camera
>> > (Canon
>> > >>>>>> A495 visible, A-490 IR (courtesy of Jeff Warren). While many will
>> > not be
>> > >>>>>> suitable, a quick look indicates we'll be able to find enough good
>> > ones for
>> > >>>>>> the mapping.
>>
>> > >>>>>> Thanks to all for participating. Special thanks to Stewart for his
>> > >>>>>> balloon and expertise.
>>
>> > >>>>>> Given the prevailing winds, we will likely need to see about access
>> > to
>> > >>>>>> the adjacent property on the other side of the creek and schedule
>> > another
>> > >>>>>> session to get the desired coverage of the filter strip/hedgerow
>> > feature and
>> > >>>>>> the area along the creek.
>>
>> > >>>>>> I haven't gotten to stitching the images into a map yet, but I am
>> > >>>>>> putting up some of the other photos and video. If in any of it,
>> > there is
>> > >>>>>> something you want removed, let me know and I'll take it down.
>>
>> > >>>>>> I started a playlist with video snippets from our session at:
>> > >>>>>>http://www.youtube.com/playlist?list=PLF788505DE175E36E
>>
>> > >>>>>> I have first ground photos at:
>> > >>>>>>http://www.flickr.com/photos/coylepdc/sets/72157627745065388/
>>
>> > >>>>>> John, one of the videos is basically just audio of our conversations
>> > >>>>>> while we tried to fly Mathew's Delta kite at the end. I think you
>> > shot
>> > >>>>>> photos and/or video of some of that, so when you can, please post
>> > that and
>> > >>>>>> let us know where it is. We might consider whether this audio make
>> > sense to
>> > >>>>>> combine with your materials. Plus we're all interested in all the
>> > other
>> > >>>>>> material you shot, so let us know where to look for it too.
>>
>> > >>>>>> I'll keep us updated as we get additional images and a draft map
>> > >>>>>> available.
>>
>> > >>>>>> Thanks again to all,
>> > >>>>>> Pat

--

Grant Fraley

--

Grant Fraley

--

Grant Fraley

[Tyler Depke]

Howdy all,

I don't know anything about putting the distortion parameters into a database, but if you want a super easy camera calibration program, there is a free one here that I am using for my thesis research. You essentially open the program, click a button to display a checkerboard pattern, take pictures of the checkerboard pattern, load them into the program and select what coefficients you want to solve for using Brown's distortion model (k1, k2, k3, p1, p2). You could probably get a calibration done in 10 minutes. The biggest thing is that for the small point and shoot cameras you will need a large projector because you want the checkerboard pattern to cover as much of the image as possible and you need to maintain your focus at infinity. Also, when you take pictures of the checkerboard pattern, make sure to rotate the camera as far as possible in a complete circle.

http://www.agisoft.ru/products/lens/

Tyler

--
Graduate Student/Teaching Assistant
Texas A&M University
High Alpine and Arctic Research Program
Department of Geology and Geophysics, MS 3115
College Station, TX 77843-3115
http://geoweb.tamu.edu/profile/TDepke

Video/Photographer
http://www.tylerdepke.com

[Grant W Fraley]

Jeff,

It looks like the Agisoft software Tyler has pointed out (also on the wiki page it looks like) would perform the same function as I would. If it doesn't meet your needs, let me know and I can certainly help out. 

I would be wary of using a projector. Tyler is right that a larger surface is better, but a flat surface is also a good thing and you may introduce some skewing or other effects from the lens of the projector. The better option would be to print a large poster and hang it on the wall if the Agisoft program would accommodate this.

Just in case, here you go...

Attached is a pattern that you can print out on an 8.5 x 11 and tape down to a white background surface. Snap ~20 pictures from various angles (including nadir / straight looking). Ensure that the edges of the checkerboard are visible in each frame. If you provide me with a download link, or ask me and I'll get you an FTP account on our server, then I can generate the calibrated camera model for you.

Grant

--

Grant Fraley

[Jeffrey Warren]

Will this work, or do I really have to print it out? http://archive.publiclaboratory.org/download/a490-model.zip

Adam, Olivia, or anyone else who has an SD1100, SD1200, or SD1300 (anything in that series ought to have the same lens, i think), would you mind printing that page out and taking these photos too? Even if we don't immediately create all the models, it'll be good to have these so we can go back and add them to the database. Maybe the easiest way would be to attach/upload them to this page: http://publiclaboratory.org/wiki/mapknitter-correcting-lens-distortion

Jeff

[Grant W Fraley]

Jeff,

The images of your screen seem to have worked alright.

The camera model calculated for the a490 is as follows:

K:

4020.8694    0.0000 1821.8153 

   0.0000 4021.4802 1385.1918 

   0.0000    0.0000    1.0000 

D:

  -0.0881 

  -0.3200 

   0.0010 

   0.0001 

   1.6784

The camera matrix K is defined as:

K =

   fx  0  cx 

   0  fy  cy 

   0  0   1 

where fx and fy are the focal length in pixels for the x and y dimension, and the point (cx, cy) is the principal point (which would be the center of the CCD in an ideal camera).

The distortion vector is defined as:

D =

  k1

  k2

  p1

  p2

  k3

where k1,k2, and k3 are the radial distortion coefficients and p1 and p2 are the tangential distortion coefficients.

If you want to checkout what the effect is of applying the correction (or "undistortion") to each of the original images you can download the output here:

http://terrapanlabs.com/publiclaboratory/a490-model_undistort.zip

For each input image there is an output of: 1) copy of the original, 2) corners detected in the original, and 3) geometrically corrected image with corners burnt in.

I apologize, but I haven't looked into the lens database format much. However, it seems that you should be able to plugin the camera model parameters directly now that you have them. Also, the spatial resolution vs altitude calculations that I posted previously will be different (i.e. more accurate) with the calibrated camera model, but I haven't updated the values at this time.

Cheers,

Grant

--

Grant Fraley

[Tyler Depke]

Jeff,

I think Grant already answered your question, but yeah, you shouldn't have to print the pattern to take pictures. All you need is a truly planar surface. Unless you are using a really old school CRT or a projector, you should get perfect results and I imagine even with a CRT or projector you are going to get values that are pretty close.

Tyler