Different approach from attempt to stitch (super-resolution?) night sky (late 2014)

[Benjamin Hill]

Super-resolution from highly overlapping photos seems hard.  Maybe impossible?

I have taken a series of night-sky images from a fixed camera over a period of several hours.  I identified keypoints (stars) in the images, and correlated the keypoints across frames.  This results in a file of ~1000 rows of 4 ints:

int FRAME_A, int STAR_B, int PIXEL_X, int PIXEL_Y (and timestamp of the frame if it would be helpful)

The challenge is that it was taken with a wide-angle lens, and I don't know the exact distortion parameters of the lens.  

Goal: use the input file to either determine the lens model, or another mathematical model (machine learning?), that will allow a very precise function that maps a "true" night sky location to the coordinates in every frame that covers that part of the sky.  It is ok if the model takes a long time to train, but the final lookup should be fast.

getAllOverlappingExactSourceLocations(x1, y1) -> 

array of frames [{source_frame:frame_a, source_pixel_x:(float)x, source_pixel_y:(float)y}, {source_frame:...]

This may be easier to think of in the inverse: given a panoramic landscape shot taken with a slowly moving motorized camera, and choosing any ray from the camera's location, get the intersections with that ray and all the photos that overlap that ray.

It feels like this should be easy using Hugin or PTGUI: At the end of the day, I've got very accurate keypoints, and I should be able to get an accurate lens model from them... right?  And once I have an accurate lens model, I should be able to get each frame's yaw/pitch/roll with sub-pixel accuracy.  And then I... er... need help.  How do I use this model + a frame's yaw/pitch/roll to get source pixel locations for the final frame, that is pretty much what the final stitcher is doing, true?  So I'd need to reimplement my own emblend?  Any easier way?

[bran...@yahoo.com]

It feels like this should be easy using Hugin or PTGUI: At the end of the day, I've got very accurate keypoints, and I should be able to get an accurate lens model from them... right?  And once I have an accurate lens model, I should be able to get each frame's yaw/pitch/roll with sub-pixel accuracy.  And then I... er... need help.  How do I use this model + a frame's yaw/pitch/roll to get source pixel locations for the final frame, that is pretty much what the final stitcher is doing, true?  So I'd need to reimplement my own emblend?  Any easier way?

What is the end goal? As for getting the correct lens numbers if you have a full 360, it should be able to get them at least when shot in daylight. I am not too sure with what you are doing how much the sky may have moved between your first shot and last shot or if it would even matter when figuring out the lens numbers.

[panostar]

In this tutorial - http://www.johnhpanos.com/starcal.htm - I describe how to evaluate lens parameters by aligning camera images with a 360x180 equirectangular star image using PTGui, though Hugin will work equally well.  However, the method is somewhat flawed since due allowance needs to be made for refraction in the earth's atmosphere, which changes the apparent position of stars as you get nearer to the horizon.  The method does work quite well using the upper part of the visible hemisphere.

John

[Matthew Petroff]

I would recommend looking at Astrometry.net [1], which does blind astrometric calibration of images of the night sky. This give you FITS World Coordinate System metadata for the images [2] that maps image pixels to celestial coordinates. However, it does have some issues with fisheye lenses, although there seem to be workarounds [3].

-Matthew

[1] http://astrometry.net/use.html
[2] http://fits.gsfc.nasa.gov/fits_wcs.html
[3] https://groups.google.com/forum/#!topic/astrometry/RumO4M7an0E