Correct SIP order (tweak oreder)

[Toni Šarić]

I was wondering is it possible to determine the correct SIP/tweak order coefficient?
Is there any method to calculate the level of distortion?
Normally I use tw=4 because it gives me minimum separation distance between stars on the image and stars from the catalog.
The reason I am asking is that I need to know RA Dec coordinates of one particular pixel in my image and that pixel is located at the very edge of my image.

[Dustin Lang]

You would have to define "correct" :)  The SIP terms are polynomials, so the more you add, the better the fit becomes -- but at some point it starts overfitting, and polynomials are known to be terrible when extrapolating outside the area they were fit on.

I think that if you measured the scatter (eg in the corr files) for different tweak levels, you would find that it would keep dropping as you increase the tweak order, but hopefully it would drop quickly until you hit the "right" order, and then only decrease a bit after that.  No matter what you do, getting the distortion at the very edge of the field is going to be tough!

I would recommend using --crpix-center also -- center the distortion solution at the center of the image.

cheers,

--dustin

--
You received this message because you are subscribed to the Google Groups "astrometry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astrometry+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/e0cea410-e891-45ca-a298-6e6ab9e83ab6n%40googlegroups.com.

[Toni Šarić]

Hi.

Yes, I already have set --crpix-center.

I understand now, thanks.

By measuring the scatter (in corr files) you mean measuring standard deviation of separation between field stars and index stars?

BR

Toni

[Dustin Lang]

Yes, exactly.

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/66241391-6369-42bf-8696-f5c7f16ca30fn%40googlegroups.com.

[Toni Šarić]

Hi Dustin.
I solved a few images with a tweak order from 0 to 5.
Below the text are the results.
The first column is the degree of tweak order. The second and third are mean separation(measured as the average on-sky separation between field and index stars) and standard deviations in arcsec, respectively:

tw=0   7.184726''   3.835836''     2.213243

tw=1   10.922670  5.403251      2.722024

tw=2   7.184726    3.835836      2.213243

tw=3   2.057162    1.731583      3.660477

tw=4   1.843406    1.531140      3.946095

tw=5   1.446841    1.407375     14.701140

So, increasing the tweak order gives better and better results.

As I mentioned before, I am interested in one particular pixel at the very edge of the image. I am comparing RADec of that pixel with a RADec of a known star near to it.

The last column gives results for that separation, in arcminutes. So, by increasing tweak order, the separation between those two points also increases, but if solving is better(more accurate as we can conclude from the separation results), I am expecting it to decrease.

BR

Toni

[Dustin Lang]

Hi,

This is exactly the behavior I would expect.

SIP is a polynomial fit, so the higher the order, the more ability it has to fit, or *over-fit*, the data.

https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html

Also, polynomials are *very bad* at extrapolating.  On that web page above, look at how the blue line wiggles in the right-hand plot, and notice how it shoots off to very large values at the left-hand side of the plot.  That is: the SIP polynomial will be fit on matched stars across your image, so *between* matched stars, the solution should be good.  But as soon as you go outside the "convex hull" of the matched stars, you should expect the SIP predictions to wobble around, worse and worse as the order increases.  Making predictions at the edge of the image is the worst-case scenario.

cheers,

--dustin

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/98fce09a-3e4b-45d6-97ca-c28edf80431an%40googlegroups.com.

[Bryan]

I had a professor who would say "Give me enough degrees of freedom and I can fit an elephant with a polynomial"

....but the standard deviation will  be horrendous, i.e., "overfit" the data.

Bryan

[Dustin Lang]

That's a famous John von Neumann quote :)

https://www.johndcook.com/blog/2011/06/21/how-to-fit-an-elephant/

https://twitter.com/fermatslibrary/status/1235582404388982785

As Toni's results show, as you add more parameters to the fit (higher SIP order), the "residuals" (difference between the SIP model and the measured star positions) will decrease.  If you add enough parameters (as many as you have data points), you can make the curve go exactly through all the data points.  But between the data points, its predictions will be *terrible*.  And outside the fitting region, its predictions will shoot off to infinity (or -infinity).

Now, what's the practical solution?  For this particular case -- making a prediction at the very edge of the image -- I think it's really tough.  Real lenses can do weird stuff at the edges of their fields, so I don't know if it is reasonable to say that the SIP order can only be, say, 3.  So to really nail it down, I think you need a data set where you actually have measurements of stars at the edges of the image.  You might have to get that by "stacking" measurements from many images so that you can place stars in a grid and nail down what the distortion is doing out there at the edge.  (There's not existing code in the Astrometry.net package to do that though...)  OR, if this is a commercial lens, there may be manufacturer or other data sheets about what the distortion is.  Measuring stars isn't always the best way to do things :)

cheers,

--dustin

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/d6497dc3-66aa-42b6-9c04-45f5831c7930n%40googlegroups.com.

[Toni Šarić]

Thanks you very much Dustin for your thorough answer and clear explanation.

I will use tweak_order=2 from now on.

I noticed in .corr file fairly large separation between index and field star of 180 arcseconds for one particular star while mean separation is around 11' with std of 22''.

Would it be smart to remove that star from solving? Will I get better accuracy if this star is not used?

I can exclude it, and other similar(bad) matches that with parameter "pixel-error", right?

BR

Toni

[Dustin Lang]

The tweak algorithm downweights matches at large radius -- there corr file should have a column named WEIGHT or something like that.

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/68d1909a-b2e0-4b73-8312-19a7ada25148n%40googlegroups.com.

[Toni Šarić]

Yes it does.

It gives weight of 0.5 for that particular star.

Just for clarification, 180 arcseconds in my image is roughly 10 pixels difference.

If I put "pixel-error" = 0.2, that star is excluded from the solving.

But still there are other matched stars with separation ~ 14 arcseconds which is more than 0.2 pixels, but it shouldn't be, if I understood that parameter correctly.

My apologies for so many questions, if you have some user guide with more detailed explanation for parameters, it would be really helpful.

BR

Toni

[Dustin Lang]

pixel-error isn't a maximum, it's a Gaussian standard deviation

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/274645c6-2ebf-4d28-84e1-a7fbaf22f944n%40googlegroups.com.

[Toni Šarić]

That makes sense, thanks for clarification.