--odds-to-solve explanations

[Maik Riechert]

Hi,

The help output of solve-field describes --odds-to-solve as "odds ratio at which to consider a field solved (default: 1e9)".

My first question is, what does it exactly mean? Which 'odds'? My feeling is the higher the odds, the higher the quality of the solve. 

The second question is about the mechanism itself: If I set it to 1e50, it will search until it finds such solution or gives up. Whereas if I leave it at 1e9, it can happen that it finds a 1e10 solution and quits even if it could have also found a 1e60 solution if it spent some more time searching, right? I also observed that even if I set it to 1e50 and no solve was found, it still writes the wcs file of the best solve it found so far. Is that right as well? If yes, then I could basically set odds-to-solve to 'infinity' and let it search for a given amount of time and then get the best solution it has found in this time, right?

Cheers

Maik

[Dustin Lang]

Good questions!

The help output of solve-field describes --odds-to-solve as "odds ratio at which to consider a field solved (default: 1e9)".

My first question is, what does it exactly mean? Which 'odds'? My feeling is the higher the odds, the higher the quality of the solve. 

Yes, exactly, to a point.  The code works by finding quadrangles of stars in your image, computing their shapes, and finding matching shapes in the index files.  Each of those is a proposed match between your image and a part of the sky.  Since there can be false matches (four-star features can look the same within error), it then predicts where *else* it should expect to find stars.  The "odds" threshold is how much better than random chance that prediction has to be.

Note that there is no "tweaking" at this point -- the prediction is evaluated entirely based on the four-star match.  The default 1e9 threshold is high enough that we basically never accept a false positive (except for cases where the stars in the image aren't randomly distributed in 2D -- eg, when there are sources in a straight line, or things like globular clusters).  But it could be that by rotating an initial solution by 0.001 degrees, we get a much better solution, so setting the odds higher doesn't guarantee you get a better final answer.

(there is an "odds-to-tune-up" option, to try to tweak up an ok-but-not-good-enough solution to see if that helps.)

Also, note that if the odds threshold is very high, then the index needs to have a lot of stars in the image to be able to hit the threshold -- the odds increase by a maximum amount per star, so a given odds threshold is also a threshold on the number of stars required, and since the density of stars is related to the size of the features in the index, that means that if you set the odds very high, it will only be able to get solutions from index files with small quads, and these will generally have more relative noise in the solution.

Given all that, I don't know if I would recommend your proposed approach!

 

The second question is about the mechanism itself: If I set it to 1e50, it will search until it finds such solution or gives up. Whereas if I leave it at 1e9, it can happen that it finds a 1e10 solution and quits even if it could have also found a 1e60 solution if it spent some more time searching, right?

Correct.

 

I also observed that even if I set it to 1e50 and no solve was found, it still writes the wcs file of the best solve it found so far. Is that right as well?

Hmm, I am surprised at that!  Is it possible the wcs file was from a previous run?  I would have expected that if it never gets a solution above the threshold, it shouldn't write a wcs file.

[Maik Riechert]

2013/12/17 Dustin Lang <dstn...@gmail.com>

Good questions!

The help output of solve-field describes --odds-to-solve as "odds ratio at which to consider a field solved (default: 1e9)".

My first question is, what does it exactly mean? Which 'odds'? My feeling is the higher the odds, the higher the quality of the solve. 

Yes, exactly, to a point.  The code works by finding quadrangles of stars in your image, computing their shapes, and finding matching shapes in the index files.  Each of those is a proposed match between your image and a part of the sky.  Since there can be false matches (four-star features can look the same within error), it then predicts where *else* it should expect to find stars.  The "odds" threshold is how much better than random chance that prediction has to be.

Thanks! This helps me a lot in understanding everything.

This quadrangle, can I see it in the results somewhere? I am wondering what the triangle in the ..-indx.png file is about.

 

Note that there is no "tweaking" at this point -- the prediction is evaluated entirely based on the four-star match.  The default 1e9 threshold is high enough that we basically never accept a false positive (except for cases where the stars in the image aren't randomly distributed in 2D -- eg, when there are sources in a straight line, or things like globular clusters).  But it could be that by rotating an initial solution by 0.001 degrees, we get a much better solution, so setting the odds higher doesn't guarantee you get a better final answer.

(there is an "odds-to-tune-up" option, to try to tweak up an ok-but-not-good-enough solution to see if that helps.)

And by tweaking, what exactly does that mean? Previously I thought tweaking=SIP, so where does rotation fit in? First rotation, then SIP? If --no-tweak is set, will odds-to-tune-up be useless? Is tune=tweak?

You know what would be really nice? A simple diagram of how the solving process works.

 

Also, note that if the odds threshold is very high, then the index needs to have a lot of stars in the image to be able to hit the threshold -- the odds increase by a maximum amount per star, so a given odds threshold is also a threshold on the number of stars required, and since the density of stars is related to the size of the features in the index, that means that if you set the odds very high, it will only be able to get solutions from index files with small quads, and these will generally have more relative noise in the solution.

I see, yes, in general I would think that bigger quads would yield a smaller error, if such quads are found. Right? Hmm, maybe not indefinitely big, because of distortion effects.

 

Given all that, I don't know if I would recommend your proposed approach!

Ok, I will think about it. In the end, my goal is to have results which have a very high quality, as the solution is afterwards used in a different process (I can only tell you by private email at the moment).

 

 

The second question is about the mechanism itself: If I set it to 1e50, it will search until it finds such solution or gives up. Whereas if I leave it at 1e9, it can happen that it finds a 1e10 solution and quits even if it could have also found a 1e60 solution if it spent some more time searching, right?

Correct.

 

I also observed that even if I set it to 1e50 and no solve was found, it still writes the wcs file of the best solve it found so far. Is that right as well?

Hmm, I am surprised at that!  Is it possible the wcs file was from a previous run?  I would have expected that if it never gets a solution above the threshold, it shouldn't write a wcs file.

Well, it could make sense I guess, as long as the .solved file isn't written which it is not.

[Dustin Lang]

The attached an.pdf are some slides from my talk about astrometry.net.  It doesn't include the "tweak" part, but it's a start.

Thanks! This helps me a lot in understanding everything.

This quadrangle, can I see it in the results somewhere? I am wondering what the triangle in the ..-indx.png file is about.

Exactly, it's drawn in the indx.png file.  (But not on the web version)

> And by tweaking, what exactly does that mean? Previously I thought tweaking=SIP, so where does rotation fit in? First rotation, then SIP? If --no-tweak is set, will odds-to-tune-up be useless?
> Is tune=tweak?

"tweak" means tuning up the solution to a given SIP polynomial order.  If tweak_order = 1, then it tunes up to linear order only (no SIP).

Odds-to-tune just tunes up to linear order, and is separate from --no-tweak.

> I see, yes, in general I would think that bigger quads would yield a smaller error, if such quads are found. Right? Hmm, maybe not indefinitely big, because of distortion effects.

 
Exactly right.

> Ok, I will think about it. In the end, my goal is to have results which have a very high quality, as the solution is afterwards used in a different process (I can only tell you by private email at the moment).

 
One thing I will suggest.  At the moment, it "tweaks" using only the stars in the index containing the matching quad.  This means that for large quads (large in the image), there are few stars so the tweak might not be very good.  However, if you re-run solve-field using the "--verify" option on the wcs file you get, it will then check that WCS file against ALL the index files, and tune up the best one.  Doing that can often produce improved results.

> Well, it could make sense I guess, as long as the .solved file isn't written which it is not.

The problem is that solve-field just calls the "astrometry-engine" (formerly "backend") program, and when that program completes and there is a *.wcs file, solve-field believes solving completed successfully.  It's ugly.  If there was an X.wcs file from a previous run, solve-field will probably claim that solving succeeded regardless of what really happened.

--dustin

[Maik Riechert]

2013/12/17 Dustin Lang <dstn...@gmail.com>

The attached an.pdf are some slides from my talk about astrometry.net.  It doesn't include the "tweak" part, but it's a start.

That's very good!

 

This quadrangle, can I see it in the results somewhere? I am wondering what the triangle in the ..-indx.png file is about.

Exactly, it's drawn in the indx.png file.  (But not on the web version)

I looked at quite a few indx.png files now and every one of them only has a triangle, with 3 stars, instead of a quadrangle in it.

One thing I will suggest.  At the moment, it "tweaks" using only the stars in the index containing the matching quad.  This means that for large quads (large in the image), there are few stars so the tweak might not be very good.  However, if you re-run solve-field using the "--verify" option on the wcs file you get, it will then check that WCS file against ALL the index files, and tune up the best one.  Doing that can often produce improved results.

I tried this (with default odds-to-solve) but at least with my images never got a result better than when using a high odds-to-solve value. The output is always like:

Verifying WCS using indices with quads of size [286.598, 4307.06] arcmin

Verifying WCS with index 1 of 6 (/.../index-4119.fits)

Verifying WCS of field 1.

 --> log-odds 7.22605

Verifying WCS with index 2 of 6 (/.../index-4118.fits)

Verifying WCS of field 1.

 --> log-odds 9.46849

Verifying WCS with index 3 of 6 (/.../index-4117.fits)

Verifying WCS of field 1.

 --> log-odds 7.12472

Verifying WCS with index 4 of 6 (/.../index-4116.fits)

Verifying WCS of field 1.

 --> log-odds 5.606

Verifying WCS with index 5 of 6 (/.../index-4115.fits)

Verifying WCS of field 1.

 --> log-odds 4.91154

Verifying WCS with index 6 of 6 (/.../index-4114.fits)

Verifying WCS of field 1.

 --> log-odds 4.22235

Got 0 solutions.

And then it starts the normal solving I guess, because the end result is the same.

For comparison, I attached a solution I got with default odds-to-solve (and --verify) which has 1e16 odds, and one with odds-to-solve=1e30 which has 1e32 odds. Just to clarify again: higher odds=higher *accuracy* meaning star matching?

 

> Well, it could make sense I guess, as long as the .solved file isn't written which it is not.

The problem is that solve-field just calls the "astrometry-engine" (formerly "backend") program, and when that program completes and there is a *.wcs file, solve-field believes solving completed successfully.  It's ugly.  If there was an X.wcs file from a previous run, solve-field will probably claim that solving succeeded regardless of what really happened.

I just checked, and at least for me it saves a wcs file and leaves it on disk (but it's not the best one it found!), and doesn't save a .solved file. If it would have been the best solve, it would be quite useful.

Cheers

Maik

[Dustin Lang]

Very strange.

About the triangles: some of the large-scale indexes have triangles rather than quads.

About the weird solving behavior: thanks for attaching images, that will help debugging!

--dustin