Trying to get command line astrometry to work as fast as web version

[Ann Marie Cody]

Hi,
 Having now gotten astrometry.net to work, I am attempting to optimize it to operate on a few thousand images, many of which are of the same field. I know my images sizes, so I specify a small range for that.
When I run an example image through the online system, I get a solution fairly quickly (<30s). However, when I run seemingly the same thing via the command line,
it either fails to solve or takes a very long time (>20 minutes). I was wondering if someone could take a look at my most recent image, which is under job 211069.

I took this image and ran the following command:
solve-field  --scale-units degwidth --scale-low 0.263 --scale-high 0.265 testimage.fits

I have downloaded the index files index-4205*.fits since my images are 15.9' wide. They were sufficient to solve another image of the same field.
Note that I also tried specifying the exact RA and Dec of the center (as reported by the online solution), and it still didn't work!

So the question is-- why does the solution pop up quite quickly and correctly on the web, whereas it fails to solve with the above on the command line?


thanks,
Ann Marie

[Dustin Lang]

Hi,

For that sample image you sent, the web version cheated -- it noticed that your image already had a WCS header, and "verified" it rather than searching anew.

You might also want to grab the 4204* index files -- each scale contains "quad" (four-star) features within a square-root-of-2 range of diameters, so it helps to have a little more overlap in scale.

You might want to check that the source extraction (star detection) looks okay -- check out the *-objs.png plots and ensure the brightest stars are detected and centered in the red circles.

You could also try re-submitting that example image, but removing the FITS WCS headers (or just change, say, CRVAL) so that the "verify" routine will fail and it won't be able to cheat.

cheers,
--dustin

[Ann Marie Cody]

Hmm, something doesn't quite make sense, because the wcs header in the original image is totally *wrong*. That's why I wanted astrometry.net to refit it,
and it indeed produced a correct WCS. So I don't see how it is verifying the existing WCS if it ultimately knows that one is incorrect.

The vast majority of my images have erroneous WCS fits, so I'm hoping to amend them. I tried erasing what I thought were the WCS header keywords
CRVAL, etc. but somehow the erroneous RA and Dec are still displaying. Perplexing!

What should help is that I know all of the field centers to within an arcminute. I tried inputting that with my test above, and it still didn't find a solution. Any idea
why that didn't work?

[Ann Marie Cody]

Hi again,
  I also tried the index-4204* files, and still no solution. I am now testing on an additional image, which you can find under job 212300. The input image had a terrible wcs solution in which east and north
were pointed in almost the same direction, and neither pointed "up" in the image. NEvertheless, the online program found the correct solution quite rapidly. When I try it via the command line, it does
tell me "Found an existing WCS header, will try to verify it." But it then finds no solution, despite my giving it the exact center RA and Dec that the online program found.
The output is:
Verifying WCS using indices with quads of size [0.255085, 2.55085] arcmin
Got 0 solutions.

This makes sense, since the initial wcs is in fact wrong.
So it then proceeds to look for new solutions and doesn't find any.
I get the same behavior (minus the verifying step) when I erase the WCS header keywords.

Any more thoughts on why this isn't working?? I'll attach part of an object file (whole thing is too big)-- it is identifying the brightest stars, but also finding some potentially spurious things.

thanks,
Ann Marie

[Andrew Hood]

Ann,
It doesn't work as well as it would with the original image, but with your screenshot I got a solution with the 4203* indexes.
CRVAL1  =        100.444735151 / RA  of reference point
CRVAL2  =        9.76968821232 / DEC of reference point
CRPIX1  =                  661 / X reference pixel
CRPIX2  =                  651 / Y reference pixel
COMMENT index id: 4203
COMMENT index healpix: 23
COMMENT index hpnside: 2
COMMENT scale: 0.704741 arcsec/pix

Is that what you expected?
Andrew

--
You received this message because you are subscribed to the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4259 / Virus Database: 3681/7019 - Release Date: 01/20/14

-- 
There's no point in being grown up if you can't be childish sometimes.
-- Dr. Who

[Andrew Hood]

Also Ann, once I knew where to start the search,  solve-field with these extra options:

-L 10 -H 20 -u amw --ra 06:41:46.760 --dec +09:46:11.233 --radius .25 -r --downsample 4

only listed 8 indexes to be tried, did not match on objects 1-10 but did on 11-20.

Fiddling with tweek-order moved the centre a bit but not a lot.

Andrew

[Dustin Lang]

Hi Ann Marie,

I'm looking at this image:
http://nova.astrometry.net/status/165258

and in the header of the uploaded image I see:

CTYPE1  = 'RA---TAN'           / Type of WCS Projection
CTYPE2  = 'DEC--TAN'           / Type of WCS Projection
CRPIX1  =           975.911057 / Reference pixel on axis 1
CRPIX2  =          1004.763634 / Reference pixel on axis 2
CRVAL1  =     100.435824392515 / Value at ref. pixel on axis 1
CRVAL2  =     9.76032881887849 / Value at ref. pixel on axis 2
CD1_1   =  -1.3051132996688E-4 / Transformation matrix element
CD1_2   =  -1.8965668603616E-6 / Transformation matrix element
CD2_1   =  1.83038256029448E-6 / Transformation matrix element
CD2_2   =  -1.3051326996567E-4 / Transformation matrix element

Which has north and east nearly orthogonal, and a correct center.

Nova.astrometry.net didn't do any blind search on that image, it just verified the original WCS.  And you can check that the answer is right by looking at the "red-green" plot,
http://nova.astrometry.net/user_images/197386#redgreen
where the green circles are objects in the reference catalog -- they encircle many many stars visible in your image.

Am I looking at the wrong image?  Why do you believe the original WCS is totally wrong?

thanks,
--dustin

[Ann Marie Cody]

Hi Dustin and Andrew,
 Thanks a lot for your comments- very helpful. I think I have figured this out now (at least it works on my three test images), so here's the summary:

--Dustin, you were right-- the image we were discussing (testimage1.fits) did not have N and E in the wrong place-- I was getting it confused with my other file, testimage.fits, which did have that problem.
 So when I simply run solve-field testimage1.fits, it does verify the wcs and come up with a solution. The problem is that I was giving it other options that were apparently preventing a solution (see next comment).

--Andrew, your solve-field options ended up showing me what the problem was: I was giving the radius too small a value. I had it set to 0.05, as I didn't realize that this is the region
of the index files to search in, and *not* the "uncertainty" on the center ra and dec. I kept giving it the precise ra and dec solution and wondering why it wasn't solving!

--Furthermore, the downsampling seems to help coax the solution out faster.

Well, now it is time to try this on the other few thousand images I have- wish me luck!

Thanks again,
Ann Marie

[Andrew Hood]

Pleasure Ann. An extra batch of "little grey cells", and eyes, never go astray.
Andrew

--
You received this message because you are subscribed to the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

No virus found in this message.
Checked by AVG - www.avg.com

Version: 2014.0.4259 / Virus Database: 3681/7023 - Release Date: 01/21/14