How to interpret data in output files

[rmp...@gmail.com]

Hi all,

I am a first time user of Astrometry.net, and I have some questions regarding the output files from the browser-based astrometry program. First, let me say thank you for the incredible functionality and ease-of-use of the program. My questions are in regards to the axy.fits and rdls.fits output files. I am attempting to read these files using IDL, and while I am able to open them, I am not familiar with FITS files, and thus do not know how to interpret the data they contain. My overall goal is to compare the terrestrial magnitudes, extraterrestrial magnitudes, and current altitudes of every unique pair of stars in the image. My primary questions are these:

1) Which fields of the axy.fits file contain the x and y coordinates of the stars found?

2) Which fields of the rdls.fits file contain the RA and Dec of the stars found?

3) Is there any reference in either of these files to the catalog numbers and/or extraterrestrial magnitudes of the stars found?

The files are attached for reference. Finally, if there is any documentation on the program which I could consult, please let me know; I was unable to find anything that discussed the output files of the browser-based program. Thank you for your help.

Aloha,

Robert

[Dustin Lang]

Hi,

Try looking at the "corr.fits" file.  It has an entry for each star in your image that we think we matched to a reference-catalog star.

It has:

field_x, field_y -- measured x,y coordinates in your image; FITS indexing, so (1,1) is the center of the first pixel
field_ra, field_dec -- those x,y coordinates converted to RA,Dec
index_ra, index_dec -- reference catalog RA,Dec (J2000) coordinates
index_x, index_y -- those RA,Dec coordinates converted into your image x,y coords
flux, background -- measured flux and background estimate of stars in your image (these are very rough measures)

Oh, but I see that it doesn't contain the reference-catalog measured magnitudes.  Hmm....

The command-line version does have the "--tag-all" option, which copies information from the reference catalog into output files... in the 4200-series of index files that includes only the J-band mag from 2MASS, which itself probably doesn't help you much.

--dustin

[rmp...@gmail.com]

Hi Dustin,

Thank you very much for the response. It seems that the corr.fits file has just about everything I need. However, I still don't understand how to interpret the data. When I print the header, it shows me that there are 13 fields of data, but the data array has dimensions [88,45]. If I print out the first column of the array I get:

  64 139 100 151   0   0   0   0  64 123  79 240  96   0   0   0  64  81  62 231 140 151 218  55  64  48 112  11 140  40 119  17  64 139  98  63 103 224 183  37  64 123 102  53  73 248 229 130  64  81  62

 184  90  30  76  26  64  48 130 127 189  34 214 193   0   0   0 133   0   0   0   1  63 240   0   0   0   0   0   0  67  37 181 164   0   0   0   0

Which of these data are the pixel number, RA, Dec, etc? I've attached the corr.fits file for reference.

Thank you,

Robert

[rmp...@gmail.com]

Sorry, here's the corr.fits file.

[Dustin Lang]

It's a binary FITS table.  There are 13 columns, using a total of 88 bytes, and 45 rows.  What IDL library are you using to read it?  mrdfits?

NAXIS1  =                   88 / Bytes in row
NAXIS2  =                   45 / no comment


I don't use IDL, but I think you can do something like,

table = mrdfits('corr.fits')
print, table.field_x

--dustin

[rmp...@gmail.com]

Hi Dustin,

Thank you, that (almost) did the trick! I had been trying to use a different FITS reading procedure, but mrdfits() is what I should have used. The only difference was that I had to write:

table = mrdfits('corr.fits',1,header)

"1" is to force the procedure to read the first extension (0 is the default), and "header" is to return a structure which includes header tags.

Thank you again,

Robert

[Andrew Hood]

On 09/12/2014 12:33 PM, John Carter wrote:
> I've tried to open all of those fits files, but both Nebulosity and
> GraphicConverter say they are broken. They won't open with a text
> editor either. What app will display them?
Not all FITS files are images. Get yourself a copy of FitsView
http://www.nrao.edu/software/fitsview/ or fdump
http://heasarc.gsfc.nasa.gov/lheasoft/ftools/futils.html

Andrew

--
There's no point in being grown up if you can't be childish sometimes.
-- Dr. Who

[Andrew Hood]

On 10/12/2014 9:04 AM, John R Carter Sr wrote:
> Thanks for the information.
> I downloaded fitsview (Windows), but it won't open any of the .fits files that nova.astrometry.net has (rdls, axy, corr).
John, these aren't images. You'd need "fdump" or "fv" to display them.

There is a Windows download for "fv"
http://heasarc.gsfc.nasa.gov/lheasoft/ftools/fv/ however you should note
it does not particularly like Windows file/directory names with spaces
or "strange"characters, and takes ages to enumerate large directories. I
suggest you install it in c:\fv and make a directory under that to hold
the files you want to examine. If you want to type full path names in
the file open box, it wants forward slashes.

As far as I can tell "fdump"
(http://heasarc.gsfc.nasa.gov/lheasoft/ftools/ftools_intro.html) isn't
available as a native Windows program. It will compile with Cygwin. eg:

$ fdump /cygdrive/c/fv/an/axy.fits STDOUT - -
SIMPLE = T / file does conform to FITS standard
BITPIX = 8 / number of bits per data pixel
NAXIS = 0 / number of data axes
EXTEND = T / FITS dataset may contain extensions
COMMENT FITS (Flexible Image Transport System) format is defined in
'Astronomy
COMMENT and Astrophysics', volume 376, page 359; bibcode:
2001A&A...376..359H
SRCFN = '/tmp/tmp.fits.Msvmn4' / Source image
SRCEXT = 1 / Source image extension (1=primary)
COMMENT Parameters used for source extraction
HISTORY Created by Astrometry.net's image2xy program.
HISTORY SVN URL: svn+ssh://astrometry.net/svn/tags/nova/2014-02-19/util/
HISTORY SVN Rev: 24486
HISTORY SVN Date: 2014-02-13 21:46:27 +0000 (Thu, 13 Feb 2014)
HISTORY Visit us on the web at http://astrometry.net/
NEXTEND = 1 / Number of extensions
LONGSTRN= 'OGIP 1.0' / The OGIP long string convention may be used
COMMENT This FITS file may contain long string keyword values that are
COMMENT continued over multiple keywords. This convention uses the '&'
COMMENT character at the end of the string which is then continued
COMMENT on subsequent keywords whose name = 'CONTINUE'.
IMAGEW = 5184 / image width
IMAGEH = 3456 / image height
ANRUN = T / Solve this field!
More?[Yes] n

Andrew

PS Please don't reply to me. Reply to the group.

[S Eigell]

25 years as the predominant Consumer OS (even serious inroads into Academia and Scientific circles) and there STILL isn't a truly functional generic FITS viewer available for Windows ??!!??

And the Science Community is left to wonder why they have to Beg for Budgets and Grants, when there is such a Disconnect between "their world" and the larger community-at-large perpetuated by the failures to reach-out and capitalize on even the nominally "Science Oriented" folks who participate in hobbies such as Amateur Astronomy.

No wonder that Science Projects are left Begging while "Reality Stars" roll in such Riches as could fund entire Science Departments...

(sorry...  </soapbox>)

On Tuesday, December 9, 2014 4:02:33 PM UTC-7, Andrew Hood wrote:

On 10/12/2014 9:04 AM, John R Carter Sr wrote:
> Thanks for the information.
> I downloaded fitsview (Windows), but it won't open any of the .fits files that nova.astrometry.net has (rdls, axy, corr).
John, these aren't images. You'd need "fdump" or "fv" to display them.

There is a Windows download for "fv"
http://heasarc.gsfc.nasa.gov/lheasoft/ftools/fv/ however you should note

it does not particularly like Windows file/directory names.

 

As far as I can tell "fdump"
(http://heasarc.gsfc.nasa.gov/lheasoft/ftools/ftools_intro.html) isn't
available as a native Windows program. It will compile with Cygwin

 

Andrew

[Dustin Lang]

I hear and understand your frustration.  But most researchers in astronomy are so deeply in the Mac/Linux world that they can't easily open a .doc file, never mind build and package a windows executable.

[S Eigell]

Feeling a bit sorry for my minor tirade yesterday, I did find one Windows App which provides access to the FITS Header Data:  "F4W2HDU" - a free Windows GUI Utility offered from a larger Commercial Suite of FITS4WIN

(Unsure about the Group's Policy on linking to Commercial Software, so you'll need to search Astroshed for this tool).

[John Carter]

I downloaded and installed F4W2HDU. It opened the fits file produced by Astrometry.net. But I don't see anything in any of the files that answers this guy's question - and mine. All I see is header information. So what tool shows me the "stuff"? I suppose that might be "fdump". That requires compiling in Cygwin, which I have, and very few such programs have actually compiled for me. - Still frustrated, but only because I'm not a scientist trained to handle this kind of stuff. I just want to be able to take a picture and KNOW what it is I'm looking at - like confirming that I'm looking at what I think I'm looking at. That's what Plate Solving is supposed to be about, right?

[Dustin Lang]

Hi,

One option for looking at FITS BINTABLE (binary tables), which is what the .corr, .rdls, .axy files are, is the "tablist" program, distributed with the Astrometry.net code.  I believe that compiles cleanly in cygwin.  If it doesn't, please report here and we'll try to get it sorted out.

I guess I'm not sure what you're looking for as far as KNOWing what you're looking at.  For images that contain NGC, IC, Tycho, and HD stars, we label them in the annotation image (the details depend on the image scale), eg,

http://nova.astrometry.net/user_images/504588#annotated

And on the right-hand side there is a zoom-in series.  Between that and the RA,Dec, image size, and rotation... what more would you like to see?

To be fair, the annotation images are often empty -- we only label a limited set of objects.  We could label things like USNO-B or 2MASS stars, but those just have "phone number" names -- basically just labelling their RA,Dec.  Would adding the RA,Dec grid lines help?  Like this:

Thanks for your input!
--dustin

PS -- it would be easy enough to provide text versions of the FITS tables.  It sounds like that could be useful to people, right?

[John Carter]

Dustin, the image you showed is a good example of identifying major components doesn't show any labeling for IC or Tycho stars, and I'm sure that if they were there, the screen would be chock full of labels. So I'm sure some choice needs to be made about how many to include. In several images that I have uploaded, nothing was labeled, not even a star, and in each of those images there were at least two Tycho stars. Here's my case in point:

http://nova.astrometry.net/user_images/490439#annotated

There is at least ONE Tycho star in that image: TYC 2877-0913-1

... at magnitude +11.62 and several GSC objects, as identified by SkySafari Pro - assuming of course, that I am actually looking at Phaethon, and three concurrent images seem to show that I am.

So, if Tycho stars are being identified, why not that one?

John C.

[Dustin Lang]

Hi,

Tycho-2 stars are labelled if the radius is less than 0.25 degrees.  (On the diagonal.)

I should probably create an interactive annotation tool for the web site....

Meanwhile, the "plotann.py" command-line tool can be used to label on your own machine, but of course on Windows that's not going to be easy to build...

--dustin

[Andrew Hood]

On 15/12/2014 4:46 AM, Dustin Lang wrote:
>
> Meanwhile, the "plotann.py" command-line tool can be used to label on
> your own machine, but of course on Windows that's not going to be easy
> to build...

Dustin,
It is fairly easy to build the GIT code with 32bit Cygwin. Just ensure
you have all the prereqs before you start.
For 64bit Cygwin, some of the optional prereqs do not seem to be 64bit
clean, but I'll give it another go soon.
Andrew