Local install

[Per Frejvall]

Hi all!

First: great software! I currently use it through my ACP/Pinpoint installation in a remote observatory tha is shared with three other rigs. We do have severe bandwidth limitations, so I am thinking of setting up a small local server with Ubuntu 14.04 LTS and a full Astrometry.net including Nova web front end.

Not being 100% at home in the environment, although with reasonable Unix and Linux experience dating 1982 and onward, I wonder if there are any more pointers.

I have managed to install the server with the prerequisites that I found, but I guess I have to suck the whole package with Git as the make install expects to get it from there.

Has anyone done this already? Any pointers?

All the best,

Per Frejvall,

Saltsjöbaden, Sweden

Remote at Gite Les Granges in Provence

[Dustin Lang]

First step would be to install the normal "solve-field" command-line program.  You can either get that from astrometry.net/downloads or clone from github: https://github.com/dstndstn/astrometry.net

Next you would need to set up your web service clone.  This is a bit involved and not well documented, though I don't mind trying to write some documentation to help you out.  The code for that is stored in the github repo (in the "net" subdirectory), but is not included in the tarball releases, so you'll have to git clone to get that.

cheers,
--dstn

[Per Frejvall]

Hi Dustin!

Thanks for getting back to me. I have an Ubuntu 14.04 LTS 64-bit server, have cloned the git repository, all dependencies in place and am right now (right very now) stuck on netpbm. It is installed but doesn't get detected by the "make reconfig" command. When I try to solve an apod it throws an error, which is totally related to netpbm not being available, which it is.

per@anet:~/astrometry.net/demo$ solve-field apod1.jpg --overwrite
Reading input file 1 of 1: "apod1.jpg"...
jpegtopnm: WRITING PPM FILE
Read file stdin: 800 x 526 pixels x 1 color(s); maxval 255
Using 8-bit output
Extracting sources...
simplexy: found 1776 sources.
cairoutils.c:729:cairoutils_read_ppm_stream: Netpbm is not available; can't read PPM images
cairoutils.c:729:cairoutils_read_ppm_stream: Netpbm is not available; can't read PPM images
libpng warning: Image width is zero in IHDR
cairoutils.c:729:cairoutils_read_ppm_stream: libpng warning: Image height is zero in IHDR
libpng error: Invalid IHDR data
*** longjmp causes uninitialized stack frame ***: /usr/local/astrometry/bin/plotxy terminated
======= Backtrace: =========
Netpbm is not available; can't read PPM images
cairoutils.c:729:cairoutils_read_ppm_stream: /lib/x86_64-linux-gnu/libc.so.6(+0x73f1f)[0x7f127c0abf1f]

Both netpbm and libnetpbm10-dev are indeed installed with apt-get.

Puzzled...

/per

[Jean-Paul Godard]

Make a search for Netpbm...

This is frequently discussed....

JPG

[Dustin Lang]

Hi,

I have been meaning to work on the netpbm detection code...  distributions keep moving it around, and there is no pkg-config for it.

In the Astrometry.net code base directory, try:

ln -s /usr/include netpbm
export NETPBM_INC="-I$(pwd)"
make reconfig


cheers,
--dustin

[Per Frejvall]

Nah, didn't do it. Which of the files should it point to? The two links below point to the proper files that I got from apt-get install:

http://packages.ubuntu.com/trusty/amd64/libnetpbm10/filelist

http://packages.ubuntu.com/trusty/amd64/netpbm/filelist 

/per

[Per Frejvall]

OK, I'm past that now. I needed libnetpbm10-dev as well... It now passes make reconfig and I will pursue further. I'll report back later...

/per 

[Per Frejvall]

Woah! Almost!

When I symlinked and forced the NETPBM_INC, did a make clean and the made it all again it compiled and runs.

There appears to be a glitch at the end, and I cannot find the coordinates in the resulting new fits, but it is progress.

  log-odds ratio 1272.75 (inf), 406 match, 135 conflict, 1760 distractors, 415 index.
  RA,Dec = (23.4748,30.6475), pixel scale 1.07568 arcsec/pix.
  Hit/miss:   Hit/miss: ++++-++++++++++++++++++++++++++++++++++++++++++-++++++++++++-+++++++++++-++++++++++++++++++++++-++++
Field 1: solved with index index-4204-01.fits.
Field 1 solved: writing to file astrometry.net/demo/M33.solved to indicate this.
Field: astrometry.net/demo/M33.fts
Field center: (RA,Dec) = (23.48, 30.65) deg.
Field center: (RA H:M:S, Dec D:M:S) = (01:33:54.002, +30:38:50.378).
Field size: 59.6236 x 44.8897 arcminutes
Field rotation angle: up is 89.6989 degrees E of N
Creating new FITS file "astrometry.net/demo/M33.new"...
Creating index object overlay plot...
Creating annotation plot...
Your field contains:
  NGC 598 / M 33 / Triangulum galaxy
augment-xylist.c:1392:augment_xylist Failed to parse WCS header from file "/tmp/tmp.sanitized.wfQbcF"
 sip_qfits.c:259:read_header_file Failed to read FITS header from file "/tmp/tmp.sanitized.wfQbcF" extension 1
 anqfits.c:517:anqfits_get_header_const failed to get header start + size for file "/tmp/tmp.sanitized.wfQbcF" extension 1
 anqfits.c:391:anqfits_header_size Failed to get header size for file "/tmp/tmp.sanitized.wfQbcF" ext 1: ext not in range [0, 1)
 anqfits.c:381:anqfits_header_start Failed to get header start for file "/tmp/tmp.sanitized.wfQbcF" ext 1: ext not in range [0, 1)

/per

[Dustin Lang]

Could you please post the troublesome M33.fits image?  And what is the command line you are running?

The new FITS file should contain a standard WCS header.  The reference pixel is given by CRPIX*, the reference RA,Dec by CRVAL*.  If you want that to be the center of the image, add --crpix-center.

cheers,
--dstn

[Per Frejvall]

Hi again!

It solves fine if I remove the previous CRVAL and CRPIX that Pinpoint put there. The file is at http://filer.frejvall.se/test1.fts It is the version that bombs out. If you strip the header items it will pass everything.

I find that the solved coordinates are slightly different to those reported by Pinpoint. Have you done a comparison or is that perhaps impossible... I use GSC-11 with Pinpoint.

Oh, the command line I used for the test was: solve-field -L 1 -H 1.2 -u app test1.fts --overwrite

So, I guess the solver works, now I need to dig into the Nova stuff so I can add a local front-end on the same machine. Lots of config items there... ;)

All the best,

Per

[Dustin Lang]

I don't have/use pinpoint, but if you want to compare and post results, go for it...

About that "glitch" you pointed out at the end: that was a bug, which I (think I) just fixed on master... could you "git pull", "make", and see if it persists?

Thanks,
--dustin

[Per Frejvall]

I will post differences once I get everything going :)

I will pull and test for you - no problem. I'm getting the hang of rebuilding that stuff - haha!

I have found another interesting item with the WCS. I have tested this with the official Nova and with my own install. Nova reports the same solution as my local install when I look in the json calibration. I have written my own client in C# (which I intend to use for my local Nova in the observatory).

When I look at the  WCS headers in the .new fits file, they are not the same. This has also been verified against Nova by downloading the WCS text file.

Solve-field says: Ra 23.4741,       Dec 30.6498
WCS says:         Ra 23.0767808716, Dec 30.7358379184

Yiekes! Which of these values are to be trusted? Pinpoint says the reported, not the WCS.

All the best,

Per

[Dustin Lang]

How are you getting the "WCS" values?  Are you looking at one of the header cards, like CRVAL?  Those only give you the location of the *reference* pixel, NOT necessarily the center of the image.

cheers,
--dstn

[Per Frejvall]

OK, so the actual solution is not in there?

I have pulled, made and installed. It now solves all my three test images with varying numbers of cards in their FITS headers. Also, the CRVAL values in the WCS file and the new fits made by the the solver match. I ran exactly the same files.

I have the notion that somewhere in the header cards there should be a full and very accurate solution. My personal interest in plate solving is for model making. I do only unguided imaging with subs up to an hour, something that requires a really good and encoded mount along with a good sky model. I shoot about 100 subs, plate solve them and sync my mount on them, which in turn causes the mount to add the syncs to the sky model.

The way I write the model making software makes use of the direct values from Astrometry.net, but I would like the headers to reflect reality as well.

The mount, by the way, is an Italian job called 10Micron, and I have two of them, a GM2000HPS II in my remote and the smaller brother, GM1000HPS, on the balcony.

/per

[Dustin Lang]

I am confused about what you are asking.

The WCS header (the just-the-header file, X.wcs, or the new FITS file with header, X.new produced by solve-field) DOES describe a full pixel-to-sky transformation.  It is defined in excessive detail in "paper 2" linked here: http://fits.gsfc.nasa.gov/fits_wcs.html
The transformation has a "reference point", described by CRPIX and CRVAL, and a transformation matrix between pixel and intermediate world coordinates, the CD matrix.  Astrometry.net also has the option of computing "SIP" polynomial distortion parameters, described by A_#_#, B_#_#.

--dstn

[Per Frejvall]

Yes, I think I understand. What I am looking for is the exact geometrical center of the image, and only that. I suspect it is what solve-field reports:

Field center: (RA,Dec) = (23.47, 30.65) deg.
Field center: (RA H:M:S, Dec D:M:S) = (01:33:53.720, +30:38:57.282).
Field size: 59.7056 x 44.9554 arcminutes
Field rotation angle: up is 89.6975 degrees E of N

I took the CRVAL values and converted to HMS and it is not the exact same as the HMS value reported per above, so I assume those are computed properly with the transformation, right? If so, they could be put in the headers as well.

/per

[Dustin Lang]

If you add "--crpix-center" to solve-field, it will make the reference pixel the center pixel in the image, and then CRVAL will be the RA,Dec of the center pixel.

Without that, the reference pixel can be anywhere in the image and the CRVAL will be the RA,Dec of that reference pixel.

There is a program, wcs-xy2rd, for converting pixel coords to RA,Dec coords.

Indeed, solve-field reports the center position, like you guess.

--dstn

[Per Frejvall]

Got it! Great! I can fiddle that into an "amateur standard" FITS word in the header :)

I now understand everything much better, and, on top of that I have realized that Pinpoint appears to do FK5 coordinates... Hmmm... I'll do some thinking, but I haven't done any real maths for ages.

So, on to the Nova part, then.

Signing off with a great Thank You!

/per 

[Dustin Lang]

There is nearly 0 documentation on how to set up a new "nova" server.  One scrap here,

http://astrometry.net/doc/net/orientation.html#running-a-local-machine-your-laptop-say

(but it won't actually solve any jobs)

and another set of notes that probably only I understand here:

https://github.com/dstndstn/astrometry.net/blob/master/net/README

I'm going to write something more complete... but try running through the first set of instructions and see how that goes.

--dstn

[Dustin Lang]

I just put together some new documentation on setting up a "nova" clone:

http://astrometry.net/doc/nova.html


Please let me know when/where you get lost.

cheers,
--dustin