Astrometry.net Debian/Ubuntu package?

[Jasem Mutlaq]

It would be great if someone can prepare a debian/ubuntu package for astrometry.net, this will make installing the package immensely easier for most people who don't want to compile and install dependencies...etc. You can even link astrometry.net subversion to Ubuntu's launchpad and have daily builds. Even the index files can be available as multiple packages.

I tried my luck with creating a debian package but faced a few difficulties, and would appreciate it if more experienced packages can step in! Thanks for the great work folks!

[Dustin Lang]

I would be happy to provide support for anyone interested in doing this.  I use ubuntu on my machines, including the servers that run the nova.astrometry.net web service.

cheers,
--dustin

[Jasem Mutlaq]

Thanks, I was actually able to build an Ubuntu package, but it requires some changes to your Makefile structure to be compatible with GNU makefile policy for the package to work. I am working from your SVN trunk and I will submit a patch to you soon :-)

[Dustin Lang]

Great, I would be pleased to receive any patches, or help you navigate the twisty paths of the Makefiles...

cheers,
--dustin

[Jasem Mutlaq]

Sure, see attached diff. Here is a summary of the changes:

1. All INSTALL_DIR was changed to DESTDIR to honour GNU Makefile policy. Without this, the package will not even build.
1. Setting INSTALL_DIR to use DESTDIR to honour GNU Makefile policy. Without this, the package will not even build.

2. I had to add math library to the linker at some places (-lm), otherwise it will not link. Oddly, this is not required if you compile it under a regular Ubuntu system, but it failed under pbuilder chroot jail, which is what is actually used to build packages.

3. Defined CONF_BASE_DIR so that the string replacement in the configuration file is done properly.

Regards,
Jasem

[Jasem Mutlaq]

Sorry I just noticed that I sent two #1s. The first one is invalid. This is what happens when you send emails at 4:30 AM :-)

[Dustin Lang]

Hi Jasem,

This is great!

I'm just reviewing the patch and have a few comments/questions:

```
Index: gsl-an/Makefile
===================================================================
--- gsl-an/Makefile    (revision 23745)
+++ gsl-an/Makefile    (working copy)
@@ -175,6 +175,7 @@
 
 # "sort" removes duplicates.
 GSL_OBJS := $(sort $(QR_OBJS) $(CHOL_OBJS) $(SVD_OBJS) $(LU_OBJS) $(EXTRA_OBJS))
+GSL_LIBS := -lm
 
 $(GSL_OBJS): config.h
 
```

I don't think that does anything... the variable GSL_LIBS does not appear anywhere else that I can see.  Were you trying to add a "-lm" everywhere the GSL library is linked?  If so, that should go in util/makefile.gsl .

But easier would be to add a line like,

LDLIBS := $(LDLIBS_DEF)
LDLIBS += -lm

like in blind/Makefile, to the other directories that link executables.  I see util/ already has one, so I'm not sure what else is required.

```
Index: catalogs/Makefile
===================================================================
--- catalogs/Makefile    (revision 23745)
+++ catalogs/Makefile    (working copy)
@@ -89,6 +89,7 @@
 
 all: ngc2000.py $(LIBCAT)
 
+PY_INSTALL_DIR := $(PY_BASE_INSTALL_DIR)/util
 install: $(PYTHON_INSTALL) $(LIBCAT) $(HEADERS)
     @echo Installing in base directory '$(INSTALL_DIR)'
     mkdir -p '$(PY_INSTALL_DIR)'
```

No, PY_INSTALL_DIR is defined earlier in that file,

PY_INSTALL_DIR := $(PY_BASE_INSTALL_DIR)/catalogs

```
Index: libkd/Makefile
===================================================================
--- libkd/Makefile    (revision 23745)
+++ libkd/Makefile    (working copy)
@@ -146,7 +146,7 @@
 
 ALL_TEST_FILES = test_libkd test_libkd_io test_dualtree_nn
 ALL_TEST_EXTRA_OBJS =
-ALL_TEST_LIBS = $(LIBKD) $(ANFILES_LIB) $(ANUTILS_LIB) $(QFITS_LIB)
+ALL_TEST_LIBS = $(LIBKD) $(ANFILES_LIB) $(ANUTILS_LIB) $(QFITS_LIB) -lm
 
 include $(COMMON)/makefile.tests
 
```

Only problem is, in util/makefile.tests, ALL_TEST_LIBS is used like this,

```
test: test.o $(COMMON)/cutest.o $(ALL_TEST_FILES_O) $(sort $(ALL_TEST_EXTRA_OBJS)) $(ALL_TEST_LIBS)
    $(CC) -o $@ $(CFLAGS) $(LDFLAGS) $^ $(ALL_TEST_EXTRA_LDFLAGS)
```

so I think you should add "-lm" to ALL_TEST_EXTRA_LDFLAGS instead.

```
Index: blind/Makefile
===================================================================
--- blind/Makefile    (revision 23745)
+++ blind/Makefile    (working copy)
@@ -214,6 +214,7 @@
 
 PYTHON_EXECS := plotann.py
 PYTHON_INSTALL := $(PYTHON_EXECS) __init__.py
+CONF_BASE_DIR := $(INSTALL_DIR)
 
 install: $(INSTALL_EXECS) $(INSTALL_LIB)
     @echo Installing in directory '$(INSTALL_DIR)'
@@ -223,8 +224,7 @@
         cp $$x '$(INSTALL_DIR)/bin'; \
     done
     mkdir -p '$(INSTALL_DIR)/etc'
-
-    python -c "import os; print open('../etc/backend.cfg-dist').read().replace('INSTALL_DIR', os.environ['INSTALL_DIR'])" > '$(INSTALL_DIR)/etc/backend.cfg'
+    python -c "import os; print open('../etc/backend.cfg-dist').read().replace('INSTALL_DIR', '$(CONF_BASE_DIR)')" > '$(INSTALL_DIR)/etc/backend.cfg'
     mkdir -p '$(INSTALL_DIR)/include'
     @for x in $(INSTALL_H); do \
         echo cp $$x '$(INSTALL_DIR)/include'; \
```

Could you explain what the issue was here?  What was wrong with using os.environ?

OHHHH, good catch!, I see that I recently commented-out the link "export INSTALL_DIR", which asks Make to put that variable in the environment when it runs recipes.  I just reverted that change.  With that fixed, I think the CONF_BASE_DIR shouldn't be necessary.  But maybe you needed that for some other reason?

```
Index: util/makefile.common
===================================================================
--- util/makefile.common    (revision 23745)
+++ util/makefile.common    (working copy)
@@ -16,7 +16,8 @@
 # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
 
 
-INSTALL_DIR ?= /usr/local/astrometry
+DESTDIR ?= /usr/local/astrometry
+INSTALL_DIR = $(DESTDIR)
 #export INSTALL_DIR
 
 # Installation subdirs
```

Is there a way to make it so that INSTALL_DIR, if specified on the command line, it is used, and likewise, if DESTDIR is specified, it is used?  Maybe something like,
```
INSTALL_DIR ?= /usr/local/astrometry
DESTDIR ?= $(INSTALL_DIR)
```

I just want this for the selfish reason that I will still think it's called INSTALL_DIR  :)  And other packagers may use INSTALL_DIR (eg, the Homebrew installer on Mac does this), and I would like it if existing documentation (little though it is) remains valid.

```
Index: util/Makefile
===================================================================
--- util/Makefile    (revision 23745)
+++ util/Makefile    (working copy)
@@ -313,10 +313,10 @@
         echo cp $$x '$(LIB_INSTALL_DIR)/'$$x; \
         cp $$x '$(LIB_INSTALL_DIR)/'$$x; \
     done
-    @echo "The following copy commands may fail; they are optional."
-    -cp _util.so '$(PY_INSTALL_DIR)'
-    -cp util.py '$(PY_INSTALL_DIR)'
-    @echo ok
+    #@echo "The following copy commands may fail; they are optional."
+    #-cp _util.so '$(PY_INSTALL_DIR)'
+    #-cp util.py '$(PY_INSTALL_DIR)'
+    #@echo ok
 
 .PHONY: install
 
```

Don't remove that, please.  Those are the python bindings for the C code -- not used by the core Astrometry.net code, I don't think, but used by other projects.  In debian/ubuntu they won't be an issue to build and I would very much appreciate them being installed.

```
@@ -478,7 +478,7 @@
     test_dfind test_ctmf test_dsmooth test_dcen3x3 test_simplexy
 # test_hd depends on hd.fits...
 ALL_TEST_EXTRA_OBJS =
-ALL_TEST_LIBS = $(LIBKD_LIB) $(ANFILES_LIB) $(ANUTILS_LIB) $(QFITS_LIB) $(GSL_LIB)
+ALL_TEST_LIBS = $(ANFILES_LIB) $(ANUTILS_LIB) $(QFITS_LIB) $(GSL_LIB) $(LIBKD_LIB) -lm
 ALL_TEST_EXTRA_LDFLAGS = $(WCSLIB_LIB)
 
 include $(COMMON)/makefile.tests
```

Be careful with the order of libraries.  Some recent compilers are very fussy that the libraries are listed in dependency order.  The dependency structure in the Astrometry.net libraries is nutty -- if I recall correctly it's ANFILES -> LIBKD -> ANUTILS -> QFITS, GSL

Are you actually going to build and run the tests in the debian package?


Sorry for nit-picking!  Thanks very much again for this patch,
--dustin

[Dustin Lang]

(Jasem sent an updated patch, which I applied in http://trac.astrometry.net/changeset/23759)

Many thanks!
--dustin

[Jan-Benedict Glaw]

Great! I'm keen to see Debian packages being buildable.

MfG, JBG

--
Jan-Benedict Glaw jbg...@lug-owl.de +49-172-7608481
Signature of: The real problem with C++ for kernel modules is:
the second : the language just sucks.
-- Linus Torvalds

[Jasem Mutlaq]

The usual workflow is:

configure
make
make install

And usually, the configure script takes care of setting those variable. e.g. ./configure --prefix=/usr --sysconfdir=/etc the Makefile uses. But in case of astrometry.net, you run "make" first and it invokes configure later on. Personally, I would dispense with the whole Makefile system and just go with CMake. It's cross-platform, portable, and much easier to maintain. I used to work directly with Makefiles but after switching to CMake, I never looked back. At any rate, CONF_DIR can be used for now.

On Fri, Oct 18, 2013 at 5:06 PM, Dustin Lang <ds...@cmu.edu> wrote:

Ahh, I see, debian would set DESTDIR=/usr, but of course you want the config file to go in /etc, not /usr/etc.  Sure.  Ok, got it.  Yes, if CONF_DIR is set (is that the standard variable name?), we can install the config file there.

I'm working on the Makefiles, etc, to rename some things, so if I come across that I'll make the change.

cheers,
--dustin

On Fri, 18 Oct 2013, Jasem Mutlaq wrote:

Yes, I agree for someone compiling from source, he could choose where ever he wants it to be installed. However, as a
systems package under Debian/Ubuntu for example, there are standard directories for libs/binaries/headers and
configuration files. So, can we make an extra option? If CONF_DIR is defined while running 'make install' for example, it
shall installs it into that particular directory? Otherwise, it would just install it as it is now.

Regards,
Jasem



On Fri, Oct 18, 2013 at 4:24 PM, Dustin Lang <ds...@cmu.edu> wrote:

      Yes on the rename; no on always putting it in /etc.  Sorry, but an install prefix means "put everything under
      HERE".  Always putting it in /etc, say, breaks normal-user installs (when you don't have permission to write
      to /etc), and the Homebrew installer.

      On Fri, 18 Oct 2013, Jasem Mutlaq wrote:

            In addition to my previous email, can you consider renaming the configuration file to something
            like astrometry.conf and
            have it installed always under /etc/astrometry.conf or /etc/astrometry.net/astrometry.conf

            This will also make it easier for programs such as KStars to parse the conf file to look for the
            data directory and find
            if the index files are sufficient for the FOV of the telescope/camera setup of a particular user.

            Regards,
            Jasem



            On Fri, Oct 18, 2013 at 3:39 PM, Jasem Mutlaq <mutl...@ikarustech.com> wrote:
                  Absolutely, I am building from trunk already. The backend lib was another issue as well
            since the name is
                  quite generic. Also, I might split the package to astrometry.net and astrometry.net-dev
            (for the headers and
                  static libs). I would assume the headers and static libs are only necessary if someone
            wants to build against
                  astrometry.net, but not required for solve-field and other binaries?

                  Regards,
                  Jasem

[Dustin Lang]

configure is used by the subset of gsl we ship.

[Dustin Lang]

Today I did a bunch of renaming and restructuring, including:

-header files go in X/include/astrometry , not X/include
-many fewer obscure executables get built and installed

-renamed "backend" to "astrometry-engine"
-renamed "backend.cfg" to "astrometry.cfg"
-renamed "build-index" to "build-astrometry-index"
-renamed "libbackend.{a,so}" to "libastrometry.{a,so}"
-solve-field "--backend-config" aliased as "--config"
-solve-field "--backend-batch" renamed to "--batch"

This appears to build correctly on mac (incl homebrew) and my ubuntu server, but further testing is always appreciated.

cheers,
--dustin

[Bryan Cashion]

Dustin:

Is this what's in the 0.45 package?  I can give it a go on Ubuntu as well.

Bryan

[Dustin Lang]

Yes, I just updated the 0.45 pre-release at astrometry.net/downloads so it contains those new changes.

thanks,
--dustin

[Jasem Mutlaq]

Great changes! I encountered a few issues. You made the replacement for INSTALL_DIR in astrometry.cfg to be BASE_CONF_DIR which defaults to INSTALL_DIR/etc. However, this means that the configuration file will point to the wrong directory where the indexes are stored, which are in INSTALL_DIR/data

Furthermore, I still needed a way to pass this directory path to the Makefile via a variable, since I can't use INSTALL_DIR or BASE_CONF_DIR for that. So I added DATA_CONF_DIR and made it default to INSTALL_DIR and in the Debian packaging I set it to /usr. This is not used for any installation, just in that one step in the string replacement of the configuration file.

Also, 'make clean' does not completely clean everything, here is what I still get after I run make clean and then do 'svn stat'. What is odd is that util/svn.c gets modified as well?

?       sdss/cutils.py
?       sdss/cutils_wrap.c
?       sdss/build
M       util/svn.c
?       util/os-features.log
?       util/os-features-makefile.log
?       blind/plotstuff
?       catalogs/ngcic-accurate-entries.c
?       catalogs/ngc2000entries.py
?       catalogs/ngc2000names.c
?       catalogs/ngc2000entries.c
?       report.txt

Finally, the make install shows the following error:

cp plotstuff.py /tmp/buildd/astrometry.net-0.45/debian/astrometry.net/usr/lib/python/astrometry/blind/plotstuff.py
cp plotstuff_c.py /tmp/buildd/astrometry.net-0.45/debian/astrometry.net/usr/lib/python/astrometry/blind/plotstuff_c.py
cp: cannot stat 'plotstuff_c.py': No such file or directory
cp _plotstuff_c.so /tmp/buildd/astrometry.net-0.45/debian/astrometry.net/usr/lib/python/astrometry/blind/_plotstuff_c.so
cp: cannot stat '_plotstuff_c.so': No such file or directory
make[3]: *** [install-extra] Error 1
make[3]: Leaving directory `/tmp/buildd/astrometry.net-0.45/blind'
make[2]: [install] Error 2 (ignored)

The deb package builds fine now nonetheless!

Regards,
Jasem

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

--

--
Best Regards,
Jasem Mutlaq

[Dustin Lang]

Sorry, I didn't see your patch until after I started putting in my own fixes...

I defined DATA_INSTALL_DIR and use it.

I added the files you mention to the Makefile "clean" commands, and try to build the _plotstuff.so before trying to install it.

svn.c gets touched every time util/ gets built, I think.  It uses the svn:keywords mechanism to insert the version whenever you do an svn up.  It's not perfect, but it avoids rebuilding everything all the time but mostly tracks the version correctly.

Thanks for the testing -- please let me know how the next round of debian builds go!

--dustin

[Dustin Lang]

ps, I re-pre-released version 0.45 with these changes.

[Jasem Mutlaq]

I can't use DATA_INSTALL_DIR in the string replacement for the configuration file because in build time, DATA_INSTALL_DIR=/tmp/debian/astrometry.net/usr/data or something akin to that, and I can't have that ending up in the configuration file. At the same time, if I set DATA_INSTALL_DIR=/usr/data the build fails because pbuilder expects all files to end up in /tmp/debian during the package build up. This is why I still need just need _one_ variable that I can set.

You can do something like this:

CONF_DATA_DIR ?= $(DATA_INSTALL_DIR)


python -c "import os; print open('../etc/astrometry.cfg-dist').read().replace('DATA_INSTALL_DIR', '$(CONF_DATA_DIR)')" > '$(CONF_BASE_DIR)/astrometry.cfg'

This should fix it and still keep DATA_INSTALL_DIR intact.

Also, I think you forgot to remove catalogs/ngc2000.py from SVN as make clean removes it, but 'svn up' restores it again.

Thanks for your work, we're almost there!! :-)

[Dustin Lang]

Oh yeah, I keep forgetting that it stages in one place and installs in another!

I just added another couple of variables for this:

'''
# If your build system stages an install in one place (say, a temp
# dir) and then copies everything to the final install place, (eg,
# debian), set *_INSTALL_DIR to the staging place and *_FINAL_DIR to
# the final destination directory.

# This should rarely be used, since hardly anything should care that
# it's going to be moved after being staged.
# (ie, the only place it's used at the moment is in blind/Makefile when
#  putting the final data directory in the config file etc/astrometry.cfg)

FINAL_DIR ?= $(INSTALL_DIR)
DATA_FINAL_DIR ?= $(FINAL_DIR)/data
'''

So I think you should be able to do:

STAGE=/path/to/temp/staging/directory
FINAL_DIR=/usr INSTALL_DIR=${STAGE}/usr ETC_INSTALL_DIR=${STAGE}/etc make install

By the way, do I need to do anything special about the files in doc/, examples/, etc?  I thought in debian they went somewhere like /usr/share/data or something?

I re-pre-released v0.45 with this change too.

cheers,
--dustin

[Jasem Mutlaq]

Great! The packages are now building and available!

To get them:

sudo apt-add-repository ppa:mutlaqja/astrometry.net

sudo apt-get update

sudo apt-get install astrometry.net

I'll work next on the data files, and those are trivial to package. The doc files shall go to /usr/share/doc/astrometry.net, but I need to update the debian packaging a bit hopefully soon.

Let me know if anyone encounters any issues installing the package.

Regards,
Jasem

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

[Dustin Lang]

Excellent!

Of course, I then went and rampaged through the makefiles, making a bit of a mess.  I'll try to clean up tomorrow; sorry if it breaks the build.

--dstn

[Jasem Mutlaq]

It did break the build :-)

See attached log.

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

[Dustin Lang]

Oops, I shouldn't check in code at midnight :)

I think I have stabilized it now -- builds in homebrew and on my ubuntu box.

I added the option to use the system-installed GSL library; see here for details:

http://astrometry.net/doc/build.html#overriding-things

by the short story is:

make SYSTEM_GSL=yes
make SYSTEM_GSL=yes py extra

Oh, and it looks like one of the build failures was because I removed the "FAQ" file, and the debian installer expects that to exist.  (I moved its contents onto the web docs, astrometry.net/doc)

I hope the next build works!

cheers,
--dustin

[Jasem Mutlaq]

It now breaks in 'undefined reference' to gsl. Check attached log.

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

[Dustin Lang]

Did you specify that the gsl dev package is a dependency?

On my machine it seems to be called: libgsl0-dev

Is it possible to do builds more frequently than daily, since I seem to be breaking it faster than that?

thanks,
--dstn

[Jasem Mutlaq]

Sorry I didn't know it was an extra dependency, now it's building fine locally :-)

I just uploaded the changes to launchpad to see if it builds there gracefully.

Regards,
Jasem

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

[Jasem Mutlaq]

Well, the i386 package built fine but it failed building 64bit due to some pointer issue. Please check the log here:

https://launchpadlibrarian.net/154341656/buildlog_ubuntu-precise-amd64.astrometry.net_0.45%2Br5-80~ubuntu12.04.1_FAILEDTOBUILD.txt.gz

Regards,
Jasem

[Dustin Lang]

Okay, fixed that.  Thanks.
--dustin

[Dustin Lang]

Of course, then I went on to make another big change reorganizing the qfits library, so may have broken it again...  Please re-run the debian build when you get a chance and I'll pick up the pieces.

(Or, is there an easy way for me to trigger a build at launchpad?)

thanks!
--dustin

[Jasem Mutlaq]

Unfortunately, I tried to create a branch on Launchpad that automatically imports from SVN several times per day, but it failed. I posted a question but didn't get any answer (https://answers.launchpad.net/launchpad/+question/237406) so I decided to import the code myself to get the build rolling.

Now I am getting the following error:

make[2]: Entering directory `/tmp/buildd/astrometry.net-0.45/util'
/bin/sh ./make-tests.sh test_bl.c test_bt.c test_endian.c test_fitstable.c test_xylist.c test_rdlist.c test_healpix.c test_fitsioutils.c test_fitsbin.c test_log.c test_scamp_catalog.c test_starutil.c test_svd.c test_hd.c test_ioutils.c test_tycho2.c test_anwcs.c test_sip-utils.c test_errors.c test_multiindex.c test_convolve_image.c test_qsort_r.c test_wcs.c test_big_tables.c test_quadfile.c test_dfind.c test_ctmf.c test_dsmooth.c test_dcen3x3.c test_simplexy.c > test.c
gcc -o test.o  -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12    -I/usr/include/libpng12      -I../util -I../libkd -I../util -I../qfits-an -I../gsl-an  -I. -c test.c
gcc -o test_bl.o  -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12    -I/usr/include/libpng12      -I../util -I../libkd -I../util -I../qfits-an -I../gsl-an  -I. -c test_bl.c
gcc -o test_bt.o  -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12    -I/usr/include/libpng12      -I../util -I../libkd -I../util -I../qfits-an -I../gsl-an  -I. -c test_bt.c
gcc -o test_endian.o  -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12    -I/usr/include/libpng12      -I../util -I../libkd -I../util -I../qfits-an -I../gsl-an  -I. -c test_endian.c
gcc -o test_fitstable.o  -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12    -I/usr/include/libpng12      -I../util -I../libkd -I../util -I../qfits-an -I../gsl-an  -I. -c test_fitstable.c
test_fitstable.c:7:19: fatal error: qfits.h: No such file or directory
 #include "qfits.h"
                   ^
compilation terminated.
make[2]: *** [test_fitstable.o] Error 1

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

[Dustin Lang]

Oops, I always forget to build the tests.  Fixed, try now please.

Thanks for your patience!

--dstn

[Jasem Mutlaq]

There wouldn't any problems at all if Debian/Ubuntu was the ONLY distro in the world! :-)

What I'm worried about is that soon there will be packages for OpenSUSE, Archlinux, Redhat and a plethora of other distros, and they most likely, the packaging will be different and some will go with the default installation directory, and others will make some modifications. Have to trust that most distros will install the config file to /etc/astrometry.conf in which case it would be parsed fine. At any rate, I added an option in KStars to specify astrometry.net data directory.

Regards,
Jasem

On Sun, Oct 27, 2013 at 9:49 PM, Dustin Lang <ds...@cmu.edu> wrote:

I don't understand the problem you are trying to solve.

You are creating the debian distro, so you *KNOW* where the config file is.  You wrote it.  The config file, in turn, says where the data files can be found.

Reading more about "/usr/share", it might actually be the right place for the index files:

  "Note, however, that '/usr/share' is generally not intended to be shared by different operating systems or by different releases of the same operating system. Any program or package which contains or requires data that doesn't need to be modified should store that data in '/usr/share' (or '/usr/local/share', if installed locally). It is recommended that a subdirectory be used in /usr/share for this purpose.""
--http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/usr.html

Again, though, I see no need to change the Astrometry.net Makefiles -- the debian index file package and the /etc/astrometry.conf file just have to agree where they go.

cheers,
--dustin

On Sun, 27 Oct 2013, Jasem Mutlaq wrote:

It wouldn't be a problem for astrometry.net, but it would be a problem for others using it like KStars & Ekos. There should be a way to reliably find the
data directory, it could be anywhere. The same for the config file, it could be under any prefix. The only thing we can use are the binaries, perhaps we
can query one binary about the prefix

like

$ solve-field --prefix
/usr

$ solve-field --conf
/etc/astrometry.conf

This way I can know the location and parse the file for the location of the data dir , or even

$ solve-field --data-dir
/usr/data

The reason I need the data dir is that I parse the file in it, and then tell the user if he is missing any index files for his/her FOV and ask him/her to
download them. At any rate, I hope I explained why I need those :-)

Regards,
Jasem



On Sun, Oct 27, 2013 at 9:28 PM, Dustin Lang <ds...@cmu.edu> wrote:

      Hi,

      As long as astrometry.conf points to the right place, it doesn't really matter where the files are, right?

      --dstn

      On Sat, 26 Oct 2013, Jasem Mutlaq wrote:

            I just remembered one thing, in the debian installation, the data are installed to /usr/share/data. Could it you make this also be
            the default behavior of
            the Makefile? i.e. it creates the data dir in INSTALL_DIR/share/data and not just INSTALL_DIR/data

            The reason I'm asking for this is that I'm going to make astrometry.net an optional dependency in KStars that shall be released as
            part of KDE 4.12 pretty
            soon. And usually when I add a dependency, you'll find packages for all distros making a package for that dependency, and I wanted
            all of the packages in
            all distros to have the same paths for consistency sake.

            Regards,
            Jasem
             


            On Sat, Oct 26, 2013 at 4:45 PM, Dustin Lang <ds...@cmu.edu> wrote:

                  It's not trivial, because the code doesn't *read* the files, it uses *mmap* to memory-map them, and then treats them as raw
            data, in the right
                  endian-ness.

                  I'm fine if we only create little-endian index files.  If anyone asks I can make big-endian ones.  Nobody has asked this for
            years, so I think
                  it's safe.

                  --dstn

                  On Sat, 26 Oct 2013, Dustin Lang wrote:


                        we don't want to ship both big-endian and little-endian -- they're already huge :)
                        On Sat, 26 Oct 2013, Jasem Mutlaq wrote:

                               Well, can this be handled at the application level? i.e. solve-field would
                               know if the machine it was compiled on is little or big endian and reads
                               the
                               files accordingly?


                               On Sat, Oct 26, 2013 at 4:27 PM, Dustin Lang <ds...@cmu.edu> wrote:

                                     the index files are actually endian-specific, so is there a way to
                                     say that the same file is good for all little-endian archs?

                                     On Sat, 26 Oct 2013, Jasem Mutlaq wrote:

                                           I think so, though even ARM supports little-endian these days.

                                           Regards,
                                           Jasem



                                           On Sat, Oct 26, 2013 at 3:36 PM, Dustin Lang <ds...@cmu.edu>
                                           wrote:

                                                 Thanks for this!

                                                 Does debian still support any big-endian CPUs?

                                                 cheers,
                                                 --dustin

                                                 On Sat, 26 Oct 2013, Jasem Mutlaq wrote:

                                                       Hi Dustin,

                                                       After looking into my data directory for
                                           astrometry, I realized I shouldn't be uploading 32 GB of data
                                           just for the
                                           sake of .deb
                                                       files. So I prepared the
                                                       recipe for you! Attached you'll find two
                                           directories, data & debian. I put dummy files in data just to
                                           test, but you
                                           need to put
                                                       ALL the 32GB index files
                                                       in there. You can see under debian/control how I
                                           divided the packages, please review that and change if
                                           necessary. You
                                           need to
                                                       install devscripts

                                                       $ sudo apt-get install devscripts

                                                       then in the extracted astrometry-data directory:

                                                       $ debuild -i -us -uc -b

                                                       If all goes well, you should have your .deb files!
                                           Those are all architecture independent. Please make them
                                           available
                                           for download
                                                       once they are built
                                                       successfully :-)

                                                       Please let me know if you face any issues.

                                                       Regards,
                                                       Jasem



                                                       On Tue, Oct 22, 2013 at 2:08 AM, Jasem Mutlaq
                                           <mutl...@ikarustech.com> wrote:
                                                             Great, see attached!


                                                       On Mon, Oct 21, 2013 at 10:30 PM, Dustin Lang
                                           <ds...@cmu.edu> wrote:

                                                             Thanks!

                                                             I will create an account for you on one of
                                           the astrometry.net machines and you can scp files there.
                                            Please send
                                           a
                                                       passphrase-protected
                                                             ssh public key.

                                                             cheers,
                                                             --dustin

                                                             On Mon, 21 Oct 2013, Jasem Mutlaq wrote:

                                                                   Great. I created a team
                                           (astrometry-team) so that multiple people can commit to the
                                           Bazaar repo. I also
                                           set you as the
                                                                   project's driver.

                                                                   https://launchpad.net/astrometry.net

                                                                   To checkout code:

                                                                   bzr checkout lp:astrometry.net

                                                                   then like svn, bzr commit

                                                                   I just uploaded the latest changes,
                                           but from now on you can push the changes yourself!! I'll work
                                           on the
                                           data files
                                                       next. I
                                                                   will need an
                                                                   FTP account or something so I can
                                           upload the massive catalogs in .deb format later on. We can
                                           make a link
                                           to them in
                                                       the
                                                                   launchpad page as
                                                                   well.

                                                                   Thanks for your work!

                                                                   Regards,
                                                                   Jasem





                                                                   On Mon, Oct 21, 2013 at 8:24 PM,
                                           Dustin Lang <ds...@cmu.edu> wrote:

                                                                         PS, I am "dstndstn" on
                                           launchpad.net, if you want to add me.

                                                                         https://launchpad.net/~dstndstn


                                                                         On Mon, 21 Oct 2013, Jasem

                                                                         --
                                                                         You received this message
                                           because you are subscribed to a topic in the Google Groups
                                           "astrometry"
                                           group.
                                                                         Visit this group at
                                           http://groups.google.com/group/astrometry.




                                                                         --
                                                                         --
                                                                         Best Regards,
                                                                         Jasem Mutlaq




                                                                         --
                                                                         --
                                                                         Best Regards,
                                                                         Jasem Mutlaq

                                                                         --
                                                                   You received this message because you

                                           are subscribed to the Google Groups "astrometry" group.

[Dustin Lang]

Again, I'm confused, why would KStars specify the astrometry.net data directory rather than just leave that to astrometry.conf ?

cheers,
--dustin

[Jasem Mutlaq]

This assumes astrometry.conf will always be in /etc, which is more than likely to be the place for most distros. However, since the _default_ astrometry.net installation puts it under INSTALL_DIR/astrometry/etc/astrometry.conf , this could pose a problem.

Actually, if someone downloads and compiles the source with default values, it will end up in /usr/local/astrometry/etc/astrometry.conf

The only reason I need the data dir is that if I know the user's FOV, I can find out if he has the necessary index files or not, and then recommend to download the missing index files. To do that, I need the data dir.

Regards,
Jasem

On Sun, Oct 27, 2013 at 10:04 PM, Dustin Lang <ds...@cmu.edu> wrote:

At any rate, I added an option in KStars to specify astrometry.net data directory.

Why?  Why not just use astrometry.conf?

--dstn

[Dustin Lang]

Ok, I see.

But you need to know where astrometry.conf is anyway, right?  Put an option in kstars for the user to specify the astrometry.conf file, and then if you want to find out where the data files are, parse the astrometry.conf file yourself.  That's my opinion, at least.  Having multiple places where a path is specified is a bad idea.  The config file format is pretty easy.

cheers,
--dustin

[Jasem Mutlaq]

Fair enough, I'll make it this way :-)

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.