solve-field only works on xyls files and not jpg or fits

[Jielai]

Hi! 

I just installed the software and am in the process of trying to get it to solve-field afitsfile.fits

I try: solve-field image.fits and get:

Reading input file 1 of 1: "image.fits"...

ERROR: Image type not recognized: Could not determine file type (does the file exist?): image.fits

augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.fWPu6M --fix-sdss --infile image.fits --uncompressed-outfile /tmp/tmp.uncompressed.STBeKD --outfile /tmp/tmp.ppm.vgDUI3 --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

similarly when trying to run the demo: 

solve-field demo/apod2.jpg

Reading input file 1 of 1: "demo/apod2.jpg"...

ERROR: Image type not recognized: Could not determine file type (does the file exist?): demo/apod2.jpg

augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.yywDLk --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile /tmp/tmp.uncompressed.oXjr4O --outfile /tmp/tmp.ppm.wa0KCh --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

I don't know if it's related to netpbm not working, but my understanding is that netpbm just allows you to plot nice things as an output and if I put "solve-field --no-plot somefits.fits" it shouldn't affect anything? 

On the other hand, "solve-field --no-plot demo/apod2.xyls " works.

I would also like to use the --use-sextractor option, but it returns the same thing:

solve-field --use-sextractor image.fits 

Reading input file 1 of 1: "image.fits"...

ERROR: Image type not recognized: Could not determine file type (does the file exist?): image.fits

augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.nohHjv --fix-sdss --infile image.fits --uncompressed-outfile /tmp/tmp.uncompressed.MOmtS1 --outfile /tmp/tmp.ppm.SwlETg --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

Am I supposed to get sextractor to output a fits bintable? I can't see how to do that, sextractor can give various ASCII tables and XML-VOTable or fits format or fits "LDAC" format.

Help! 

Thanks

[Dustin Lang]

Thanks for posting your question here.

Could you please try running solve-field, adding "--verbose --no-delete-temp"

You do need netpbm installed, because we use "jpegtopnm" to convert to pnm and then to FITS.  The error message should be better!

--dstn

[Jielai]

Thanks for replying so quickly Dustin!

----------------------------------------- about netpbm ------------------------------------------

I spent lots of time to try and get netpbm working, in particular, following: http://oldforum.astrometry.net/index.php?p=/discussion/10/problem-with-netpbm-on-mac-osx/#Item_10

I tried, each time re-downloading the source code or after rm util/os-features-config.h util/makefile.os-features util/cairoutils.o

• make, make extra NETPBM_INC="-I/opt/local/include" NETPBM_LIB="-L/opt/local/lib", make install   >> couldn't find pam.h
  
• Type this in terminal:
• NETPBM_INC="-I/opt/local/include/netpbm" (this seemed to allow the finding of pam.h)
• NETPBM_LIB="-L/opt/local/lib"
• add above two lines to astrometrysourcefolder/makefile.netpbm 
• make
• this gave output:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm
   NETPBM_LIB is -L/opt/local/lib -lnetpbm
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm" os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -lnetpbm" >> os-features.log && \
  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp
ld: warning: directory not found for option '-L/opt/local/lib -lnetpbm'
Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:
      _main in os-features-test-ZPce9R.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------
mv os-features-config.h.tmp os-features-config.h
Config results:
------------------------------
cat os-features-config.h
#define NEED_CANONICALIZE_FILE_NAME 1
#define NEED_DECLARE_QSORT_R 1
#define NEED_QSORT_R 0
#define NEED_SWAP_QSORT_R 0
#define HAVE_NETPBM 0
------------------------------

I also tried

• Type this in terminal:
• NETPBM_INC="-I/opt/local/include/netpbm" (this seemed to allow the finding of pam.h)
• NETPBM_LIB="-L/opt/local/lib"
• add above two lines to astrometrysourcefolder/makefile.netpbm
• make
• gave same output as above, so I edited astrometrysourcefolder /util/makefile.os-features and took away the hash sign in line: "# HAVE_NETPBM := yes" 
• make extra 

is it the "make extra" that gets everything working using netpbm?

------------------------------------------ running solve-field again with --verbose -----------------------------------------

I tried 

>> solve-field --verbose --no-delete-temp 83F010612_57_light.fits 

Reading input file 1 of 1: "83F010612_57_light.fits"...

Base: "./83F010612_57_light", basefile "83F010612_57_light.fits", basedir ".", suffix "fits"

Checking if file "83F010612_57_light.fits" is xylist or image: image

  (not xyls because: Missing required columns: : FITS extension 1 in file 83F010612_57_light.fits is not a table (or there was an error opening the file): error: cannot find data start in [83F010612_57_light.fits]:[1])

Running: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.YAVD74 --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.po7064 --outfile /tmp/tmp.ppm.kIfdtB --ppm

ERROR: Image type not recognized: Could not determine file type (does the file exist?): 83F010612_57_light.fits

augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.YAVD74 --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.po7064 --outfile /tmp/tmp.ppm.kIfdtB --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

thanks so much for your help!

[Dustin Lang]

Hi,

Notice that compile warning:

ld: warning: directory not found for option '-L/opt/local/lib -lnetpbm'

Does /opt/local/lib actually exist?  Where did netpbm install its libraries?

Also, is /opt/local/bin (or wherever it installed jpegtopnm, etc) in your PATH?  Can you run "jpegtopnm" from the command-line?

cheers,
--dstn

[Jielai]

ls -d $PWD/*pbm*

/opt/local/lib/libnetpbm.11.61.dylib /opt/local/lib/libnetpbm.dylib

/opt/local/lib/libnetpbm.11.dylib    /opt/local/lib/libpbm.dylib

/opt/local/lib/libnetpbm.a

on the website: http://oldforum.astrometry.net/index.php?p=/discussion/10/problem-with-netpbm-on-mac-osx/#Item_10

you suggested: NETPBM_LIB="/lib/libnetpbm.a"

so I actually also tried:

>>NETPBM_LIB="-L/opt/local/lib/libnetpbm.a"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is -L/opt/local/lib/libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm"  os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib/libnetpbm.a" >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

ld: warning: path '/opt/local/lib/libnetpbm.a' following -L not a directory

Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:

      _main in os-features-test-UVO3wf.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------
mv os-features-config.h.tmp os-features-config.h
Config results:
------------------------------
cat os-features-config.h
#define NEED_CANONICALIZE_FILE_NAME 1
#define NEED_DECLARE_QSORT_R 1
#define NEED_QSORT_R 0
#define NEED_SWAP_QSORT_R 0
#define HAVE_NETPBM 0
------------------------------

I guess that doesn't work because NETPBM_LIB="-L .... specifies a directory in which libnetpbm lives? (Sorry, I don't have a very strong background in this!)

I also tried  (just in case... though it probably doesn't make sense... )

>>NETPBM_LIB="-L/opt/local/lib -libnetpbm.a"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is -L/opt/local/lib -libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm"  os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -libnetpbm.a" >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

ld: warning: directory not found for option '-L/opt/local/lib -libnetpbm.a'

Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:

      _main in os-features-test-BAhNDj.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------

-libnetpbm.a I guess is not an option! 

>>NETPBM_LIB="-L/opt/local/lib -lnetpbm"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm
   NETPBM_LIB is -L/opt/local/lib -lnetpbm
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm" os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -lnetpbm" >> os-features.log && \
  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp
ld: warning: directory not found for option '-L/opt/local/lib -lnetpbm'
Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:
      _main in os-features-test-ZPce9R.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------

What does -lnetpbm actually do?

Thank you so much for helping!

[adren...@gmail.com]

It handles the conversion of some bitmap formats, including jpeg, to the intermediate pnm format, required for ultimate conversion to/from fits.

It better be an option, it runs on _everything_.  Have you considered downloading, building, and installing it, or if you're running a common distribution, just downloading the package and letting it install?

If you do a find / -name libnetpbm* -print  do you see a libnetpbm.a anywhere? .so?

Mine's installed in /usr/lib on one machine where I pulled the package, usr/local/lib on a machine where I built it, opt/local/lib on a server we have.

One way or another, you have to get the basic requirements installed, or built and installed.

Best Regards,

--- Jodie

On Wednesday, April 10, 2013 7:50:21 PM UTC-7, Jielai wrote:

ls -d $PWD/*pbm*

/opt/local/lib/libnetpbm.11.61.dylib /opt/local/lib/libnetpbm.dylib

/opt/local/lib/libnetpbm.11.dylib    /opt/local/lib/libpbm.dylib

/opt/local/lib/libnetpbm.a

on the website: http://oldforum.astrometry.net/index.php?p=/discussion/10/problem-with-netpbm-on-mac-osx/#Item_10

you suggested: NETPBM_LIB="/lib/libnetpbm.a"

so I actually also tried:

>>NETPBM_LIB="-L/opt/local/lib/libnetpbm.a"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is -L/opt/local/lib/libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm"  os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib/libnetpbm.a" >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

ld: warning: path '/opt/local/lib/libnetpbm.a' following -L not a directory

Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:

      _main in os-features-test-UVO3wf.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------
mv os-features-config.h.tmp os-features-config.h
Config results:
------------------------------
cat os-features-config.h
#define NEED_CANONICALIZE_FILE_NAME 1
#define NEED_DECLARE_QSORT_R 1
#define NEED_QSORT_R 0
#define NEED_SWAP_QSORT_R 0
#define HAVE_NETPBM 0
------------------------------

I guess that doesn't work because NETPBM_LIB="-L .... specifies a directory in which libnetpbm lives? (Sorry, I don't have a very strong background in this!)

I also tried  (just in case... though it probably doesn't make sense... )

>>NETPBM_LIB="-L/opt/local/lib -libnetpbm.a"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is -L/opt/local/lib -libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm"  os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -libnetpbm.a" >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

ld: warning: directory not found for option '-L/opt/local/lib -libnetpbm.a'

Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:

      _main in os-features-test-BAhNDj.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------

-libnetpbm.a I guess is not an option! 

>>NETPBM_LIB="-L/opt/local/lib -lnetpbm"

then make returned:

Testing netpbm...

   NETPBM_INC is -I/opt/local/include/netpbm
   NETPBM_LIB is -L/opt/local/lib -lnetpbm
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm" os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -lnetpbm" >> os-features.log && \
  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp
ld: warning: directory not found for option '-L/opt/local/lib -lnetpbm'
Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:
      _main in os-features-test-ZPce9R.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------

What does -lnetpbm actually do?

[Dustin Lang]

Ok, hang on a second.

The NETPBM_LIB variable gets inserted into the gcc command-line when linking.

You can use either the static (.a) or dynamic (.so, or, since you're on Mac, .dylib) lib.  They should look like:

Static: just provide the path to the .a file:

NETPBM_LIB=/opt/local/lib/libnetpbm.a

Dynamic: you need to tell it where to find it with "-L", and which library to link, with "-l", without the "lib" or ".dylib":

NETPBM_LIB="-L/opt/local/lib -lnetpbm"

When using dynamic linking, you ALSO need the directory containing the library to be in the dynamic loader's search path.  With dynamic linking, the library gets loaded at runtime, so at runtime the system has to be able to find it.  That is done, on MacOS, via the DYLD_LIBRARY_PATH variable.  (On Linux it's LD_LIBRARY_PATH.)  So you need something like,

export DYLD_LIBRARY_PATH=/opt/local/lib

And you probably want to put that in your ~/.bash_profile  so it gets set all the time.  But maybe fink or macports or whatever you are using already does that.


There is one thing I really don't understand in the error message you show above:

((cc -o os-features-test-netpbm \
....
"-I/opt/local/include/netpbm"  os-features-test.c
...
"-L/opt/local/lib -libnetpbm.a" >> os-features.log &&
...
ld: warning: directory not found for option '-L/opt/local/lib -libnetpbm.a'

It looks like you somehow have an extra set of "" characters in the variable name.

What do you get when you do:

> echo $NETPBM_LIB

It should be something like:

-L/usr/local/lib -lnetpbm

NOT

"-L/usr/local/lib -lnetpbm"

And, what does this say:

echo $SHELL

cheers,
dustin

On Wednesday, April 10, 2013 10:50:21 PM UTC-4, Jielai wrote:

ls -d $PWD/*pbm*

/opt/local/lib/libnetpbm.11.61.dylib /opt/local/lib/libnetpbm.dylib

/opt/local/lib/libnetpbm.11.dylib    /opt/local/lib/libpbm.dylib

/opt/local/lib/libnetpbm.a

on the website: http://oldforum.astrometry.net/index.php?p=/discussion/10/problem-with-netpbm-on-mac-osx/#Item_10

you suggested: NETPBM_LIB="/lib/libnetpbm.a"

so I actually also tried:

>>NETPBM_LIB="-L/opt/local/lib/libnetpbm.a"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is -L/opt/local/lib/libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm"  os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib/libnetpbm.a" >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

ld: warning: path '/opt/local/lib/libnetpbm.a' following -L not a directory

Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:

      _main in os-features-test-UVO3wf.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------
mv os-features-config.h.tmp os-features-config.h
Config results:
------------------------------
cat os-features-config.h
#define NEED_CANONICALIZE_FILE_NAME 1
#define NEED_DECLARE_QSORT_R 1
#define NEED_QSORT_R 0
#define NEED_SWAP_QSORT_R 0
#define HAVE_NETPBM 0
------------------------------

I guess that doesn't work because NETPBM_LIB="-L .... specifies a directory in which libnetpbm lives? (Sorry, I don't have a very strong background in this!)

I also tried  (just in case... though it probably doesn't make sense... )

>>NETPBM_LIB="-L/opt/local/lib -libnetpbm.a"

then make returned:

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is -L/opt/local/lib -libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm"  os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -libnetpbm.a" >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

ld: warning: directory not found for option '-L/opt/local/lib -libnetpbm.a'

Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:

      _main in os-features-test-BAhNDj.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------

-libnetpbm.a I guess is not an option! 

>>NETPBM_LIB="-L/opt/local/lib -lnetpbm"

then make returned:

Testing netpbm...

   NETPBM_INC is -I/opt/local/include/netpbm
   NETPBM_LIB is -L/opt/local/lib -lnetpbm
((cc -o os-features-test-netpbm \
  -I/opt/local/include -I/opt/local/include -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1   -I./util -I/usr/local/netpbm/include  -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM "-I/opt/local/include/netpbm" os-features-test.c -L/opt/local/lib -L/opt/local/lib -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline "-L/opt/local/lib -lnetpbm" >> os-features.log && \
  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp
ld: warning: directory not found for option '-L/opt/local/lib -lnetpbm'
Undefined symbols for architecture x86_64:
  "_pm_init", referenced from:
      _main in os-features-test-ZPce9R.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--------------- End of expected error messages -----------------

What does -lnetpbm actually do?

[Jielai]

>>echo $SHELL 

/bin/bash

I started with a fresh copy of the source code and:

>>NETPBM_INC=-I/opt/local/include/netpbm 

>>NETPBM_LIB=/opt/local/lib/libnetpbm.a

and changed /util/makefile.netpbm to have above two lines instead

>>make

AND tried:

I started with a fresh copy of the source code and:

>>NETPBM_INC=-I/opt/local/include/netpbm 

>>NETPBM_LIB=/opt/local/lib/libnetpbm.dylib

and changed /util/makefile.netpbm to have above two lines instead

>>make

both worked!!! Thank you (and thank you for explaining)!! (I have both libnetpbm.a and libnetpbm.dylib in /opt/local/lib) below is the output for using libnetpbm.a

Testing netpbm...
   NETPBM_INC is -I/opt/local/include/netpbm 

   NETPBM_LIB is /opt/local/lib/libnetpbm.a
((cc -o os-features-test-netpbm \
  -I/usr/local/scisoft/include/ -I/usr/local/scisoft/include/ -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/usr/local/scisoft/packages/freetype-2.4.2/include/freetype2 -I/usr/local/scisoft/packages/freetype-2.4.2/include -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12 -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1    -I/usr/local/scisoft/Packages/libpng-1.2.42/include/libpng12   -I../gsl-an   -I../qfits-an/include -I../libkd -I. -DTEST_NETPBM -I/opt/local/include/netpbm  os-features-test.c -L/usr/local/scisoft/lib/ -L/usr/local/scisoft/lib/ -g -Wall -ffinite-math-only  -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline /opt/local/lib/libnetpbm.a >> os-features.log && \

  ./os-features-test-netpbm >> os-features.log && \
  echo "#define HAVE_NETPBM 1") \
|| echo "#define HAVE_NETPBM 0") >> os-features-config.h.tmp

--------------- End of expected error messages -----------------
mv os-features-config.h.tmp os-features-config.h
Config results:
------------------------------
cat os-features-config.h
#define NEED_CANONICALIZE_FILE_NAME 1
#define NEED_DECLARE_QSORT_R 1
#define NEED_QSORT_R 0
#define NEED_SWAP_QSORT_R 0

#define HAVE_NETPBM 1

os-features-config.h has #define HAVE_NETPBM 1

BUT... :( I still can't use solve-field properly

Now I go onto run in /astrometrysourcefolder

>> solve-field --verbose util/apod2.jpg

Reading input file 1 of 1: "demo/apod2.jpg"...
Base: "demo/apod2", basefile "apod2.jpg", basedir "demo", suffix "jpg"
Checking if file "demo/apod2.jpg" is xylist or image: image
  (not xyls because: Failed to open FITS table demo/apod2.jpg: Failed to read primary FITS header from demo/apod2.jpg: error: qfits_get_hdrinfo failed.: error: error adding file "demo/apod2.jpg" to cache: error: file "demo/apod2.jpg" is not FITS (does not start with "SIMPLE  ="))
Running: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.SAX87A --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile /tmp/tmp.uncompressed.lN4Ft0 --outfile /tmp/tmp.ppm.ZmEb3W --ppm
ERROR: Image type not recognized: Could not determine file type (does the file exist?): demo/apod2.jpg
augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.SAX87A --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile /tmp/tmp.uncompressed.lN4Ft0 --outfile /tmp/tmp.ppm.ZmEb3W --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

similarly when I try solve-field on a fits image in a folder with the fits image:

>> solve-field --verbose 83F010612_57_light.fits 

Reading input file 1 of 1: "83F010612_57_light.fits"...
Base: "./83F010612_57_light", basefile "83F010612_57_light.fits", basedir ".", suffix "fits"
Checking if file "83F010612_57_light.fits" is xylist or image: image
  (not xyls because: Missing required columns: : FITS extension 1 in file 83F010612_57_light.fits is not a table (or there was an error opening the file): error: cannot find data start in [83F010612_57_light.fits]:[1])

Running: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.hxY1R8 --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.nU1pDa --outfile /tmp/tmp.ppm.B8j14Q --ppm

ERROR: Image type not recognized: Could not determine file type (does the file exist?): 83F010612_57_light.fits

augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.hxY1R8 --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.nU1pDa --outfile /tmp/tmp.ppm.B8j14Q --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255 

when I run in /astrometrysrcfolder

after I downloaded and put in your new fits2fits.py mentioned in: https://groups.google.com/forum/?fromgroups=#!topic/astrometry/k36gTwowjGc (and re-made everything in case...)

>>solve-field --verbose demo/apod2.xyls 

Reading input file 1 of 1: "demo/apod2.xyls"...

Base: "demo/apod2", basefile "apod2.xyls", basedir "demo", suffix "xyls"

Checking if file "demo/apod2.xyls" is xylist or image: xyls

Running: /usr/local/astrometry/bin/fits2fits.py --verbose demo/apod2.xyls /tmp/tmp.sanexyls.cqCLgq

Filename: demo/apod2.xyls

No.    Name         Type      Cards   Dimensions   Format

0    PRIMARY     PrimaryHDU      19  ()            uint8

1    SOURCES     BinTableHDU     24  232R x 3C     [E, E, E]

Traceback (most recent call last):

  File "/usr/local/astrometry/bin/fits2fits.py", line 119, in <module>

    sys.exit(main())

  File "/usr/local/astrometry/bin/fits2fits.py", line 112, in main

    verbose=options.verbose)

  File "/usr/local/astrometry/bin/fits2fits.py", line 44, in fits2fits

    logging.info('Header has %i cards' % len(hdr))

AttributeError: Header instance has no attribute '__len__'

augment-xylist.c:600:run Failed to run command: /usr/local/astrometry/bin/fits2fits.py --verbose demo/apod2.xyls /tmp/tmp.sanexyls.cqCLgq

 ioutils.c:669:run_command_get_outputs Command failed: return value 1

but when I run in /astrometrysrcfolder

>> sudo solve-field demo/apod2.xyls

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid
Reading input file 1 of 1: "demo/apod2.xyls"...
Header has 19 cards
Header has 24 cards
(null): Error reading magic number from Netpbm image stream.  Most often, this means your input file is empty.
(null): Error reading magic number from Netpbm image stream.  Most often, this means your input file is empty.
solve-field.c:332:plot_source_overlay Plotting command failed
 solve-field.c:138:run_command Command was: "/usr/local/astrometry/bin/plotxy -I /tmp/tmp.ppm.rcEKtx -i demo/apod2.axy -C red -w 2 -N 50 -x 1 -y 1 -P | /usr/local/astrometry/bin/plotxy -i demo/apod2.axy -I - -w 2 -r 3 -C red -n 50 -N 200 -x 1 -y 1 > demo/apod2-objs.png"
 solve-field.c:137:run_command Command exited with exit status 1
Solving...
Reading file "demo/apod2.axy"...
Field 1 did not solve (index index-4210.fits, field objects 1-10).
Field 1 did not solve (index index-4209.fits, field objects 1-10).
Field 1 did not solve (index index-4208.fits, field objects 1-10).
Field 1 did not solve (index index-4210.fits, field objects 11-20).
Field 1 did not solve (index index-4209.fits, field objects 11-20).
Field 1 did not solve (index index-4208.fits, field objects 11-20).
Field 1 did not solve (index index-4210.fits, field objects 21-30).
  log-odds ratio 312.484 (5.12738e+135), 143 match, 7 conflict, 62 distractors, 270 index.
  RA,Dec = (84.5763,-2.75371), pixel scale 20.255 arcsec/pix.
  Hit/miss:   Hit/miss: ++++++++++++++++++++++++-++++++++++-++++++--++++++-++++-+++++++++++++++++-+-++++-+++++-++-++++++-+-+
Field 1: solved with index index-4209.fits.
Field 1 solved: writing to file demo/apod2.solved to indicate this.
Field: demo/apod2.xyls
Field center: (RA,Dec) = (84.58, -2.755) deg.
Field center: (RA H:M:S, Dec D:M:S) = (05:38:18.524, -02:45:19.089).
Field size: 4.50137 x 3.3791 degrees
Field rotation angle: up is 90.2125 degrees E of N

why does it make a difference and solve when I use sudo? although it still doesn't output all the files mentioned on: http://astrometry.net/doc/readme.html#output-files  

THANK YOU!

[Dustin Lang]

Hi,

both worked!!! Thank you

Good!

BUT... :( I still can't use solve-field properly

Now I go onto run in /astrometrysourcefolder

>> solve-field --verbose util/apod2.jpg

Reading input file 1 of 1: "demo/apod2.jpg"...

You must have a cut-and-paste error here -- I assume you mean "demo/apod2.jpg", not "util/apod2.jpg".

Can you please run these commands and tell me what they print:

file demo/apod2.jpg

file -b -N -L -k -r demo/apod2.jpg

Could you also please try running:

/usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile sane.fits --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile uncomp --outfile out.pnm --ppm --verbose

Thanks!
--dstn

similarly when I try solve-field on a fits image in a folder with the fits image:

 

>> solve-field --verbose 83F010612_57_light.fits 

Reading input file 1 of 1: "83F010612_57_light.fits"...
Base: "./83F010612_57_light", basefile "83F010612_57_light.fits", basedir ".", suffix "fits"
Checking if file "83F010612_57_light.fits" is xylist or image: image
  (not xyls because: Missing required columns: : FITS extension 1 in file 83F010612_57_light.fits is not a table (or there was an error opening the file): error: cannot find data start in [83F010612_57_light.fits]:[1])
Running: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.hxY1R8 --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.nU1pDa --outfile /tmp/tmp.ppm.B8j14Q --ppm
ERROR: Image type not recognized: Could not determine file type (does the file exist?): 83F010612_57_light.fits
augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.hxY1R8 --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.nU1pDa --outfile /tmp/tmp.ppm.B8j14Q --ppm
 ioutils.c:669:run_command_get_outputs Command failed: return value 255 

Can you run:

liststruc 83F010612_57_light.fits

when I run in /astrometrysrcfolder

after I downloaded and put in your new fits2fits.py mentioned in: https://groups.google.com/forum/?fromgroups=#!topic/astrometry/k36gTwowjGc (and re-made everything in case...)

>>solve-field --verbose demo/apod2.xyls 

...

Reading input file 1 of 1: "demo/apod2.xyls"...

  File "/usr/local/astrometry/bin/fits2fits.py", line 119, in <module>

    sys.exit(main())

  File "/usr/local/astrometry/bin/fits2fits.py", line 112, in main

    verbose=options.verbose)

  File "/usr/local/astrometry/bin/fits2fits.py", line 44, in fits2fits

    logging.info('Header has %i cards' % len(hdr))

AttributeError: Header instance has no attribute '__len__'

augment-xylist.c:600:run Failed to run command: /usr/local/astrometry/bin/fits2fits.py --verbose demo/apod2.xyls /tmp/tmp.sanexyls.cqCLgq

 ioutils.c:669:run_command_get_outputs Command failed: return value 1

but when I run in /astrometrysrcfolder

>> sudo solve-field demo/apod2.xyls

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid
Reading input file 1 of 1: "demo/apod2.xyls"...
Header has 19 cards
Header has 24 cards
(null): Error reading magic number from Netpbm image stream.  Most often, this means your input file is empty.
(null): Error reading magic number from Netpbm image stream.  Most often, this means your input file is empty.
solve-field.c:332:plot_source_overlay Plotting command failed
 solve-field.c:138:run_command Command was: "/usr/local/astrometry/bin/plotxy -I /tmp/tmp.ppm.rcEKtx -i demo/apod2.axy -C red -w 2 -N 50 -x 1 -y 1 -P | /usr/local/astrometry/bin/plotxy -i demo/apod2.axy -I - -w 2 -r 3 -C red -n 50 -N 200 -x 1 -y 1 > demo/apod2-objs.png"
 solve-field.c:137:run_command Command exited with exit status 1
Solving...
Reading file "demo/apod2.axy"...
Field 1 did not solve (index index-4210.fits, field objects 1-10).

 

why does it make a difference and solve when I use sudo? although it still doesn't output all the files mentioned on: http://astrometry.net/doc/readme.html#output-files  

 

The output files may be in the demo/ directory.

There is really no reason you should run the solver with sudo!

But the difference is probably in the environment variable settings.  It looks like it is probably picking up a different version of pyfits.  Try:

python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"
sudo python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"

[Jielai]

oops! Yep, I got so used to typing util/ that I typed it here, but in the terminal, it was demo/

>>file demo/apod2.jpg

demo/apod2.jpg: JPEG image data, JFIF standard 1.02, comment: "Adobe ImageReady"

>>file -b -N -L -k -r demo/apod2.jpg

ERROR: line 5: regexec error 17, (illegal byte sequence)

>>/usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile sane.fits --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile uncomp --outfile out.pnm --ppm --verbose

"file" command failed.  Command: "file -b -N -L -k -r demo/apod2.jpg"
   ERROR: line 5: regexec error 17, (illegal byte sequence)
   
Could not determine file type of "demo/apod2.jpg"
"file" command failed.  Command: "file -b -N -L -k -r demo/apod2.jpg"
   ERROR: line 5: regexec error 17, (illegal byte sequence)
   
ERROR: Image type not recognized: Could not determine file type (does the file exist?): demo/apod2.jpg

>>liststruc 83F010612_57_light.fits 

HDU #1  Array:  NAXIS = 2,  BITPIX = 16
   NAXIS1 = 3352
   NAXIS2 = 2532

>>python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"

2.3.1
/usr/local/scisoft/packages/python/lib/python2.6/site-packages/pyfits/__init__.py 

>>sudo python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid
Password:
3.0.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/__init__.pyc

oh! How did that happen? If I shouldn't use sudo, which makes sense, does that mean the /usr/local/scisoft/packages/python/lib/python2.6/ is not working properly? 

I tried solving the DYLD message that keeps coming up every time I sudo, but no luck. (https://discussions.apple.com/thread/4143805?start=0&tstart=0)

Thanks so much for getting back to me so quickly in this process!!

[Dustin Lang]

>>file -b -N -L -k -r demo/apod2.jpg

ERROR: line 5: regexec error 17, (illegal byte sequence)

Weird, I've never seen that before!

What do you get for:

    which file
    file --version

 

>>/usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile sane.fits --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile uncomp --outfile out.pnm --ppm --verbose

"file" command failed.  Command: "file -b -N -L -k -r demo/apod2.jpg"
   ERROR: line 5: regexec error 17, (illegal byte sequence)   

Ok, that makes sense.

 

>>liststruc 83F010612_57_light.fits 

HDU #1  Array:  NAXIS = 2,  BITPIX = 16
   NAXIS1 = 3352
   NAXIS2 = 2532

>>python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"

2.3.1
/usr/local/scisoft/packages/python/lib/python2.6/site-packages/pyfits/__init__.py 

>>sudo python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid
Password:
3.0.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/__init__.pyc

 

oh! How did that happen?

Different python versions maybe?

  which python
  sudo which python

 

If I shouldn't use sudo, which makes sense, does that mean the /usr/local/scisoft/packages/python/lib/python2.6/ is not working properly?

The pyfits API has changed a lot between 2.3 and 3.0 / 3.1.  Try installing a newer one, or using that 3.0.7.

OR, you can try adding   "solve-field --no-fits2fits" to avoid that "fits2fits" step.

cheers,
dustin

[Jielai]

>>which file

/opt/local/bin/file

>>file --version

file-5.12

magic file from /opt/local/share/misc/magic

hmm... it's the same python:

>>which python

/opt/local/bin/python

>>sudo which python

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid

Password:

/opt/local/bin/python

>>solve-field --no-fits2fits demo/apod2.xyls

works in that it now solves for WCS without needing sudo... but it means I don't get the <base>ngc.png nor the <base>.new files and the <base>-objs.png is zero bytes. 

<base>-ngc.png	an annotation of the image.
<base>.wcs	a FITS WCS header for the solution.
<base>.new	a new FITS file containing the WCS header.
<base>-objs.png	a plot of the sources (stars) we extracted from the image.
<base>-indx.png	sources (red), plus stars from the index (green), plus the skymark (“quad”) used to solve the image.
<base>-indx.xyls	a FITS BINTABLE with the pixel locations of stars from the index.
<base>.rdls	a FITS BINTABLE with the RA,Dec of sources we extracted from the image.
<base>.axy	a FITS BINTABLE of the sources we extracted, plus headers that describe the job (how the image is going to be solved).
<base>.solved	exists and contains (binary) 1 if the field solved.
<base>.match	a FITS BINTABLE describing the quad match that solved the image.
<base>.kmz	(optional) KMZ file for Google Sky-in-Earth. You need to have “wcs2kml” in your PATH. See http://code.google.com/p/wcs2kml/downloads/list http://code.google.com/p/google-gflags/downloads/list

Dustin, how do I --use-sextractor and also how do I run solve-field on fits files?

>>solve-field --no-fits2fits --verbose 83F010612_57_light.fits 

Reading input file 1 of 1: "83F010612_57_light.fits"...
Base: "./83F010612_57_light", basefile "83F010612_57_light.fits", basedir ".", suffix "fits"
Checking if file "83F010612_57_light.fits" is xylist or image: image
  (not xyls because: Missing required columns: : FITS extension 1 in file 83F010612_57_light.fits is not a table (or there was an error opening the file): error: cannot find data start in [83F010612_57_light.fits]:[1])

Running: /usr/local/astrometry/bin/image2pnm.py --no-fits2fits --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.faxOoa --outfile /tmp/tmp.ppm.G51c1g --ppm

ERROR: Image type not recognized: Could not determine file type (does the file exist?): 83F010612_57_light.fits

augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --no-fits2fits --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.faxOoa --outfile /tmp/tmp.ppm.G51c1g --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

>>/usr/local/astrometry/bin/image2pnm.py --no-fits2fits --fix-sdss --infile 83F010612_57_light.fits --uncompressed-outfile /tmp/tmp.uncompressed.faxOoa --outfile /tmp/tmp.ppm.G51c1g --ppm --verbose
"file" command failed.  Command: "file -b -N -L -k -r 83F010612_57_light.fits"

   ERROR: line 5: regexec error 17, (illegal byte sequence)
   

Could not determine file type of "83F010612_57_light.fits"
"file" command failed.  Command: "file -b -N -L -k -r 83F010612_57_light.fits"

   ERROR: line 5: regexec error 17, (illegal byte sequence)
   

ERROR: Image type not recognized: Could not determine file type (does the file exist?): 83F010612_57_light.fits

Thank you sooo much for helping! I just want to be able to run it on these fits files. 

p.s. My supervisor is Roberto Abraham and he saids hi! Apparently he knows you:)

[Dustin Lang]

It looks to me like your "file" program (or probably its "magic" file) is broken.  I would suggest you report it to macports or fink or wherever it came from.  For what it's worth, /usr/bin/file on my mac osx 10.8.2, file-5.04, works fine.

hmm... it's the same python:

>>which python

/opt/local/bin/python

>>sudo which python

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid

Password:

/opt/local/bin/python

So maybe the difference is in the PYTHONPATH then?

>>solve-field --no-fits2fits demo/apod2.xyls

works in that it now solves for WCS without needing sudo... but it means I don't get the <base>ngc.png nor the <base>.new files and the <base>-objs.png is zero bytes. 

Right, I guess the plotting is broken for xyls inputs.

Dustin, how do I --use-sextractor and also how do I run solve-field on fits files?

Uh, install sextractor, making sure the "sex" executable is in your PATH, and add --use-sextractor to the solve-field command.

As for FITS files, again, it's your "file" program failing:

"file" command failed.  Command: "file -b -N -L -k -r 83F010612_57_light.fits"

cheers,
dustin

[Jielai]

Thanks Dustin again, for your help!

I re-named my /opt/local/bin/file to something else and then tried

solve-field --no-fits2fits --verbose 83F010612_57_light.fits

and it worked! It was a problem with my file that I installed with macports, but I have no idea what it is. Neither can I uninstall it because other ports depend on it, so I just renamed it for now...

Thanks!

[Dustin Lang]

It looks like this is a known issue (recently reported);

https://trac.macports.org/ticket/38771

[Jielai]

that was me, taking your advice :)

[Jielai]

Update: I updated my file port using macports from file 5.12 to 5.14

The problem persisted when I ran solve-field:

>>solve-field --verbose demo/apod2.jpg 

Reading input file 1 of 1: "demo/apod2.jpg"...
Base: "demo/apod2", basefile "apod2.jpg", basedir "demo", suffix "jpg"
Checking if file "demo/apod2.jpg" is xylist or image: image
  (not xyls because: Failed to open FITS table demo/apod2.jpg: Failed to read primary FITS header from demo/apod2.jpg: error: qfits_get_hdrinfo failed.: error: error adding file "demo/apod2.jpg" to cache: error: file "demo/apod2.jpg" is not FITS (does not start with "SIMPLE  ="))

Running: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.Rek1Ko --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile /tmp/tmp.uncompressed.c8wxmu --outfile /tmp/tmp.ppm.zBjwyH --ppm
ERROR: Image type not recognized: Could not determine file type (does the file exist?): demo/apod2.jpg
augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.Rek1Ko --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile /tmp/tmp.uncompressed.c8wxmu --outfile /tmp/tmp.ppm.zBjwyH --ppm

 ioutils.c:669:run_command_get_outputs Command failed: return value 255

>>/usr/local/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.Rek1Ko --fix-sdss --infile demo/apod2.jpg --uncompressed-outfile /tmp/tmp.uncompressed.c8wxmu --outfile /tmp/tmp.ppm.zBjwyH --ppm --verbose

"file" command failed.  Command: "file -b -N -L -k -r demo/apod2.jpg"

   ERROR: line 5: regexec error 17, (illegal byte sequence)
   

Could not determine file type of "demo/apod2.jpg"
"file" command failed.  Command: "file -b -N -L -k -r demo/apod2.jpg"

   ERROR: line 5: regexec error 17, (illegal byte sequence)
   

ERROR: Image type not recognized: Could not determine file type (does the file exist?): demo/apod2.jpg 

but...

>>file -b -N -L -k -r demo/apod2.jpg

JPEG image data, JFIF standard 1.02, comment: "Adobe ImageReady??"

 

so then i went into python and did this:

>>cd blahblahblah/astrometry/util

>>python

Python 2.7.3 (default, Nov 17 2012, 19:54:34) 

[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import run_command

>>> run_command.run_command('file -b -N -L -k -r run_command.py')

(0, 'Python script text executable Python script, ASCII text executable\n', '')

>>> run_command.run_command('file -b -N -L -k -r /Users/jlzhang/src/astrometry.net-0.43-2/demo/apod2.jpg')

(-100, 'ERROR: line 5: regexec error 17, (illegal byte sequence)\n', '')

>>> run_command.run_command('file -b -N -L -r /Users/jlzhang/src/astrometry.net-0.43-2/demo/apod2.jpg')

(0, 'JPEG image data, JFIF standard 1.02, comment: "Adobe ImageReady"\n', '')

>>> run_command.run_command('file -b -N -L -r -k /Users/jlzhang/src/astrometry.net-0.43-2/demo/apod2.jpg')

(-100, 'ERROR: line 5: regexec error 17, (illegal byte sequence)\n', '')

 I'm not sure what's going on... but I have now gone and changed /usr/local/astrometry/python/astrometry/util/filetype.py as follows: 

    #filecmd = 'file -b -N -L -k -r %s'

    filecmd = 'file -b -N -L -r %s'

and solve-field runs on fits and jpgs now! 

what does the -k do? .. finds the best match instead of the first match... I'm not entirely sure what it's matching. 

-k, --keep-going

             Don't stop at the first match, keep going.  Subsequent matches

             will be have the string `\012- ' prepended.  (If you want a new-

             line, see the -r option.)  The magic pattern with the highest

             strength (see the -l option) comes first.

Let me know if the change to filetype.py was a stupid thing to do?

[Dustin Lang]

Hi,

> but...
> >>file -b -N -L -k -r demo/apod2.jpg
>       JPEG image data, JFIF standard 1.02, comment: "Adobe ImageReady??"

Hmm, image2pnm.py just runs the "file" command, so it will run whatever is first
in your PATH.

>  I'm not sure what's going on... but I have now gone and changed
> /usr/local/astrometry/python/astrometry/util/filetype.py as follows:
>     #filecmd = 'file -b -N -L -k -r %s'
>     filecmd = 'file -b -N -L -r %s'
>

> and it runs on fits and jpgs now!

>
> what does the -k do? .. finds the best match instead of the first match... I'm not entirely sure what it's matching.
> -k, --keep-going
>              Don't stop at the first match, keep going.  Subsequent matches
>              will be have the string `\012- ' prepended.  (If you want a new-
>              line, see the -r option.)  The magic pattern with the highest
>              strength (see the -l option) comes first.
>
> Let me know if the change to filetype.py was a stupid thing to do?

Hey, if it works for you, that's fine.  I put the '-k' in there because 'file'
sometimes gets confused about the filetype, and it thinks it's a Minix
filesystem, OR maybe it's a jpeg.

cheers,
dustin

[Jielai]

Looks like it's all working now after taking out the -k (including all the output files), thanks Dustin!