Need full CD scale/rotation matrix solution but without distortion terms

[Philip Massey]

Hi, Dustin---

We're now at Las Campanas using the astrometry.net software I finally got installed on my MacBook with your copious help a few months ago.  We're now needing to try to optimize the solutions for our situation.  Here's the problem:

The full default solution with all of its A, B, AP, BP terms is too poorly determined for us---our fields aren't sufficiently well populated to give a robust solution to the edges.  On the other hand, setting "no-tweak" or "tweak 1" forces the scales to be identical in both x and y and the axes strictly orthogonal, i.e.,  CD1_1 = CD2_2 and CD1_2 = -CD2_1.   This isn't realistic as our observations are at modestly high airmasses thus differential refraction will result in slight differences in the scale (and non-othogonality) of the axis.  

The full (default) solution returned by solve-field doesn't impose this restriction on the scales (CDn_m are determined independently), but adds in these poorly determined distortion terms (A, B, AP, BP).

Is there some way to let CDn_m be independently determined (as in the full solution) but without the actual distortion terms?  I'm guessing from the words in the headers this might be equivalent to 0th order for A_ORDER, B_ORDER, AP_ORDER, BP_ORDER.

How do I specify this, and is my inference correct?

Thanks!

---phil m.

[Dustin Lang]

Good question, and easy answer:

--tweak-order 1

Hope that works for you!
--dustin

[Philip Massey]

Hi, Dustin---

I'm afraid this didn't do it.  Here's the header from a frame run with tweak-order 1.  You'll see that CD1_1 = CD2_2 and CD1_2 = -CD2_1. exactly.   Here's the command used to solve the frame:

solve-field --overwrite --scale-units arcsecperpix --scale-low 0.43 --scale-high 0.44 --ra  22.78 --dec  -73.53 --radius 0.3  --use-sextractor --tweak-order 1 ccd032.fits

COMMENT --Start of Astrometry.net WCS solution--

COMMENT --Put in by the new-wcs program--

COMMENT

WCSAXES =                    2 / no comment

CTYPE1  = 'RA---TAN-SIP' / TAN (gnomic) projection + SIP distortions

CTYPE2  = 'DEC--TAN-SIP' / TAN (gnomic) projection + SIP distortions

EQUINOX =               2000.0 / Equatorial coordinates definition (yr)

LONPOLE =                180.0 / no comment

LATPOLE =                  0.0 / no comment

CRVAL1  =        22.7450772136 / RA  of reference point

CRVAL2  =       -73.5244895833 / DEC of reference point

CRPIX1  =         1116.6690709 / X reference pixel

CRPIX2  =        1491.19696571 / Y reference pixel

CUNIT1  = 'deg     ' / X pixel scale units

CUNIT2  = 'deg     ' / Y pixel scale units

CD1_1   =   -0.000120924821431 / Transformation matrix

CD1_2   =    4.78950698201E-07 / no comment

CD2_1   =   -4.78950698201E-07 / no comment

CD2_2   =   -0.000120924821431 / no comment

IMAGEW  =                 2046 / Image width,  in pixels.

IMAGEH  =                 3148 / Image height, in pixels.

A_ORDER =                    0 / Polynomial order, axis 1

B_ORDER =                    0 / Polynomial order, axis 2

AP_ORDER=                    0 / Inv polynomial order, axis 1

BP_ORDER=                    0 / Inv polynomial order, axis 2

HISTORY Created by the Astrometry.net suite.

HISTORY For more details, see http://astrometry.net .

HISTORY Subversion URL

HISTORY   http://astrometry.net/svn/trunk/src/astrometry/util/

HISTORY Subversion revision 22921

HISTORY Subversion date 2013-06-02 12:07:59 -0700 (Sun, 02 Jun

HISTORY   2013)

HISTORY This WCS header was created by the program "blind".

COMMENT Original WCS key: "DATE"

_ATE    = '2013-09-21T20:50:07' / Date this file was created.

COMMENT -- blind solver parameters: --

COMMENT Index(0):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4207-08.

COMMENT   fits

COMMENT Index(1):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4206-08.

COMMENT   fits

COMMENT Index(2):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4205-08.

COMMENT   fits

COMMENT Index(3):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4204-32.

COMMENT   fits

COMMENT Index(4):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4203-32.

COMMENT   fits

COMMENT Index(5):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4202-32.

COMMENT   fits

COMMENT Index(6):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4201-32.

COMMENT   fits

COMMENT Index(7):

COMMENT   /Volumes/pepe/astromindicies/2MASS-indicies/index-4200-32.

COMMENT   fits

COMMENT Index(8):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-207.fits

COMMENT Index(9):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-206.fits

COMMENT Index(10):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-205.fits

COMMENT Index(11):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-204-08.fi

COMMENT   ts

COMMENT Index(12):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-203-08.fi

COMMENT   ts

COMMENT Index(13):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-202-08.fi

COMMENT   ts

COMMENT Index(14):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-201-08.fi

COMMENT   ts

COMMENT Index(15):

COMMENT   /Volumes/pepe/astromindicies/USNO-indicies/index-200-08.fi

COMMENT   ts

COMMENT Field name: ./ccd032.axy

COMMENT Field scale lower: 0.43 arcsec/pixel

COMMENT Field scale upper: 0.44 arcsec/pixel

COMMENT X col name: X_IMAGE

COMMENT Y col name: Y_IMAGE

COMMENT Start obj: 10

COMMENT End obj: 20

COMMENT Solved_in: (null)

COMMENT Solved_out: ./ccd032.solved

COMMENT Solvedserver: (null)

COMMENT Solved_out: ./ccd032.solved

COMMENT Solvedserver: (null)

COMMENT Parity: 2

COMMENT Codetol: 0.01

COMMENT Verify pixels: 1 pix

COMMENT Maxquads: 0

COMMENT Maxmatches: 0

COMMENT Cpu limit: 600.000000 s

COMMENT Time limit: 0 s

COMMENT Total time limit: 0 s

COMMENT Total CPU limit: 0.000000 s

COMMENT Tweak: yes

COMMENT Tweak AB order: 1

COMMENT Tweak ABP order: 1

COMMENT --

COMMENT -- properties of the matching quad: --

COMMENT index id: 204

COMMENT index healpix: 8

COMMENT index hpnside: 1

COMMENT log odds: 223.667

COMMENT odds: 1.37187e+97

COMMENT quadno: 1334267

COMMENT stars: 139349,136974,139367,139342

COMMENT field: 13,2,1,0

COMMENT code error: 0.00127288

COMMENT nmatch: 40

COMMENT nconflict: 6

COMMENT nfield: 359

COMMENT nindex: 45

COMMENT scale: 0.435333 arcsec/pix

COMMENT parity: 1

COMMENT quads tried: 65

COMMENT quads matched: 36

COMMENT quads verified: 0

COMMENT objs tried: 14

COMMENT cpu time: 0.002219

COMMENT --

COMMENT

COMMENT --End of Astrometry.net WCS--

COMMENT --(Put in by the new-wcs program)--

COMMENT

[Dustin Lang]

Hmm, yes, I confirm that behavior.

Ok, I think I see what is going on, but I don't really have time to dig into it today.  Maybe you can give it a shot.

The relevant code is blind/solver.c : solver_tweak2 which calls blind/tweak2.c : tweak2().

You'll see that there are nested loops where we increase the order (SIP order) and the "annealing" size.  Note that there is special handling for the linear (order == 1) case.  Try removing that and see what happens!

--dustin

[David W Hogg]

warning: degeneracies available.  Just be careful out there.

--
You received this message because you are subscribed to the Google Groups "astrometry" group.
Visit this group at http://groups.google.com/group/astrometry.

--
David W. Hogg ~ CCPP, NYU; MPIA ~ http://cosmo.nyu.edu/hogg/

[Philip Massey]

Hi, Dustin---

I'm afraid I'll have to wait until you can dig into this.  It took all of my limited skills just to compile this on my Mac using BREW.  I'm a FORTRAN guy, and while I've occasionally edited C code, BREW doesn't even seem to have left the source code around.  Besides, I'd probably break the current installation and while the solutions not quite as good as we need, it's giving us a reasonable idea of what we're getting.  We'll muddle along tonight using what we have and hope you'll have a chance to look at it soon.  Many thanks!

regards,

phil

[Philip Massey]

Hi, Dustin---By any chance were you able to look into this yet?  We'd find it very useful to be able to use this in real-time at the telescope to understand how detection limits.  Thanks!  ---phil

[Dustin Lang]

Hi Phil,

It looks like just removing the special-casing for the linear order worked; at least, in one test I got a not-orthogonal CD matrix.

You can grab that in the latest snapshot,
http://astrometry.net/downloads/astrometry.net-23559.tar.gz

or possibly with:

brew uninstall astrometry.net
brew install --HEAD astrometry.net

cheers,
dustin

[Philip Massey]

Thanks very much, Dustin!  We will give it a try tonight!  It's very kind of you to deal with this in real time, and we're very grateful for being able to use this amazing software!

[Dustin Lang]

No problem.  Good luck out there, and please do report whether it works or not, when you get a chance.

--dustin

[Philip Massey]

Hi, Dustin---

I'm afraid I'm having a bit of an installation problem.  I've made brew happy---at least brew doctor is saying i'm ready to brew, and I've

done an uninstall astrometry.net.  

But when I go to do a brew install --HEAD astrometry.net I get this:

[d102-46:~] massey% brew install --HEAD astrometry.net

==> Checking out http://astrometry.net/svn/trunk/src/astrometry

svn: E155036: Please see the 'svn upgrade' command

svn: E155036: Working copy '/Library/Caches/Homebrew/astrometry.net--svn-HEAD' is too old (format 10, created by Subversion 1.6)

Error: Failure while executing: svn up -q --force /Library/Caches/Homebrew/astrometry.net--svn-HEAD

I've done the svn upgrade but nothing changes.  Suggestions?  Thanks!

[Philip Massey]

I did try to use the snapshot---after all, I've already got all the libraries installed---but when I try to do the make it dies thusly:

make -C qfits-an/src

gcc  -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE   -c -o anqfits.o anqfits.c

anqfits.c: In function ‘anqfits_readpix’:

anqfits.c:806:11: warning: variable ‘planesize’ set but not used [-Wunused-but-set-variable]

     off_t planesize;

           ^

anqfits.c:805:15: warning: variable ‘NY’ set but not used [-Wunused-but-set-variable]

     off_t NX, NY;

               ^

anqfits.c:805:11: warning: variable ‘NX’ set but not used [-Wunused-but-set-variable]

     off_t NX, NY;

           ^

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1075:no such instruction: `vxorpd %xmm0, %xmm0,%xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1087:no such instruction: `vmovsd %xmm0, 48(%r13)'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1090:no such instruction: `vmovsd LC10(%rip), %xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1096:no such instruction: `vmovsd %xmm0, 40(%r13)'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1438:no such instruction: `vxorpd %xmm0, %xmm0,%xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1450:no such instruction: `vmovsd %xmm0, 48(%rbx)'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1453:no such instruction: `vmovsd LC10(%rip), %xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1459:no such instruction: `vmovsd %xmm0, 40(%rbx)'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3758:no such instruction: `vxorpd %xmm4, %xmm4,%xmm4'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3759:no such instruction: `vmovsd 48(%rbx), %xmm1'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3760:no such instruction: `vcomisd %xmm4, %xmm1'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3762:no such instruction: `vmovsd 40(%rbx), %xmm2'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3763:no such instruction: `vcomisd LC10(%rip), %xmm2'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3767:no such instruction: `vmovsd LC10(%rip), %xmm2'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3782:no such instruction: `vcomisd LC10(%rip), %xmm2'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3783:no such instruction: `vxorpd %xmm5, %xmm5,%xmm5'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3794:no such instruction: `vcomisd %xmm5, %xmm1'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3805:no such instruction: `vxorpd %xmm0, %xmm0,%xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3846:no such instruction: `vcvtsi2sdq %r9, %xmm0,%xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3850:no such instruction: `vmulsd %xmm2, %xmm0,%xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3852:no such instruction: `vaddsd %xmm0, %xmm1,%xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3889:no such instruction: `vmovss (%rcx), %xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3891:no such instruction: `vcvtps2pd %xmm0, %xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3902:no such instruction: `vmovsd (%rcx), %xmm0'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3995:no such instruction: `vcvttsd2siq %xmm0, %rdi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3997:no such instruction: `vcvtsi2sdq %r9, %xmm3,%xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:3998:no such instruction: `vcvttsd2siq %xmm3, %rdi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4006:no such instruction: `vmovapd %xmm0, %xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4008:no such instruction: `vcvtsi2sdq %r9, %xmm3,%xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4010:no such instruction: `vmovddup %xmm3, %xmm6'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4011:no such instruction: `vcvtpd2psx %xmm6, %xmm6'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4012:no such instruction: `vmovss %xmm6, (%rdx)'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4018:no such instruction: `vmovapd %xmm0, %xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4020:no such instruction: `vcvtsi2sdq %r9, %xmm3,%xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4022:no such instruction: `vmovsd %xmm3, (%rdx)'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4028:no such instruction: `vcvttsd2si %xmm0, %edi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4030:no such instruction: `vcvtsi2sdq %r9, %xmm3,%xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4031:no such instruction: `vcvttsd2si %xmm3, %edi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4037:no such instruction: `vcvttsd2si %xmm0, %edi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4039:no such instruction: `vcvtsi2sdq %r9, %xmm3,%xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4040:no such instruction: `vcvttsd2si %xmm3, %edi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4048:no such instruction: `vcvttsd2si %xmm0, %edi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4050:no such instruction: `vcvtsi2sdq %r9, %xmm3,%xmm3'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4051:no such instruction: `vcvttsd2si %xmm3, %edi'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4194:no such instruction: `vmovsd 40(%rbx), %xmm2'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4204:no such instruction: `vxorpd %xmm7, %xmm7,%xmm7'

/var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:4205:no such instruction: `vcomisd %xmm7, %xmm0'

make[1]: *** [anqfits.o] Error 1

make: *** [qfits-an] Error 2

[Philip Massey]

Dustin---Nevermind.  I got homebrew to work.  All I needed to do was to first cd to /Library/Caches/Homebrew/astrometry.net--svn-HEAD

and THEN run svn upgrade.  Once I did that brew install --HEAD astrometry.net worked perfectly, and indeed --tweak-order 1 is giving us just

what we want. MANY THANKS!!!!  We'll soon know if this solved our subtraction problem or not.

[Dustin Lang]

I would be interested in trying to sort out that compilation problem once you're back from the mountain, if you're up for it.

cheers,
dustin

[Philip Massey]

Hi, Dustin---Sure, I'd be happy to help. I fly back tomorrow, but then go up to the MMT (on a different project) but I should have enough time to work on this with you while I'm there.  thanks!  ---phil

[Dustin Lang]

Thanks!  No rush.  It looked to me like some bit of code was trying to use assembly instructions for the wrong chipset.

The first thing I would ask for is for you to cd into the snapshot directory, and then do:

make clean
make > log 2>&1
make -k > log2 2>&1
make report

and then send or post the "log", "log2", and "report.txt" files.

Also a description of your machine (MacBook Pro with osx 10.8.3, or whatever).

Thanks,
--dustin

[Jan-Benedict Glaw]

On Tue, 2013-09-24 21:11:14 -0700, Philip Massey <philip....@gmail.com> wrote:
> I did try to use the snapshot---after all, I've already got all the
> libraries installed---but when I try to do the make it dies thusly:
> make -C qfits-an/src
> gcc -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native
> -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -c -o anqfits.o anqfits.c

[...]

What kind of machine is this actually? I think the problem here is
that GCC is forced to use the "native" instruction set of your CPU (as
it detects it.)

So, GCC seems to detect a machine that can so some form of vector
processing. Like SSE and the like.

> /var/folders/mv/gvc101b94zz4pyyn_xy8m8tm0000gn/T//cc4gKpWj.s:1075:no such
> instruction: `vxorpd %xmm0, %xmm0,%xmm0'

[...]

...but your assembler (`gas' I guess?) doesn't understand those. So I
guess you'd update your `binutils' package. If you want to find out
what your binutils will understand, you'd probably just call
`objdump --help': at its output's end, you'll find a list of what's
supported.

MfG, JBG

--
Jan-Benedict Glaw jbg...@lug-owl.de +49-172-7608481
Signature of: Arroganz verkürzt fruchtlose Gespräche.
the second : -- Jan-Benedict Glaw

[Philip Massey]

Hi, Dustin---

Done! I attach here the log, log2, and report.txt files.  I'm running on a MacBook Air

under 10.8.5.  Hope this helps!  For report.txt I had to do a make report > report.txt.

cheers,

phil

[Dustin Lang]

Thanks!

I see that gcc is /usr/local/bin/gcc -- do you know where that came from?  Homebrew?  Self-installed?

As Jan-Benedict was saying, those "vxorpd, vmovsd" and other missing instructions are "AVX" instructions.  It looks like gcc is using those instructions but then the assembler isn't understanding them (the tempfile ending with ".s" is the assemly code gcc is writing out).  Some kind of mismatch between compiler and assembler / gcc and binutils?

Can you compile other packages with this setup?  It seems like a pretty fundamental problem, and the astrometry.net code isn't doing anything particularly fancy here.

cheers,
--dustin

[Philip Massey]

Hi, Dustin---

The gcc was installed by myself, and it comes from the HPC website, specifically

the gcc-mlion.tar.gz link from http://hpc.sourceforge.net

I've used it for years, and certainly within the last day or two used it to compile the C code

for wcstools from the CfA and "hotpants" (image subtraction software) from UW without issues.

I did notice that both of these provided a specific Makefile.osx that set LDFLAGS and other

things.  Is there any chance there's something in (or missing) from your 

Makefile that might lead to the wrong architecture, or that sort of thing?  I'm happy to try any

other tests you can suggest.

cheers,

phil

--
You received this message because you are subscribed to a topic in the Google Groups "astrometry" group.

[Dustin Lang]

Hi,

As shown in the log file you sent, the flags we're using are:

gcc -I/usr/local/include -I/usr/local/include -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fPIC -Winline -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE   -c -o anqfits.o anqfits.c

The relevant one here is "-march=native", which means, quoting from the GCC manual,

-march=cpu-type

Generate instructions for the machine type cpu-type. In contrast to -mtune=cpu-type, which merely tunes the generated code for the specified cpu-type, -march=cpu-type allows GCC to generate code that may not run at all on processors other than the one indicated. Specifying -march=cpu-type implies -mtune=cpu-type.

The choices for cpu-type are:

‘native’ -- This selects the CPU to generate code for at compilation time by determining the processor type of the compiling machine. Using -march=native enables all instruction subsets supported by the local machine (hence the result might not run on different machines). Using -mtune=native produces code optimized for the local machine under the constraints of the selected instruction set.

This is obviously much better than a Makefile trying to figure out what architecture it's running on and then telling the compiler to optimize for that.  If anybody knows what architecture it's running on, it should be the compiler!

Gcc 4.8.1 was released in May 2013, so you certainly haven't been using it for "years"  :).  But it's good that it works with other software...

Could you please try:

make clean
CFLAGS="-v" make > log3 2>&1

and post that log3 ?

And do you think you could post one of those Makefile.osx files you mention?

thanks,
--dustin

[Dustin Lang]

Phil sent me the attached files by email.

Actually, ggroups won't let me attach them, so they're at
http://broiler.astrometry.net/~dstn/temp/log3

[Dustin Lang]

Oh, Phil, could I request one more thing: what do you get when you run:

which as

as -v -V < /dev/null

I am guessing that "gcc" is just running "as" without specifying the full path.  Maybe that is picking up the system /usr/bin/as version, which apparently doesn't know about core i7-avx extensions.  Just a guess.

That would be a generic problem with the compiler, but maybe you've never hit it before because other software hasn't asked for "-march=native", but instead asked for something more specific that actually is supported by your assembler.

If that is what is going on, it might be solvable by modifying your PATH variable so that the "as" that is packaged with your "gcc" appears before /usr/bin.  Or we report it to the HPC folks and see what they say.

cheers,
--dstn

[Philip Massey]

Hi, Dustin---

Here's what I get:

+ /usr/bin/../libexec/as/x86_64/as -v -V 

Apple Inc version cctools-846.2.1, GNU assembler version 1.38

Is this what you expected?

---phil

[Dustin Lang]

Hi Phil,

Thanks for that.

I just remembered a workaround that might... um... work... around... this problem for you.

ARCH_FLAGS="-march=corei7" make

or maybe

ARCH_FLAGS="" make


I'm no expert at gcc/assembler problems.  This seems like an issue to take up with the HPC folks.  I can start that conversation, if you like.  It seems like their gcc should ship with a compatible "as" assembler.  Maybe they do -- is there an "as" in /usr/local/bin alongside your gcc?  What do you get for

echo $PATH

?  It seems like /usr/local/bin must be before /usr/bin, since "which gcc" says /usr/local/bin and not /usr/bin...

Confusedly,
--dustin

[Philip Massey]

Hi, Dustin---SUCCESS!!  Doing an ARCH="" make worked!

For the record, here's what echo $PATH shows:

/opt/intel/composer_xe_2013_sp1.0.074/bin/ia32:/opt/intel/composer_xe_2013_sp1.0.074/mpirt/bin/ia32:/opt/intel/composer_xe_2013_sp1.0.074/debugger/gdb/intel64_mic/py27/bin:/opt/intel/composer_xe_2013_sp1.0.074/debugger/gdb/intel64/py27/bin:/opt/intel/composer_xe_2013_sp1.0.074/bin:/usr/local/bin:/usr/local/astrometry/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/applications/exelis/idl82/bin:/usr/X11r6/bin:/usr/bin:.

---phil