FIXME in solver/query-starkd.c

[Kendy Kutzner]

Hi

there is a FIXME in solver/query-starkd.c. To fix it, I suggest the patch below.

Would you prefer a pull request instead?

Thanks for your consideration
Kendy




diff --git a/solver/query-starkd.c b/solver/query-starkd.c
index 060e15e4..7063cfec 100644
--- a/solver/query-starkd.c
+++ b/solver/query-starkd.c
@@ -219,7 +219,8 @@ int main(int argc, char **argv) {
exit(-1);
}
itemsize = fits_get_atom_size(type) * arraysize;
- data = fitstable_read_column_array_inds(tagalong, col,
type, inds, N, NULL);
+ // Type could be anything. Let's convert to double for
display purposes.
+ data = fitstable_read_column_array_inds(tagalong, col,
fitscolumn_double_type(), inds, N, NULL);
if (!data) {
ERROR("Failed to read data for column \"%s\" in index", col);
exit(-1);
@@ -229,18 +230,16 @@ int main(int argc, char **argv) {
}

for (i=0; i<N; i++) {
- //int j;
+ int j;
printf("%g, %g", radec[i*2+0], radec[i*2+1]);
if (getinds)
printf(", %i", inds[i]);

- //// FIXME -- print tag-along data of generic type.
- /*
- for (j=0; j<pl_size(tagdata); j++) {
- double* data = pl_get(tagdata, j);
- printf(", %g", data[i]);
- }
- */
+ for (j=0; j<pl_size(tagdata); j++) {
+ // We converted to 'double' above.
+ double* data = pl_get(tagdata, j);
+ printf(", %g", data[i]);
+ }

printf("\n");
}

[Dustin Lang]

Hi,

Thanks, a PR would be very convenient!

Have you tested this with different data types in the table?  Does it work with integer types too?

cheers,

--dustin

--
You received this message because you are subscribed to the Google Groups "astrometry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astrometry+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/CABnrD-XNMcH%3D-2EPe9gbDMUOTHwm2rwuuZ1c1pneVZ4-ZPG3XQ%40mail.gmail.com.

[Kendy Kutzner]

On Mon, Nov 15, 2021 at 9:38 PM Dustin Lang <dstn...@gmail.com> wrote:
> Thanks, a PR would be very convenient!

Might take a bit more time.

> Have you tested this with different data types in the table? Does it work with integer types too?

I've tested with float->double. For testing integers, I'll need to
create such a file first. Or do you happen to have one handy?

Kendy

[Dustin Lang]

Thank you, much appreciated!

There are some example big kd-trees here with many tag-along columns,

https://portal.nersc.gov/project/cosmo/data/legacysurvey/dr9/

the files named "*.kd.fits", eg https://portal.nersc.gov/project/cosmo/data/legacysurvey/dr9/survey-ccds-90prime-dr9.kd.fits

would be great.

cheers,

--dustin

[Kendy Kutzner]

Thank you for the file pointer.

$ solver/query-starkd -r 0 -d 0 -R 180 -t image_filename -I
survey-ccds-90prime-dr9.kd.fits | head -n 5
Searching kdtree survey-ccds-90prime-dr9.kd.fits at RA,Dec = (0,0),
radius 180 deg.
Got 132312 results.
# RA, Dec, index, image_filename
226.176, 29.4839, 0, 57
225.529, 29.4899, 1, 48

$ solver/query-starkd -r 0 -d 0 -R 180 -T -I
survey-ccds-90prime-dr9.kd.fits | head -n 5
Searching kdtree survey-ccds-90prime-dr9.kd.fits at RA,Dec = (0,0),
radius 180 deg.
Got 132312 results.
# RA, Dec, index, image_filename, image_hdu, camera, expnum, plver,
procdate, plprocid, ccdname, object, propid, filter, exptime, mjd_obs,
airmass, fwhm, width, height, ra_bore, dec_bore, crpix1, crpix2,
crval1, crval2, cd1_1, cd1_2, cd2_1, cd2_2, yshift, ra, dec, skyrms,
sig1, ccdzpt, zpt, ccdraoff, ccddecoff, ccdskycounts, ccdskysb,
ccdrarms, ccddecrms, ccdphrms, phrms, ccdnastrom, ccdnphotom, ccd_cuts
226.176, 29.4839, 0, 57, 2, 57, 7.81902e+07, 86, 50, 57, 67, 48, 66,
114, 69.118, 57819.5, 1.02, 2.91, 4032, 4096, 225.856, 29.75,
-79.7794, 4272.19, 225.856, 29.75, -6.87479e-07, -0.000125887,
-0.000125955, 5.70173e-07, 0, 226.176, 29.4839, 0.376602, 0.0228618,
25.5748, 25.5785, 0.00207243, -0.0407273, 10.7085, 22.7257, 0.0159893,
0.0444845, 0.0127651, 0.0136485, 1023, 1207, 0
225.529, 29.4899, 1, 48, 4, 48, 7.81902e+07, 50, 48, 53, 67, 48, 65,
114, 69.118, 57819.5, 1.02, 3.03, 4032, 4096, 225.856, 29.75,
-57.8975, -189.386, 225.856, 29.75, -8.13454e-07, -0.000125929,
-0.000125972, 8.94379e-07, 0, 225.529, 29.4899, 0.386112, 0.0231362,
25.577, 25.5785, 0.00481899, -0.0370977, 10.709, 22.7257, 0.0165358,
0.0422175, 0.0137878, 0.0136485, 919, 1066, 0

I don't know that data. Does the output look reasonable to you?

Kendy

[Dustin Lang]

Thanks -- looks like it's doing the right thing for ints, floats, and doubles.  Strings, not so much :)  And presumably, array types.

But regardless, I appreciate this effort and will happily accept a PR if it's not too much trouble!

thanks,

--dustin

[Kendy Kutzner]

On Tue, Nov 16, 2021 at 1:29 AM Dustin Lang <dstn...@gmail.com> wrote:
> Thanks -- looks like it's doing the right thing for ints, floats, and doubles. Strings, not so much :) And presumably, array types.

yep, type A (char) is also converted to double. Printing "strings" is
harder, because one never knows what is in such a column
(NUL-terminated? UTF8? UTF16? some binary data?)

For array types, silently only the first value is printed.

> But regardless, I appreciate this effort and will happily accept a PR if it's not too much trouble!

https://github.com/dstndstn/astrometry.net/pull/243 for your consideration.

Kendy

[Dustin Lang]

Thanks, I merged your PR.  I believe the FITS standard says that A type (strings) have to be ASCII, and probably a limited set of characters.  FITS was mostly defined before unicode was much of a concern.  They're not necessarily NULL-terminated, but rather defined by the length of the array field.  I believe the standard says what you're supposed to put in any empty spaces; it'll either be a space or a NUL.  But certainly some FITS files break the standard (eg, the Astrometry.net index files abuse the standard and use "A" array to contain binary data), so printing would have to be done *carefully*.

Anyway, for query-starkd, I *think* the "-o out.fits" option lets you write the tag-along data in FITS format, and then there are utilities like "tablist" from CFITSIO that will print out binary tables (correctly dealing with strings, arrays, and all data types).  So no real need to reinvent that particular wheel!

cheers,

--dustin