Improper MOSFIRE FITS File

[khru...@gmail.com]

Hi Nick,

While reducing a mask I ran across an issue with the first step (flats). During afternoon calibrations the Sidecar server crashed, and since the bars were already moved to the next mask (COSMOS_1_v1) we decided to continue on with calibrations on that mask, and do the calibrations on the previous mask (the one on which the server crashed) at the end of the night. However I am now finding out that the DRP doesn't think the flats are in the proper MOSFIRE format. I tried playing around with the header information, using the header from another set of flats (that did reduce fine), and replace the relevant information, and saving that into the corrupt file. However that didn't work and I'm starting to suspect it might be an issue with other parts of the fits file, namely the extensions. I tried opening the MOSFIRE fits files in multi-extension mode on DS9 but I didn't find any header or image data.

I've posted one of the flat files online for you to examine here:
http://ifa.hawaii.edu/~jasonchu/m140209_0132.fits

And the full output from the DRP is here:

[supernova:COSMOS_1_v1/2014feb9/K] jasonchu% mospy COSMOS_1_K.py
Loading: Flat.txt
['/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0132.fits', '/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0133.fits', '/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0134.fits', '/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0135.fits', '/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0136.fits', '/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0137.fits', '/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0138.fits']
Traceback (most recent call last):
File "COSMOS_1_K.py", line 20, in <module>
Flats.handle_flats('Flat.txt', maskname, band, flatops)
File "/Users/jasonchu/MOSFIRE/mosdrp/MOSFIRE/Flats.py", line 64, in handle_flats
hdr, dat, bs = IO.readmosfits(fname, options, extension=extension)
File "/Users/jasonchu/MOSFIRE/mosdrp/MOSFIRE/IO.py", line 372, in readmosfits
raise Exception("Improper MOSFIRE FITS File: %s" % path)
Exception: Improper MOSFIRE FITS File: /Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0132.fits
[supernova:COSMOS_1_v1/2014feb9/K] jasonchu%

Any suggestions? Thanks in advance!

[khru...@gmail.com]

Sorry link is weird, try going here and downloading m140209_0132.fits:

http://ifa.hawaii.edu/~jasonchu/

[Nick Konidaris]

Hi Jason,

Yes, this sort of problem (incorrect FITS _Extensions_) can happen. The extension is written after the FITS file is written, thus, if sidecar crashes the FITS File can be written, but the extension is not. The pipeline handles sidecar crashes, but it's not well documented. 

All you have to do is set the extension parameter. Pick a file with the proper extension and set the line

Flats.handle_flats('Flat.txt', maskname, band, flatops)

to

Flats.handle_flats('Flat.txt', maskname, band, flatops, extension="/path/to/file/with/proper/extension")

The extension parameter applies to any of the pipeline steps if you need it.

N

--
You received this message because you are subscribed to the Google Groups "MOSFIRE Data Reduction PIpeline" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mosfire-drp...@googlegroups.com.
To post to this group, send email to mosfi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mosfire-drp/907d74ea-e461-4cd1-9016-567b6acbf011%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
   +1 831 704 6425

[khru...@gmail.com]

Thank you Nick! For the K band I was able to reduce it all the way up to line 40: Wavelength.apply_lambda_simple(maskname, band, obsfiles, waveops).

However, for line 41 I added the extension keyword like such:
Wavelength.apply_lambda_simple(maskname, band, 'Ne.txt', waveops, extension="/Users/jasonchu/MOSFIRE/raw_data/2014feb09/m140209_0312.fits")

but python complained: TypeError: apply_lambda_simple() got an unexpected keyword argument 'extension'

If I remove it then I get the same problem as before (Improper MOSFIRE FITS file). Is there a way around this? Again thanks for your continued support!

-Jason

[Nick Konidaris]

apply_lambda_simple doesn't need the extension parameter. In fact, only the flats should need an extension parameter (because flats are the malformed files). ~n

--
You received this message because you are subscribed to the Google Groups "MOSFIRE Data Reduction PIpeline" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mosfire-drp...@googlegroups.com.
To post to this group, send email to mosfi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mosfire-drp/3b3fe796-0bf0-4602-9e60-c0d49762ebc8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[khru...@gmail.com]

Actually all the calibration files for that mask were malformed: flats, Ne, and Ar for both H and K band. I had to use the extension= on several other commands where the Ne fits files were used, and I think it just so happened that apply_lambda_simple doesn't have the extension= option. Is there a way around it? Thanks!

[khru...@gmail.com]

Sorry I forgot to mention this is for K band reduction. Thanks!

[khru...@gmail.com]

Ok so I was able to fix the problem and finished reducing the data (which looked good as far as I can tell). Here is what I did:

In Wavelength.py, in the subroutine apply_lambda_simple (appx. line 830) I added an additional keyword called as such:

header, drop, bs = IO.readmosfits(wavenames[0], options, extension=extension)

On the line where apply_lambda_simple is defined (appx. line 819) I added extension=None as such:

longslit=None, smooth=True, neon=None, extension=None):

This allowed me to pass on an intact MOSFIRE extension onto the Neon calibration fits files. NOTE: This is only needed for K-band reduction, since the other bands are wavelength-calibrated using sky lines.