Oversized FITS files created from SunPy?

[Tom Bridgman]

I'm trying to generate the SunPy equivalent of a level 1.5 FITS file from a level 1 file using SunPy.

However, after performing many of the steps in aiapy - update_pointing, register, correct_degradation and deconvolve, writing this FITS file is about 10x larger than the original FITS file.  (Note these steps fail under AstroPy 5.0 but work under AstroPy 4.2.1)

I find no options for compression as part of the sunpy.io.fits.write().

I do receive a warning about the data array being converted to float64.

Is there an option for reducing the size of this file?

Thanks,

Tom

[Tom Bridgman]

Actually, are there any issues with forcing the data array to a smaller size?  Are all the dependent keywords updated appropriately?

Tom

[Chris Bethge]

You should be able to write compressed files by passing the CompImageHDU class from astropy with the hdu_type parameter:

from astropy.io.fits import CompImageHDU

sunpy.io.fits.write('compressed_file.fits', data, header, hdu_type=CompImageHDU)

Cheers,

Chris

[Larry Manley]

register does update the 'BITPIX' header value to reflect the float64 data type.  If you cast the data back to integer (uint16) be sure to change aiamap.meta['BITPIX'] back to 16.

If you use CompImageHDU (recommended as above), the default compression (Rice) and floating point quantization level (16) yields a ~16MB file for full-disk AIA images and is visually lossless.

See the notes for CompImageHDU: https://docs.astropy.org/en/stable/io/fits/api/images.html

Regards,

   Larry

[Tom Bridgman]

The CompImageHDU option worked very well.  Original files were 12.2MB, level 1.5 output file now 14.3MB.  Thanks for that pointer.

But it does raise the question for me of how 'interchangeable' the AstroPy and SunPy FITS routines are.  I wanted to use the info() method available in AstroPy FITS, but apparently not in SunPy FITS.  I figured the SunPy FITS handler was, at worst, a subclass of the AstroPy FITS handler and all methods available for AstroPy FITS files would work for SunPy.  But that seems to not be the case.  Just how far does this mismatch go? 

Thanks,

Tom

[Chris Bethge]

There is no mismatch - they are different things. sunpy.io.fits.read returns a (data, header) tuple which uses astropy.io.fits internally, astropy.io.fits.open() returns an HDU. Think of sunpy.io.fits.read and write as convenience functions implemented in sunpy to add different functionality geared towards how solar physicists interact with fits files, not a replica of astropy.io.fits. If the .info() method is needed, astropy.io.fits can be used.

-Chris

[Stuart Mumford]

Like Chris said, if sunpy.io is useful for you then use it and be happy.

I would say however, that the primary objective isn't to provide a fully featured interface to FITS as Astropy has that. The main objective is to provide a simple and consistent API the Map and Timeseries factories can use internally.

Thanks,
Stuart

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.