Gzip compression

[Hrishi Chandanpurkar]

Hi folks,

I recently became acquainted with xray, and absolutely loved it! I have already made it my to-go tool for reading nc/ hdf files.

Currently I use h5py for writing hdf5 files; this supports compression such as gzip. Most of my data have some masked grids and hence I see a big reduction in size if compressed.

I wonder is there a way to compress the output files with xray?

Thanks,

hrishi

[Joe Hamman]

We've recently added this functionality to xray's `Dataset.to_netcdf` method: http://xray.readthedocs.org/en/latest/generated/xray.Dataset.to_netcdf.html#xray.Dataset.to_netcdf

Using the netCDF4 engine, you can specify any of the per Variable compression keywords (http://unidata.github.io/netcdf4-python/#netCDF4.Dataset.createVariable).

xref: https://github.com/xray/xray/pull/589

Joe

Joe Hamman
Research Assistant and Graduate Student
Civil & Environmental Engineering - University of Washington
Phone:  (206) 498-5497
Email:  jham...@uw.edu

Web:  joehamman.com

--
You received this message because you are subscribed to the Google Groups "xray" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xray-dev+u...@googlegroups.com.
To post to this group, send email to xray...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xray-dev/d6c72449-cb81-4058-bb9c-dfaf2a93eac5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hrishi Chandanpurkar]

Thanks, Joe. Apologies for delay in getting back. 

The zlib compression didn't work in the ds.to_netcdf() method as it resulted in the  'unexpected encoding parameters for 'netCDF4' backend:..' error. However, updating encoding prior to writing the file, as shown below, works well.

dataset=tmp.to_dataset(name='tmp')

dataset.tmp.encoding.update({'zlib': True})

dataset.to_netcdf('tmp.nc',engine='netcdf4')

Thanks!

hrishi