Help with translator development

[pari...@ornl.gov]

Greetings all. I'm working on writing translators for SEM/STEM EDX (energy dispersive X-ray spectroscopy) measurements.

I think I'm on the right track, but I want to make sure I'm getting the details of the .h5 write correct.

I'm getting the following .h5, with attributes hung off the datasets:

The attributes on Raw_Data are:

For the Position_Values, etc., attributes look like this:

The relevant method is below. I can share the code to anyone who has code.ornl.gov access; send me a message.

What am I missing? Anything I need to add? 

Thanks!

 

  

def write_h5(self, out_file_name = [], compress_value=2):

        if out_file_name == []:
            out_file_name = input('.h5 file to write: ')


        print('Trying to write    {}'.format(out_file_name))


        # find a chunk size compatible with the number of pixels (positions)
        ls = [32, 16, 8, 4, 2, 1]
        for test_chunk in ls:
            if self._npixels % test_chunk == 0:
                chunk_size = test_chunk
                break


        with h5py.File(out_file_name, 'w') as outfile:
            outfile.create_group('/Measurement_000')
            group_name = '/Measurement_000'
            rawdata = outfile.create_dataset(
                    group_name + '/Raw_Data',
                    data=self._specdata, compression=compress_value,
                    chunks=(chunk_size, self._nchannels) )


            Pos_values = outfile.create_dataset(
                    group_name + '/Position_Values',
                    data=self._Position_Values)
            Pos_idx = outfile.create_dataset(
                    group_name + '/Position_Indices',
                    data=self._Position_Indices)


            Spec_values = outfile.create_dataset(
                    group_name + '/Spectroscopic_Values',
                    data=self._Spectroscopic_Values)
            Spec_idx = outfile.create_dataset(
                    group_name + '/Spectroscopic_Indices',
                    data=self._Spectroscopic_Indices)


            rawdata.attrs['quantity'] = 'X-ray counts'
            rawdata.attrs['units'] = 'Counts'
            rawdata.attrs['Position_Values'] = Pos_values.ref
            rawdata.attrs['Position_Indices'] = Pos_idx.ref
            rawdata.attrs['Spectroscopic_Values'] = Spec_values.ref
            rawdata.attrs['Spectroscopic_Indices'] = Spec_idx.ref
           
            Spec_values.attrs['labels'] = [b'X-ray energy']
            Spec_values.attrs['units'] = [
                    np.string_(self._Spectroscopic_Unit),
                    np.string_(self._Spectroscopic_Unit)]
            Spec_idx.attrs['labels'] = [b'X-ray detector channel']
            Spec_idx.attrs['units'] = [b'Channel']


            Pos_values.attrs['labels'] = [b'Y', b'X']
            Pos_values.attrs['units'] = [np.string_(self._Position_Unit),
                    np.string_(self._Position_Unit)]
            Pos_idx.attrs['labels'] = [b'Y', b'X']
            Pos_idx.attrs['units'] = b'Pixels'

        print('Success!           {}'.format(out_file_name))

[Chris Smith]

Everything looks ok there, but I would recommend using our Dimension objects and the write_main_dataset function from pyUSID.  If your Positions or Spectral Dimensions are too complicated to use the Dimension class, you can also use the write_ind_val_dsets function to build them.

Chris Smith

[pycro...@gmail.com]

Looks OK to me too. Few comments:

1. You are not required to follow the convention of /Measurement_000/Channel_000/ but that is something we follow for consistency with other data types. 

2. I would agree with Chris that you can make things a lot easier for yourself if you used write_main_dataset, write_simple_attrs, create_indexed_group, etc. You can see how these functions are used in this tutorial.