How to open HDF files on MATLAB?

[Akiema Forbes]

Great day to start a new topic!

I downloaded data from an online source. The data is zipped and I extract them into one folder. The files are H5F.
 I want to use all these files at once on MATLAB. I begin to by pasting the directory into the address bar.
The files end with h5. I use the command  ending with uiopen (and the location of file), but it states the editor does
not have enough memory. Also I use the command info=hdf5info(file name here.)

[Adam Atia]

Hey Akiema,

Have you tried the h5read function? 

Here is an example of how to read multiple files (it also depends on which data you want to extract):

cd 'My_Directory_Path'   % Type in the path to your folder in place of My_Directory_Path, but keep the single quotes

list=dir('*.h5f');   % Stores a list of all your data files ending with .h5f

list_end=length(list); % Stores the number of files on the list

for i=1:list_end

fname=list(i).name

 data(:,:,i)= h5read(fname,'DATASETNAME')   % You must know the actual names of the datasets stored in the files to fill in DATASETNAME. You can look at the metadata from the data source website, or you can use                   %%%%%%%%%%%%%%%%%%%%%%%h5disp('filename') to see the metadata. This line will store multiple MxN matrices  (if your list has 5 files, this should store 5 matrices.)

end

Try this out first and see how it works for you. Usually you have to apply some conversion factor or scaling factor to the data to transform values to their real values (e.g. temperature might be stored as 3000 degrees Celsius, but you need to divide by 100 to get the actual value of 30 degrees Celsius.) Any necessary conversion factors should be noted in the metadata. So the key here is.... make sure you look at the metadata! :)

Let me know if you're trying to extract data for just one point or over a large area. The code above was just a quick example of how to read all the files, but if you just want one pixel from each data file, you would rewrite the h5read line as:

data(row, column,i)= h5read(fname,'DATASETNAME') %%% row is the row index of the pixel of interest, and column is the column index of the pixel of interest.

Give it a shot, and keep me updated on how it goes! Also, let me know if you still have memory issues. You might need to use a computer that has more RAM. 

Take care,

Adam A

[Adam Atia]

Sorry, I just realized that the example below is for h5 files! How about this: attach your code (.m file) and one example h5f file here. I've never worked with h5f files personally, so I'd like to test things it out in MATLAB myself first. Then we could take it from there. :)

Adam A  

[Adam Atia]

One more thing--- here is a link to the H5F file functions available in Matlab:

http://www.mathworks.com/help/matlab/file-h5f.html

This should be helpful.

On Friday, October 3, 2014 1:14:14 PM UTC-4, Akiema Forbes wrote:

[Akiema Forbes]

Hey Adam,

 The explanation of your matlab code is great. I understand how to use the code when you directly suggest what I need to pull from the manual.

 

The matlab code runs until the loop finishes. I'm not sure how to code in the material I need to find the pixel. Is it the longitude and latitude?

Akiema

 

[Adam Atia]

Hi Akiema,

Sorry for the delayed response. To locate the pixel that you're interested in, first, you'll need to extract both the latitude and longitude matrices. You would just need to know the exact name given to these matrices (e.g. 'lat' or 'long'). You have to check the metadata for these names, or you might be able to look at the file info to see what the names are.

Once you have those latitude and longitude matrices, you need to locate the pixel that most closely matches with your desired coordinates. One way to do this is to calculate the distance formula for each pixel:

Example:

distance=sqrt((latitude-my_latitude).^2 + (longitude-my_longitude).^2);

Now you have a new matrix called 'distance.' You want to find the pixel that has the lowest distance value. In other words, the minimum distance value corresponds to the pixel that is closest to your desired coordinates. Here is how you can do that:

 

[min_value, index_position]=min(dist(:)); %%% This will give two outputs: min_value is the actual value of the minimum distance, and index_position is the index of the pixel you're looking for in the matrix.

[row, column]=ind2sub(size(distance), index_position); %%% The ind2sub function can convert your index to the corresponding row and column of the pixel you want. 

 

Actually, the first thing you should do in your loop is to get your pixel location. Then your computations later on can be done more quickly by just specifying the row and column that you care about. For example, let's say you want to see the temperature for just your pixel, and you need to apply conversion factor of 100 to get the value in Celsius:

my_pixel_temperature= temperature(row, column)*100

One more thing-- I realized that one line I posted previously was incorrect:

data(row, column,i)= h5read(fname,'DATASETNAME') %%% row is the row index of the pixel of interest, and column is the column index of the pixel of interest.

This should be something like this:

data= h5read(fname,'DATASETNAME') %%% row is the row index of the pixel of interest, and column is the column index of the pixel of interest.

my_pixel_data(i,1)=data(row,column);

Let me know if you have any questions or issues with this. Also, if any of the other REU students need assistance, please refer them here (I mentioned this to everyone before, but I'm not sure that people remembered.) I'd like everyone to interact and learn from each other in this forum. Good luck with your work!

Take care,

Adam A

[Aaron Davitt]

Morning,

Here is a website with a list of examples you can try fo rmatlab (you may have to look around a bit):

http://hdfeos.org/examples/index.php

Here is a specific example taken from the website:

http://hdfeos.org/zoo/LPDAAC/AGNS100_v003_64__089_0001_h5.m

I used this website to open hdf files and I had to play with it a few times before I understood exactly what was happening. 

Good luck.

Cheers,

Aaron

[joel chapman]

Hey Adam, I was looking up how to read HDF files and I found this. I need your help on the pixel location method. If you do read this and reply, I will highly appreciate it! Thanks!

You can also just email me at chapman...@gmail.com