How to create a table with numpy array as col, where the numpy shape are inconsistent?

15 views
Skip to first unread message

Wei Jiang

unread,
Oct 15, 2019, 2:34:18 PM10/15/19
to pytables-users
Hi:

I want to store images into a h5 file using pytables. The problem is that the images are at different resolution as numpy.ndarray.
I tried using the "create_table" with a cutomized class, and I tried to set the Col type as "tables.Col.from_dtype(np.dtype(np.uint8))". But when I tried to pass an image in, the error is "TypeError: invalid type (<class 'numpy.ndarray'>) for column ``rgb``"
I also tried to use "create_array" or "create_earray", but seems bot method requires the image to be the same resolution? For example, I can do "shape=(0, 256, 256, 3)", but I cannot store image with different resolutions with error "ValueError: the shapes of the appended object and the ``/rgb`` EArray differ in non-enlargeable dimension 1".

I'd appreciate it if you could provide some hints to the problem?
Thanks!

Kyle Robertson

unread,
Oct 16, 2019, 4:24:24 PM10/16/19
to pytables-users
Some example code that reproduces your problem would be useful, otherwise it's difficult to help you.

I wouldn't try to store numpy arrays as elements in a Table, thats not really what Tables are built for. I would instead use the hierarchical format to store each individual image as a new leaf, perhaps underneath a group. So, the table structure would look like:

/root
   /images
     /image1_array_leaf
     /image2_array_leaf
     /image3_array_leaf
     ...

Does that make sense? You should be able to store as many images as you want this way, and there will be no requirement that they all have the same size.

Wei Jiang

unread,
Oct 16, 2019, 7:56:49 PM10/16/19
to pytables-users
Thanks for the help! I'm able to put all images as seperate leaf into a single H5 file now.
Reply all
Reply to author
Forward
0 new messages