If you have a cdef class, when you call dir() on an instance of the class,
all the module-level functions are also listed. When you try to get one of
these functions using either dot access or getattr(), python crashes. I
don't want them to show up in dir() since they are not methods of the
class. I think this is a bug.
In any case, what I am trying to do is create HDF5 files by walking through
nested data structure with classes and cdef classes as well as classes
nested in lists and dictionaries. It is really quite a mess.
On Sat, Sep 29, 2012 at 11:21 PM, Ian Bell <ian.h.b...@gmail.com> wrote:
> If you have a cdef class, when you call dir() on an instance of the class,
> all the module-level functions are also listed. When you try to get one of
> these functions using either dot access or getattr(), python crashes. I
> don't want them to show up in dir() since they are not methods of the
> class. I think this is a bug.
> In any case, what I am trying to do is create HDF5 files by walking
> through nested data structure with classes and cdef classes as well as
> classes nested in lists and dictionaries. It is really quite a mess.
> In any case, what I am trying to do is create HDF5 files by walking through
> nested data structure with classes and cdef classes as well as classes
> nested in lists and dictionaries.
Are you writing HDF5 by hand? or calling the HDF5 libs?
If the former, are you working on an implimentation or really just
something that will handle you particular use-case of HDF?
If the later -- why not use one of the existing wrappers? (pyhdf and PyTables)
-- just curious, and wondering if you may be making strides towards a
python-native netecdf4 implementation...
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
On Sun, Sep 30, 2012 at 1:52 PM, Chris Barker <chris.bar...@noaa.gov> wrote:
> Hi,
> It seems you have solved your problem, but:
> > In any case, what I am trying to do is create HDF5 files by walking
> through
> > nested data structure with classes and cdef classes as well as classes
> > nested in lists and dictionaries.
> Are you writing HDF5 by hand? or calling the HDF5 libs?
> If the former, are you working on an implimentation or really just
> something that will handle you particular use-case of HDF?
> If the later -- why not use one of the existing wrappers? (pyhdf and
> PyTables)
> -- just curious, and wondering if you may be making strides towards a
> python-native netecdf4 implementation...
Chris,
For my purposes I am using h5py and my own recursive writer to build the
HDF5 file. It can handle native data types - lists, dictionaries, integer,
float, string, etc.. I ignore bound and unbound methods. It works for my
purposes and isn't really all that complicated.There are some obvious and
less-obvious limitations (dictionaries can only have string-based keys for
instance). I've attached my code for this part to this email. There is
some other garbage for my project, but the HDF5Writer class is the one you
want.
Are there better tools out there that could theoretically take a very
deeply nested structure and automatically write it all in an approximately
parallel data structure within the HDF5 file? It looks like neither pyhdf
nor PyTables have the level of automaticness that I need.
On Mon, Oct 1, 2012 at 11:57 PM, Ian Bell <ian.h.b...@gmail.com> wrote:
> For my purposes I am using h5py and my own recursive writer to build the
> HDF5 file. It can handle native data types - lists, dictionaries, integer,
> float, string, etc.. I ignore bound and unbound methods. It works for my
> purposes and isn't really all that complicated.There are some obvious and
> less-obvious limitations (dictionaries can only have string-based keys for
> instance). I've attached my code for this part to this email. There is some
> other garbage for my project, but the HDF5Writer class is the one you want.
> Are there better tools out there that could theoretically take a very deeply
> nested structure and automatically write it all in an approximately parallel
> data structure within the HDF5 file?
I don't think so, no.
It sounds like a pretty cool idea, though.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
