Crash on accessing module level functions in cdef class

[Ian Bell]

If you have a cdef class, when you call dir() on an instance of the class, all the module-level functions are also listed.  When you try to get one of these functions using either dot access or getattr(), python crashes.  I don't want them to show up in dir() since they are not methods of the class.  I think this is a bug. 

In any case, what I am trying to do is create HDF5 files by walking through nested data structure with classes and cdef classes as well as classes nested in lists and dictionaries.  It is really quite a mess.

Ian

[Ian Bell]

Sorry, please disregard message.  User error.  I created an infinitely recursive loop.  Oops

Ian

[Chris Barker]

Hi,

It seems you have solved your problem, but:

> In any case, what I am trying to do is create HDF5 files by walking through
> nested data structure with classes and cdef classes as well as classes
> nested in lists and dictionaries.

Are you writing HDF5 by hand? or calling the HDF5 libs?

If the former, are you working on an implimentation or really just
something that will handle you particular use-case of HDF?

If the later -- why not use one of the existing wrappers? (pyhdf and PyTables)

-- just curious, and wondering if you may be making strides towards a
python-native netecdf4 implementation...

-Chris
--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Ian Bell]

On Sun, Sep 30, 2012 at 1:52 PM, Chris Barker <chris....@noaa.gov> wrote:

Hi,

It seems you have solved your problem, but:

> In any case, what I am trying to do is create HDF5 files by walking through
> nested data structure with classes and cdef classes as well as classes
> nested in lists and dictionaries.

Are you writing HDF5 by hand? or calling the HDF5 libs?

If the former, are you working on an implimentation or really just
something  that will handle you particular use-case of HDF?

If the later -- why not use one of the existing wrappers? (pyhdf and PyTables)

-- just curious, and wondering if you may be making strides towards a
python-native netecdf4 implementation...

Chris,

For my purposes I am using h5py and my own recursive writer to build the HDF5 file.  It can handle native data types - lists, dictionaries, integer, float, string, etc..  I ignore bound and unbound methods.  It works for my purposes and isn't really all that complicated.There are some obvious and less-obvious limitations (dictionaries can only have string-based keys for instance).  I've attached my code for this part to this email. There is some other garbage for my project, but the HDF5Writer class is the one you want.

Are there better tools out there that could theoretically take a very deeply nested structure and automatically write it all in an approximately parallel data structure within the HDF5 file?  It looks like neither pyhdf nor PyTables have the level of automaticness that I need.

Regards,
Ian

[Chris Barker]

On Mon, Oct 1, 2012 at 11:57 PM, Ian Bell <ian.h...@gmail.com> wrote:

> For my purposes I am using h5py and my own recursive writer to build the
> HDF5 file. It can handle native data types - lists, dictionaries, integer,
> float, string, etc.. I ignore bound and unbound methods. It works for my
> purposes and isn't really all that complicated.There are some obvious and
> less-obvious limitations (dictionaries can only have string-based keys for
> instance). I've attached my code for this part to this email. There is some
> other garbage for my project, but the HDF5Writer class is the one you want.
>
> Are there better tools out there that could theoretically take a very deeply
> nested structure and automatically write it all in an approximately parallel
> data structure within the HDF5 file?

I don't think so, no.

It sounds like a pretty cool idea, though.