cdef and collections.defaultdict

[John Ehresman]

Is it possible to cdef something as a default dict? Trying to assign a
defaultdict to something cdef'd as a dict, results in an exception
because the PyDict_CheckExact check fails.

Thanks,

John

[Robert Bradshaw]

On Tue, Dec 21, 2010 at 1:11 PM, John Ehresman <j...@wingware.com> wrote:
> Is it possible to cdef something as a default dict?

Nope.

> Trying to assign a
> defaultdict to something cdef'd as a dict, results in an exception because
> the PyDict_CheckExact check fails.

We use PyDict_CheckExact to exclude subclasses that may be
incompatible with calling the C API functions directly (e.g. a
specialized __getitem__). Similar obstructions to the set/frozenset
relationship. Note that defaultdict is not even a builtin or available
for Python < 2.5. While such optimizations would be nice, I don't
think it's on anyones roadmap for the immediate future.

Of course you can always call the C API functions directly, if
applicable. I don't know your application, but I would also recommend
profiling to see if these objects being untyped are really your
bottlenecks.

- Robert

[Justin Peel]

It worked for me to just typecast the defaultdict as a dict. For instance,

cdef dict d = <dict>defaultdict(int)

[John Ehresman]

On 12/21/10 4:20 PM, Robert Bradshaw wrote:
> We use PyDict_CheckExact to exclude subclasses that may be
> incompatible with calling the C API functions directly (e.g. a
> specialized __getitem__). Similar obstructions to the set/frozenset
> relationship. Note that defaultdict is not even a builtin or available
> for Python< 2.5. While such optimizations would be nice, I don't
> think it's on anyones roadmap for the immediate future.

Understood. I may just switch to dicts or try typecasting.

Also, is there a guide to what works in pure mode? I generally try
things in the .py file, look at the C code (I know my way around the C
api), find it doesn't work, and then add to the .pxd file.

Thanks,

John

[Robert Bradshaw]

On Tue, Dec 21, 2010 at 1:22 PM, Justin Peel <pee...@gmail.com> wrote:
> It worked for me to just typecast the defaultdict as a dict. For instance,
> cdef dict d = <dict>defaultdict(int)

The <type> cast does not do any type checking, so something like
<dict>range() would "work" as well. Of course it may segfault or have
undesirable semantics if you actually try to use it, so only do this
if you're sure that all the Python/C API dict functions you call have
the right behavior.

- Robert

[Robert Bradshaw]

http://docs.cython.org/src/tutorial/pure.html and
https://github.com/cython/cython/blob/master/Cython/Shadow.py are the
best two resources out there, such as they are. What kind of things do
you "find out don't work" for you?

- Robert

[Justin Peel]

A defaultdict is just a dict with the default_factory attribute and the __missing__ method defined, so it should be fine. I tested it as a counter for instance and it worked just fine. Of course, it may not work so well for another use case.

[John Ehresman]

On 12/21/10 4:32 PM, Robert Bradshaw wrote:
> http://docs.cython.org/src/tutorial/pure.html and
> https://github.com/cython/cython/blob/master/Cython/Shadow.py are the
> best two resources out there, such as they are. What kind of things do
> you "find out don't work" for you?

The class and function decorators (@cython.cclass, @cython.cfunc) seem
to have no effect or at least a differect effect than declarations in
the .pxd file. Also, it's unclear how instance attributes should be
declared.

It may just be that some of these aren't yet implemented. Using a .pxd
works reasonably well, though I'd like to avoid duplication.

Thanks,

John

[Robert Bradshaw]

On Tue, Dec 21, 2010 at 1:39 PM, John Ehresman <j...@wingware.com> wrote:
> On 12/21/10 4:32 PM, Robert Bradshaw wrote:
>>
>> http://docs.cython.org/src/tutorial/pure.html and
>> https://github.com/cython/cython/blob/master/Cython/Shadow.py are the
>> best two resources out there, such as they are. What kind of things do
>> you "find out don't work" for you?
>
> The class and function decorators (@cython.cclass, @cython.cfunc) seem to
> have no effect or at least a differect effect than declarations in the .pxd
> file.

I don't think those were ever implemented, just decided on. I should
clarify that in the docs.

> Also, it's unclear how instance attributes should be declared.

As cdef classes currently must be declared in .pxd files, you need to
declare them there.

> It may just be that some of these aren't yet implemented.  Using a .pxd
> works reasonably well, though I'd like to avoid duplication.

Yes. Pure mode was mostly written in one big burst, just enough to
make it possible to use a single codebase for the compiled and
non-compiled versions of a project, but there's a lot more that could
still be done to make it more convenient.

- Robert