Automatic Type Conversion

[Tony Barbieri]

Hello again,

I'm trying to wrap my head completely around how to best utilize Cython.  Right now I have my .pxd files where I list all of the public methods of my C++ class.  I then write up a .pyx file that has the python implementation of each of those classes.

Now I'm wondering what the best way to move between my custom Class objects and PythonObjects.  Some of my classes in C++ are custom Dict and List objects.  Is there a way to write up an "automatic" way to convert from my class types and PyObjects?  I of course would write the implementation but I'm looking for a way such as when I call a method from my pyx class within a python session, my returned C++ Dict object is automatically converted to a Py_Dict and if I pass a Py_Dict to my pyx class it automatically converts to my custom C++ Dict class to be used by the C++ library....I believe I've seen something similar in sip.  A way where you write up the conversion process and map PyObject types to C++ types.

I hope that makes sense.  I feel like I'm missing something fundamental here....

--
-tony

[Robert Bradshaw]

No, there's not a way to declare a set of automatic conversions.
Cython is not a wrapper-generator. You can write utility functions to
do the conversions yourself and call them manually (being careful, of
course, to get the memory allocation semantics of these more
complicated structs correct). This also affords the opportunity to
give your wrapping a more Pythonic interface; often a 1:1 mapping
between C++ and Python is not the best API.

- Robert

[Tony Barbieri]

I'm still a bit stumped about this.  Here is an example of what I would like to acheive.  Maybe I wasn't very clear in my last email (after reading it I don't think I was).

pxd:

cdef cppclass MyCustomDict:

MyCustomDict()

MyCustomDict(string, CustomValue)

pyx:

from cMyCustomDict cimport MyCustomDict as cMyCustomDict

from cpython.ref cimport PyObject

cdef class MyCustomDict:

cdef cMyCustomDict *thisptr

def __cinit__(self, *args):

if args and isinstance(args[0], PyObject):

#if args is a python type convert to my CustomValue type somehow

self.thisptr = new MyCustomDict(convert_to_customvaltype(args[0]))

else:

self.thisptr = new MyCustomDict()

This is where I think I'm missing something fundamental about Cython and how it works internally.  Ideally I could also do <CustomValue>PyObject and it would call a conversion method that I've written to convert any python type into my custom C++ type...Are any of these things possible or even the right way to go?

Thanks!

-tony 

--
-tony

[Robert Bradshaw]

On Wed, Nov 9, 2011 at 7:42 AM, Tony Barbieri <grea...@gmail.com> wrote:
> I'm still a bit stumped about this.  Here is an example of what I would like
> to acheive.  Maybe I wasn't very clear in my last email (after reading it I
> don't think I was).
> pxd:
> cdef cppclass MyCustomDict:
>
> MyCustomDict()
> MyCustomDict(string, CustomValue)
>
> pyx:
> from cMyCustomDict cimport MyCustomDict as cMyCustomDict
> from cpython.ref cimport PyObject
>
> cdef class MyCustomDict:
>
> cdef cMyCustomDict *thisptr
> def __cinit__(self, *args):
>
> if args and isinstance(args[0], PyObject):
>
> #if args is a python type convert to my CustomValue type somehow
> self.thisptr = new MyCustomDict(convert_to_customvaltype(args[0]))
>
> else:
>
> self.thisptr = new MyCustomDict()
>
> This is where I think I'm missing something fundamental about Cython and how
> it works internally.  Ideally I could also do <CustomValue>PyObject and it
> would call a conversion method that I've written to convert any python type
> into my custom C++ type...Are any of these things possible or even the right
> way to go?

It is not possible to register custom conversions to be used on
casts; only a fixed number of basic conversions (essentially numeric
values and char*) are supported. The way you've written it, calling
convert_to_customvaltype (that you wrote) explicitly, is correct.
(Note that args[0] is always going to be a python object if it's
there, unless by PyObject you mean something else.)

- Robert

[Lisandro Dalcin]

On 9 November 2011 14:13, Robert Bradshaw <robe...@math.washington.edu> wrote:
>
> It is not possible to  register custom conversions to be used on
> casts;
>

But would be nice to have, right?

--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

[Robert Bradshaw]

On Wed, Nov 9, 2011 at 1:09 PM, Lisandro Dalcin <dal...@gmail.com> wrote:
> On 9 November 2011 14:13, Robert Bradshaw <robe...@math.washington.edu> wrote:
>>
>> It is not possible to  register custom conversions to be used on
>> casts;
>>
>
> But would be nice to have, right?

The Zen of Python: explicit is better than implicit.

But, yes, if this can be done right, it could be nice.

- Robert

[Stefan Behnel]

Robert Bradshaw, 10.11.2011 05:58:
> On Wed, Nov 9, 2011 at 1:09 PM, Lisandro Dalcin wrote:

>> On 9 November 2011 14:13, Robert Bradshaw wrote:
>>> It is not possible to register custom conversions to be used on
>>> casts;
>>
>> But would be nice to have, right?
>
> The Zen of Python: explicit is better than implicit.

+1

> But, yes, if this can be done right, it could be nice.

-0

It's not only "when done right" on the compiler side but also depends a lot
on the user side. This could get us too close to C++ code.

Stefan

[Lisandro Dalcin]

The problem with C++ is that things are many times implicit. For
example, you have to use explicitly use "explicit" in constructors to
avoid implicit conversions. But if the user EXPLICITLY ask Cython to
use a custom to/from C<->Python converters, I do not think that's too
bad.

For example, I have a little use case from my own code: In petsc4py, I
wrap PETSc. PETSc has a numeric data type "PetscScalar" typedefed to
"double" or "double complex" depending on how you built PETSc. In
Cython, I cannot declare a type that could be either "double" or
"double complex" at C compile time. Then I have to EXPLICITLY call a
toScalar()/asScalar() almost everywhere. That's really annoying, I
would really like to say: "PetscScalar" is number-like, use
toScalar()/asScalar() for converting to C/from Python.

[Robert Bradshaw]

On Thu, Nov 10, 2011 at 11:14 AM, Lisandro Dalcin <dal...@gmail.com> wrote:
> On 10 November 2011 03:28, Stefan Behnel <stef...@behnel.de> wrote:
>> Robert Bradshaw, 10.11.2011 05:58:
>>>
>>> On Wed, Nov 9, 2011 at 1:09 PM, Lisandro Dalcin wrote:
>>>>
>>>> On 9 November 2011 14:13, Robert Bradshaw wrote:
>>>>>
>>>>> It is not possible to  register custom conversions to be used on
>>>>> casts;
>>>>
>>>> But would be nice to have, right?
>>>
>>> The Zen of Python: explicit is better than implicit.
>>
>> +1
>>
>>> But, yes, if this can be done right, it could be nice.
>>
>> -0
>>
>> It's not only "when done right" on the compiler side but also depends a lot
>> on the user side. This could get us too close to C++ code.
>>
>
> The problem with C++ is that things are many times implicit.

Yes.

> For
> example, you have to use explicitly use "explicit" in constructors to
> avoid implicit conversions. But if the user EXPLICITLY ask Cython to
> use a custom to/from C<->Python converters, I do not think that's too
> bad.

It's implicit where it's used. It's unclear what the scope of such a
declaration would be.

> For example, I have a little use case from my own code: In petsc4py, I
> wrap PETSc. PETSc has a numeric data type "PetscScalar" typedefed to
> "double" or "double complex" depending on how you built PETSc. In
> Cython, I cannot declare a type that could be either "double" or
> "double complex" at C compile time. Then I have to EXPLICITLY call a
> toScalar()/asScalar() almost everywhere. That's really annoying, I
> would really like to say: "PetscScalar" is number-like, use
> toScalar()/asScalar()  for converting to C/from Python.

The real vs. complex decided at C compile time seems like an even
thornier issue.

- Robert