TypeError: can't apply this __setattr__

[Shriramana Sharma]

For practice, I am implementing a subclass of builtins.list in Cython
which will store only str-s and has one data member "approved" which
should take only a boolean. It's entirely Python syntax except that it
has a cdef at the head.

However I am getting a TypeError saying "can't apply this __setattr__"
when I try to initialize an object of this type:

>>> from strlist import strlist
>>> a=strlist()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "strlist.pyx", line 24, in strlist.strlist.__init__ (strlist.c:1215)
self.approved = False
File "strlist.pyx", line 33, in strlist.strlist.__setattr__ (strlist.c:1417)
super().__setattr__ ( name, value )
TypeError: can't apply this __setattr__ to strlist.strlist object

OTOH when I try the same using the pure Python implementation of the
class (i.e. just removed cdef) I have no problem:

>>> from strlist_py import strlist
>>> a=strlist()
>>> a
strlist(<super: <class 'strlist'>, <strlist object>>,approved=False)

The code is attached. Can anyone please help me figure out what I'm
doing wrong? Thanks.

--
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

[Björn Dahlgren]

You could declare approved (that fixes your example for me at least):

cdef class strlist ( list ) :

        cdef public bool approved

just remember to add:

from cpython cimport bool

at the top.

By the way, you might want to check out PEP 8 for "coding convention" for writing python
 and generally spaces (4) are preferred nowadays in favour of tabs.. (nitpicking)

Cheers,
/Björn

[Stefan Behnel]

Björn Dahlgren, 13.05.2013 22:22:

> On Sunday, 12 May 2013 09:24:41 UTC+2, Shriramana Sharma wrote:
>>
>> For practice, I am implementing a subclass of builtins.list in Cython
>> which will store only str-s and has one data member "approved" which
>> should take only a boolean. It's entirely Python syntax except that it
>> has a cdef at the head.
>>
>> However I am getting a TypeError saying "can't apply this __setattr__"
>> when I try to initialize an object of this type:
>>
>> >>> from strlist import strlist
>> >>> a=strlist()
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File "strlist.pyx", line 24, in strlist.strlist.__init__
>> (strlist.c:1215)
>> self.approved = False
>> File "strlist.pyx", line 33, in strlist.strlist.__setattr__
>> (strlist.c:1417)
>> super().__setattr__ ( name, value )
>> TypeError: can't apply this __setattr__ to strlist.strlist object
>>
>> OTOH when I try the same using the pure Python implementation of the
>> class (i.e. just removed cdef) I have no problem:
>>
>> >>> from strlist_py import strlist
>> >>> a=strlist()
>> >>> a
>> strlist(<super: <class 'strlist'>, <strlist object>>,approved=False)

Extension types are not identical to Python types. Specifically, they do
not have a __dict__, which prevents the normal setattr() mechanism from
working.

Luckily, you really don't want setattr() here. The proposal below is the
right approach.

>> The code is attached. Can anyone please help me figure out what I'm
>> doing wrong? Thanks.
>

> You could declare approved (that fixes your example for me at least):
>
> cdef class strlist ( list ) :
> cdef public bool approved
>
> just remember to add:
>
> from cpython cimport bool
>
> at the top.

That's the right way of doing it but not the right type, because it pays
the entire overhead of a Python bool object just to store one bit of
information. Instead, use the "bint" type to declare this property.

Stefan

[Shriramana Sharma]

Thanks, people, for your replies.

Thanks for this hint, but I disagree with the PEP and am glad that
Python doesn't actually enforce that.

I do not know why one would prefer spaces to tabs, I mean for example
all those backspaces you'd have to use to go up a level, and I think
it is just the personal style of somebody who contributed a lot to the
core libraries or such, and it was made a standard *as far as the core
libraries are concerned* because mixing up different indenting systems
cannot be sensibly handled due to the syntactical importance of
whitespace.

However I am not capable and hence do not intend to contribute to the
core libraries, and whatever code I write for my own project I will
use my own preferred indenting style. I have been using tabs ever
since my earliest days of C programming and will continue to use them
for the foreseeable future.

On Wed, May 15, 2013 at 10:59 AM, Stefan Behnel <stef...@behnel.de> wrote:
>
> That's the right way of doing it but not the right type, because it pays
> the entire overhead of a Python bool object just to store one bit of
> information. Instead, use the "bint" type to declare this property.

I don't get this -- what overhead can a bool object in *any* language
entail? It is possibly the simplest of data types, no?

[Lisandro Dalcin]

On 18 May 2013 06:33, Shriramana Sharma <sam...@gmail.com> wrote:
>
> I don't get this -- what overhead can a bool object in *any* language
> entail? It is possibly the simplest of data types, no?
>

In [1]: import struct

In [2]: struct.calcsize("?")
Out[2]: 1

In [3]: struct.calcsize("i")
Out[3]: 4

In [4]: True.__sizeof__()
Out[4]: 24



--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

[Shriramana Sharma]

On Sat, May 18, 2013 at 9:44 PM, Lisandro Dalcin <dal...@gmail.com> wrote:
>
> In [4]: True.__sizeof__()
> Out[4]: 24

Wow, but *why* is the question?!

[Pauli Virtanen]

18.05.2013 19:23, Shriramana Sharma kirjoitti:
> On Sat, May 18, 2013 at 9:44 PM, Lisandro Dalcin <dal...@gmail.com> wrote:
>>
>> In [4]: True.__sizeof__()
>> Out[4]: 24
>
> Wow, but *why* is the question?!

(i) class information, (ii) garbage collection information. I'd imagine
the situation is the same in Java and any other language with boxed values.

--
Pauli Virtanen

[Nikita Nemkin]

On Wed, May 15, 2013 at 10:59 AM, Stefan Behnel <stef...@behnel.de>
wrote:
> That's the right way of doing it but not the right type, because it pays
> the entire overhead of a Python bool object just to store one bit of
> information. Instead, use the "bint" type to declare this property.

In CPython, True and False are statically allocated singletons.
As a result, Python bool objects (when properly typed in Cython)
are _almost_ as efficient as bint.

bint is preferred because:
1) bool objects have to be refcounted, which means every bool assignment
has an extra INCREF/DECREF.
(Cython also inserts an unnecessary typecheck when constants are
assigned).
2) On x64, bool takes 8 bytes (like PyObject *), while bint is 4 (like
int).
3) bool check is 2 pointer comparisons + (unlikely) PyObject_IsTrue call.
This requires a few more instructions compared to one zero check in case
of bint.


Best regards,
Nikita Nemkin

[Chris Barker - NOAA Federal]

On Sat, May 18, 2013 at 10:02 AM, Nikita Nemkin <nik...@nemkin.ru> wrote:
> 2) On x64, bool takes 8 bytes (like PyObject *), while bint is 4 (like int).

kind of makes you wonder why a bint isn't 1 byte (bchar?)

- Chris


--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Nikita Nemkin]

On Mon, 20 May 2013 21:44:58 +0600, Chris Barker - NOAA Federal
<chris....@noaa.gov> wrote:

> On Sat, May 18, 2013 at 10:02 AM, Nikita Nemkin <nik...@nemkin.ru> wrote:
>> 2) On x64, bool takes 8 bytes (like PyObject *), while bint is 4 (like
>> int).
>
> kind of makes you wonder why a bint isn't 1 byte (bchar?)

1) Operating on quantities smaller than machine word requires additional
instructions. (32 bit operands are a norm for x64 so there is no
penalty).
2) Lonely bchar structure member or bchar variable on the stack will have
to be padded anyway.
You will only win some bytes when there are multiple bchars following
each other. And then 1) applies in full...

Here is a nice on-topic article:
http://blogs.msdn.com/b/oldnewthing/archive/2008/11/26/9143050.aspx


Best regards,
Nikita Nemkin

[Chris Barker - NOAA Federal]

On Mon, May 20, 2013 at 8:53 AM, Nikita Nemkin <nik...@nemkin.ru> wrote:
>>> 2) On x64, bool takes 8 bytes (like PyObject *), while bint is 4 (like
>>> int).
>>
>> kind of makes you wonder why a bint isn't 1 byte (bchar?)
>
> 1) Operating on quantities smaller than machine word requires additional
> instructions. (32 bit operands are a norm for x64 so there is no
> penalty).
> 2) Lonely bchar structure member or bchar variable on the stack will have
> to be padded anyway.
> You will only win some bytes when there are multiple bchars following
> each other.

That's a pretty common case -- at least for when it matters (i.e. lots
of bools...)

> And then 1) applies in full...

Does it? processors are really fast compared to memory access these
days. So if you are pushing a big array of bools, it may be faster to
save that factor of four memory.

> Here is a nice on-topic article:
> http://blogs.msdn.com/b/oldnewthing/archive/2008/11/26/9143050.aspx

well, that is addressing bits vs ints, which is a bigger difference,
though similar. But given the memory band-width issue, bits may be a
better way to go.

But you'd have to:

a) be smarter that me about this stuff!
b) profile!

to be sure.

-Chris

[Robert Bradshaw]

On Mon, May 20, 2013 at 10:51 AM, Chris Barker - NOAA Federal
<chris....@noaa.gov> wrote:
> On Mon, May 20, 2013 at 8:53 AM, Nikita Nemkin <nik...@nemkin.ru> wrote:
>>>> 2) On x64, bool takes 8 bytes (like PyObject *), while bint is 4 (like
>>>> int).
>>>
>>> kind of makes you wonder why a bint isn't 1 byte (bchar?)
>>
>> 1) Operating on quantities smaller than machine word requires additional
>> instructions. (32 bit operands are a norm for x64 so there is no
>> penalty).
>> 2) Lonely bchar structure member or bchar variable on the stack will have
>> to be padded anyway.
>> You will only win some bytes when there are multiple bchars following
>> each other.
>
> That's a pretty common case -- at least for when it matters (i.e. lots
> of bools...)

If you have enough bools that char vs int makes a difference, you
might want to jump for the 32x savings of storing them as bits rather
than bytes. But the real reason bint is an int is because that's the
the result type of relational and equality operators in C. The value
of bint primarily lies in its coercion to/from Python.

>> And then 1) applies in full...
>
> Does it? processors are really fast compared to memory access these
> days. So if you are pushing a big array of bools, it may be faster to
> save that factor of four memory.
>
>> Here is a nice on-topic article:
>> http://blogs.msdn.com/b/oldnewthing/archive/2008/11/26/9143050.aspx
>
> well, that is addressing bits vs ints, which is a bigger difference,
> though similar. But given the memory band-width issue, bits may be a
> better way to go.
>
> But you'd have to:
>
> a) be smarter that me about this stuff!
> b) profile!
>
> to be sure.
>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris....@noaa.gov
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Chris Barker - NOAA Federal]

On Fri, May 24, 2013 at 11:03 AM, Robert Bradshaw

> If you have enough bools that char vs int makes a difference, you
> might want to jump for the 32x savings of storing them as bits rather
> than bytes.

True -- but that introduces a whole other level of hassle and
confusing performance implications....

> But the real reason bint is an int is because that's the
> the result type of relational and equality operators in C. The value
> of bint primarily lies in its coercion to/from Python.

Thanks for that factoid -- nice to know.

[Stefan Behnel]

Am 18.05.2013 19:02, schrieb Nikita Nemkin:
> 1) bool objects have to be refcounted, which means every bool assignment
> has an extra INCREF/DECREF.
> (Cython also inserts an unnecessary typecheck when constants are
> assigned).

Compiler smartification patches are welcome.

Stefan