Bug or Feature?
Stephan Diehl
following behaviour:
Python 2.3.1
============
>>> class Int(int):pass
...
>>> a = Int(7)
>>> b = Int(8)
>>> c = a + b
>>> type(c)
<type 'int'>
Basicly: Int is not closed under it's defined operations. :-(
by contrast:
>>> class myset(Set):pass
...
>>> a = myset([1,2])
>>> b = myset([2,3])
>>> c = a & b
>>> type(c)
<class '__main__.myset'>
subclasses of Set are closed under the set operations. :-)
Should I consider this as a bug or something I have to live with?
Example, why this can be problematic:
Consider, I want to define an integer subtype that holds integers modulo N.
The straight forward way would be:
class mod7(int):
def __new__(cls,value):
return int.__new__(cls,value%7)
Since addition, substraction, etc. doesn't work as expected, one must
redefine all relevant operators.
This is not what I'd consider object oriented behaviour.
Stephan
P.S.: for the mind police out there: Other than that, Python is great.
Michele Simionato
> I was playing around with defining new types and have seen the
> following behaviour:
>
> Python 2.3.1
> ============
>
> >>> class Int(int):pass
> ...
> >>> a = Int(7)
> >>> b = Int(8)
> >>> c = a + b
> >>> type(c)
> <type 'int'>
>
> Basicly: Int is not closed under it's defined operations. :-(
>
You want my "innermethods" module. Originally I wrote it as a recipe
for the cookbook, but then I forgot to post it ;)
"""
A method of a class ``C`` is said to be an ``inner method`` of ``C``
if it returns an instance of ``C``. For instance, the ``.replace``
method of the string class is an inner method of ``str``, since
invoked by a string, it returns a string:
>>> s='s'
>>> r=s.replace('s','r')
>>> print r,type(r)
r <type 'str'>
In general, the property of being an inner method is not preserved
by inheritance. For instance, if I define a subclass of the string class,
>>> class Str(str):
... pass
then ``Str`` has still a ``.replace`` method (inherited from ``str``),
but it is not an inner method of ``Str``:
>>> s=Str('s')
>>> r=s.replace('s','r')
>>> print r,type(r) # the type of r is 'str', not 'Str'!
r <type 'str'>
In many circumstances, this is not a problem. However, sometimes
it is important to keep the inner methods safe under inheritance.
If we are writing the methods from scratch, this can be done
by returning ``self.__class__(returnvalue)`` instead of the
simple ``returnvalue``. However, if the methods have been written
by others and we are simply inhering them, we need a more sophisticated
solution.
This module provides a ``protect_inner_methods`` metaclass-like function
which guarantees that the inner methods of the class invoking it are
safe under inheritance. As an exemplication and a commodoty, the
module also provides a ``Str`` class with protected versions of
all the inner methods of the builtin ``str`` class. The only
exception are ``__str__`` and ``__repr__``: it is better than
they stay unprotected, so the string representation of subclasses
of ``str`` is a simple plain string, not an enhanced one. In
this way ``str(x)`` continues to convert ``x`` instances in ``str``
instances. If ``Str`` instances are wanted, use ``Str()``!
Here is an example, showing that ``Str.replace`` is an inner method
of ``Str``:
>>> from innermethods import Str
>>> s=Str('s')
>>> r=s.replace('s','r') # .replace invoked from an instance of 'Str'
>>> print r,type(r) # returns an instance of 'Str', i.e. it is inner method
r <class 'innermethods.Str'>
Let me show that the inner methods of ``Str`` are protected under
inheritance, i.e. they continues to be inner methods of the
subclasses:
>>> class MyStr(Str):
... pass
>>> s=MyStr('s')
>>> r=s.replace('s','r') # .replace invoked from an instance of 'Str'
>>> print r,type(r) # returns an instance of 'Str', i.e. it is inner method
r <class 'MyStr'>
"""
def protect_inner__method(cls,methname):
"""Given a class and an inner method name, returns a wrapped
version of the inner method. Raise an error if the method is not
defined in any ancestors of the class. In the case of duplicated
names, the first method in the MRO is taken."""
def _(self,*args,**kw):
supermethod=getattr(super(cls,self),methname)
return self.__class__(supermethod(*args,**kw))
return _
def protect_inner_methods(name,bases,dic):
"""Metaclass-like function. Returns a new class with inner methods
protected under inheritance. The calling class must provide a
``__innermethods__`` attribute containing the list of the
method names to be protected, otherwise nothing will be done."""
cls=type(name,bases,dic) # generate the new class
innermethods=dic.get('__innermethods__',[])
# look for the list of methods to wrap and wrap them all
for methname in innermethods:
setattr(cls,methname,protect_inner__method(cls,methname))
return cls
class Str(str):
"""A string class with wrapped inner methods."""
__metaclass__=protect_inner_methods
__innermethods__="""__add__ __mod__ __mul__ __rmod__ __rmul__ capitalize
center expandtabs join ljust lower lstrip replace rjust rstrip strip
swapcase title translate upper zfill""".split()
def __radd__(self,other): # inner method
"""Guarantees ``'a'+Str('b')`` returns a ``Str`` instance.
Also ``s='s'; s+=Str('b')`` makes ``s`` a ``Str`` instance."""
return self.__class__(other.__add__(self))
Stephan Diehl
> Stephan Diehl <stepha...@gmx.net> wrote in message
> news:<bpsik5$ni7$07$1...@news.t-online.com>...
>> I was playing around with defining new types and have seen the
>> following behaviour:
>>
>> Python 2.3.1
>> ============
>>
>> >>> class Int(int):pass
>> ...
>> >>> a = Int(7)
>> >>> b = Int(8)
>> >>> c = a + b
>> >>> type(c)
>> <type 'int'>
>>
>> Basicly: Int is not closed under it's defined operations. :-(
>>
>
> You want my "innermethods" module. Originally I wrote it as a recipe
> for the cookbook, but then I forgot to post it ;)
>
Yes, thanks, this is most helpfull.
Although, the question was more along the line, if (in the light of the
principle of least surprise) this behaviour makes sense.
My private opinion would be, that the usefullness of builtin types as real
types would be much better, if they were safe under inheritance.
Michael Hudson
> Although, the question was more along the line, if (in the light of the
> principle of least surprise) this behaviour makes sense.
> My private opinion would be, that the usefullness of builtin types as real
> types would be much better, if they were safe under inheritance.
The problem is that it's impossible for the int type to know about the
requirements of your subclass' constructor. Explicit is better than
implicit, and all that.
For the example you posted, I don't see much of an advantage to
inheriting from int.
Cheers,
mwh
--
If a train station is a place where a train stops, what's a
workstation? -- unknown (to me, at least)
Duncan Booth
> Although, the question was more along the line, if (in the light of
> the principle of least surprise) this behaviour makes sense.
> My private opinion would be, that the usefullness of builtin types as
> real types would be much better, if they were safe under inheritance.
>
There are some tradeoffs here.
I think that the majority of users would say that the speed of operations
using 'int' is more important than the ease of inheriting from them. If you
can make the 'int' operations work this way, but keep them at least as fast
as they are today, then there may be a case for changing the behaviour.
The other problem though is that this would break backwards compatibility.
For example if any code were to currently add booleans, it would suddenly
get a boolean result. The (untested) code below, currently counts how many
items in a list are greater than a specified value, but under your scheme
it would return a boolean.
def HowManyGreaterThan(aList, aValue):
return reduce(operator.add, [ x > aValue for x in aList ])
I think what would be most useful would be new subclasses of int etc. which
have the properties you desire, but you can write them yourself easily
enough.
What rules, BTW, are you proposing for operations between two separate
subclasses of int? e.g. given your mod7 class, and a corresponding mod9
class should this print 3, 6, 24 or throw an exception?
x = mod7(3)
y = mod9(8)
print x*y
and does this do the same?
print y*x
--
Duncan Booth dun...@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
Stephan Diehl
> Stephan Diehl <stepha...@gmx.net> wrote in
> news:bpt5do$nls$04$1...@news.t-online.com:
>
>> Although, the question was more along the line, if (in the light of
>> the principle of least surprise) this behaviour makes sense.
>> My private opinion would be, that the usefullness of builtin types as
>> real types would be much better, if they were safe under inheritance.
>>
> There are some tradeoffs here.
>
> I think that the majority of users would say that the speed of operations
> using 'int' is more important than the ease of inheriting from them. If
> you can make the 'int' operations work this way, but keep them at least as
> fast as they are today, then there may be a case for changing the
> behaviour.
O.K., since the stuff I'm concerned about here is of no relevance for
99.9 % of all python programmers, execution speed is a very valid reason for
this (and I'll consider this behaviour as a feature :-)
>
> The other problem though is that this would break backwards compatibility.
> For example if any code were to currently add booleans, it would suddenly
> get a boolean result. The (untested) code below, currently counts how many
> items in a list are greater than a specified value, but under your scheme
> it would return a boolean.
>
> def HowManyGreaterThan(aList, aValue):
> return reduce(operator.add, [ x > aValue for x in aList ])
Which, of course, would bring us to the discussion if a bool should be a
subtype of int (and since this was decided long ago, it should not be
discussed, of course)
>
> I think what would be most useful would be new subclasses of int etc.
> which have the properties you desire, but you can write them yourself
> easily enough.
>
> What rules, BTW, are you proposing for operations between two separate
> subclasses of int? e.g. given your mod7 class, and a corresponding mod9
> class should this print 3, 6, 24 or throw an exception?
>
> x = mod7(3)
> y = mod9(8)
> print x*y
>
> and does this do the same?
>
> print y*x
>
tough question. I'd probably throw an exception.
thanks for your input
Stephan
Stephan Diehl
> Stephan Diehl <stepha...@gmx.net> writes:
>
>> Although, the question was more along the line, if (in the light of the
>> principle of least surprise) this behaviour makes sense.
>> My private opinion would be, that the usefullness of builtin types as
>> real types would be much better, if they were safe under inheritance.
>
> The problem is that it's impossible for the int type to know about the
> requirements of your subclass' constructor. Explicit is better than
> implicit, and all that.
>
> For the example you posted, I don't see much of an advantage to
> inheriting from int.
This is probably more a theoretical question, but if I decide to have an
operation within one specific domain, I want the result in that domain too.
Otherwise, there would be no point at all to define such a numeric class at
all.
Stephan
>
> Cheers,
> mwh
>
Mel Wilson
Michael Hudson <m...@python.net> wrote:
>Stephan Diehl <stepha...@gmx.net> writes:
>
>> Although, the question was more along the line, if (in the light of the
>> principle of least surprise) this behaviour makes sense.
>> My private opinion would be, that the usefullness of builtin types as real
>> types would be much better, if they were safe under inheritance.
>
>The problem is that it's impossible for the int type to know about the
>requirements of your subclass' constructor. Explicit is better than
>implicit, and all that.
Not strictly true. A numeric operation can access the
objects class, as:
def __mul__ (self, other):
if not isinstance (other, Decimal):
other = Decimal (other)
return self.__class__ (self._v * other._v, self._e + other._e)
so that operations on a descendant of the Decimal class will
return objects of the descendants class. (Strange that I
didn't code `isinstance (other, self.__class__)`, etc. Must
think out the consequences of that.)
What this would mean in runtime for the bazillions of int
operations that run every day, I don't know.
>For the example you posted, I don't see much of an advantage to
>inheriting from int.
True. With int working as it does, it might be better to
pull out all the stops and write an Int class that supports
full inheritance, and work from there.
Regards. Mel.
Dang Griffith
<stephan.d...@gmx.net> wrote:
...
>This is probably more a theoretical question, but if I decide to have an
>operation within one specific domain, I want the result in that domain too.
>Otherwise, there would be no point at all to define such a numeric class at
>all.
>
>Stephan
I understand what you're saying, but there's no universal rule that
says that the result of operations between members of a set are also
members of the set. Obvious examples include division on the set of
integers, and division over the set of real numbers, i.e.division by
zero is not a real number.
--dang
Michael Hudson
> In article <m3llq5z...@pc150.maths.bris.ac.uk>,
> Michael Hudson <m...@python.net> wrote:
> >Stephan Diehl <stepha...@gmx.net> writes:
> >
> >> Although, the question was more along the line, if (in the light of the
> >> principle of least surprise) this behaviour makes sense.
> >> My private opinion would be, that the usefullness of builtin types as real
> >> types would be much better, if they were safe under inheritance.
> >
> >The problem is that it's impossible for the int type to know about the
> >requirements of your subclass' constructor. Explicit is better than
> >implicit, and all that.
>
> Not strictly true.
Um, did you actually read what I said? Again, with emphasis:
> >The problem is that it's impossible for the int type to know about the
> >*REQUIREMENTS OF YOUR SUBCLASS' CONSTRUCTOR*.
> A numeric operation can access the objects class, as:
>
> def __mul__ (self, other):
> if not isinstance (other, Decimal):
> other = Decimal (other)
> return self.__class__ (self._v * other._v, self._e + other._e)
>
> so that operations on a descendant of the Decimal class will
> return objects of the descendants class.
Yes, but now you're constrainting future subclasses' constructors.
> What this would mean in runtime for the bazillions of int
> operations that run every day, I don't know.
There would be ways to avoid that, I think.
> >For the example you posted, I don't see much of an advantage to
> >inheriting from int.
>
> True. With int working as it does, it might be better to
> pull out all the stops and write an Int class that supports
> full inheritance, and work from there.
I think someone has written such a UserInt somewhere...
It seems to be generally good advice to only subclass builtin types in
Python if you want to ADD behaviour, not change it. Not 100% true,
but close.
Cheers,
mwh
--
LINTILLA: You could take some evening classes.
ARTHUR: What, here?
LINTILLA: Yes, I've got a bottle of them. Little pink ones.
-- The Hitch-Hikers Guide to the Galaxy, Episode 12
Stephan Diehl
If we have a look at the mathematical definition of numbers, the interesting
thing is not the set of these numbers, but the set combined with some
usefull operators (like addition and multiplication).
It is (mathematicaly speaking) not possible to have a result under these
operations that are outside the definition.
In that sense, there IS this universal rule, you were talking about. (and
division by zero is just not allowed by definition).
Stephan
Tim Peters
> I was playing around with defining new types and have seen the
> following behaviour:
>
> Python 2.3.1
> ============
>
> >>> class Int(int):pass
> ...
> >>> a = Int(7)
> >>> b = Int(8)
> >>> c = a + b
> >>> type(c)
> <type 'int'>
>
> Basicly: Int is not closed under it's defined operations. :-(
Yes, and that's intentional. The difficulty is that the base class can't
know how to construct a subclass instance -- there's no guarantee, for
example, that a subclass constructor even accepts an argument list
consisting of one int. There's also no guarantee that, e.g., an instance S
of a subclass of str *wants*
S + ""
to return S identically (if type(S) is str, then S+"" is S). So substantial
work was actually done to disable optimizations (like S+"" is S) for
base-class operations performed on instances of subclasses of builtin types.
So the implementation strives never to screw you, but it may not do what you
want, either. If you want Ints to be closed under an operation, you need to
supply an appropriate implementation of that operation as a method of Int.
Then you control the optimizations you're willing to accept, and can
construct your Int instances so as to satisfy Int invariants (something the
base class can't know how to do, since it can't have any idea of what
user-defined invariants may be -- although it's hard to see the point to
this in the Int example as given, because the Int class is trivial).
> by contrast:
>
> >>> class myset(Set):pass
> ...
> >>> a = myset([1,2])
> >>> b = myset([2,3])
> >>> c = a & b
> >>> type(c)
> <class '__main__.myset'>
>
> subclasses of Set are closed under the set operations. :-)
If you let Guido see that, it won't in 2.4 <wink>.
Stephan Diehl
[...]
>
> So the implementation strives never to screw you, but it may not do what
> you
> want, either. If you want Ints to be closed under an operation, you need
> to supply an appropriate implementation of that operation as a method of
> Int. Then you control the optimizations you're willing to accept, and can
> construct your Int instances so as to satisfy Int invariants (something
> the base class can't know how to do, since it can't have any idea of what
> user-defined invariants may be -- although it's hard to see the point to
> this in the Int example as given, because the Int class is trivial).
Sounds very reasonable. The triviality of the Int example was the reason I
had expected differently (and the fact that int,str,... are immutable what
let them stand out somehow)
>
>> by contrast:
>>
>> >>> class myset(Set):pass
>> ...
>> >>> a = myset([1,2])
>> >>> b = myset([2,3])
>> >>> c = a & b
>> >>> type(c)
>> <class '__main__.myset'>
>>
>> subclasses of Set are closed under the set operations. :-)
>
> If you let Guido see that, it won't in 2.4 <wink>.
Could we please delete this thread? What have I done?
Delaney, Timothy C (Timothy)
> >>
> >> >>> class myset(Set):pass
> >> ...
> >> >>> a = myset([1,2])
> >> >>> b = myset([2,3])
> >> >>> c = a & b
> >> >>> type(c)
> >> <class '__main__.myset'>
> >>
> >> subclasses of Set are closed under the set operations. :-)
> >
> > If you let Guido see that, it won't in 2.4 <wink>.
>
> Could we please delete this thread? What have I done?
The Right Thing(TM).
"In the face of ambiguity, refuse the temptation to guess." (Go Tim Peters!).
Tim Delaney
David Eppstein
"Delaney, Timothy C (Timothy)" <tdel...@avaya.com> wrote:
> > >> subclasses of Set are closed under the set operations. :-)
> > >
> > > If you let Guido see that, it won't in 2.4 <wink>.
> >
> > Could we please delete this thread? What have I done?
>
> The Right Thing(TM).
>
> "In the face of ambiguity, refuse the temptation to guess." (Go Tim Peters!).
It is very useful that ImmutableSet is closed under set operations.
By making the set operations copy the types of their arguments, the
author of sets.py was able to achieve this without duplicating each set
operation's implementation. If you don't like this behavior you can
always override it...
--
David Eppstein http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science
Delaney, Timothy C (Timothy)
>
> It is very useful that ImmutableSet is closed under set operations.
> By making the set operations copy the types of their arguments, the
> author of sets.py was able to achieve this without
> duplicating each set
> operation's implementation. If you don't like this behavior you can
> always override it...
So what happens when I make a set subclass that takes no parameters (other than self) to __init__?
Tim Delaney
David Eppstein
"Delaney, Timothy C (Timothy)" <tdel...@avaya.com> wrote:
> than self) to init ?
What do you expect to happen in general when you define a class that
does not adhere to some protocol, and then use it in that protocol
anyway?
>>> import sets
>>> class FunkySet(sets.Set):
... def __init__(self):
... sets.Set.__init__(self)
...
>>> f=FunkySet()
>>> f.add(1)
>>> g=FunkySet()
>>> g.add(2)
>>> g.add(3)
>>> f&g
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/s
ets.py", line 205, in __and__
return self.__class__(common)
TypeError: __init__() takes exactly 1 argument (2 given)
Andrew Dalke
> > So what happens when I make a set subclass that takes no parameters
(other
> > than self) to init ?
David Eppstein:
> What do you expect to happen in general when you define a class that
> does not adhere to some protocol, and then use it in that protocol
> anyway?
It depends on what you deem to be "the protocol". Things which
use the file protocol often only need the read or readline methods and
don't care about having flush or a given way to call the constructor.
The list protocol is usually only [] and __len__ or __iter__ and less
often get or setdefault or extend.
Going extreme the other way, the behaviour of the builtin
list is that type(x) is types.ListType, which is not true of any
other list-like object.
As for me personally, my answer to TCD is that sets.Set implements
the set protocol and a bit more, to make it useful for derived
classes. I see the case he's talking about as outside the set
protocol and hence implementation defined.
Andrew
da...@dalkescientific.com
Dang Griffith
<stephan.d...@gmx.net> wrote:
And for the set of whole numbers, some divisions are allowed, but any
division that results in a fraction would not be allowed. By
definition, i.e. a combination of operators and operands that result
in a value not in the same domain is not a valid combination. It
works, but seems circular. :-)
--dang
Stephan Diehl
You might have noticed, I was talking about Mathematics, not the imperfect
implementation of it in computers :-)
To have division as a meaningfull operation, the set in question must be a
group under the '*' operation (See for example
"http://en2.wikipedia.org/wiki/Group_(mathematics)" )
The point is, that division is the reverse operation to multiplication.
Cheers, Stephan
Gonçalo Rodrigues
<stephan.d...@gmx.net> wrote:
[snip]
>
>You might have noticed, I was talking about Mathematics, not the imperfect
>implementation of it in computers :-)
>To have division as a meaningfull operation, the set in question must be a
>group under the '*' operation (See for example
>"http://en2.wikipedia.org/wiki/Group_(mathematics)" )
>The point is, that division is the reverse operation to multiplication.
But when talking of "numbers" a group is not enough, you want a field
- a set with two operations (in some situations something weaker, like
integral domains, is enough). In fields, such as real numbers, complex
numbers, p mod integers, p-adic fields, etc. every number has a
multiplicative inverse *except* the zero.
Pedantically yours,
G. Rodrigues
Mel Wilson
"Andrew Dalke" <ada...@mindspring.com> wrote:
>Timothy C Delaney:
>> > So what happens when I make a set subclass that takes no parameters
>(other
>> > than self) to init ?
>
>David Eppstein:
>> What do you expect to happen in general when you define a class that
>> does not adhere to some protocol, and then use it in that protocol
>> anyway?
>
>It depends on what you deem to be "the protocol". Things which
>use the file protocol often only need the read or readline methods and
>don't care about having flush or a given way to call the constructor.
>The list protocol is usually only [] and __len__ or __iter__ and less
>often get or setdefault or extend.
Files are mutable, and most file methods change the
object in-place. There's no question of creating a
different file, modified. Looks like there's more
difference between mutable and immutable class instances
than just __new__.
It's nice to respect a descendant's choice of __init__,
but if you blow off all the descendant's own methods as a
consequence, it's not so nice.
>[ ... ]
>As for me personally, my answer to TCD is that sets.Set implements
>the set protocol and a bit more, to make it useful for derived
>classes. I see the case he's talking about as outside the set
>protocol and hence implementation defined.
I think behaviour of __init__ is just one more class
characteristic to be documented.
Regards. Mel.
Patrick Maupin
> I think someone has written such a UserInt somewhere...
There have been a couple of UserInts floating around.
I think the most recent was mine.
The bad news is that my UserInt module actually returns
ints for all operations.
The good news is that, being a lazy programmer, I didn't
actually write my "UserInt" module -- I let my computer
do it for me, by writing a script which would generate
the module.
The main reason I posted the script is because it can
easily be adapted to create any sort of UserXXX classes,
as well as to add any sort of behavior mods to those
classes. I think it would be trivial to create a UserInt
class which returns other instances of itself for arithmetic
operations by using the script:
Regards,
Pat