overload builtin operator
Shaun
I'm trying to overload the divide operator in python for basic arithmetic.
eg. 10/2 ... no classes involved.
I am attempting to redefine operator.__div__ as follows:
# my divide function
def safediv(a,b):
return ...
# reassign buildin __div__
import operator
operator.__div__ = safediv
The operator.__dict__ seems to be updated OK but the '/' operator still
calls buildin __div__
Does anyone know if this is possible and if I'm going along the correct
path with my attempts above?
Is it possible to do this using a C extention?
Regards,
Shaun.
Robert Kern
> Hi,
>
> I'm trying to overload the divide operator in python for basic arithmetic.
> eg. 10/2 ... no classes involved.
>
> I am attempting to redefine operator.__div__ as follows:
>
> # my divide function
> def safediv(a,b):
> return ...
>
> # reassign buildin __div__
> import operator
> operator.__div__ = safediv
>
> The operator.__dict__ seems to be updated OK but the '/' operator still
> calls buildin __div__
Actually,
int1 / int2
is, I believe, equivalent to
int.__div__(int1, int2)
operator.__div__ does not get called.
> Does anyone know if this is possible and if I'm going along the correct
> path with my attempts above?
> Is it possible to do this using a C extention?
No, you can't replace the methods on builtin types.
--
Robert Kern
rk...@ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
Reinhold Birkenfeld
> Hi,
>
> I'm trying to overload the divide operator in python for basic arithmetic.
> eg. 10/2 ... no classes involved.
>
> I am attempting to redefine operator.__div__ as follows:
>
> # my divide function
> def safediv(a,b):
> return ...
>
> # reassign buildin __div__
> import operator
> operator.__div__ = safediv
>
> The operator.__dict__ seems to be updated OK but the '/' operator still
> calls buildin __div__
It won't work that way. You cannot globally modify the behaviour of an operator,
but you can customize how an operator works for your type.
Consider:
class safeint(int):
def __div__(self, other):
return safediv(self, other)
safeint(10)/2
Reinhold
Bengt Richter
You are right that you cannot globally modify the behaviour of an operator in the
sense the OP seems to be envisioning, but with some trouble I think it would be possible
to interfere with the translation of '<numerator-term>/<denominator-term>' to become
'safediv(<numerator-term>, <denominator-term>)' by transforming the AST during a custom
import process, such that wherever a Div node is found, a CallFunc node is substituted. E.g.,
for a node like
Div((Name('numerator'), Name('denominator')))
substitute another node like
CallFunc(Name('safediv'), [Name('numerator'), Name('denominator')], None, None)
where the Name('numerator') and Name('denominator') in the latter are actually
the Div node's .left and .right, so as to operate on the same args (which can
be abitrary expression-representing subtrees).
Of course, you probably also want to replace
AugAssign(Name('a'), '/=', Name('b'))
with
Assign([AssName('a', 'OP_ASSIGN')], CallFunc(Name('safediv'), [Name('a'), Name('b')], None, None))
Unfortunately, the AST tree does not seem to be designed to be edited easily wrt
_replacing_ nodes found by walking via flattened lists vs. just _using_ them in order.
I think I can create special walker that can do replacements of nodes found anywhere, and
calling node-type-specific or default supplied callbacks to supply replacements for original nodes
passed to them, but this will require a number of specialized visitors etc., so I will have to get back
to this later. But at least the OP and you nudged me into thinking about AST rewriting again ;-)
(also affects my @@sourcedeco ideas ;-)
Regards,
Bengt Richter
Shaun
I'll take a different tack.
The exact problem I am trying to solve here is to avoid the
ZeroDivisionError in division.
I have c++ code which delegates to python to calculate expressions on
table cells. The values of the table cell are arbitary numbers and the
expressions to be calculated are fairly simple python arithmetic and
math functions.
The problem being that some users want an expression like '(100/x)+ 3'
where x=0 to return 3. So that dividing a number by zero results in 0.
Apart from parsing the expression string myself and checking for divide
by zero I can't find another way to solve the problem.
Hopefully someone out there has some ideas.
Thanks,
Shaun.
Robert Kern
> Thanks for your replies, obviously this isn't a simple thing to do so
> I'll take a different tack.
>
> The exact problem I am trying to solve here is to avoid the
> ZeroDivisionError in division.
> I have c++ code which delegates to python to calculate expressions on
> table cells. The values of the table cell are arbitary numbers and the
> expressions to be calculated are fairly simple python arithmetic and
> math functions.
>
> The problem being that some users want an expression like '(100/x)+ 3'
> where x=0 to return 3. So that dividing a number by zero results in 0.
You have silly users. There's only one place I can think of where
division of a finite, nonzero number by zero can sensibly result in zero
(underdetermined least squares via SVD). I'd be curious as to why they
want that behavior.
> Apart from parsing the expression string myself and checking for divide
> by zero I can't find another way to solve the problem.
It might be easier to parse the expression and wrap all of the numbers
by a subclass of float that does division like you want.
Bengt Richter
You are right that you cannot globally modify the behaviour of an operator in the
sense the OP seems to be envisioning, but with some trouble I think it would be possible
to interfere with the translation of '<numerator-term>/<denominator-term>' to become
'safediv(<numerator-term>, <denominator-term>)' by transforming the AST during a custom
import process, such that wherever a Div node is found, a CallFunc node is substituted. E.g.,
for a node like
Div((Name('numerator'), Name('denominator')))
substitute another node like
CallFunc(Name('safediv'), [Name('numerator'), Name('denominator')], None, None)
where the Name('numerator') and Name('denominator') in the latter are actually
the Div node's .left and .right, so as to operate on the same args (which can
be abitrary expression-representing subtrees).
Of course, you probably also want to replace
AugAssign(Name('a'), '/=', Name('b'))
with
Assign([AssName('a', 'OP_ASSIGN')], CallFunc(Name('safediv'), [Name('a'), Name('b')], None, None))
Unfortunately, the AST tree does not seem to be designed to be edited easily wrt
_replacing_ nodes found by walking via flattened lists vs. just _using_ them in order.
I think I can create special walker that can do replacements of nodes found anywhere, and
calling node-type-specific or default supplied callbacks to supply replacements for original nodes
passed to them, but this will require a number of specialized visitor methods etc., so I will
have to get back to this later. But at least the OP and you nudged me into thinking about
AST rewriting again ;-) (also affects my @@sourcedeco ideas ;-)
... this didn't go out due to news server problem, so I'll add some clips from what I did re ast editing:
----< testdiv.py >-------------------------------------------------------------------------
def test():
print 1.0/2.0
print 12/3
a=12; b=3
print a/b
print 2**a/(b+1)
try:
print 'print 1/0 =>'
print 1/0
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'print a/(b*(a-12)) =>'
print a/(b*(a-12))
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'def foo(x=1/(b-3)): return x =>'
def foo(x=1/(b-3)): return x
print 'print foo() =>'
print foo()
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'b /= (a-12) =>'
b /= (a-12)
print 'b after b/=(a-12):', b
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
if __name__ == '__main__': test()
-------------------------------------------------------------------------------------------
Outputfrom run without conversion of / :
0.5
4
4
1024
print 1/0 =>
ZeroDivisionError: integer division or modulo by zero
print a/(b*(a-12)) =>
ZeroDivisionError: integer division or modulo by zero
def foo(x=1/(b-3)): return x =>
ZeroDivisionError: integer division or modulo by zero
b /= (a-12) =>
ZeroDivisionError: integer division or modulo by zero
----< import_safediv.py >------------------------------------------------------------------
# import_safediv.py
from importast import importast
from compiler.ast import Div, CallFunc, Name
from compiler.ast import AugAssign, Assign, AssName
def safediv(num, den):
if den==0: result = 0*num
else: result = num/den
print 'safediv(%r, %r) => %r'%(num, den, result)
return result
def div2safediv(divnode):
"""replace Div nodes with CallFunc nodes calling safediv with same args"""
return CallFunc(Name('safediv'),[divnode.left, divnode.right], None, None, divnode.lineno)
def divaugass2safediv(auganode):
if auganode.op != '/=': return auganode
assname =auganode.node.name
return Assign([AssName(assname, 'OP_ASSIGN')],
CallFunc(Name('safediv'),[Name(assname), auganode.expr], None, None, auganode.lineno))
callbacks = {Div:div2safediv, AugAssign:divaugass2safediv}
def import_safediv(modname, verbose=False):
return importast(modname, callbacks, verbose)
if __name__ == '__main__':
import sys
modname, args = sys.argv[1], sys.argv[2:]
verbose = (args[0:] and args[0]=='-verbose') or False
modsafe = import_safediv(modname, verbose)
modsafe.safediv = safediv
if hasattr(modsafe, 'test'): modsafe.test()
-------------------------------------------------------------------------------------------
Result from running the above and specifying testdiv as the module
to import and enforce safediv on, and run test() on:
safediv(1.0, 2.0) => 0.5
0.5
safediv(12, 3) => 4
4
safediv(12, 3) => 4
4
safediv(4096, 4) => 1024
1024
print 1/0 =>
safediv(1, 0) => 0
0
print a/(b*(a-12)) =>
safediv(12, 0) => 0
0
def foo(x=1/(b-3)): return x =>
safediv(1, 0) => 0
print foo() =>
0
b /= (a-12) =>
safediv(3, 0) => 0
b after b/=(a-12): 0
Note that the exceptions went away ;-)
It was much messier than it should have been, and the code is not very efficient,
but at least it's a kind of proof of concept ;-)
Regards,
Bengt Richter
Bengt Richter
[... a response to the OP's apparent desire to "overload the divide operator" with a call
to his safediv function ...]
The part that rewrote the the AugAssign could only work for plain name Augassign targets, so
I introduced a helper function to generate a suitable assignment target node for attributes,
subscripts, and slices as well. So the test (still very alpha) looks like the following now:
print 's=[15]; s[0] /= 3; print s =>'
s=[15]; s[0] /= 3; print s
class T(list):
def __getitem__(self, i): print i; return 42
def __setitem__(self, i, v): print i, v
def __div__(self, other): print self, other; return 4242
t = T()
print 't.x=15; t.x /= 3; print t.x =>'
t.x=15; t.x /= 3; print t.x
print 't=T([15, 27]); t /= 3; print t =>'
t=T([15, 27]); t /= 3; print t
def foo(x, y):
"""for dis.dis to show code"""
z = x/y
x /= y
x.a /= y
x[z] /= y
x[:] /= y
if __name__ == '__main__': test()
------------------------------------------------------------------------------------------
Outputfrom run without conversion of / :
----< import_safediv.py >------------------------------------------------------------------
# import_safediv.py
from arborist import import_editing_ast, get_augassign_tgt
from compiler.ast import Div, CallFunc, Name, AugAssign, Assign
def safediv(num, den):
if den==0: result = 0*num
else: result = num/den
print 'safediv(%r, %r) => %r'%(num, den, result)
return result
def div2safediv(divnode):
"""replace Div nodes with CallFunc nodes calling safediv with same args"""
return CallFunc(Name('safediv'),[divnode.left, divnode.right], None, None, divnode.lineno)
def divaugass2safediv(auganode):
if auganode.op != '/=': return auganode
return Assign([get_augassign_tgt(auganode)],
CallFunc(Name('safediv'),[auganode.node, auganode.expr], None, None, auganode.lineno))
callbacks = {Div:div2safediv, AugAssign:divaugass2safediv}
def import_safediv(modname, verbose=False):
modsafe = import_editing_ast(modname, callbacks, verbose)
modsafe.safediv = safediv
return modsafe
if __name__ == '__main__':
modsafe = import_safediv('testdiv', verbose=True)
modsafe.test()
-------------------------------------------------------------------------------------------
Result from using the above interactively:
[14:21] C:\pywk\ut\ast>py24
Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32
Type "help", "copyright", "credits" or "license" for more information.
First import as usual and run test()
>>> import testdiv
>>> testdiv.test()
0.5
4
4
1024
print 1/0 =>
ZeroDivisionError: integer division or modulo by zero
print a/(b*(a-12)) =>
ZeroDivisionError: integer division or modulo by zero
def foo(x=1/(b-3)): return x =>
ZeroDivisionError: integer division or modulo by zero
b /= (a-12) =>
ZeroDivisionError: integer division or modulo by zero
s=[15]; s[0] /= 3; print s =>
[5]
t.x=15; t.x /= 3; print t.x =>
5
t=T([15, 27]); t /= 3; print t =>
[15, 27] 3
4242
Now import the import_safediv importer, to import with
conversion of ast to effect translation of divides to safediv calls:
Import and runs test() much as before:
>>> from import_safediv import import_safediv
>>> tdsafe = import_safediv('testdiv')
>>> tdsafe.test()
safediv(1.0, 2.0) => 0.5
0.5
safediv(12, 3) => 4
4
safediv(12, 3) => 4
4
safediv(4096, 4) => 1024
1024
print 1/0 =>
safediv(1, 0) => 0
0
print a/(b*(a-12)) =>
safediv(12, 0) => 0
0
def foo(x=1/(b-3)): return x =>
safediv(1, 0) => 0
print foo() =>
0
b /= (a-12) =>
safediv(3, 0) => 0
b after b/=(a-12): 0
s=[15]; s[0] /= 3; print s =>
safediv(15, 3) => 5
[5]
t.x=15; t.x /= 3; print t.x =>
safediv(15, 3) => 5
5
t=T([15, 27]); t /= 3; print t =>
[15, 27] 3
safediv([15, 27], 3) => 4242
4242
Look at the code generated by normal import first
>>> import dis
>>> dis.dis(testdiv.foo)
40 0 LOAD_FAST 0 (x)
3 LOAD_FAST 1 (y)
6 BINARY_DIVIDE
7 STORE_FAST 2 (z)
41 10 LOAD_FAST 0 (x)
13 LOAD_FAST 1 (y)
16 INPLACE_DIVIDE
17 STORE_FAST 0 (x)
42 20 LOAD_FAST 0 (x)
23 DUP_TOP
24 LOAD_ATTR 3 (a)
27 LOAD_FAST 1 (y)
30 INPLACE_DIVIDE
31 ROT_TWO
32 STORE_ATTR 3 (a)
43 35 LOAD_FAST 0 (x)
38 LOAD_FAST 2 (z)
41 DUP_TOPX 2
44 BINARY_SUBSCR
45 LOAD_FAST 1 (y)
48 INPLACE_DIVIDE
49 ROT_THREE
50 STORE_SUBSCR
44 51 LOAD_FAST 0 (x)
54 DUP_TOP
55 SLICE+0
56 LOAD_FAST 1 (y)
59 INPLACE_DIVIDE
60 ROT_TWO
61 STORE_SLICE+0
62 LOAD_CONST 1 (None)
65 RETURN_VALUE
and then corresponding import_safediv (with same line numbers):
>>> dis.dis(tdsafe.foo)
40 0 LOAD_GLOBAL 0 (safediv)
3 LOAD_FAST 0 (x)
6 LOAD_FAST 1 (y)
9 CALL_FUNCTION 2
12 STORE_FAST 3 (z)
41 15 LOAD_GLOBAL 0 (safediv)
18 LOAD_FAST 0 (x)
21 LOAD_FAST 1 (y)
24 CALL_FUNCTION 2
27 STORE_FAST 0 (x)
42 30 LOAD_GLOBAL 0 (safediv)
33 LOAD_FAST 0 (x)
36 LOAD_ATTR 4 (a)
39 LOAD_FAST 1 (y)
42 CALL_FUNCTION 2
45 LOAD_FAST 0 (x)
48 STORE_ATTR 4 (a)
43 51 LOAD_GLOBAL 0 (safediv)
54 LOAD_FAST 0 (x)
57 LOAD_FAST 3 (z)
60 BINARY_SUBSCR
61 LOAD_FAST 1 (y)
64 CALL_FUNCTION 2
67 LOAD_FAST 0 (x)
70 LOAD_FAST 3 (z)
73 STORE_SUBSCR
44 74 LOAD_GLOBAL 0 (safediv)
77 LOAD_FAST 0 (x)
80 SLICE+0
81 LOAD_FAST 1 (y)
84 CALL_FUNCTION 2
87 LOAD_FAST 0 (x)
90 STORE_SLICE+0
91 LOAD_CONST 1 (None)
94 RETURN_VALUE
I guess this shows that code munging by AST rewriting is somewhat feasible,
but you can probably expect to have to dig into dark corners ;-)
I need to think through a few things I'm doing by intuition before I go much further...
Regards,
Bengt Richter
Terry Reedy
like develop a customized interpreter that has alternate definitions for
builtin operators. So maybe the OP should ask again in a year or two.
TJR
Iain King
Robert Kern wrote:
> Shaun wrote:
> > Thanks for your replies, obviously this isn't a simple thing to do so
> > I'll take a different tack.
> >
> > The exact problem I am trying to solve here is to avoid the
> > ZeroDivisionError in division.
> > I have c++ code which delegates to python to calculate expressions on
> > table cells. The values of the table cell are arbitary numbers and the
> > expressions to be calculated are fairly simple python arithmetic and
> > math functions.
> >
> > The problem being that some users want an expression like '(100/x)+ 3'
> > where x=0 to return 3. So that dividing a number by zero results in 0.
>
> You have silly users.
You mean you don't? Damn. Can I have some of yours?
Iain
Robert Kern
> Robert Kern wrote:
>>You have silly users.
>
> You mean you don't? Damn. Can I have some of yours?
No, you may not. Mine! All mine!
Bengt Richter
OTOH, did you notice that I provided (although I didn't post the underlying
walker framework) a way (that works now) of importing a module with
the / operator effectively converted to a safediv call, as the OP apparently wanted?
;-)
BTW, I posted an update that handles non-simple-name-target /= augassigns as well,
but though it shows on google groups, I can't see it yet. Still some news server
problem, apparently. Or maybe it's just slogging to catch up after previous problem...
Regards,
Bengt Richter