__str__ and __repr__ confusion

[Riccardo Gori]

Hello,
I'm working at fixing issue 888. It is easy to fix but I don't understand why
in sympy/utilities/lambdify.py I found that lambdastr function does use
__str__ for expr.

def lambdastr(args, expr):
"""
Returns a string that can be evaluated to a lambda function.

>>> from sympy import symbols
>>> x,y,z = symbols('xyz')
>>> lambdastr(x, x**2)
'lambda x: (x**2)'
>>> lambdastr((x,y,z), [z,y,x])
'lambda x,y,z: ([z, y, x])'
"""
# Transform everything to strings.
expr = str(expr)
if isinstance(args, str):
pass
elif hasattr(args, "__iter__"):
args = ",".join(str(a) for a in args)
else:
args = str(args)

return "lambda %s: (%s)" % (args, expr)

But in python doc we can read:

(http://docs.python.org/ref/customization.html)
################################
__repr__( self)
Called by the repr() built-in function and by string conversions (reverse
quotes) to compute the ``official'' string representation of an object. If at
all possible, this should look like a valid Python expression that could be
used to recreate an object with the same value (given an appropriate
environment). If this is not possible, a string of the form "<...some useful
description...>" should be returned. The return value must be a string
object. If a class defines __repr__() but not __str__(), then __repr__() is
also used when an ``informal'' string representation of instances of that
class is required.

This is typically used for debugging, so it is important that the
representation is information-rich and unambiguous.

__str__( self)
Called by the str() built-in function and by the print statement to
compute the ``informal'' string representation of an object. This differs
from __repr__() in that it does not have to be a valid Python expression: a
more convenient or concise representation may be used instead. The return
value must be a string object.
################################

As I understand __repr__ should be used to have a pythonic valid expression
and __str__ should be a normal string. But sympy Basic __repr__
implementation returns unicode text with repr_level_2 so we can't use it in
eval().

This is also discussed in issue 697.

I suggest to change this behaviour, so we can make lambdify more robust. Is
someone working at this?

Thank you

Riccardo

[Ondrej Certik]

Hi Riccardo,

sorry for a late reply, I am at a PMAA08[1] conference now, so I get
to emails sporadically.

please go ahead and fix the __repr__ and __str__ methods. As you
rightly cited, let's follow the python policy, i.e. __repr__ to be
something evaluable (or at least something that sympify can handle)
and __str__ a normal string. And get rid of set_repr_level()
completely. Let's use

pprint

for nice unicdoe asciiart printing and other methods for other style
of printing.

The only thing I am not sure now -- what should be the difference
between __str__ and __repr__ then? I know we discussed it in the
issues, but I don't think we came to a conclusion. I think the __str__
should return a good looking string, but this behaviour should not
change using some global state (set_repr_level). Then there should be
functions like pprint, or print_tree or print_unicode to do other
kinds of printing. And as to __repr__, I know it should be something
evaluable, but the quesiton is, whether

(sympify(1)/2).__repr__

should return "1/2" or "sympify(1)/2" (or something else). Note, that
in isympy, we use the "pprint" method, so the comfort way of using
sympy will not change by how __repr__ looks like. If you use regular
python shell, you can also easily hook up pprint, e.g.

>>> from sympy import pprint
>>> import sys
>>> sys.displayhook = pprint
>>> from sympy.abc import x
>>> x
x
>>> x**2
2
x

so it seems to me, that __repr__ should really return something that
you can directly evaluate, i.e.

(sympify("1/2").__repr_

should return sympify(1)/2, or Integer(1)/2, or something like that.

Ondrej

[Ondrej Certik]

[1] http://www.dcs.bbk.ac.uk/pmaa08/

[Riccardo Gori]

Hello,
I think that repr() should return an evaluable expression that returns the
same type of the called object:
i.e.

In [1]: Basic.set_repr_level(0)
Out[1]: 1

In [2]: repr(sympify("1/2"))
Out[2]: Half(1, 2)

It should not returns 1/2 because evaluating 1/2 returns a float:

In [3]: Basic.set_repr_level(1)
Out[3]: 0

In [4]: repr(sympify("1/2"))
Out[4]: 1/2

In [5]: type(eval(repr(sympify("1/2"))))
Out[5]: <type 'float'>

In this particoular case the problem is that Half is not imported in the
namespace so we can't evaluate it, so a solution is to implement __repr__
that returns a function that returns an Half(): something like:

In: repr(sympify("1/2"))
Out: Rational(1,2)
In: type(Rational(1,2))
Out: <class 'sympy.core.numbers.Half'>

>
> Ondrej
>

Riccardo

[Ondrej Certik]

Yes, I agree with this.

Also, let's get rid of set_repr_level()? This is quite an important
change, so I'd like to hear opinions of others as well.

Ondrej

[Kirill Smelkov]

We already had this discussion in the past:

http://code.google.com/p/sympy/issues/detail?id=697

And I think repr(x) should be "Symbol('x')", str(x) should be "x" and
there should be no Basic.set_repr_level.

--
Всего хорошего, Кирилл.

[Ondrej Certik]

>> Also, let's get rid of set_repr_level()? This is quite an important
>> change, so I'd like to hear opinions of others as well.
>
> We already had this discussion in the past:
>
> http://code.google.com/p/sympy/issues/detail?id=697
>
> And I think repr(x) should be "Symbol('x')", str(x) should be "x" and
> there should be no Basic.set_repr_level.

Agree. So you mean this, right Kirill?

>>> repr(x**2/2)
Mul(Half(1, 2), Pow(Symbol('x'), Integer(2)))
>>> str(x**2/2)
'(1/2)*x**2'

If we all agree, let's do it.

Ondrej

[Pearu Peterson]

On Jun 20, 11:42 am, "Ondrej Certik" <ond...@certik.cz> wrote:
> >> Also, let's get rid of set_repr_level()? This is quite an important
> >> change, so I'd like to hear opinions of others as well.
>
> > We already had this discussion in the past:
>
> >http://code.google.com/p/sympy/issues/detail?id=697
>
> > And I think repr(x) should be "Symbol('x')", str(x) should be "x" and
> > there should be no Basic.set_repr_level.
>
> Agree. So you mean this, right Kirill?
>
> >>> repr(x**2/2)
>
> Mul(Half(1, 2), Pow(Symbol('x'), Integer(2)))

There is a problem with this representation - it will
be implementation dependent (say, one will change
the Half to something else like Rational).

Another solution to the repr probelm is to return

sympify('x**2/2')

that will satisfy the basic assumption behind repr:

eval(repr(obj)) == obj

(assuming that the parser works correctly) and
is well readable.

Pearu

[Riccardo Gori]

Commited in Python 3k beta_1:
http://www.python.org/dev/peps/pep-3138/

[Kirill Smelkov]

On Fri, Jun 20, 2008 at 05:30:29AM -0700, Pearu Peterson wrote:
>
>
>
> On Jun 20, 11:42 am, "Ondrej Certik" <ond...@certik.cz> wrote:
> > >> Also, let's get rid of set_repr_level()? This is quite an important
> > >> change, so I'd like to hear opinions of others as well.
> >
> > > We already had this discussion in the past:
> >
> > >http://code.google.com/p/sympy/issues/detail?id=697
> >
> > > And I think repr(x) should be "Symbol('x')", str(x) should be "x" and
> > > there should be no Basic.set_repr_level.
> >
> > Agree. So you mean this, right Kirill?
> >
> > >>> repr(x**2/2)
> >
> > Mul(Half(1, 2), Pow(Symbol('x'), Integer(2)))

Yes, exactly.

> There is a problem with this representation - it will
> be implementation dependent (say, one will change
> the Half to something else like Rational).

I don't think there is a problem that repr is implementation dependent --
in fact repr is usually used for debugging and it is more convenient
that it'll tell us what exactly some symbolic tree is.

So, if repr says there is a Half - I think it is good that we could
distinguish it from some Rational.

> Another solution to the repr probelm is to return
>
> sympify('x**2/2')
>
> that will satisfy the basic assumption behind repr:
>
> eval(repr(obj)) == obj
>
> (assuming that the parser works correctly) and
> is well readable.

I think we could have

eval(repr(obj)) == obj

with the above approach too.

At present

eval('Half')

produces NameError, but I think this should be fixed.

----

I'm for:

>>> repr(x**2/2)

Mul(Half(1, 2), Pow(Symbol('x'), Integer(2)))

--
Всего хорошего, Кирилл.

[Ondrej Certik]

This is fixed trivially just by importing Half:

In [1]: sympify(Half)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)

/home/ondra/sympy/<ipython console> in <module>()

NameError: name 'Half' is not defined

In [2]: from sympy.core.numbers import Half

In [3]: sympify(Half)
Out[3]: <class 'sympy.core.numbers.Half'>

>
>
> ----
>
> I'm for:
>
> >>> repr(x**2/2)
>
> Mul(Half(1, 2), Pow(Symbol('x'), Integer(2)))

The Pearu's proposition has the advantage that it's shorther. However
I think the repr should be in a way that you can copy & paste for
example into our test suite. Neither sympify() nor Mul(Half(1, 2),
Pow(Symbol('x'), Integer(2))) could (I mean of course you can do that,
but it looks ugly). Another option is this:

>>> repr(x**2/2)
Symbol("x")**2/2
>>> repr(x**2/2 /x**2)
sympify(1)/2
>>> repr(x/x)
sympify(1)
>>> repr(sin(x).series(x))
Symbol("x") - Symbol("x")**3/6 + Symbol("x")**5/120 + O(Symbol("x")**6)

And actually, how about using "x" instead of "Symbol("x")" for all
symbols that are loaded by "from sympy.abc import *" ? The repr would
return a very natural expressions in 95% of cases and it could always
(100%) be evaluated by:

from sympy.abc import *
from sympy import *
eval( repr(something) )

Would that be pythonic? If not, we may for example import "x" (maybe
also y and z) by "from sympy import *", so that x, y, z would be just
that and all other symbols by Symbol("a").

Ondrej

[Kirill Smelkov]

I think before we are going to answer what repr output should be, it would be
better to remember what repr does in the first place.

Ondrej, I think you are forgetting one important thing -- repr is used
often for debugging.

I quote http://docs.python.org/ref/customization.html#l2h-183

__repr__(self):

Called by the repr() built-in function and by string conversions (reverse
quotes) to compute the ``official'' string representation of an object. If at
all possible, this should look like a valid Python expression that could be
used to recreate an object with the same value (given an appropriate
environment). If this is not possible, a string of the form "<...some useful
description...>" should be returned. The return value must be a string object.
If a class defines __repr__() but not __str__(), then __repr__() is also used
when an ``informal'' string representation of instances of that class is
required.

This is typically used for debugging, so it is important that the
representation is information-rich and unambiguous.

So, say about Symbol('x') -> 'x' -- symbols could be Dummy, Temporary, they
could have non-default assumption, etc.

When we are looking at repr(e) we want to know it's internal representation --
that's why repr should be verbose, not terse.

For all other use-case we have various printers which could be plugged into
interactive Python session either in IPython or in pure puthon with the help of
sympy.interactive.

I'm still for repr to return verbose internal form. What do you think?

--
Всего хорошего, Кирилл.

[Fredrik Johansson]

On Fri, Jun 27, 2008 at 10:56 PM, Kirill Smelkov
<ki...@landau.phys.spbu.ru> wrote:

> I quote http://docs.python.org/ref/customization.html#l2h-183
>
> __repr__(self):
>
> Called by the repr() built-in function and by string conversions (reverse
> quotes) to compute the ``official'' string representation of an object. If at
> all possible, this should look like a valid Python expression that could be
> used to recreate an object with the same value (given an appropriate
> environment). If this is not possible, a string of the form "<...some useful
> description...>" should be returned. The return value must be a string object.
> If a class defines __repr__() but not __str__(), then __repr__() is also used
> when an ``informal'' string representation of instances of that class is
> required.

Note "given an appropriate environment". The appropriate environment
is use within SymPy so I think it is actually sufficient if
sympify(repr(x)) == x holds, sympify taking the role of eval. It is
fine in my opinion if repr displays rationals as 2/3, etc.

> This is typically used for debugging, so it is important that the
> representation is information-rich and unambiguous.
>
>
> So, say about Symbol('x') -> 'x' -- symbols could be Dummy, Temporary, they
> could have non-default assumption, etc.

Yes, and this is something I have pointed out before. It is very
problematic that expressions can have "hidden" properties, that don't
show up in the output of either repr() or str(). My suggested solution
is to remove assumptions from expressions and get rid of
Dummy/Temporary/Wild, just having symbols.

Fredrik

[Kirill Smelkov]

Why woudn't str(expr) serve for this?

Guys, you are just forcing someone to invent another pairs of methods
for which repr/__repr__ were originally invented for.

If repr is not that informative there will be some point in time, where
this information would be needed, and one (probably me) will have to
write say debug/__debug__, but why can't we have this sort of output
right from repr? What are cons of this? My point is that there is
str(expr) and also we can always use sympy.interactive and ipython.

--
Всего хорошего, Кирилл.

[Ondrej Certik]

Sorry for top posting, writing from a cell phone. I think we need to
implement all printing ways we mentioned in this thread, probably in
the Printer class just like pretty printing or latex. Then we will
just call one in Basic.__str__ and another in Basic.__repr__ and
remove all other str and repr methods. We can of course discuss which
one to call from where, but then we'll be just discussing a change of
exactly one line of code. As to other things, I agree with Fredrik.
Fredrik, will you come to euroscipy or scipy2008, so that we can fix
this together? It be awesome. :) All students here, please apply for
sponsorship to scipy2008, so that enthought has some good students to
choose from and that we can hack on sympy at Caltech.:) Ondrej

[Kirill Smelkov]

All,

Ok, after a bit of thinking, and good idea from Fredrik about
"appropriate context" in repr description, here is what I propose:

pprint(expr) -- nice looking 2D pretty-printed formula

str(expr) -- if possible nice loocking 1D formula, and

sympify(str(expr)) == expr invariant holds

repr(expr) -- internal representation of expr with class names and
other details, and

eval(repr(expr)) == expr invariant holds

Sounds reasonable?

--
Всего хорошего, Кирилл.

[Ondrej Certik]

On Sun, Jun 29, 2008 at 2:16 PM, Kirill Smelkov

Sounds ok to me. But as I said, all printing ways should be really
easy to access and also very easy to switch, ideally just one line of
code in Basic.__repr__().

Ondrej

[Riccardo Gori]

On Sunday 29 June 2008 14:16:24 Kirill Smelkov wrote:
> Ok, after a bit of thinking, and good idea from Fredrik about
> "appropriate context" in repr description, here is what I propose:
>
> pprint(expr) -- nice looking 2D pretty-printed formula
>
> str(expr) -- if possible nice loocking 1D formula, and
>
> sympify(str(expr)) == expr invariant holds
>
>
> repr(expr) -- internal representation of expr with class names and
> other details, and
>
> eval(repr(expr)) == expr invariant holds
>
>
> Sounds reasonable?

I agree 100%

Riccardo

[Ondrej Certik]

Great. If you'd like to give it a shot, go ahead. This thread should
give enough information how to do it, if not, just ask.

Ondrej