Recommendations for creating symbols

[Joachim Durchholz]

Hi all,

I'm wondering about the section on creating symbols in
https://github.com/sympy/sympy/wiki/Idioms-and-Antipatterns#strings-as-input
.

It is mildly discouraging importing from sympy.abc because an accidental
"from sympy.abc import *" would clobber I and Q (and possibly others),
and recommends
>>> import symbols
>>> x, y, z = symbols('x y z')
instead.

Now in https://github.com/sympy/sympy/pull/9219 , I applied this advice to
>>> from sympy.abc import t, w, x, y, z, n, k, m, p, i
and got
>>> t, w, x, y, z, n, k, m, p, i = symbols('t w x y z n k m p i')
I think that's actually really dangerous, because it's easy to swap two
letters without noticing so you essentially get the equivalent of
>>> p = Symbol('i')
>>> i = Symbol('p')
with the devious consequence that everything would still work but any
outputs would be utterly confusing because it would print 'p' wherever
the user expects 'i' and vice versa.

So... should I change the wiki to recommend sympy.abc over symbols()?

Regards,
Jo

[Francesco Bonazzi]

On Monday, March 30, 2015 at 11:26:17 AM UTC+2, Joachim Durchholz wrote:

Now in https://github.com/sympy/sympy/pull/9219 , I applied this advice to
 >>> from sympy.abc import t, w, x, y, z, n, k, m, p, i
and got
 >>> t, w, x, y, z, n, k, m, p, i = symbols('t w x y z n k m p i')
I think that's actually really dangerous, because it's easy to swap two
letters without noticing so you essentially get the equivalent of
 >>> p = Symbol('i')
 >>> i = Symbol('p')
with the devious consequence that everything would still work but any
outputs would be utterly confusing because it would print 'p' wherever
the user expects 'i' and vice versa.

What about a new code quality test checking for such mistakes? 

[Joachim Durchholz]

Am 30.03.2015 um 13:28 schrieb Francesco Bonazzi:
>
>> [with `symbols`,] it's easy to swap two

>> letters without noticing so you essentially get the equivalent of
>> >>> p = Symbol('i')
>> >>> i = Symbol('p')
>> with the devious consequence that everything would still work but any
>> outputs would be utterly confusing because it would print 'p' wherever
>> the user expects 'i' and vice versa.
>>
> What about a new code quality test checking for such mistakes?

Sounds like an excellent idea to me.
Solves the basic problem for code in SymPy, so this should definitely be
done (or put up as an Issue so it is not forgotten).

For user code, this does not help, unfortunately.

[Aaron Meurer]

If it's part of a doctest you will catch it immediately. If it's part
of a regular test and never actually uses the name of the symbol, then
I suppose it won't matter, although it will be confusing if someone is
investigating a failing test.

symbols() supports commas, so an easy thing to do here is to use the
form symbols(' t, w, x, y, z, n, k, m, p, i'), so that the left-hand
side of the assignment exactly matches the right-hand side.

Aaron Meurer

>
> So... should I change the wiki to recommend sympy.abc over symbols()?
>
> Regards,
> Jo
>

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/551916B5.8090808%40durchholz.org.
> For more options, visit https://groups.google.com/d/optout.

[Joachim Durchholz]

Am 30.03.2015 um 19:58 schrieb Aaron Meurer:
> symbols() supports commas, so an easy thing to do here is to use the
> form symbols(' t, w, x, y, z, n, k, m, p, i'), so that the left-hand
> side of the assignment exactly matches the right-hand side.

Ah, I knew it could do commas but I didn't know that it allows the
comma-plus-space combination as well.
That's going to make replacing `from sympy.abc import ...` with
`symbols('...')` easy enough (and I think I'll fix up any existing calls
as well, just to set the precedent).

Question is: Is this what we want?
grep reports 2200+ mentions of 'abc' in SymPy, that's an awful lot of
work, both for fixing and for review.

My personal main motive that I even started actually doing this has been
to get some more warnings out of the system - static analysis does not
know about the symbols that the exec calls in sympy.abc generates.
I could imagine fixing that by rewriting sympy.abc as

a = Symbol('a')
b = Symbol('b')
...
z = Symbol('z')
A = Symbol('A')
B = Symbol('B')
...
Z = Symbol('Z')
from sympy.printing.pretty.pretty_sombology import g, G
alpha = Symbol(g('alpha'))
beta = Symbol(g('beta'))
...
omega = Symbol(g('omega'))
ALPHA = Symbol(G('ALPHA'))
ΒETA = Symbol(G('BETA'))
...
OMEGA = Symbol(G('OMEGA'))

It's dull, it's dumb, but it gets the work done, and this needs to be
maintained so rarely that it shouldn't be a problem.
It should also make the import of sympy.abc fast (I guess importing
sympy.abc is slow because module initialization runs ~100 one-liners
through the Python compiler, though I haven't checked that it is indeed
as slow as `sympy/__init__.py` suggests).

However, that's just something that I came up with tonight, I'll be
happy to follow whatever course people see as best.

[Aaron Meurer]

I would definitely test the different alternatives for sympy.abc to
see what's faster.

Also, I wonder what the effect of sympy.abc on the cache is. We have
an LRU cache, and assumedly importing it messes with that, at least
for a little bit.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sympy/55199B7D.5030505%40durchholz.org.

[Joachim Durchholz]

Am 30.03.2015 um 21:33 schrieb Aaron Meurer:
> I would definitely test the different alternatives for sympy.abc to
> see what's faster.
>
> Also, I wonder what the effect of sympy.abc on the cache is. We have
> an LRU cache, and assumedly importing it messes with that, at least
> for a little bit.

Preliminary findings:

Once isympy is started up (which takes quite a while), doing
>>> from sympy import abc
takes negligible time, whether via exec_ or directly.

Doing
$ python bin/sympy_time.py
and
$ python bin/sympy_time_cache.py
both fail, though with different messages.
(I'm being called off and can't investigate more.)

[Joachim Durchholz]

Am 30.03.2015 um 21:33 schrieb Aaron Meurer:

> I would definitely test the different alternatives for sympy.abc to
> see what's faster.

I just ran bin/test with `a = Symbol('a')` etc.
Runtime went from 56:29 to 56:37, i.e. below the noise threshold.
That was with Python 2.7. I guess the Pyton compiler is "fast enough"
after all.
It's not too surprising actually: symbol creation happens just once at
first module import, for ~75 symbols, which isn't particularly often.

Other considerations
--------------------

Error situation:
Importing abc offers the trap that `import * from sympy.abc` overwrites
the COSINEQ single-letter definitions from other SymPy modules. It's a
trap that everybody walks into once, has trouble figuring out, then
knows how to avoid.
For ..., n, m, ... = symbols('... m, n, ...')" we have a trap that
everybody walks into, has trouble figuring out, then fixes easily and
knows how to avoid... essentially the same situation as with a star
import. Well, worse because the problem is harder to diagnose with more
varied failure mode, less bad because it bites only those who import a
lot of names from abc.

Static analysis:
Importing from sympy.abc will confuse all static analysis tools because
sympy.abc uses exec with a constructed string.
Using symbols() will not cause that problem.

Performance:
Using symbols() in all contexts might have performance ramifications,
creating new Symbol() objects means more memory pressure than reusing
precreated symbols from sympy.abc (which happen 521 times in SymPy
itself, hopefully just in test code).

Implementation effort:
Switching to symbols() wherever sympy.abc is used would mean touching
521 imports, and checking whether symbol reuse is actually okay for all
uses of the imported names.

Alternate implementation
------------------------
Implement sympy.abc using this:
--
a, b, c, d, e, f, g, h, i, j = symbols('a, b, c, d, e, f, g, h, i, j')
k, l, m, n, o, p, q, r, s, t = symbols('k, l, m, n, o, p, q, r, s, t')
u, v, w, x, y, z = symbols('u, v, w, x, y, z')
A, B, C, D, E, F, G, H, I, J = symbols('A, B, C, D, E, F, G, H, I, J')
K, L, M, N, O, P, Q, R, S, T = symbols('K, L, M, N, O, P, Q, R, S, T')
U, V, W, X, Y, Z = symbols('U, V, W, X, Y, Z')
alpha, beta, gamma = symbols('alpha, beta, gamma')
delta, epsilon, zeta = symbols('delta, epsilon, zeta')
eta, theta, iota = symbols('zeta, eta, theta, iota')
kappa, lamda, mu = symbols('kappa, lamda, mu')
nu, xi, omicron = symbols('nu, xi, omicron')
pi, rho, sigma = symbols('pi, rho')
tau, upsilon, phi = symbols('sigma, tau, upsilon, phi')
chi, psi, omega = symbols('phi, chi, psi, omega')
--
Error situation: As with importing.
Static analysis: As with symbols().
Performance: As with importing (minimally faster, perhaps).
Implementation effort: Very, very easy.

> Also, I wonder what the effect of sympy.abc on the cache is. We have
> an LRU cache, and assumedly importing it messes with that, at least
> for a little bit.

How can I test this?

[James Crist]

Performance:
Using symbols() in all contexts might have performance ramifications,
creating new Symbol() objects means more memory pressure than reusing
precreated symbols from sympy.abc (which happen 521 times in SymPy
itself, hopefully just in test code).

We cache symbol creation, so Symbol('a') is Symbol('a') is True. This means no additional memory is consumed.

In [1]: from sympy import *

In [2]: a, b, c = symbols('a, b, c')

In [3]: A, B, C = symbols('a, b, c')

In [4]: a is A
Out[4]: True

In [5]: id(a)
Out[5]: 139703386464976

In [6]: id(A)
Out[6]: 139703386464976

[Joachim Durchholz]

Am 02.04.2015 um 18:46 schrieb James Crist:
>
>>
>> Performance:
>> Using symbols() in all contexts might have performance ramifications,
>> creating new Symbol() objects means more memory pressure than reusing
>> precreated symbols from sympy.abc (which happen 521 times in SymPy
>> itself, hopefully just in test code).
>>
>
> We cache symbol creation, so Symbol('a') is Symbol('a') is True. This means
> no additional memory is consumed.

Ah, I wasn't aware of that.
What happens with assumptions - do these persist? (I'm a bit worried
about tests here.)

[James Crist]

Symbols are cached by name and assumptions, and like everything else in SymPy, are immutable. Thus:

In [1]: a = symbols('a')

In [2]: a2 = symbols('a', positive=True)

In [3]: a is a2

Out[3]: False

In [4]: hash(a) == hash(a2)

Out[4]: False

So no, you should have no problems with assumptions.

--
You received this message because you are subscribed to a topic in the Google Groups "sympy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sympy/dbOIFUeec3s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sympy+unsubscribe@googlegroups.com.

To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/551D7457.6000306%40durchholz.org.