Why does subs() not work as expected ?

[Bruce Allen]

I have a very basic sympy question, which has me stumped, and am hoping
that someone here can set me straight. I have an expression for which
subs() seems to have no effect:

>>> a=Symbol('a', real=True, positive=True)

>>> q=u[7]

>>> q
-a*(45*a**18 - 120*a**16 + 240*a**12 - 504*a**8 + 1440*a**4 - 2700*a**2
- 4645)/4050

>>> q.subs(a, Rational(1,2))
-a*(45*a**18 - 120*a**16 + 240*a**12 - 504*a**8 + 1440*a**4 - 2700*a**2
- 4645)/4050

>>> srepr(q)
"Mul(Integer(-1), Rational(1, 4050), Symbol('a', real=True,
positive=True), Add(Mul(Integer(45), Pow(Symbol('a', real=True,
positive=True), Integer(18))), Mul(Integer(-1), Integer(120),
Pow(Symbol('a', real=True, positive=True), Integer(16))),
Mul(Integer(240), Pow(Symbol('a', real=True, positive=True),
Integer(12))), Mul(Integer(-1), Integer(504), Pow(Symbol('a', real=True,
positive=True), Integer(8))), Mul(Integer(1440), Pow(Symbol('a',
real=True, positive=True), Integer(4))), Mul(Integer(-1), Integer(2700),
Pow(Symbol('a', real=True, positive=True), Integer(2))), Integer(-4645)))"

But if I just cut and paste the same expression into the variable, then
subs() works as expected:

>>> q=-a*(45*a**18 - 120*a**16 + 240*a**12 - 504*a**8 + 1440*a**4 -
2700*a**2 - 4645)/4050

>>> q.subs(a, Rational(1,2))
1371514291/2123366400

The internal representation is identical:

>>> srepr(q)
"Mul(Integer(-1), Rational(1, 4050), Symbol('a', real=True,
positive=True), Add(Mul(Integer(45), Pow(Symbol('a', real=True,
positive=True), Integer(18))), Mul(Integer(-1), Integer(120),
Pow(Symbol('a', real=True, positive=True), Integer(16))),
Mul(Integer(240), Pow(Symbol('a', real=True, positive=True),
Integer(12))), Mul(Integer(-1), Integer(504), Pow(Symbol('a', real=True,
positive=True), Integer(8))), Mul(Integer(1440), Pow(Symbol('a',
real=True, positive=True), Integer(4))), Mul(Integer(-1), Integer(2700),
Pow(Symbol('a', real=True, positive=True), Integer(2))), Integer(-4645)))"

Why does subs() work in one case, and not in the other?

Cheers,
Bruce

[Oscar Benjamin]

I'm not sure. I can't reproduce this without u though. I tried making
u=(the srepr output) but then the substitution worked fine.


Oscar

[David Bailey]

Oscar,

That question made me wonder if

Symbol('a', real=True, positive=True) and Symbol('a')

Are the same objects. Logically I imagine they can't be - either one symbol is mutated or they remain distinct.

David

[Oscar Benjamin]

> That question made me wonder if
>
> Symbol('a', real=True, positive=True) and Symbol('a')
>
> Are the same objects. Logically I imagine they can't be - either one symbol is mutated or they remain distinct.

They are distinct:

In [25]: Symbol('a', positive=True) == Symbol('a', positive=True)
Out[25]: True

In [26]: Symbol('a', positive=True) == Symbol('a')
Out[26]: False

Symbols are immutable so they can not be mutated. Symbols created with
the same name and assumptions are considered equivalent. What is less
obvious is that this also applies where the assumptions can be
inferred as equivalent. For example there is no need to specify
real=True at the same time as positive=True because positive implies
real:

In [27]: Symbol('a', positive=True) == Symbol('a', positive=True, real=True)
Out[27]: True

In Bruce's example both symbols seem to have the same assumptions so
this shouldn't be the issue.


Oscar

[David Bailey]

On 07/04/2021 12:28, Oscar Benjamin wrote:
>
> In Bruce's example both symbols seem to have the same assumptions so
> this shouldn't be the issue.
>
>
> Oscar

I just checked and you can make two non-identical symbols that way:

Symbol('a')+Symbol('a',Positive=True)
a + a

I almost wonder if that should not flag an error.

David

[Bruce Allen]

David, Oscar,

Thank you for your help.

Oscar, the list 'u' was created in the course of a calculation, and
saved as a .pkl file. I then reloaded it and want to manipulate the
saved equations, of which u[7] is an example. I found an even cleaner
example, see below.

I have the impression that when the .pkl file was reloaded, the identity
of the object

Symbol('a', real=True, positive=True)

contained within u[3] was set to a memory address and when I created the
object

Symbol('a', real=True, positive=True)

in the current session, that was left pointing to a *different* address.
(See below.)

I can use "id(a)" to see what address the "current session variable a"
is pointing to. Is there a simple way for me to understand what address
a=Symbol('a',real=True,positive=True) in the u[3] object is pointing to?
Presumably these are different, even though the two variables have the
same name and the same properties.

If that behavior is correct, I may need some help to revise my mental
model of how sympy/python works (:-).

Cheers,
Bruce


>>> u[3]
2*a

>>> srepr(u[3])
"Mul(Integer(2), Symbol('a', real=True, positive=True))"

>>> x=2*a

>>> srepr(x)

"Mul(Integer(2), Symbol('a', real=True, positive=True))"

>>> x==u[3]
False

[Oscar Benjamin]

On Wed, 7 Apr 2021 at 14:43, 'Bruce Allen' via sympy
<sy...@googlegroups.com> wrote:
>
> David, Oscar,
>
> Thank you for your help.
>
> Oscar, the list 'u' was created in the course of a calculation, and
> saved as a .pkl file. I then reloaded it and want to manipulate the
> saved equations, of which u[7] is an example. I found an even cleaner
> example, see below.
>
> I have the impression that when the .pkl file was reloaded, the identity
> of the object
>
> Symbol('a', real=True, positive=True)
>
> contained within u[3] was set to a memory address and when I created the
> object
>
> Symbol('a', real=True, positive=True)
>
> in the current session, that was left pointing to a *different* address.
> (See below.)
>
> I can use "id(a)" to see what address the "current session variable a"
> is pointing to. Is there a simple way for me to understand what address
> a=Symbol('a',real=True,positive=True) in the u[3] object is pointing to?
> Presumably these are different, even though the two variables have the
> same name and the same properties.
>
> If that behavior is correct, I may need some help to revise my mental
> model of how sympy/python works (:-).

I don't think you need to adjust your mental model :)

I think this is just a bug to do with pickling symbols that have assumptions:
https://github.com/sympy/sympy/issues/21121

Oscar

[Bruce Allen]

Dear Oscar,

Thank you.

On the upside, this shows that sympy has a robust user base -- the bug
was picked up just 20 days ago and here I am stumbling on it independently.

On the downside, this makes it hard to work on a calculation which
requires large numbers of CPU hours and needs to be checkpointed and saved.

Do you know of any workarounds? Or is there an alternative to
pickle.save() and pickle.load() for this, which works well with sympy?
The only reason I am using pickle is because it is what popped up when I
googled 'save python data'.

Cheers,
Bruce

[Oscar Benjamin]

On Wed, 7 Apr 2021 at 15:27, 'Bruce Allen' via sympy
<sy...@googlegroups.com> wrote:
>
> On the downside, this makes it hard to work on a calculation which
> requires large numbers of CPU hours and needs to be checkpointed and saved.
>
> Do you know of any workarounds? Or is there an alternative to
> pickle.save() and pickle.load() for this, which works well with sympy?
> The only reason I am using pickle is because it is what popped up when I
> googled 'save python data'.

The simplest workaround is to use `srepr` and `eval`.


Oscar

[Chris Smith]

Assuming symbols with the same names but different assumptions are not being used, couldn't one just "refresh" the unpickled expression by pairing the pickled symbols with current session variables by name? Does this work?

unpickle = a # an expression that was unpickled

reps= {}

for i in unpickle.atoms(Symbol):

    reps[i] = var(str(i), **i.assumptions0)

refresh = unpickle.xreplace(reps)

[David Bailey]

On 07/04/2021 15:27, 'Bruce Allen' via sympy wrote:
> Dear Oscar,
>
> Thank you.
>
> On the upside, this shows that sympy has a robust user base -- the bug
> was picked up just 20 days ago and here I am stumbling on it
> independently.
>
> On the downside, this makes it hard to work on a calculation which
> requires large numbers of CPU hours and needs to be checkpointed and
> saved.
>
> Do you know of any workarounds? Or is there an alternative to
> pickle.save() and pickle.load() for this, which works well with sympy?
> The only reason I am using pickle is because it is what popped up when
> I googled 'save python data'.

I suppose I would just let it run, and use hibernate rather than
shutdown to switch off - then when you restart the machine the process
will continue where it left off. That certainly works well under Windows
(for calculations of any sort, partially watched videos, etc).

I have the feeling that adding extra steps like pickling is an
invitation for something to go wrong!

David

[Oscar Benjamin]

I have made what I think is a fix for this bug here:
https://github.com/sympy/sympy/pull/21260#issuecomment-815221036

If you know how to check out that PR and can test it then that would be helpful.

Otherwise if anyone can review the PR that would be good. I'm hoping
to put out a new release ASAP and that could go in.


Oscar

[Aaron Meurer]

This seems like a bug, but without full steps to reproduce it, there's
not much we can do. If the srepr() output is the same, it should work
the same. If you are able to reproduce this consistently, that would
be helpful. Typically this sort of thing is caused by a being
actually a different symbol with different assumptions, but based on
the srepr output, it should be the same. It seems like some odd
situation where something got messed up in the internal SymPy state,
and my guess is this will not be easy to reproduce (i.e., you won't
see it again).

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/4b8b8ea4-bd9f-d155-41f9-80fcbd2d5a38%40googlemail.com.

[Aaron Meurer]

I did not see that this thread was continued under a different
subject. Yes, unfortunately, there are currently a lot of bugs related
to pickling and unpickling SymPy expressions. We need to make a
concerted effort to iron them all out. I would also suggest using
cloudpickle, as it works better than pickle in some cases. Although in
general, you may have to avoid pickle with SymPy at the present moment
if you run into bugs like this.

Aaron Meurer

On Wed, Apr 7, 2021 at 4:33 PM Oscar Benjamin

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxRxSrqRaEMUTNNEsraNbxANeUHxpWFuCWdsRQP6jNu3uw%40mail.gmail.com.