How to prevent a specific substitution in cse?

73 views
Skip to first unread message

Jason Moore

unread,
Nov 8, 2026, 5:35:47 AM (12 days ago) Nov 8
to sympy
Hi,

I have an expression (matrix) which I apply cse to and, after cse, it looks like:

Matrix([
[                                       z0,                                         0,    -1,     0, -z0,   0, 0, 0, z10*(xi - xp)],
[                                        0,                                        z0,     0,    -1,   0, -z0, 0, 0, z10*(yi - yp)],
[(-2*z1*z2**2 - 2.5*z2)*Derivative(z1, xi), (-2*z1**2*z2 - 2.5*z1)*Derivative(z2, yi), 2*uxi, 2*uyi,   0,   0, 0, 0,             0]])


With the replacements being:

[(z0, 1/h), (z1, wxixi), (z2, wyiyi), (z10, -1/h**2)]

I want to prevent cse from ever replacing the arguments of Derivative(), so that Derivative(wxixi, xi) should never be transformed into z1 = wxixi and Derivative(z1, xi).

There is an ignore keyword argument in cse that may do this, but I have not succeeded in using that kwarg.

Is there a simple way to force cse to ignore any Derivative() objects?

And, secondarily, I am surprised that single variable substitutions are made, e.g. z1 = wixi. Shouldn't only sub-expressions be generated? Subbing a single variable shouldn't be necessary. I notice that a single variable substitution doesn't seem to occur for a simple example:

In [7]: a, b = sm.symbols('a, b')

In [8]: expr = sm.Derivative(a, b)*sm.Derivative(a, b) + sm.Derivative(a, b)

In [9]: expr
Out[9]: Derivative(a, b)**2 + Derivative(a, b)

In [10]: sm.cse(expr)
Out[10]: ([(x0, Derivative(a, b))], [x0**2 + x0])

In [11]: mexpr = sm.Matrix([sm.Derivative(a, b), expr])

In [12]: sm.cse(mexpr)
Out[12]:
([(x0, Derivative(a, b))],
 [Matrix([
  [        x0],
  [x0**2 + x0]])])



Thanks for any help,

Jason
moorepants.info

Chris Smith

unread,
Nov 8, 2026, 9:33:46 PM (11 days ago) Nov 8
to sympy
I don't reproduce your example when running cse on the reconstructed Matrix. Could you paste of reproducing snippet?

```
>>> r

[(z0, 1/h), (z1, wxixi), (z2, wyiyi), (z10, -1/h**2)]
>>> str(m)
'Matrix([[z0, 0, -1, 0, -z0, 0, 0, 0, z10*(xi - xp)], [0, z0, 0, -1, 0, -z0, 0, 0, z10*(yi - yp)], [(-2*z1*z2**2 - 2.5*z2)*Derivative(z1, xi), (-2*z1**2*z2 - 2.5*z1)*Derivative(z2, yi), 2*uxi, 2*uyi, 0, 0, 0, 0, 0]])'
>>> cse(m.xreplace(dict(r)))
([(x0, 1/h), (x1, -x0), (x2, h**(-2))], [Matrix([
[                                                   x0,                                                     0,    -1,     0, x1,  0, 0, 0, -x2*(xi - xp)],
[                                                    0,                                                    x0,     0,    -1,  0, x1, 0, 0, -x2*(yi - yp)],
[(-2*wxixi*wyiyi**2 - 2.5*wyiyi)*Derivative(wxixi, xi), (-2*wxixi**2*wyiyi - 2.5*wxixi)*Derivative(wyiyi, yi), 2*uxi, 2*uyi,  0,  0, 0, 0,             0]])])
```
As to how a symbol can show up as a replacement: it can happen if a non-symbol collapses to a symbol when evaluating. I am looking at a fix to keep that from inadvertently be happening with an UnevaluatedExpr.

Until I see the reproducer I would just suggest a masking operation on your original matrix M:

from sympy.solvers.solveset import _masked
M, restore = _masked(M, Derivative)  # masked handles nested Derivatives
reps, [M] = cse(M)

for d, e in restore:
    reps = [(s, rhs.xreplace({d: e})) for s, rhs in reps]
    M = M.xreplace({d: e})


The `ignore` tells what variables (symbols) should not appear in any identified patterns.

>>> eq=exp(y) + Derivative(f(x) + Derivative(f(x), x), x) + 1/(exp(y) + 1)
>>> cse(eq)
([(x0, exp(y)), (x1, f(x))], [x0 + Derivative(x1 + Derivative(x1, x), x) + 1/(x0 + 1)])
>>> cse(eq,ignore=[x])
([(x0, exp(y))], [x0 + Derivative(f(x) + Derivative(f(x), x), x) + 1/(x0 + 1)])
>>> cse(eq,ignore=[y])
([(x0, f(x))], [exp(y) + Derivative(x0 + Derivative(x0, x), x) + 1/(exp(y) + 1)])

Why don't you want Derivatives in the cse? Is it because you then have to special case replacement of them before doing doit? That could be solved by wrapping Derivatives in Subs as in Derivative(f(x), x) -> Subs(Derivative(x0, x), x0, f(x))

>>> Subs(Derivative(x0, x), x0, f(x))
Subs(Derivative(x0, x), x0, f(x))
>>> _.doit()
Derivative(f(x), x)

/c

Chris Smith

unread,
Nov 9, 2026, 12:04:49 AM (11 days ago) Nov 9
to sympy
In  cse Derivative restoration by smichr · Pull Request #31002 · sympy/sympy you can now pass the remove_cse_from_derivative post-processor to the cse call to preserve Derivatives.

/c

Chris Smith

unread,
Nov 9, 2026, 12:22:01 AM (11 days ago) Nov 9
to sympy
If you want to maintain a non-collapsing symbolic Derivative, see  cse function postprocess by smichr · Pull Request #31003 · sympy/sympy

/c

Jason Moore

unread,
Nov 9, 2026, 2:04:04 AM (11 days ago) Nov 9
to sy...@googlegroups.com
Thanks Chris!

I'll send a reproducer. I have to extract it.

Jason


--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sympy/b21b471e-14c4-4190-876d-fa7cc13f963cn%40googlegroups.com.

Jason Moore

unread,
Nov 9, 2026, 2:31:43 AM (11 days ago) Nov 9
to sy...@googlegroups.com
Dear Chris,

When trying to extract a reproducer, I realized that we have a custom wrapper around cse (for doing forward Jacobians) and that wrapper is what is causing the undesired subexpressions.


It uses cse, but I'll have to dig more on the bug. I'll try your fix within the wrapper to see if that is useful.

Jason

Reply all
Reply to author
Forward
0 new messages