Feedback requested on new deprecations policy

[Aaron Meurer]

We are currently in the process of revamping our deprecations policy
at https://github.com/sympy/sympy/pull/22900, and we would like any
feedback from the community, both users and developers. Our current
policy that this would replace is on the wiki
https://github.com/sympy/sympy/wiki/Deprecating-policy.

In particular, the new policy would have

1. An official period of at least 1 year for all deprecations to last.
Previously there was no official period for deprecations, and
deprecated functionality was removed more or less whenever we felt
like it.

2. Deprecations will still raise a SymPyDeprecationWarning, but the
proposed policy is to make the warnings contain much more verbose and
helpful messages. In addition, all warnings will be documented in the
respective docstrings, listed in a separate "all active deprecations"
document in the docs, and listed in the release notes for each release
(currently only the last of these is done).

3. The current silly "deprecation removal issue" thing that we (tried)
to do will be removed. All documentation for deprecations will be in
the documentation.

4. I have added some text to the document going over when backwards
compatibility breaks should be made (tl;dr: sparingly), and what does
and doesn't require deprecation to change. The document also has
developer instructions on how to add a deprecation to the code.

One question we have for users is if you are happy with our current
SymPyDeprecationWarnings, which are relatively loud by default (they
use a warnings filter that always turns them on). Would you prefer
more silent warnings, like warnings that are documented but aren't
accompanied by a warning printed to the screen?

Aaron Meurer

[Peter Stahlecker]

Hi Aaron,

As a user of mostly symy physics mechanics I definitely prefer the 'loud' warnings.

NB: this is a GREAT package!

Thanks, Peter

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6JkAa0jbEDKPHy7p6yVpjB8pBomcinV%3Dp-UdaBD_hiQhw%40mail.gmail.com.

--

Best regards,

Peter Stahlecker

[Chris Smith]

A concern I have about the policy is that "positional argument names are not Public API". I hope this is not correct, because if a function `f(a, b)` is changed to `f(b, a)` this would start failing without warning. I'm not even sure that `f(a, b, c=1, d=2)` should be allowed to be changed to swap the order of `c` and `d` since the API allows all 4 to be given and does not require the defaults to be given. So if `f(1, 2, 3, 4)` started returning (silently) the result for `f(1, 2, 4, 3)`, that would be a problem.

/c

[Aaron Meurer]

On Fri, Feb 18, 2022 at 7:39 PM Chris Smith <smi...@gmail.com> wrote:

A concern I have about the policy is that "positional argument names are not Public API". I hope this is not correct, because if a function `f(a, b)` is changed to `f(b, a)` this would start failing without warning.

The names are different from the order. The name of a positional argument doesn't matter but the order obviously does.

I'm not even sure that `f(a, b, c=1, d=2)` should be allowed to be changed to swap the order of `c` and `d` since the API allows all 4 to be given and does not require the defaults to be given. So if `f(1, 2, 3, 4)` started returning (silently) the result for `f(1, 2, 4, 3)`, that would be a problem.

It depends on the specific keyword argument. In some cases it's not really something you should ever use without using the keyword name. 

It would be good to be more explicit about this by using keyword-only arguments. If you write 

def f(a, b, *, c=1, d=2):

    ...

Then f(1, 2, 3, 4) is not allowed. You have to write f(1, 2, a=3, b=4). You can also do a similar thing with / to make things positional only starting in Python 3.8. Keyword only OTOH has been available ever since we dropped Python 2.

I think we should use keyword only arguments a lot more, because it makes the signature future proof against adding more keyword arguments. Making more things positional only could be good too, although I rarely see people unnecessarily calling a positional argument as a keyword argument. 

I started to do this in https://github.com/sympy/sympy/pull/21803 but some felt that changing existing signatures was itself a compatibility break (I disagree, but even if we decide this, it should be possible to make the change with a custom decorator that just deprecates the old way). Either way it's a huge undertaking to fix all the signatures in SymPy, but worth it IMO. 

Aaron Meurer 

--

You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/bc9ff2d6-540f-4942-9e9c-e79774d20088n%40googlegroups.com.

[S.Y. Lee]

However, in the future, we may need have a complete review guideline even for setting up the parameter names,
or to check if the bad naming choices are not marked as positional-only arguments.
Although I believe that there are a lot of immature sympy commits before,
but this is quite difficult to change the old code for this, so I'd leave them alone.

[Jason Moore]

1 year seems too short from my perspective. As a downstream package maintainer, I release many packages less often than SymPy makes releases and keeping up with deprecations disappearing effectively each SymPy release would be stressful. I would advocate for at least 2 years, which equates to at least a couple of SymPy releases. Or even stating something like "deprecations can be removed if they've existed both for two years and two SymPy releases".

I also think the current method of leaving the deprecations in indefinitely is fine and preferable. I don't think having a policy written in a way that encourages people to constantly look for deprecations that are just over the minimum time period to immediately remove them. They should really only be removed when they both reach the minimum AND are causing some issue with new code refactoring or designs.

Jason

moorepants.info
+01 530-601-9791

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6JkAa0jbEDKPHy7p6yVpjB8pBomcinV%3Dp-UdaBD_hiQhw%40mail.gmail.com.

[Jason Moore]

I now see that this is a thread from a year ago and that the PR is already merged. S.Y. Lee's comment made me think this was a new thread. I probably wouldn't have written this if I new this was old.

Jason

moorepants.info
+01 530-601-9791

[Oscar Benjamin]

On Sun, 15 Jan 2023 at 12:26, Jason Moore <moore...@gmail.com> wrote:
>
> 1 year seems too short from my perspective. As a downstream package maintainer, I release many packages less often than SymPy makes releases and keeping up with deprecations disappearing effectively each SymPy release would be stressful. I would advocate for at least 2 years, which equates to at least a couple of SymPy releases. Or even stating something like "deprecations can be removed if they've existed both for two years and two SymPy releases".

To be clear the policy says "at least one year" i.e. that should be
considered a minimum. I don't think that we should routinely remove
deprecated features after only 1 year. Maybe 3 years is a better time
frame for that. In some cases we should do it sooner but there should
be some reason for doing it more quickly.

> I also think the current method of leaving the deprecations in indefinitely is fine and preferable. I don't think having a policy written in a way that encourages people to constantly look for deprecations that are just over the minimum time period to immediately remove them. They should really only be removed when they both reach the minimum AND are causing some issue with new code refactoring or designs.

The point of deprecating things is so that they can eventually be
removed or changed. Perhaps fewer things should be deprecated but
saying that deprecated things should be left indefinitely defeats the
purpose.

--
Oscar

[Peter Stahlecker]

Just to increase my understanding: why would one deprecate 'things' which work?

Is this related to python's development?

Thanks!

( I am nearly 68, and learned about python, its great libraries and open source in general only during Corona, when I had very little work)

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxRysLdxK%2BToPTP5%2Bn%2BesMTm50Qbd0Ufp-SUQSeRNC6QJw%40mail.gmail.com.

[Oscar Benjamin]

On Sun, 15 Jan 2023 at 16:12, Peter Stahlecker
<peter.st...@gmail.com> wrote:
>
> Just to increase my understanding: why would one deprecate 'things' which work?
> Is this related to python's development?

Sometimes things sort of work in some cases but don't really work
properly and it isn't possible to fix them without breaking someone's
code.

Here is an example:

In [9]: t, s = symbols('t, s')

In [10]: L = LaplaceTransform(exp(t), t, s)

In [11]: print(L)
LaplaceTransform(exp(t), t, s)

In [12]: print(L.doit())
(1/(s - 1), 1, True)

Here L is an unevaluated LaplaceTransform object representing the
Laplace transform of exp(t). The LaplaceTransform.doit() method
attempts to return a tuple of 3 things representing not just the
transformed function but its region of convergence and any other
convergence conditions. That sort of makes sense until you realise
that the whole point of an unevaluated LaplaceTransform object is that
you can use it in expressions e.g.:

In [13]: print(2*L + 1)
2*LaplaceTransform(exp(t), t, s) + 1

In [14]: print((2*L + 1).doit())
~/current/sympy.git/sympy/core/operations.py:458: SymPyDeprecationWarning:

Using non-Expr arguments in Mul is deprecated (in this case, one of
the arguments has type 'Tuple').

If you really did intend to use a multiplication or addition operation with
this object, use the * or + operator instead.

See https://docs.sympy.org/latest/explanation/active-deprecations.html#non-expr-args-deprecated
for details.

This has been deprecated since SymPy version 1.7. It
will be removed in a future version of SymPy.
...
[snip longer error message]
...
AttributeError: 'Tuple' object has no attribute 'as_coeff_Mul'

There are several problems here that demonstrate different points. The
fact that LaplaceTransform.doit turns an Expr until a Tuple means that
it will basically always do the wrong thing when used as part of an
expression. On the other hand anyone who might be using
LaplaceTransform.doit will have written their code to expect this
tuple so if we change it to return an expression instead of a tuple
then it will break their code.

So it is difficult to say precisely whether LaplaceTransform.doit is
"working". It is probably working for someone and they are using it to
do something useful. However it can't work in other situations and it
is clearly broken by design and needs to be changed. It isn't possible
to fix it without causing someone's code to break though.

Sometimes in this situation it would be possible to add a warning that
code will break in future so someone who is using
LaplaceTransform.doit() would see that warning. If we let that warning
get printed out for a few years then at that point we could change
SymPy's behaviour in a new release and hopefully people will have seen
the warning and already fixed their code. This is what it means to
"deprecate" something.

What this example also shows though is that in this case there is no
reasonable way for us to deprecate this and add a warning because we
can't make some alternative to the doit method which is a fundamental
part of SymPy. The only option is to make the change directly and that
will mean that some code that some other people have written might
stop working as a result. For some of those people it might seem as if
the current behaviour is "working" because the problems with the
current behaviour might not affect them.

Also what this example shows is one of the costs of leaving deprecated
functionality around which is that the call to doit should really
raise an immediate TypeError because a Tuple is not something that can
be multiplied. It does not though because the code that should raise
an error only gives a warning "Using non-Expr arguments in Mul is
deprecated". You can see why *that* is deprecated: the Mul code
expects Expr and expects its args to have methods like as_coeff_Mul so
allowing non-Expr here will lead to obscure failures. Following the
warning we have a confusing error message about a missing attribute
though when the correct one would be "Tuple cannot be in a Mul". We
don't currently give that error message because for now putting a
Tuple in a Mul is only "deprecated" rather than disallowed.

--
Oscar

[Aaron Meurer]

Just for context, here is the current policy https://docs.sympy.org/latest/contributing/deprecations.html.

I agree that 1 year is just a minimum. If something isn't really bothering anyone, it can be left longer. Sometimes you can't progress on something until a deprecation is removed. See the full text of the length policy https://docs.sympy.org/latest/contributing/deprecations.html#how-long-should-deprecations-last

Aaron Meurer

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxRysLdxK%2BToPTP5%2Bn%2BesMTm50Qbd0Ufp-SUQSeRNC6QJw%40mail.gmail.com.

[Peter Stahlecker]

Thank you VERY much for this clear and detailed explanation!

This really helped me to understand the issue of deprecation.

Peter

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxQ2iobyp89Adm6g2KDBmcmapZDUqw_GVC6aq7HQz_nzTQ%40mail.gmail.com.