Drop support for Theano and support Aesara instead

[Oscar Benjamin]

Hi all,

Does anyone use SymPy with Theano?

There is a PR which proposes to drop Theano support in favour of Aesara:
https://github.com/sympy/sympy/pull/21087

The PR looks fine to me but I don't actually know what Theano/Aesara
are. Apparently Theano is no longer actively maintained and Aesara is
a fork that should be considered a natural successor.

If anyone has any thoughts on this then please say so (here or on the
PR). Otherwise I'm inclined to merge the PR because the author clearly
knows more than I do about this and says that this is a reasonable
approach.

Oscar

[Aaron Meurer]

I would also like more context on what the status of Theano and Aesara
is. The pull request as it is breaks compatibility by renaming the
function and making it no longer work with theano. I think it would be
better to add a new function, aesara_code, and keep the old
theano_code() intact. I'm not clear if it should also be deprecated.
Theano is currently installed in the tests, meaning it presumably
still works. Do we know at what point it will stop working?

Perhaps someone can forward this thread to one of the Aesara
developers so they can comment here.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxTCV3U2YxE5%3D3-us5RbtkfF8ZaqmvKA-8oM9QkPdXJUxg%40mail.gmail.com.

[Brandon T. Willard]

I'm the lead developer/manager of the Aesara project and a core developer in the PyMC group.

Theano has officially stopped development and currently exists in an effectively frozen state.  As stated in Theano's README, its successor is our fork, Aesara.

If there are users who still rely on legacy Theano and SymPy's Theano features, then it would make sense to maintain the existing Theano code.  I'm not personally aware of any projects that do, so I can't say, but I do know that Aesara has considerably more potential for developing SymPy interactions (e.g. see some preliminary symbolic work here) in the nearterm.

[Aaron Meurer]

As long as Theano itself still works in versions of Python that SymPy
supports, it would be better to deprecate the function rather than
just remove it. The function theano_code is public API so just
removing it would be a backwards compatibility break, which we try to
avoid without deprecations. See
https://github.com/sympy/sympy/wiki/Deprecating-policy

As to whether it should be deprecated or just left alone, I guess it
depends. To me it would make sense to deprecate it if theano truly is
a dead project.

Is Aesara a drop-in replacement for Theano? Why was a fork done,
instead of just continuing development on Theano itself?

The symbolic work on Aesara looks interesting. To be clear, I am not
at all opposed to supporting Aesara in SymPy. My only concern is with
the backwards compatibility break from removing support for Theano.

As an aside, one thing that has always annoyed me with Theano,
especially when SymPy imports it for its tests, is that Theano does a
lot of things at import time, such as running compilers to see if they
work and printing many warnings (see
https://github.com/sympy/sympy/pull/14747). I think it would be better
to do these things at runtime, e.g., after the first function is
called. I hope Aesara can improve this situation over Theano.

Aaron Meurer

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/d079659c-e249-4d22-a231-8caa2a101716n%40googlegroups.com.

[Oscar Benjamin]

On Mon, 15 Mar 2021 at 22:01, Aaron Meurer <asme...@gmail.com> wrote:
>
> As long as Theano itself still works in versions of Python that SymPy
> supports, it would be better to deprecate the function rather than
> just remove it. The function theano_code is public API so just
> removing it would be a backwards compatibility break, which we try to
> avoid without deprecations. See
> https://github.com/sympy/sympy/wiki/Deprecating-policy

I think that if Theano itself is now unmaintained legacy software then
it's fine to say that if someone wants to use it with sympy they
should use an older version of sympy.

> As an aside, one thing that has always annoyed me with Theano,
> especially when SymPy imports it for its tests, is that Theano does a
> lot of things at import time, such as running compilers to see if they
> work and printing many warnings (see
> https://github.com/sympy/sympy/pull/14747). I think it would be better
> to do these things at runtime, e.g., after the first function is
> called. I hope Aesara can improve this situation over Theano.

This shows precisely that supporting something like Theano has a
nonzero cost even if that support is "deprecated". When migrating our
CI I had to work through many different projects that I do not know in
order to keep the test suite ticking over. Each of those makes general
project maintenance harder. I'm much happier about supporting a
maintained project that has a future and has maintainers willing to
help us provide support.

I'm interested to hear from anyone using SymPy with Theano though (if
anyone on this particular mailing list is).


Oscar

[Aaron Meurer]

These are fair points. It sounds like we should definitely not leave
it as is. We should either deprecate it or remove it.

The existing pull request only does a name change, suggesting there
aren't very many differences between theano and aesara yet. So a
deprecation wouldn't be that hard. If that changes in the future, it
may be easier to just remove theanocode rather than trying to keep it
around.

>
> I'm interested to hear from anyone using SymPy with Theano though (if
> anyone on this particular mailing list is).

Same here. I also am interested in more background on the fork. Theano
itself was supposed to be "dead" a few years ago when the original
developers stopped working on it, but then the pymc developers picked
up maintenance. Could the same thing happen again?

Also, I had heard that pymc was dropping theano in favor of tensorflow
(do I remember this correctly?). Is this still planned, or has that
changed to be aesara now?

Aaron Meurer

>
>
> Oscar
>
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxSoee41vDUk1oXzY-n-jgFi18z3Q15Beu6AP%2B5hOx-Thw%40mail.gmail.com.

[Brandon T. Willard]

The package layout and internal functions, objects, etc., have changed enough to warrant a fork, and our planned changes will only make it diverge even more.  We had an intermediate form of the fork named Theano-PyMC; that project attempted to preserve the Theano package name, but after a lot of old Theano bug fixes, additional features, and important refactoring, a name change was justified.

The load times are one reason for making these internal changes, but the refactoring we've done up to now is just enough to start make the _real_ refactoring possible: e.g. large-scale changes to the C compilation backend and the configuration initialization process.  Those changes will really fix the noticeable import delays.

Otherwise, at the user-level, Aesara can be very close to a drop-in replacement for Theano, aside from the superficial name change.  The largest non-name change at the user-level is the lack of internal object exposure at package/sub-package scope (e.g. no more `[theano|aesara].*` for nearly everything in the library).  This change was necessary for our refactoring efforts, since there were far too many unnecessary cross sub-package dependencies that made it very difficult to work with.

Anyway, the established deprecation policy is enough to justify keeping the existing code and adding Aesara support as a new module.

[Aaron Meurer]

I guess I'm unclear then. If no user level changes are being made,
then why was a fork needed? Couldn't this refactor just be done on
theano itself? When I saw there was a fork, I assumed it was because
you intended to make significant changes to the library that would
break the API with theano anyway.

Aaron Meurer

>
> Anyway, the established deprecation policy is enough to justify keeping the existing code and adding Aesara support as a new module.
>

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/95e13769-c372-46d1-bd70-bc58ef080221n%40googlegroups.com.

[Brandon T. Willard]

Same here. I also am interested in more background on the fork. Theano
itself was supposed to be "dead" a few years ago when the original
developers stopped working on it, but then the pymc developers picked
up maintenance.

The PyMC group are also maintainers of the original Theano repo, but, per request of the original maintainers, we are not making changes to the Theano repo. The Theano repo really should be frozen, but I'm assuming that we're leaving that up to the original maintainers.

Could the same thing happen again?

Could development stop on Aesara as it did with Theano? If that's the question you're asking, the answer is that it's always possible, but unlikely to occur any time soon. There's still a real place for a primarily Python-driven tensor library with a (relatively) accessible codebase. Also, for PyMC's purposes and future, we need to work more and more at the tensor library level, so it's important that the development requirements not skyrocket, and that the underlying library not make detrimental design and/or implementation changes.

Also, I had heard that pymc was dropping theano in favor of tensorflow
(do I remember this correctly?). Is this still planned, or has that
changed to be aesara now?

Since this announcement, we've dropped all plans to implement the next version of PyMC in TensorFlow. The reason: it's possible for us to have the kind of tensor library described above and use other backends, when desired. I wrote a quick example of how we can convert Theano graphs to JAX "graphs", and that was enough to prove the point.

My original plan was to use SymPy for all this, but it would've resulted in a distinct project anyway, because doing all that work via changes to SymPy would've most likely taken too long.

[Brandon T. Willard]

I guess I'm unclear then. If no user level changes are being made,
then why was a fork needed? Couldn't this refactor just be done on
theano itself? When I saw there was a fork, I assumed it was because
you intended to make significant changes to the library that would
break the API with theano anyway.

There actually are some important user-level changes, but, right now, not enough to make the transition from Theano to Aesara difficult. Otherwise, custom classes and other less user-level interactions with Aesara cannot be ported as directly.

More importantly, we have some very large changes underway that will only increase the divide between Aesara and Theano, so we found it better to make the change sooner than later.

[Aaron Meurer]

On Mon, Mar 15, 2021 at 5:07 PM Brandon T. Willard
<brandon...@gmail.com> wrote:
>
> Same here. I also am interested in more background on the fork. Theano
> itself was supposed to be "dead" a few years ago when the original
> developers stopped working on it, but then the pymc developers picked
> up maintenance.
>
> The PyMC group are also maintainers of the original Theano repo, but, per request of the original maintainers, we are not making changes to the Theano repo. The Theano repo really should be frozen, but I'm assuming that we're leaving that up to the original maintainers.

So I guess I was misinformed. I had understood that PyMC was taking
over development of theano. But if you can't actually make any changes
or releases for it, then you can't actually do that, and have no
choice but to fork.

>
> Could the same thing happen again?
>
> Could development stop on Aesara as it did with Theano? If that's the question you're asking, the answer is that it's always possible, but unlikely to occur any time soon. There's still a real place for a primarily Python-driven tensor library with a (relatively) accessible codebase. Also, for PyMC's purposes and future, we need to work more and more at the tensor library level, so it's important that the development requirements not skyrocket, and that the underlying library not make detrimental design and/or implementation changes.

No I meant could theano be revived again? But it sounds like it wasn't
actually "revived" in the first place.

>
> Also, I had heard that pymc was dropping theano in favor of tensorflow
> (do I remember this correctly?). Is this still planned, or has that
> changed to be aesara now?
>
> Since this announcement, we've dropped all plans to implement the next version of PyMC in TensorFlow. The reason: it's possible for us to have the kind of tensor library described above and use other backends, when desired. I wrote a quick example of how we can convert Theano graphs to JAX "graphs", and that was enough to prove the point.
>
> My original plan was to use SymPy for all this, but it would've resulted in a distinct project anyway, because doing all that work via changes to SymPy would've most likely taken too long.

I tend to agree with this decision. SymPy uses unevaluated expressions
(or "graphs" as the tensor people call it, which is a term I'm not
fond of), but it's really not designed to build arbitrary symbolic
systems. The main thing it is designed for is representing
mathematical expression. So you'd run into a lot of problems with
SymPy. For example, the core SymPy expressions automatically evaluate
and don't distinguish certain equivalent expressions. It's lower
fidelity than a pure AST, because the primary thing it cares about is
mathematics. You would end up building all of the expression classes
from scratch, in which case, you would be better off just building
your own thing instead of building on top of SymPy.

With that being said, some of these limitations also limit what SymPy
itself can do, so we still are interested in removing them. But they
are indeed a lot of work.

Aaron Meurer

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/9edbf828-2510-4aa0-a3f8-88a35d60263cn%40googlegroups.com.

[Brandon T. Willard]

> But if you can't actually make any changes
or releases for it, then you can't actually do that, and have no
choice but to fork.

Yeah, that was essentially the situation.

> SymPy uses unevaluated expressions

(or "graphs" as the tensor people call it, which is a term I'm not

fond of)...

Aren't you still talking about expression graphs, though?  As all this symbolic work relates to term rewriting (and all that), graphs seems like a pretty standard terminology--especially when one considers anything involving multiple, dependent terms.

[Aaron Meurer]

On Wed, Mar 31, 2021 at 12:18 PM Brandon T. Willard
<brandon...@gmail.com> wrote:
>
>
> > But if you can't actually make any changes
> or releases for it, then you can't actually do that, and have no
> choice but to fork.
>
> Yeah, that was essentially the situation.
>
> > SymPy uses unevaluated expressions
> (or "graphs" as the tensor people call it, which is a term I'm not
> fond of)...
>
> Aren't you still talking about expression graphs, though? As all this symbolic work relates to term rewriting (and all that), graphs seems like a pretty standard terminology--especially when one considers anything involving multiple, dependent terms.

To me a graph is a general data structure consisting of nodes and
edges between those nodes. It doesn't generally have anything to do
with symbolic expressions. As far as I'm concerned, a symbolic
expression being represented as a "graph" is an implementation detail.
There are other ways to represent expressions. For example, a
polynomial may be represented as a list of coefficients, or a sparse
mapping from monomials to coefficients. Even in SymPy, expression
trees do not map one-to-one to what the expression looks like to the
end user. An expression like x - y is represented as Add(x, Mul(-1,
y)) (there is no subtraction class). The important thing is that the
expression itself is symbolic, or unevaluated. That also makes it much
clearer what is actually going on. But then again these are the same
people who prefer to use the scary (and also somewhat incorrect) name
"tensor" instead of "nd-array". </rant>

Aaron Meurer

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/973e11b4-5cf5-4e3c-837e-8d496d657cffn%40googlegroups.com.