Enabling SymPy for external dependencies

[Joachim Durchholz]

I just hit another situation where I would really have liked to use an
external (small) library that was packaging a huge amount of research
and knowledge in a relatively small but well-done codebase.
(See the end of this message for details.)
My realization is that the Python ecosystem has (finally) started to
accumulate libraries that are actually worth reusing: Less work to learn
and integrate than to write the thing ourselves.

So I'd really like to get us into a shape where we can easily use
external dependencies.

What are the problems?
Can we turn them into tasks?
How to we complete the tasks?

I.e. I want the problems listed with an eye towards "how to we solve
that", not with a perspective of "that's why we do not do that".
Obviously, it's not going to be easy, but probably easier than dealing
with decorators.

Regards,
Jo

--
FWIW the latest use case that I had (I had more in the past): Decorators.
I spend an ungodly amount of time trying to understand them (easy) and
their implication (TONS of research, decorators need to understand all
protocols that are involved in classes, calls, and introspection, plus
there are differences between Python versions). During that research, I
found Graham Dumpleton's essay about doing decorators right:
http://blog.dscpl.com.au/2014/01/how-you-implemented-your-python.html
Turns out there were issues I *still* wasn't aware of. (That guy is
simply awesome.)
Even better, he went on and wrote wrapt:
https://github.com/GrahamDumpleton/wrapt
It's making writing decorators easier. You write a decorator function as
before, but you get not just the function you wrap, you get its class,
and the resulting decorator will properly deal with functions, methods,
static methods, and class methods, it will properly forward __name__,
__module__, it will work around incomplete/buggy decorator protocol
implementations in the standard library, and probably a few other
details that I didn't notice.
That guy is awesome. I couldn't do what he did, and I know I can do
quite a lot.
And I want his code.

[Joachim Durchholz]

(Replying instead of making this part of the initial post, to keep the
"questions part" separate from my and other people's answers.)

> What are the problems?
> Can we turn them into tasks?
> How to we complete the tasks?

Here's my stab at this.


Fragile downloads
-----------------

Problem: External dependencies are typically downloaded from another
website, which may be offline or unreachable.

Solution: Upload all dependencies into the GitHub repository. Either as
separate repositories, or into a subdirectory of the main repo. Both
ways would work, the workflows and standard git commands would be
slightly different.
We would need to adapt the build process to avoid packing the
dependencies into the installable bundle. (Or maybe we actually want
them in. It's something that can be decided upon.)

Severity: Critical. We're already being bitten for the few external
dependencies that we have.


Unreliable libraries
--------------------

Problem: It is hard to assess the quality of a library without actually
using it. Backing out of using a library is more work than adding it in,
to projects tend to accumulate dependencies on crappy libraries over time.

Solution: Require that each and every assumption about a library's
behaviour is documented in a test in SymPy before we allow it to go in.
New PRs that use parts of the library need to be checked whether they
use a new aspect of the library, and have a test added if not.
Be very, very, very reluctant about adding new libraries.

Severity: Zero on the short term, critical on the long term.


Library versionitis
-------------------

Problem: We cannot test all combinations of all versions of all used
libraries. (We don't even manage to do that for our existing set of
dependencies.)

Solution: Again, tests for all aspects of all libraries that we use.
Also, define a list of versions that we have tested. Run our tests
against all versions. Subscribe the SymPy mailing list to the release
mailing lists for all external dependencies; each of them should be
turned into a GitHub issue to run the test suite against the new
version, fix any problems that we find, and extend the list of supported
versions.
Never add a library that we aren't confident we can run it through this
procedure within the week or so. (We need to be really thorough with our
tests to reach that level of confidence.)
If it comes to the worst (external library develops into something that
doesn't benefit us anymore), we can fork the latest-known-working
version of the library and either set up a forked project, or integrate
the fork in source form into SymPy.

Severity: Medium.


More time spent on trade-offs
-----------------------------

Problem: Being more open to external dependencies means we have a large
overhead for automatic testing.
It also means more decisions about whether we want a specific external
library or not.

Solution: Adding a new external dependency carries more-or-less the same
implications as adding a new subsystem to SymPy: It's a large piece of
code that we need to integrate, we need lots of tests, we need to commit
to maintaining it (the code itself if subsystem, the interface code and
tests if external library).
We simply need to weigh the advantages of the stuff that the library
brings against the disadvantages of having to maintain the interface,
the tests, and keeping up with the versions against the advantages of
not having to deal with some particular library.

Severity: Low, we're already dealing with this kind of stuff.