Future of Long Term Support (LTS) releases

[Thomas Robitaille]

Hi everyone,

As you know, we currently treat every x.0 release as being LTS, and
this happens every two years. Six months after a new LTS version is
released, we start to maintain two release branches, and backport bug
fixes to the LTS branch for two years whenever possible. However,
maintaining an LTS branch is not always trivial as the codebase starts
to diverge quite a bit from the main branch after a while which means
that fewer and fewer backports will succeed automatically and will
require manual intervention, and this causes us to split effort
between maintaining the LTS branch and the current release branch when
these are different, so it is not without cost.

In addition, it's not always clear how we should handle deprecations
in our current system - for instance, if we want to deprecate
something when we release 6.1 or 6.2, people on LTS won't see the
deprecation warning until we release 7.0 which would mean we can only
remove it after 7.0 is out, unless we backport deprecations (which we
mostly don't do at the moment), so then those features that have been
deprecated have to be supported for yet two more years before being
removed from the LTS branch.

I do not know of any other package in the scientific Python stack that
has LTS releases, even core libraries such as numpy and scipy. At the
start of astropy, I think this made a lot of sense given how much
refactoring and changes in API there were between versions. However, I
think we are now at a stage in stability where I think we should at
least re-evaluate whether maintaining an LTS branch makes sense.

Therefore, I would like to propose an alternative: we do not tag any
releases as LTS but we keep the same release calendar (a x.0 release
every two years) and we follow semantic versioning rules which means
that we do not break any API except in those x.0 releases. This means
we can introduce deprecations at any point in the other releases, but
we only remove things or break things in the x.0 releases. For users
of LTS, this would be essentially equivalent in many ways to our
current LTS system - they would know (as would all users in the new
system) that upgrading from 6.0 to 6.1 to 6.2 to 6.3 will not ever
break any API (modulo unintentional bugs of course) and once they are
ready to update to 7.0 they should first install 6.3 and resolve any
deprecation warnings, then update to 7.0. If we wanted, we could even
choose, close to the 7.0 release date, that we don't actually need to
break API and release 6.4 instead and have the 6.x major version last
longer than two years (so we could think of the two years as a
minimum).

I think such a system would make things a lot cleaner for us - only
one release branch to maintain at any one time, and clear rules on
when to introduce deprecations and break API. We would still be
following the same release calendar as we have in the last few years.

One small technical note - if we agree that we would basically be
doing semantic versioning, then we should start calling the first
major releases e.g. 6.0.0 instead of 6.0, which might be a good idea
anyway.

I am curious what people think about this idea - in particular those
of you who might be at organizations relying on LTS?

If people are generally in favor of this, I can try and draft an APE.

Cheers,
Tom

[Tom Aldcroft]

Hi Tom,

Generally this sounds like a good idea to me. The devil is in the details though, in particular about the strictness with regards to API changes only once every two years.

A somewhat subtle situation is for API changes that are somewhat narrow in scope and fix a minor design problem or make a minor improvement in behavior. Examples from the change log:

• Multiplying a LogQuantity like Magnitude with dimensionless physical units by an array will no longer downcast to Quantity. [#12579]

• When reading an ECSV file, changed the type checking to issue an InvalidEcsvDatatypeWarning instead of raising a ValueError exception if the datatype of a column is not recognized in the ECSV standard.

• Change the repr of the Table object to replace embedded newlines and tabs with r'\n' and r'\t' respectively. This improves the display of such tables. [#12631]

• Changed behavior when a structured numpy.ndarray is added as a column to a Table.

Sometimes the original behavior could be labeled as a bug, but this is a semantic issue that does not change the fact that someone's code might depend on the original API and break. Historically we have accepted narrow-scope API changes in minor version updates like 5.1 with the idea that breakage would be limited and any critical pipelines would be using LTS or be well-tested.

Personally it might be a bit demoralizing to know that a PR that changes the API even slightly might need to wait for 2 years for release even if it is narrow in scope and impact.

As a data point for an organization that uses astropy for mission critical tools, we update to the latest stable version of astropy and other core packages once a year. We rely on fairly extensive unit, integration, and regression testing to ensure that astropy (or numpy or ...) hasn't broken anything. We do regularly find at least a few things in our local packages that need to be updated, along with many new deprecation warnings that are sometimes actionable.

- Tom

--
You received this message because you are subscribed to the Google Groups "astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/astropy-dev/CAGMHX_0ut7xF%2BE%3Droem40TaQjqusCwqUw4220p-medU5B4sYOg%40mail.gmail.com.

[Marten van Kerkwijk]

Hi All,

Looking at the changelog for 5.0 bugfix editions, I see fairly large
changes for 5.0.1 and 5.0.2, which are within the time range we would
have bugfix releases for any version., but for 5.0.3 and 5.0.4 (don't
know where 5.0.5 went in changes.rst...) it is all really minor apart
from updates to wcslib and cfitsio. I'm not sure the minor fixes
brought much benefit to LTS, while aguably the updates of libraries do
not belong: if you are OK with the risk of updating those libraries (or
numpy, ...), then why not update astropy as a whole?

I guess my view is that given how stable astropy now is, we can be more
like numpy and not have LTS at all -- with the logic being that it
generically is fine to just keep using an older version as after half a
year there are very unlikely to be bugs in it that affect running the
same code time and again.

If so, we could just stop releasing LTS without making any adjustment in
our process otherwise. And just be somewhat ad-hoc about when changes
are big enough that they warrant an increase in major version number --
it seems clear it will be fairly rare anyway... (apparently, numpy 2.0
is really going to happen, once the dtype changes are all in).

All the best,

Marten

[Pey Lian Lim]

But other more popular packages like numpy and scipy are not using strict semver, right? Do we have example of a package that actually follows a strict semver? I am afraid bumping the major release number too often would have psychological scarring on some people (or the opposite is people have to wait a long time for any major changes, as mentioned above).

[Pey Lian Lim]

Also should we put this down as potential topic for Coordination Meeting, or is that happening too late?

[Marten van Kerkwijk]

Indeed, they seem to do roughly what we do: have some not-very-large
changes and improvements with every minor version update. Which is what
I think we should continue to do too...
-- Marten

[Benjamin Weaver]

Hello y'all,

I'm generally supportive of this idea because I understand as an insider what the issues are, BUT....

In the last few weeks I've told several teams of pipeline developers that Astropy has this great feature called LTS that allows you to develop your pipeline on a stable version for an extended period of time.

My impression is that people *perceive* numpy, scipy, etc. packages as having long-term support even if they don't *actually* do that.  Alternatively, the numpy version may be more-or-less permanently, but indirectly, set by the Astropy LTS version, without much further thought about it.

So part of any proposed APE should be an extended discussion of how to treat versions in the absence of a specially-designated LTS.  I'm sure that's under consideration, but it should be written with the intent to communicate to developers who are not directly part of the Astropy community.

This could be a good thing, because it could also include a broader discussion of how to handle new versions of other packages like numpy, scipy, etc., along the lines of what Tom Aldcroft just wrote. In other words, describing in some detail what an ongoing development and testing process would look like, including routine testing of newer versions of all packages.

Kia ora koutou,

Benjamin Alan Weaver

--
You received this message because you are subscribed to the Google Groups "astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/astropy-dev/878rh6bpuj.fsf%40weasel.astro.utoronto.ca.

--

With great responsibility seldom comes great power.

[Thomas Robitaille]

Hi all,

Thanks for the replies so far! Based on the feedback I'm seeing, let's ignore the idea of more strictly following semantic versioning.

Perhaps indeed something like what Marten is suggesting would be ok: to simply stop maintaining LTS versions but keep the rest the same.

We should think about whether the x.0 would have any special meaning in the absence of LTS - maybe those could be versions where we make big API changes if needed, so for instance a change similar to when we finally removed the clobber= argument in astropy.io.fits (which we knew would impact a lot of users), or looking forward if we were to e.g. remove astropy.samp from the core package. So then we could still have small API changes in minor versions like we do now, but for more significant/impactful changes we could deprecate in minor releases and then remove/change in the next x.0 release.

Cheers,

Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/astropy-dev/CAMUmennz-i2SDkZZjwtzPyFbTnNMi%2Bp5Fi1gGXZcL2KyWAWETw%40mail.gmail.com.

[Emir Karamehmetoglu]

In general packages like numpy or scipy do not have as many breaking changes as in astropy. For scipy, it "upstreams" i.e. absorbs lots of changes that in Astropy are released as affiliated packages, and contributions for a long time were evaluated on whether they break existing user code. For Astropy, pipelines and analysis tools built on a combination of affiliated and semi-affiliated/dependent packages can quickly fall out of date with the changes and improvements made to Astropy core, and require bumping the version, which then breaks user code. I have personally seen many examples of this among colleagues and students.

I would like to offer a solution that may help. Currently, although Astropy has some of the most user-friendly installation and development documentation for a large python package that I've ever seen, we do not help with taking advantage of the semversion system.

The most we say is try to use a conda environment with anaconda default (https://docs.astropy.org/en/stable/install.html#using-conda), which is extremely unreproducible and this generic, all-in-one environment does not facilitate more frequent updates. Updates will break things, especially older code, and leave users bewildered. And we actually recommend running `conda update anaconda` in this environment, which can be not nice to recover from.

I believe this is the wrong approach. We should show and encourage users to create a NEW python environment for a given specific task, with FIXED versions of astropy. Such an environment can be exactly re-created when the user moves to the next postdoc and gets a new computer in 1.5 years, allowing their old code to keep working, or needs to come back and run the same model 5 years later upon request from a new paper wanting to build on the results.

If people are familiar with `python poetry`, it is now in a stage where it works well alongside conda, to easily create version locked manageable virtual environments. I think the conda users were initially ignored by the target demographic of poetry, but the support has been mostly forced onto it now through relentless community requests. It is not the only tool for creating reproducible environments, but I think it may ultimately be the most user-friendly. In any case:

If we instructed users how to create reproducible environments for their code (a concept they are hopefully familiar with from writing clear instructions on how to reproduce their scientific results 😉), then they could lock their script for paper xyz or pipeline code with astropy==4.1 photutils==0.9 specutils==1.0.1. Just locking only these versions will go a long way, as I believe astropy devs currently backport breaking dependency changes, except for that one jinja2 incident (I may be wrong about this). But even if only locking the astropy related packages is not always enough (due to unlocked secondary dependencies like numpy or jinja), then we could go one step further and instruct on how to create an environment dump, using conda/poetry/pipenv etc. and then also how to re-install from this conda environment yaml or poetry/pip pyproject.toml/lockfile (or even requirements.txt). 

I believe this would allow for astropy to not bother as much with LTS and backports, as the number of requests from users would go down significantly. 

Ultimately, the biggest problem is creating broken complex dependencies for users to just run their previously working code again. Getting new features into old versions is a nice to have that does not really have to be entertained. The previous goal is best achieved by a combination of instructing users how to actually create environment that are impossible to break, and being considerate in not making too many breaking changes too quickly.

Best regards,
Emir

---

Emir Karamehmetoglu

ORCID

To view this discussion on the web visit https://groups.google.com/d/msgid/astropy-dev/CAGMHX_0Ox5hGWEGzwG04esBhO7wczrzOAYn-R-v7qHWZW7AXmg%40mail.gmail.com.