PEP and APE processes

[Thomas Robitaille]

Hi everyone,

This is a side discussion that I thought would be interesting to have based on the experience I have gained so far with the PEP process. The PEP process is as follows:

* One or more person writing the PEP discuss the idea in the community and see if there is at least some degree of support for an idea, and there may be some discussions etc about approaches, collaborative writing, etc

* A pull request is opened to the PEP repo with a first complete draft

* The PR undergoes *editorial* review only, that is how the formatting/text could be improved for readability, compliance with PEP guidelines and so on. No discussion on content happens.

* The PEP is assigned a number and the PR is then merged. At this point the PEP is published as a draft

* The PEP is advertised in a discussion forum and discussion happens there

* The PEP is then updated via a PR if any changes are required, and undergoes another editorial review

* The PR is merged and a new discussion thread is started

* Eventually, after potentially several rounds of discussion/updates, a decision is made about whether to accept or reject the PEP, and the status is updated

The main difference for APEs is that merging the PR = accepting the PEP, so in fact we don't really have any permanent record of rejected ideas, and PRs also tend to linger open. Permanent numbers are assigned early on in the process so there is no confusion, and there are no 'gaps' in the numbering.

I personally quite like the PEP process in this regard, and I am curious what people think about it? Essentially what we could change is to have PRs be editorially reviewed and merge drafts and discuss them on the mailing list.

I suppose if we wanted to change this I might have to write an APE? >.<

Cheers,

Tom

[Thomas Robitaille]

Just a small correction:

The main difference for APEs is that merging the PR = accepting the PEP

This should read 'accepting the APE'!

Cheers,

Tom 

[Pey Lian Lim]

Thanks for the feedback, Tom!

Yes, this APE will need an overhaul: https://github.com/astropy/astropy-APEs/blob/main/APE1.rst

Doing it the PEP way, on the extreme side, we will get a bunch of unactionable ideas, we have to merge them all, and then the repo will be full of things we couldn't accept anyway. Our APE numbering are still in the 20s while PEP already in the 700s (granted PEP is way more popular and has higher traffic, but they also have more staff). My worry is that we do not have resources to curate the APE repo properly if we go this route.

[Marten van Kerkwijk]

Hi Tom,

I agree that we should move to the PEP style. Indeed, I'm surprised
that is not what we do! A major advantage is that it is much more
obvious how to suggest changes - just open a PR on the relevant APE
(rather than on someone's personal repository where it can easily get
lost).

All the best,

Marten

[Thomas Robitaille]

On Fri, 7 Feb 2025 at 13:59, Pey Lian Lim <p3y...@gmail.com> wrote:

Thanks for the feedback, Tom!

Yes, this APE will need an overhaul: https://github.com/astropy/astropy-APEs/blob/main/APE1.rst

Doing it the PEP way, on the extreme side, we will get a bunch of unactionable ideas, we have to merge them all, and then the repo will be full of things we couldn't accept anyway. Our APE numbering are still in the 20s while PEP already in the 700s (granted PEP is way more popular and has higher traffic, but they also have more staff). My worry is that we do not have resources to curate the APE repo properly if we go this route.

I don't think it would be this extreme, there are currently only five unmerged APEs and I don't think switching to the PEP process would suddenly create many more APEs. It still takes a lot of effort to write one and the editorial review would still ensure we don't have half baked ideas being merged in. The APEs would still need to have a certain threshold of quality to be merged in. If we did go this route we could also follow the PEP guideline that every PEP has to have the backing of a core developer (in our case I suppose it would be a voting member) so that we don't get many proposals from random people external to the project.

Cheers,

Tom

On Friday, February 7, 2025 at 6:43:02 AM UTC-5 Thomas Robitaille wrote:

Just a small correction:

The main difference for APEs is that merging the PR = accepting the PEP

This should read 'accepting the APE'!

Cheers,

Tom 

--
You received this message because you are subscribed to the Google Groups "astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/astropy-dev/9dec430c-081d-4f3b-ad16-fcadf1d13d7en%40googlegroups.com.

[Brigitta Sipőcz]

Hi,

For what it's worth, we also switched to a more PEP-style approach with the Scientific Python SPECs. So, unfinished ones don't linger in PR status for years, and we don't get number mess-ups due to out-of-order merging.

We haven't had to deal with a total random and spammy one coming from outside of the project, I suppose those won't get automatically merged as drafts anyway, but again this process helps with drafting and handing over if an idea loses momentum from the original authors. So for APEs I suppose it's a good criterion that it has something to do with astropy, maybe the support of a couple of core maintainers/voting members, but it doesn't have to be a fully worked out content for getting a number and thus permanent reference.

Cheers,

 Brigitta

To view this discussion visit https://groups.google.com/d/msgid/astropy-dev/CAGMHX_1GsfhfjRq%3DcgA-Xv%3DRmkOWLhBwC-FcNi2JrD%2B-Q%2B18Jw%40mail.gmail.com.

[Marten van Kerkwijk]

Just as another point of reference, numpy NEPs follow roughly the PEP
procedure. I would not worry too much about numbers becoming very
large, but it does mean one has to organize the list somehow, separating
accepted from in progress from rejected, as in https://numpy.org/neps/

I think that can probably be nearly automatic...

-- Marten

[Pey Lian Lim]

Thanks for the clarifications! I think my concern above is addressed. APE 1 will need overhaul. And then we have to revisit open PRs to see which ones we can merge already. And then there was that one rejected APE, do we merge that now with this new process but then label as rejected?

[Aldcroft, Tom]

Having read this discussion, I'm also in favor of moving to the PEP-style process for APEs.

- Tom

--

You received this message because you are subscribed to the Google Groups "astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/astropy-dev/7fc73c22-7a42-4746-af9d-bce928bb59a4n%40googlegroups.com.

[Pey Lian Lim]

Tom R, are you planning to submit PR to modify APE 1?

[Pey Lian Lim]

So, Tom R opened https://github.com/astropy/astropy-APEs/pull/115 during Coordination Meeting 2025. I have since taken over. Very little community inputs since. CoCo is doing a final review. Please give it a careful read. This might be your last chance before merge. Thank you all!