barriers to Warehouse contribution

[Sumana Harihareswara]

TL;DR: what should we fix or set up to make it easier for you to hack on
Warehouse, especially with a goal of a Warehouse/packaging sprint at
PyCon in May?


The Warehouse team wants to make PyPI more sustainable, including
getting more people as regular code contributors. It's not bad as is --
there's a Docker Compose developer environment setup[0] and we're
reasonably responsive on GitHub and on IRC in #pypa-dev on Freenode. But
I want to check whether there are other things we ought to do to attract
and retain contributors (who are already skilled at Python and
contributing to FLOSS more generally -- we have to walk before we can run).

I'm looking for smaller tasks since we do need to concentrate most of
our efforts in the next few months on the immediate launch. In
Sumana's-gut-prioritization order:

1) Setup: have any of you had trouble in the last few weeks when trying
to get a developer setup going? If so, we should fix our
configuration/documentation.

2) Testing: Are there things that would be easy to add to some linters
or our continuous integration/codecov configuration that would help
contributors get faster and better feedback on pull requests?

3) Documentation: What bits of Warehouse feel mysterious or hard to
navigate? Developers who are new to a codebase need is to know *the
design rationale* of confusing bits -- why it was made this way, what
decisions are embedded in particular choices, whether particular
components are the result of a feature request, a quick fix after an
outage, an experiment, etc.

Here I think the next step is to use the answers to this question,
update our application structure overview[1] and/or to write something
like Zulip's architecture summary[2] or curate a list-of-links
(conference talks, blog posts, etc.) that would get us 30% of the way
towards a history and application overview like MediaWiki's.[3]

4) Setup, part II: could we do what Zulip does[4] and get setup with
turnkey hosted droplets/VMs for use at sprints and for people with
lower-end laptops and network connections?

And of course if I've missed some category of barrier that's gotten in
your way, please speak up.


[0] https://warehouse.readthedocs.io/development/getting-started/
[1] https://warehouse.readthedocs.io/application/
[2]
https://zulip.readthedocs.io/en/latest/overview/architecture-overview.html
[3] http://aosabook.org/en/mediawiki.html
[4] https://zulip.readthedocs.io/en/latest/development/request-remote.html
--
Sumana Harihareswara
Changeset Consulting
https://changeset.nyc

[Jeremy Stanley]

On 2018-01-16 14:29:32 -0500 (-0500), Sumana Harihareswara wrote:
> TL;DR: what should we fix or set up to make it easier for you to
> hack on Warehouse, especially with a goal of a Warehouse/packaging
> sprint at PyCon in May?

[...]

Perhaps this is more of a religious war since I have to believe
there was a good reason I'm simply unaware of but... having just
spent far too long earlier this week trying to figure out how to
subscribe to the ML without associating a Google account I feel like
moving to mail.python.org like a "typical" Python community mailing
list would help reduce the barrier to entry some. Being G*-avoidant
has previously prevented me from even participating in here but when
it was suggested that there are discussions relevant to the Python
packaging ecosystem happening here instead of on distutils-sig, I
decided to put on a pair of gloves and deal with it.
--
Jeremy Stanley

[mba...@gmail.com]

I've commented on #pypa-dev on IRC but Sumana asked me to post here.

I like the idea of creating resources to enable developers to understand the product architecture better. I have been working at getting up to speed on warehouse development and could have used just such a thing. I'm not a web developer but do work with Python so understanding why/how some decisions have been made about the design of the application would go a long way to grokking the overall project. Also maybe providing a list of technologies and methodologies that would be good to know to be most effective when working on the project would help people level up on things they should know in advance. Finally, having non-core warehouse dev people on IRC available to give suggestions when new contributors are getting errors would be helpful too. I know the primary dev folks here are busy but having a few resources who could triage error messages and point the way to understanding a problem would be useful.

The last item above is a large ask, I understand that. But a welcoming community needs to be perceived in a way that someone is (almost) always available when people are hacking on things and can't figure out a particular issue. When I'm more advanced I'd volunteer for something like this, but at the moment I'm much lower on the learning curve.

Lastly, related to Jeremy's topic about multiple mailing lists, I also didn't even know about this google group until yesterday. Having fractured communication channels makes it impractical to keep tabs on all the conversations going on. I even find it hard to keep up on distutils-sig alone. I think most open source projects have one IRC channel and one mailing list type thing(whether it be mailman, discourse, google-groups, etc.) More just makes it hard to keep up and hard for newcomers to decide where to post a question.

Thanks,

-Matt

[mba...@gmail.com]

Maybe what PyPa needs is a developer advocate type position:

https://medium.com/@ashleymcnamara/what-is-developer-advocacy-3a92442b627c

Does the PSF even have someone in this position?

[Brett Cannon]

On Wed, 17 Jan 2018 at 08:40 <mba...@gmail.com> wrote:

Maybe what PyPa needs is a developer advocate type position:

https://medium.com/@ashleymcnamara/what-is-developer-advocacy-3a92442b627c

Does the PSF even have someone in this position?

Nope. The PSF only has staff for administration, PyCon, and IT (to the best of my knowledge).

-Brett

[Jeremy Stanley]

On 2018-01-17 21:08:00 +0000 (+0000), Brett Cannon wrote:
> On Wed, 17 Jan 2018 at 08:40 <mba...@gmail.com> wrote:
> > Maybe what PyPa needs is a developer advocate type position:
> >
> > https://medium.com/@ashleymcnamara/what-is-developer-advocacy-3a92442b627c
> >
> > Does the PSF even have someone in this position?
>
> Nope. The PSF only has staff for administration, PyCon, and IT (to
> the best of my knowledge).

[...]

Not to derail the thread, but the OpenStack Foundation started
budgeting for a couple of developer advocate positions roughly two
years ago and it's worked out well for us so far. Generally just
ends up being long-time technical contributors to the community
acting as semi-detached and unaffiliated full-time "floaters" going
from project to project where help is badly needed while also
picking up side tasks related to community health. Granted the
makeup of the PSF isn't identical to the OSF, but it seems like a
lot of the dynamics may be similar.
--
Jeremy Stanley

[Ian Stapleton Cordasco]

Except the osf had orders of magnitude more money

Sent from my phone with my typo-happy thumbs. Please excuse my brevity

[Nick Coghlan]

The OSF is a commercial trade association that charges Platinum
sponsors US$500k a year for a board seat, so they have quite a bit
more recurring revenue to work with than the PSF does :)

As a public interest charity, the PSF's current paid staff positions
are Director of Operations (main point of contact between the Board
and the rest of the staff), Event Manager (mainly PyCon US),
Infrastructure Manager (part-time IT lead), and an assistant treasurer
(also part-time). While this may have changed since I was on the
Board, I'd expect the next paid position to be along the lines of a
grants management/fundraising lead type role, with a view towards
building up the PSF's ability to invest in additional community
support roles (like the developer advocate activities suggested here)
by increasing recurring annual contributions.

Cheers,
Nick.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia

[Sumana Harihareswara]

To reply to part of Matt's request:

On 01/17/2018 10:29 AM, mba...@gmail.com wrote:
> I like the idea of creating resources to enable developers to understand
> the product architecture better. I have been working at getting up to speed
> on warehouse development and could have used just such a thing. I'm not a
> web developer but do work with Python so understanding why/how some
> decisions have been made about the design of the application would go a
> long way to grokking the overall project. Also maybe providing a list of
> technologies and methodologies that would be good to know to be most
> effective when working on the project would help people level up on things
> they should know in advance.

I've just submitted a pull request
https://github.com/pypa/warehouse/pull/2937 that starts addressing this,
and welcome reviews.

And I've created some open issues for some further types of
documentation in
https://github.com/pypa/warehouse/issues?q=is%3Aopen+is%3Aissue+label%3Adocumentation
in case folks want to comment, +1, etc.

I'm also working on the other things Matt mentioned -- having more
people available to answer questions synchronously and clearing up the
inventory of communication channels. Thanks!