Warehouse/PyPI update: launch, project wrapup approaching

[Sumana Harihareswara]

On Monday, we launched Warehouse and redirected browser and API traffic so Warehouse is now the codebase, and http://pypi.org/ is the site, serving nearly everyone who requests files from PyPI. The old codebase is now up at http://legacy.pypi.org/ temporarily. We had a few hiccups (incident report) and are now fixing up some search, indexing, caching, encoding, API compatibility, and UI issues. We're monitoring incoming bug reports and, so far, don't see anything new that absolutely needs fixing before we shut down the legacy site on Monday, April 30th.

https://github.com/pypa/warehouse/projects/1 is the rollout board you can watch to see our progress, and our weekly meeting notes are up. Please continue to report bugs -- if you know they're in PyPI, file them against Warehouse, and if you're not sure, file them in the "packaging problems" repository.

(The "nearly everyone" in that first sentence above is because of this User-Agent exclusion, and because I'm sure a few users are specifying legacy.pypi.org in their requests right now while working on forwards compatibility.)

We've made a number of user-visible improvements in the past couple weeks. For instance, we added a "switch to desktop version" link in the mobile view, fixed dropdowns for accessibility, added user help for folks affected by the TLS 1.0/1.1 deprecation (mea culpa, we should have done that sooner), and created a page thanking our sponsors.

And we've made many backend improvements to performance, API compatibility with legacy, sorting and indexing, instrumentation for metrics, and security -- and Donald implemented email sending via SES. Thanks to Noah Kantrowitz for reporting a privacy concern regarding Gravatar URLs and leaking users' email addresses (fixed).

It is not feasible for me to summarize all the work that volunteers put in, as testers, coders, code reviewers, writers, and user support helpers within the last two weeks. We have an embarrassment of riches here. Since April 3rd (two weeks ago) we've merged 98 PRs to Warehouse; thanks to ymyzk, reaperhulk, glasnt, alex, RazerM, bskinn, saxenanurag, hugovk, waseem18, cheungnj, contrepoint, yeraydiazdiaz, jonparrott, jMuzsik, and aalmazan for those. And I'd also like to thank the many people who, on their own, provided and continue to provide help to affected users on IRC, Twitter, StackOverflow, and elsewhere.

We don't have any virtual office hours coming up, but we have some other events planned. The Talk Python to Me podcast just interviewed Dustin, Nicole, and Ernest for an  upcoming episode. Dustin will be speaking on PyPI and packaging in general at PyCon NA in May and at SciPy in July, and we're sprinting at PyCon and EuroPython -- join us? And if you're in New York City, you can join Laura and me for a Warehouse sprint night on Thursday, April 26th.

And the project's starting to wrap up. Our MOSS funding (thanks to Mozilla for their support for the PyPI & Warehouse

work!) is nearly finished; after we shut down the legacy site on April 30th, the general pace of Warehouse development may slow down. (Warehouse has far more volunteer contributors than it did when we started MOSS-funded work in early December, but maintainer time available will diminish.) So we're seeking further funding, to speed up security and accessibility work, (potentially) localization, group/organization support, better notifications, better staging/testing workflow for project maintainers, GitHub signon, and more (see the "Post-Legacy shutdown" milestone and "cool but not urgent" milestone). We have submitted a few more grant proposals and are waiting to hear back. And donations to the Python Software Foundation’s Packaging Working Group, which works to sustain PyPI, pip, setuptools, and all other Python Packaging Ecosystem efforts, can now be made on a recurring basis! Please check out https://donate.pypi.org/  and consider pitching in or spreading the word. I'm working on writing up and sharing a more structured list explaining what we could do at various levels of funding.

Thanks as always. Please keep the kind words, bug reports, and -- I hope -- funding coming! :)

--

Sumana Harihareswara

Warehouse/PyPI project manager

Changeset Consulting

s...@changeset.nyc