2021H1 Roadmap

[Stu Hood]

Hey folks!

Late last year the Pants committers kicked off a prioritization effort to help decide the project’s roadmap. We gathered your ideas, and your feedback, and then… one of the strangest holiday seasons of all our lives happened! We hope that you and your loved ones have been healthy and happy.

But, finally: long after finishing the process of tabulating the raw results -- and mostly delayed by determining committer availability relative to our other efforts! -- we are able to talk about the plan for the next couple of quarters. Please keep in mind that estimation is challenging, and ongoing shifts in the community might affect priorities as we go.

From the complete list of submitted ideas, the items that we hope to complete by the end of Q2 are (in descending priority order): 

1. Improve cache hit rates by resolving exactly once per lockfile, and then filtering per target.

2. Improve the performance of running pytest

3. Support for running Pants (with pantsd) concurrently in multiple terminals in the same repository

4. "Init Goal": aka Goal to display files that are not tracked by pants, but could/should be, and etc

5. Adding support for having multiple Python lockfiles/resolves per repository.

6. Document how to export a virtualenv and codegen, and how to import into common IDEs

7. Add tools for managing cache (e.g. size, etc) and pantsd. Make it easy to answer "How much disk space is pants taking up?" and "What is the status of pants runs and pantsd?"

8. Make it so that running multiple goals at once (like test, lint, typecheck) causes them to run in parallel rather than sequentially.

9. Support "live reload"/"auto restart" usecases in `repl`, `run`, etc (via changes to `--loop` / "watch mode").

10. Improve the performance of running mypy

11. Improve RuleGraph user experience

12. Support for exporting a virtualenv and codegen to generically support most IDEs

13. Make the Pants CLI (rather than the tests/processes it is running) faster by porting more of it to Rust

While choosing what order to tackle the ideas in, we paid close attention to the raw preference data (thank you for your ideas and votes!), but there are some notable cases where we diverge from a strict order that are worth calling out.

• The “export a virtualenv and codegen” item (#11152) was split into an item to document the current approach for various IDEs (#11338), and the generic solution was pushed back a bit to provide time to complete PEX’s `venv` execution mode.

• Although the “Docker Support” item (#2648) was one of the highest voted items, it was pushed out of the next few months. There are at least three potentially disjoint features of a Docker integration, and we feel that it’s an item that is highly amenable to being a contribution by the community. We’d love to work with anyone to help add Docker support in Pants core, or even just to more fully flesh out the usecase.

• We feel that the “Init Goal” item (#11164) is important to help get more new users started with Pants, and so we prioritized it more highly than the votes would suggest.

• Support for a second language (Java and Go were submitted as ideas) is notably missing, but this is in alignment with the votes -- likely because the most active current community members are using pure-Python repositories. We nonetheless think that adding support for the JVM is very important (for Java and Scala, in particular), and we’re very interested in working with any contributors who would like to partner with the committers on implementing JVM support (or other languages!) in Pants v2.

If you feel that we missed something about any of the submitted ideas, please let us know by commenting on the issues with more details, or by reaching out in Slack.

As always: Pants is an open source project! We’d be happy to help anyone and everyone make a contribution to Pants, particularly to help to scratch the itch of an idea that didn’t end up quite as high in the list as they’d like. And we’d also love to work with any contributor to move toward committer status in order to help drive the direction of the project.

Thank you again for your participation! We look forward to working with you to build a better Pants.

----

@stuhood on behalf of the Pants Build committers