Quick updates on the PyCon sprint that wasn't.
10 views
Skip to first unread message
Jim Fulton
May 30, 2017, 9:21:30 AM5/30/17
to buildout-d...@googlegroups.com
FWIW, I think buildout, especially with wheel support, has potential to really benefit Python. I consider the traditional way of installing Python packages to be flawed. Unfortunately, fewer and fewer people are aware of buildout and why they might care about it.
We have some technical challenges, but I think we have even bigger PR challenges. I wonder if there's enough energy to fix and promote buildout. I hope so.
I wonder if it's practical to have a buildout sprint somewhere. FWIW, I live on the water and would be happy to host a small sprint in my home. :) I'm also willing to travel.
Leonardo Rochael Almeida
May 30, 2017, 10:55:57 AM5/30/17
to Buildout Development
Hi Jim
On 30 May 2017 at 10:21, Jim Fulton <j...@jimfulton.info> wrote:
No one showed up, so I worked on other projects.
A pity... couldn't make it to PyCon this time, or I would have worked on buildout.
Maksym Shalenyi and I are planning to work together a bit sometime next month.
Cool, do share the specifics when you have them.
FWIW, I think buildout, especially with wheel support, has potential to really benefit Python. I consider the traditional way of installing Python packages to be flawed. Unfortunately, fewer and fewer people are aware of buildout and why they might care about it.
The workflow that buildout allows is nice, but it's currently intimately coupled with the way setuptools installs packages. Since setuptools unbundled it's dependencies, the ergonomy of using setuptools, and by side-effect buildout, got somewhat worse: it's not possible to use a buildout with setuptools newer than 33.1.1 to install anything that depends on versions of dependencies of setuptools (appdirs, six, ...) that are incompatible with the versions that setuptools needs, and the resulting breakage is really hard to make sense of, if you haven't seen it before.
Wheel support, the way it's implemented today, certainly enhances the buildout experience, but since we're using `wheel` directly to install wheels, we risk running into packages that were only tested with `pip` and misbehave with `wheel`:
(I ran into the issue with the comment above in one of my wheel-enabled buildouts. I had to figure out a way of blacklisting that specific wheel so buildout picks the sdist instead).
And wheel support has no impact in the setuptools unbundling breakage...
This is to say that, no matter how better we think the buildout way of doing things is compared to the Python status quo, if we deviate too much from it, we're going to keep bumping into problems at the same time as we make it hard to explain and sell buildout to other developers.
We have some technical challenges, but I think we have even bigger PR challenges. I wonder if there's enough energy to fix and promote buildout. I hope so.
I believe there is a way to decouple the buildout workflow from setuptools, use pip, and keep the most of the buildout benefits intact while making things familiar to pip users.
This would make it easier to sell the buildout idea without bumping into the setuptools drawbacks.
And with some development in other packages (namely pip), I think we can achieve all of the current buildout benefits.
I'll start a separate thread to talk about my idea.
I wonder if it's practical to have a buildout sprint somewhere. FWIW, I live on the water and would be happy to host a small sprint in my home. :) I'm also willing to travel.
I'd like to take part in a buildout sprint, but no current budget for it, unless it also happens to be in a conference I'm going (and for now, I don't have any specific plans for a conference to go to).
Cheers,
Leo
Jim Fulton
May 30, 2017, 11:14:29 AM5/30/17
to buildout-d...@googlegroups.com
On Tue, May 30, 2017 at 10:55 AM, Leonardo Rochael Almeida <leoro...@gmail.com> wrote:
Hi JimOn 30 May 2017 at 10:21, Jim Fulton <j...@jimfulton.info> wrote:No one showed up, so I worked on other projects.A pity... couldn't make it to PyCon this time, or I would have worked on buildout.Maksym Shalenyi and I are planning to work together a bit sometime next month.Cool, do share the specifics when you have them.FWIW, I think buildout, especially with wheel support, has potential to really benefit Python. I consider the traditional way of installing Python packages to be flawed. Unfortunately, fewer and fewer people are aware of buildout and why they might care about it.The workflow that buildout allows is nice, but it's currently intimately coupled with the way setuptools installs packages. Since setuptools unbundled it's dependencies, the ergonomy of using setuptools, and by side-effect buildout, got somewhat worse: it's not possible to use a buildout with setuptools newer than 33.1.1 to install anything that depends on versions of dependencies of setuptools (appdirs, six, ...) that are incompatible with the versions that setuptools needs, and the resulting breakage is really hard to make sense of, if you haven't seen it before.Wheel support, the way it's implemented today, certainly enhances the buildout experience, but since we're using `wheel` directly to install wheels, we risk running into packages that were only tested with `pip` and misbehave with `wheel`:(I ran into the issue with the comment above in one of my wheel-enabled buildouts. I had to figure out a way of blacklisting that specific wheel so buildout picks the sdist instead).And wheel support has no impact in the setuptools unbundling breakage...This is to say that, no matter how better we think the buildout way of doing things is compared to the Python status quo, if we deviate too much from it, we're going to keep bumping into problems at the same time as we make it hard to explain and sell buildout to other developers.
I think this is fixable.
This was the main technical challenge I refer to below.
We have some technical challenges, but I think we have even bigger PR challenges. I wonder if there's enough energy to fix and promote buildout. I hope so.I believe there is a way to decouple the buildout workflow from setuptools, use pip, and keep the most of the buildout benefits intact while making things familiar to pip users.
I may not be understanding you, but while I'm not opposed to recipes that use pip, I think the package-path approach is superior and where I would want to focus.
This would make it easier to sell the buildout idea without bumping into the setuptools drawbacks.And with some development in other packages (namely pip), I think we can achieve all of the current buildout benefits.
Not sure I understand what you're getting at here.
I'll start a separate thread to talk about my idea.
OK.
Somewhat aside:
- I'm in favor of moving away from setuptools, building on a library ideally shared with pip (and/or conda?).
- The ongoing headache of setuptools unvendoring its dependencies is a good illustration, IMO, of Python's broken package-install approach. In fact, seeing new posts on https://github.com/pypa/setuptools/issues/980 is what spurred me to write this.
Jim
Reply all
Reply to author
Forward
0 new messages