virtualenv homepage advertising wrong mailinglists?
112 views
Skip to first unread message
ssba...@redhat.com
Sep 28, 2017, 8:15:13 AM9/28/17
to virtualenv
After spending a lot of effort trying to fix a bug in virtualenv I was hinted by a friend that I should also try the user mailinglist instead of the development one. Well, I hope this would work even if it is clear that reviewing and merging a patch is a development issue.
The bug is documented at https://github.com/pypa/virtualenv/issues/1029 and I have two patches for it one that solves it and one that assures that there is no regression (details on bug on why this needs to go separate).
The second problem is lack of feedback which is came as a big surprise for me as I am usually impressed by how quickly other Python projects maintainers are reacting.
- got zero feedback on both IRC channels #pypa and #pypa-dev
- no replies on official pypa-dev mailing list even after 7 days - https://groups.google.com/forum/#!topic/pypa-dev/8iVHDOqsj9M
- found a worrying report regarding a Trojan reported yesterday, this one with no reply too. -- https://groups.google.com/forum/#!topic/pypa-dev/4Bsh6EN0Wm4 -- i only hope that didn't appear from the distro itself.
Does virtualenv needs some extra maintainer workforce?
Thanks
Sorin
Paul Moore
Sep 28, 2017, 10:05:54 AM9/28/17
to Virtualenv
I've seen your reports. Personally, I haven't responded because I'm a
Windows developer and I don't really have the knowledge of Unix to
reliably assess fixes like this. Normally, I'd be wiling to make some
level of common-sense judgement, but virtualenv is something of a
special case. Specifically:
1. The virtualenv test suite is very limited. We don't really test a
significant portion of the functionality, and in particular edge
cases, so the risk that any given PR has unintended consequences is
rather high.
2. At its core, virtualenv is a set of pretty nasty hacks handling
special case behaviour for a variety of operating systems, Python
implementations, and customisations introduced by distribution vendors
into core Python. As a result, "it works on my system" is a
significantly worse predictor of quality than for other projects.
3. Virtualenv is a core of many people's workflows, and any breakage
will have a significant effect. So we *have* to be conservative.
Conversely, the fact that a significant majority of the Python
community routinely use virtualenv without issue means that what
problems we *do* get reported probably aren't as widespread as the
poster believes (which is not to diminish the impact on that
individual).
https://xkcd.com/1172/ is highly relevant here...
Also, the PyPA team is 100% volunteers, and an extremely small team,
with a lot of much higher-profile work demanding our effort. So
regrettably, virtualenv is lower priority.
With regard to the specifics of your issue, it's to do with the
activate script, which is a non-essential part of virtualenv. It's
completely possible to write your own activate script to replace it -
I know, because I have done precisely that. I didn't like the way the
supplied Windows activate scripts worked, so I wrote my own. Also, the
issue is when the script is used in bash with the non-standard strict
mode. That's not a common use case, and I, for one, don't know whether
every shell that is used by virtualenv users supports the "${PS1:-}"
syntax proposed in the fix (I know someone is going to claim "but it's
standard POSIX" - but we probably have users using other shells like
ksh, zsh, ancient Bourne shells on obsolete hardware, restricted
environments like boot loaders, ...). So while I could say "sure, it
looks simple, let's apply it" - who deals with the problems when the
change is released and we get a flood of Solaris users reporting
breakage with their shell? (This is of course an argument for complete
stagnation, and that's *not* my point - I'm just trying to emphasise
that we have to be careful and exercise judgement in ways that aren't
immediately obvious to someone on a mainstream system like Linux).
So - after all those negative points, what *can* be done?
1. Improve the test suite. Patches that improve test coverage, work
out ways to test on non-standard hardware/shells/OSes (and no, I have
no idea how we'd do that via travis) would be fantastic.
2. Improve the docs. Specifically by *reducing* users' expectations.
Docs explaining the role of the activate scripts, how they are
optional and don't do anything magical, and how to write your own
variation, would help people having difficulties with the standard
scripts. At its core, all virtualenv does is create a directory with a
copy of a python interpreter in it. Pretty much everything else is
optional (convenient, and essential for a usable user experience, but
replaceable). A better explanation of that might help people unhappy
with something delivered in the standard package (or it might not -
having to support their own customisations is not something everyone
has the time or resources to do).
3. More difficult to do, but look beyond the "works for me" scope of a
PR. Test on multiple environments (and document what you've done in
the PR), ask for people to test on their environments, etc. A PR that
says "tested on (10 different systems, including some uncommon ones)"
is much more likely to be merged than one that says "works on my
Kubuntu 17.04 environment".
Also, for Python 3, I'd strongly recommend looking at the core venv
module for virtual environment management. Its approach is supported
by the core interpreter code, so it's much more robust than
virtualenv, and it's probably the long-term solution to virtual
environment creation. Having said that, I doubt their activate scripts
are any better than ours ;-)
Paul
> --
> You received this message because you are subscribed to the Google Groups
> "virtualenv" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to python-virtual...@googlegroups.com.
> To post to this group, send email to python-v...@googlegroups.com.
> Visit this group at https://groups.google.com/group/python-virtualenv.
> For more options, visit https://groups.google.com/d/optout.
Windows developer and I don't really have the knowledge of Unix to
reliably assess fixes like this. Normally, I'd be wiling to make some
level of common-sense judgement, but virtualenv is something of a
special case. Specifically:
1. The virtualenv test suite is very limited. We don't really test a
significant portion of the functionality, and in particular edge
cases, so the risk that any given PR has unintended consequences is
rather high.
2. At its core, virtualenv is a set of pretty nasty hacks handling
special case behaviour for a variety of operating systems, Python
implementations, and customisations introduced by distribution vendors
into core Python. As a result, "it works on my system" is a
significantly worse predictor of quality than for other projects.
3. Virtualenv is a core of many people's workflows, and any breakage
will have a significant effect. So we *have* to be conservative.
Conversely, the fact that a significant majority of the Python
community routinely use virtualenv without issue means that what
problems we *do* get reported probably aren't as widespread as the
poster believes (which is not to diminish the impact on that
individual).
https://xkcd.com/1172/ is highly relevant here...
Also, the PyPA team is 100% volunteers, and an extremely small team,
with a lot of much higher-profile work demanding our effort. So
regrettably, virtualenv is lower priority.
With regard to the specifics of your issue, it's to do with the
activate script, which is a non-essential part of virtualenv. It's
completely possible to write your own activate script to replace it -
I know, because I have done precisely that. I didn't like the way the
supplied Windows activate scripts worked, so I wrote my own. Also, the
issue is when the script is used in bash with the non-standard strict
mode. That's not a common use case, and I, for one, don't know whether
every shell that is used by virtualenv users supports the "${PS1:-}"
syntax proposed in the fix (I know someone is going to claim "but it's
standard POSIX" - but we probably have users using other shells like
ksh, zsh, ancient Bourne shells on obsolete hardware, restricted
environments like boot loaders, ...). So while I could say "sure, it
looks simple, let's apply it" - who deals with the problems when the
change is released and we get a flood of Solaris users reporting
breakage with their shell? (This is of course an argument for complete
stagnation, and that's *not* my point - I'm just trying to emphasise
that we have to be careful and exercise judgement in ways that aren't
immediately obvious to someone on a mainstream system like Linux).
So - after all those negative points, what *can* be done?
1. Improve the test suite. Patches that improve test coverage, work
out ways to test on non-standard hardware/shells/OSes (and no, I have
no idea how we'd do that via travis) would be fantastic.
2. Improve the docs. Specifically by *reducing* users' expectations.
Docs explaining the role of the activate scripts, how they are
optional and don't do anything magical, and how to write your own
variation, would help people having difficulties with the standard
scripts. At its core, all virtualenv does is create a directory with a
copy of a python interpreter in it. Pretty much everything else is
optional (convenient, and essential for a usable user experience, but
replaceable). A better explanation of that might help people unhappy
with something delivered in the standard package (or it might not -
having to support their own customisations is not something everyone
has the time or resources to do).
3. More difficult to do, but look beyond the "works for me" scope of a
PR. Test on multiple environments (and document what you've done in
the PR), ask for people to test on their environments, etc. A PR that
says "tested on (10 different systems, including some uncommon ones)"
is much more likely to be merged than one that says "works on my
Kubuntu 17.04 environment".
Also, for Python 3, I'd strongly recommend looking at the core venv
module for virtual environment management. Its approach is supported
by the core interpreter code, so it's much more robust than
virtualenv, and it's probably the long-term solution to virtual
environment creation. Having said that, I doubt their activate scripts
are any better than ours ;-)
Paul
> You received this message because you are subscribed to the Google Groups
> "virtualenv" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to python-virtual...@googlegroups.com.
> To post to this group, send email to python-v...@googlegroups.com.
> Visit this group at https://groups.google.com/group/python-virtualenv.
> For more options, visit https://groups.google.com/d/optout.
ssba...@redhat.com
Sep 29, 2017, 8:20:28 AM9/29/17
to virtualenv
Well, as of today, Python activate scripts are much better than the virtualenv ones.
I guess that should be a good indication that we need to also fix the the issue in virtualenv.
I think is time to review https://github.com/pypa/virtualenv/pull/1078
BTW, Look at the changes, I even fixed an obvious bug which was a missing "$" on one of the checks. Line https://github.com/pypa/virtualenv/pull/1078/files#diff-01d9e5f8d9d6bb4a6b8fd4a3db2e6557L58 --- I am wondering how did this went in as it was very easy to spot that the condition was not working as intended, always giving the same result.
Regarding helping with PYPA projects, I would not mind giving some help here, especially on reviewing code on non-Windows platforms.
Krzysztof Rosiński
Nov 29, 2017, 3:45:44 AM11/29/17
to virtualenv
I thought I'd follow up on the discussion. The PR has been waiting for a while, and it looks like a good step forward. Can a maintainer look into it? There are a few reviews and approvals already.
Thanks,
KR
Roman Levin
Dec 12, 2017, 5:45:11 AM12/12/17
to virtualenv
Any way to move this along? It's just waiting a PR waiting to be merged.
Reply all
Reply to author
Forward
0 new messages