A shell-agnostic "activate"

Showing 1-14 of 14 messages
A shell-agnostic "activate" datagrok 3/31/12 10:07 AM
Hello all,

I wrote a brief diatribe, "Virtualenv's bin/activate is Doing It Wrong," documenting an approach to activating virtual environments that avoids sourcing bin/activate into one's shell, does not require multiple scripts to be maintained for every possible (unix) shell, and avoids several other annoyances I have encountered with virtualenv's bin/activate script:


I posted it as an issue on the virtualenv Github page and I'm looking for some more feedback about the idea. I've already made some improvements based on the feedback I received there, but the more the better. Maybe I can change this from a frustrated rant into a useful improvement for the community.


I'm browsing through the list history and finding something similar to my experience: virtualenv newbies are introduced to the tool through the "activate" convenience script, they don't realize how little magic is actually required to make a virtualenv go, and they stumble over the fact and implications of needing to source it into their shell to use it.

I'd like to propose that with, a subshell-launching technique as documented in that gist above, the code becomes easier to maintain and newbies will have an easier time reasoning about "activate" and how to use it.

My apologies if this note comes through as some horrible pastiche of HTML garbage. I can't find anywhere in the Google Groups interface to send in plain text.
Re: [venv] A shell-agnostic "activate" Carl Meyer 3/31/12 8:32 PM
On 03/31/2012 11:07 AM, datagrok wrote:
> I wrote a brief diatribe, "Virtualenv's bin/activate is Doing It Wrong,"
> documenting an approach to activating virtual environments that avoids
> sourcing bin/activate into one's shell, does not require multiple
> scripts to be maintained for every possible (unix) shell, and avoids
> several other annoyances I have encountered with
> virtualenv's |bin/activate| script:
>
> https://gist.github.com/2199506
>
> I posted it as an issue on the virtualenv Github page and I'm looking
> for some more feedback about the idea. I've already made some
> improvements based on the feedback I received there, but the more the
> better. Maybe I can change this from a frustrated rant into a useful
> improvement for the community.
>
> https://github.com/pypa/virtualenv/issues/247

Seems convincing to me that this is a cleaner way to handle shell
activation, and reduces the activation script maintenance burden as
well. As mentioned in the ticket, I'm inclined to add it under a
different name and then consider deprecating the existing activation
scripts over a period of several releases. I'd be very interested in
hearing from anyone who has an argument in favor of the current
activation scripts over subshell activation.

Carl

Re: A shell-agnostic "activate" Marcus Smith 3/31/12 10:35 PM
+1.
looks better and simpler than the current activate/deactivate stuff.

Marcus
Re: [venv] A shell-agnostic "activate" Hal 4/2/12 10:04 PM


On Saturday, March 31, 2012 8:32:39 PM UTC-7, Carl Meyer wrote:

Seems convincing to me that this is a cleaner way to handle shell
activation, and reduces the activation script maintenance burden as
well. 

I'd be very interested in


hearing from anyone who has an argument in favor of the current
activation scripts over subshell activation.

Okay - this would break my workflow. :)

AIUI, the subshell approach forces me into a "stack" of virtual envs, while I use a fanout model currently. Here's a typical scenario for me - I keep one terminal window open for my various "python projects", and activate/deactivate their associated venvs as I switch work. For each venv, I might have a couple of vim buffers open for each venv. (Messy, but it's the way I work)

Currently, I can move between the venvs in any order I choose - and those editor buffers are always available. With the subshell approach, I'd be constrained to a stack of venvs. Worse, my editor buffers are hidden - I have to abandon my current set of buffers. I can't speak to how much maintenance effort this change would save. That's certainly a major consideration. If I read the OP's blog correctly, one can wrap the existing code with the subshell approach. I'm not sure the reverse is true.

Assuming you move ahead with this, I think it's a major change, and worthy of bumping the major version number. Thanks for your work on virtualenv!

--Hal

Re: [venv] A shell-agnostic "activate" Doug Hellmann 4/3/12 4:56 AM

On Apr 3, 2012, at 1:04 AM, Hal wrote:



On Saturday, March 31, 2012 8:32:39 PM UTC-7, Carl Meyer wrote:

Seems convincing to me that this is a cleaner way to handle shell
activation, and reduces the activation script maintenance burden as
well. 

I'd be very interested in
hearing from anyone who has an argument in favor of the current
activation scripts over subshell activation.

Okay - this would break my workflow. :)

This change would completely break the "workon" command from virtualenvwrapper and make virtualenv nearly useless for me.

A huge part of the appeal of virtualenv is that it *does* affect my current shell, and I *don't* have to start a new process. I can switch between projects and update not only the runtime environment for commands run in that shell (maintaining my command history), but also replace the buffers in my editor, start and stop services, and all manner of other triggered actions (some of which *also* change the settings of the current shell).

By all means add this as another way to activate an environment, but please do not remove the existing behavior in the process.


AIUI, the subshell approach forces me into a "stack" of virtual envs, while I use a fanout model currently. Here's a typical scenario for me - I keep one terminal window open for my various "python projects", and activate/deactivate their associated venvs as I switch work. For each venv, I might have a couple of vim buffers open for each venv. (Messy, but it's the way I work)

Currently, I can move between the venvs in any order I choose - and those editor buffers are always available. With the subshell approach, I'd be constrained to a stack of venvs. Worse, my editor buffers are hidden - I have to abandon my current set of buffers. I can't speak to how much maintenance effort this change would save. That's certainly a major consideration. If I read the OP's blog correctly, one can wrap the existing code with the subshell approach. I'm not sure the reverse is true.

Assuming you move ahead with this, I think it's a major change, and worthy of bumping the major version number. Thanks for your work on virtualenv!

--Hal


--
You received this message because you are subscribed to the Google Groups "virtualenv" group.
To view this discussion on the web visit https://groups.google.com/d/msg/python-virtualenv/-/cDPev3NLbk4J.
To post to this group, send email to python-v...@googlegroups.com.
To unsubscribe from this group, send email to python-virtual...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/python-virtualenv?hl=en.

Re: [venv] A shell-agnostic "activate" datagrok 4/3/12 9:28 AM
I keep one terminal window open for my various "python projects", and activate/deactivate their associated venvs as I switch work. For each venv, I might have a couple of vim buffers open for each venv.
Currently, I can move between the venvs in any order I choose - and those editor buffers are always available. With the subshell approach, I'd be constrained to a stack of venvs.

Can you describe this a bit more for me? Or tell me if I'm understanding your workflow correctly: you use (terminal-mode) vim (not gvim), you open some buffers, then you background vim, activate/deactivate other virtual environments, resume vim and keep working? I can't quite see how employing a subshell would hide your buffers. (FWIW I'm a vim user as well.)

Do you typically not "deactivate" when moving between venvs?
Re: [venv] A shell-agnostic "activate" datagrok 4/3/12 9:47 AM
This change would completely break the "workon" command from virtualenvwrapper and make virtualenv nearly useless for me.

"Completely break" because virtualenvwrapper's tools are defined as shell functions that don't get exported to the subshell?

I can switch between projects and update not only the runtime environment for commands run in that shell (maintaining my command history), but also replace the buffers in my editor, start and stop services, and all manner of other triggered actions (some of which *also* change the settings of the current shell).

I can imagine ways to implement "workon"'s functionality, including triggering actions upon activation, in a way that works with the subshell method.

Maintaining shell history is indeed a bit difficult, but The Internets tell me it can be done: http://ptspts.blogspot.com/2011/03/how-to-automatically-synchronize-shell.html

What mechanism do you employ to replace buffers in your editor?
 
By all means add this as another way to activate an environment, but please do not remove the existing behavior in the process.

I certainly don't intend to break anyone's existing workflow. None of my comments above should be taken to mean that I disagree with you about this. :-)
Re: [venv] A shell-agnostic "activate" Doug Hellmann 4/3/12 4:47 PM

On Apr 3, 2012, at 12:47 PM, datagrok wrote:

This change would completely break the "workon" command from virtualenvwrapper and make virtualenv nearly useless for me.

"Completely break" because virtualenvwrapper's tools are defined as shell functions that don't get exported to the subshell?

"Completely break" because workon changes the settings of the current shell and so switching between environments is inexpensive and retains history. 

Always creating a new subshell would mean that the common pattern of:

$ workon env1
… do something ...
$ workon env2
… do something ...
$ workon env3
… do something ...

would create 3 new subshells, each with its own command history. Following that with a "workon env1" would not restore the state of the shell to be pointing to env1 but would create a fourth new subshell instead. Logging out of a system after using workon an arbitrary number of times would then also require the same number of "exit" calls (or whatever) to quit all of the nested processes.

I can't see a way that a workon command could safely deactivate one environment before activating a new one without leaving one of those intermediate shell processes in an invalid or inconsistent state. 

I can switch between projects and update not only the runtime environment for commands run in that shell (maintaining my command history), but also replace the buffers in my editor, start and stop services, and all manner of other triggered actions (some of which *also* change the settings of the current shell).

I can imagine ways to implement "workon"'s functionality, including triggering actions upon activation, in a way that works with the subshell method.

Triggering actions on activation is the easy case. :-)


Maintaining shell history is indeed a bit difficult, but The Internets tell me it can be done: http://ptspts.blogspot.com/2011/03/how-to-automatically-synchronize-shell.html

What mechanism do you employ to replace buffers in your editor?

I tell emacs to save a desktop file in the predeactivate hook and then tell it to load a new one in the postactivate hook. By using a path for that file relative to the virtualenv root, I can have a separate desktop file for each environment. IDEs usually have similar "project" files which could be loaded and unloaded in a similar way.

By all means add this as another way to activate an environment, but please do not remove the existing behavior in the process.

I certainly don't intend to break anyone's existing workflow. None of my comments above should be taken to mean that I disagree with you about this. :-)

Good! :-)


--
You received this message because you are subscribed to the Google Groups "virtualenv" group.
To view this discussion on the web visit https://groups.google.com/d/msg/python-virtualenv/-/8tLiJDq7WloJ.

To post to this group, send email to python-v...@googlegroups.com.
To unsubscribe from this group, send email to python-virtual...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/python-virtualenv?hl=en.

Re: [venv] A shell-agnostic "activate" Carl Meyer 4/3/12 4:54 PM
On 04/03/2012 05:47 PM, Doug Hellmann wrote:
>>     This change would completely break the "workon" command from
>>     virtualenvwrapper and make virtualenv nearly useless for me.
>>
>>
>> "Completely break" because virtualenvwrapper's tools are defined as
>> shell functions that don't get exported to the subshell?
>
> "Completely break" because workon changes the settings of the current
> shell and so switching between environments is inexpensive and retains
> history.
>
> Always creating a new subshell would mean that the common pattern of:
>
> $ workon env1
> … do something ...
> $ workon env2
> … do something ...
> $ workon env3
> … do something ...
>
> would create 3 new subshells, each with its own command history.
> Following that with a "workon env1" would not restore the state of the
> shell to be pointing to env1 but would create a fourth new subshell
> instead. Logging out of a system after using workon an arbitrary number
> of times would then also require the same number of "exit" calls (or
> whatever) to quit all of the nested processes.
>
> I can't see a way that a workon command could safely deactivate one
> environment before activating a new one without leaving one of those
> intermediate shell processes in an invalid or inconsistent state.

Thanks, this is exactly the kind of feedback I was looking for. I wasn't
sure how onerous it would be to adjust virtualenvwrapper to subshell
activation - sounds like it would be a serious problem. I definitively
have no interest in making life difficult for virtualenvwrapper, so the
existing activate scripts will stay.

The remaining question is, since from my perspective much of the benefit
of subshell activation would have been to simplify maintenance of all
these different activate scripts, if we don't gain that, is there enough
remaining rationale to add it at all?

Carl

Re: [venv] A shell-agnostic "activate" datagrok 4/3/12 8:46 PM

The remaining question is, since from my perspective much of the benefit
of subshell activation would have been to simplify maintenance of all
these different activate scripts, if we don't gain that, is there enough
remaining rationale to add it at all?

Oh ho I'm not giving up just yet. With the benefit of the feedback everyone has provided so far, I've had another whack at the problem. I've got a script in progress that, while a bit more complex than my earlier the 5-line subshell launcher, should be able to support both the "subshell" and the "source"-style workflows, and both unix and windows platforms.

My new goal is to move all of the logic currently duplicated among the activate.* scripts into a Python script that knows just enough of your shell's language to set and unset environment variables. The 'activate.sh' script could become a one-liner that 'eval's the output of the python script. I'm hoping this allows me to add the subshell capability, reducing logic duplication between the activate scripts, with no backwards incompatibility for anybody.

I'll follow up when it's good enough to try out, with clearer documentation of its use. Rough work-in-progress:

Re: [venv] A shell-agnostic "activate" Jakub Vysoky 4/4/12 1:38 AM
hello,

this is amazing. i always had feeling we should do something with
those scripts. having multiple versions (bash, csh, windows) works,
but it is ugly.

from similar reason i started project rvritualenv [1] with different
approach than the original. i just disagree with some of virtualenv's
design decisions, but that is for another discussion.

[1] https://github.com/kvbik/rvirtualenv

i do not need activate scripts at all, but it is a convenient
entry point for virtualenvs. virtualenvwrapper use this and it is a nice
piece of software (you can also use rvirtualenv with virtualenvwrapper).

i do something similar you described but in a very simpler manner.
i just generate activate scripts in python using the original ones
(only for bash and windows) as templates. not saying you should do the
same - i am looking forward with what you'll come out!

will you be intersted in adding you work to rvirtualenv as well? it would
be really nice combination ;).

to add an opinion and not just advertising my project. i think you need
always both approaches (subshell and sourcing). even ssh-agent mentioned
by you has a wrapper keychain, that generates proper activation scripts
for you and you can just source them eg in you .bashrc.

cheers, jakub.

> --
> You received this message because you are subscribed to the Google Groups
> "virtualenv" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/python-virtualenv/-/KlsG-irsztsJ.

>
> To post to this group, send email to python-v...@googlegroups.com.
> To unsubscribe from this group, send email to
> python-virtual...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/python-virtualenv?hl=en.

--
Jakub Vysoky

mob: +420 605 852 377
jab: jakub....@gmail.com
twit: https://twitter.com/kvbik

Re: [venv] A shell-agnostic "activate" Doug Hellmann 4/4/12 4:53 AM

This sounds like a better approach, and is similar to what I do with the hook loader in virtualenvwrapper. One thing to watch out for is the Python interpreter startup time is longer than a typical shell. I get complaints about the initialization process for virtualenvwrapper.sh from time to time, but I'm also using pkg_resources to load plugins so that may be part of the problem.

Doug

Re: [venv] A shell-agnostic "activate" datagrok 4/4/12 7:33 PM
On Wednesday, April 4, 2012 4:38:23 AM UTC-4, Jakub Vysoky wrote:
hello,

this is amazing.

 

will you be intersted in adding you work to rvirtualenv as well? it would be really nice combination ;).

Absolutely! I'll try it out soon, as soon as my virtualenv patch is ready.
 

to add an opinion and not just advertising my project.

On the contrary, I'm glad you introduced me to it. Thank you!

Re: [venv] A shell-agnostic "activate" Hal 4/5/12 4:02 PM

I tend to run multiple copies of vim (terminal, not gvim), each having the buffer set for that project. The vim process is part of the invoking shell's process group, and pushing into a new subshell creates a new process group which owns the terminal. There's no way to give control back to the original process group until the subshell exits. Here's how to recreate:
  $ function show() { echo "$SHLVL job list:" ; jobs; }
  $ export -f show
  $ vim /tmp/file & # launch vim in background
  $ show
  $ bash # go into subshell
  $ show # can't get to vim job
  $ exit
  $ show
  $ fg # return to vim session

Do you typically not "deactivate" when moving between venvs?
 
Yes (as virtualenv's "workon" command does that for me). As Doug pointed out, that only changes some environment variables - it leaves my vim process(es) alone.