Running a subprocess in a venv

[Larry Martell]

I have a python script, and from that I want to run another script in
a subprocess in a venv. What is the best way to do that? I could write
a file that activates the venv then runs the script, then run that
file, but that seems messy. Is there a better way?

[Johannes Findeisen]

How do you do that? It sounds messy but not wrong...

I would activate the venv and then run my Python script. In the Python
script you can call another python script in a subprocess like this:

import sys
import subprocess

# https://docs.python.org/3/library/subprocess.html#popen-constructor
proc = subprocess.Popen([sys.executable, "/path/to/an/otherscript.py"])

# https://docs.python.org/3/library/subprocess.html#popen-objects
# Do your process communication/handling... proc.communicate(),
# proc.wait(), proc.terminate(), proc.kill() etc.

Is this the answer you are looking for?

Detailed docs: https://docs.python.org/3/library/subprocess.html

Regards,
Johannes

[Roel Schroeven]

Larry Martell via Python-list schreef op 21/10/2023 om 15:01:

> I have a python script, and from that I want to run another script in
> a subprocess in a venv. What is the best way to do that? I could write
> a file that activates the venv then runs the script, then run that
> file, but that seems messy. Is there a better way?

Activating a venv it is practical when you're working in a shell, but
not actually needed. You can execute the python in the venv with the
script as parameter.

Have a look in the venv directory: there will be a Script subdirectory
(on Windows) or bin subdirectory (on Unix-like systems). Within that
directory are several executables, one of which will be python or
python3. That's the one you need.

So use something like

    subprocess.run(['/path/to/venv/bin/python3', 'yourscript.py',
possible other arguments])

--
"Binnen een begrensde ruimte ligt een kritiek punt, waar voorbij de vrijheid
afneemt naarmate het aantal individuen stijgt. Dit gaat evenzeer op voor mensen
in de begrensde ruimte van een planetair ecosysteem, als voor de gasmoleculen
in een hermetisch gesloten vat. Bij mensen is het niet de vraag hoeveel er
maximaal in leven kunnen blijven in het systeem, maar wat voor soort bestaan
mogelijk is voor diegenen die in leven blijven.
-- Pardot Kynes, eerste planetoloog van Arrakis"
-- Frank Herbert, Duin

[Larry Martell]

On Sat, Oct 21, 2023 at 9:49 AM Johannes Findeisen <mai...@hanez.org> wrote:
>
> On Sat, 21 Oct 2023 09:01:18 -0400
> Larry Martell via Python-list <pytho...@python.org> wrote:
>

> > I have a python script, and from that I want to run another script in
> > a subprocess in a venv. What is the best way to do that? I could write
> > a file that activates the venv then runs the script, then run that
> > file, but that seems messy. Is there a better way?
>

> How do you do that?

How? Open a file and write the commands I need then invoke that.

> It sounds messy but not wrong...
>
> I would activate the venv and then run my Python script. In the Python
> script you can call another python script in a subprocess like this:
>
> import sys
> import subprocess
>
> # https://docs.python.org/3/library/subprocess.html#popen-constructor
> proc = subprocess.Popen([sys.executable, "/path/to/an/otherscript.py"])
>
> # https://docs.python.org/3/library/subprocess.html#popen-objects
> # Do your process communication/handling... proc.communicate(),
> # proc.wait(), proc.terminate(), proc.kill() etc.
>
> Is this the answer you are looking for?
>
> Detailed docs: https://docs.python.org/3/library/subprocess.html

I know how to use Popen. What I was missing was running the script
using sys.executable. Thanks.

[Johannes Findeisen]

sys.executable is the path to the actual Python binary, e.g.
"/usr/bin/python". You could add "/usr/bin/python" there manually but
this is not portable to Windows for example.

When you add a shebang line to your other script and the file is
executable, you may not need to add sys.executable as first argument to
Popen but using sys.executable is the most reliable way to do this... ;)

Regards,
Johannes

[Larry Martell]

I need the path to whichever venv is being used so sys.executable works for me.

[Thomas Passin]

On 10/21/2023 11:32 AM, Larry Martell via Python-list wrote:
> On Sat, Oct 21, 2023 at 9:49 AM Johannes Findeisen <mai...@hanez.org> wrote:
>>
>> On Sat, 21 Oct 2023 09:01:18 -0400
>> Larry Martell via Python-list <pytho...@python.org> wrote:
>>
>>> I have a python script, and from that I want to run another script in
>>> a subprocess in a venv. What is the best way to do that? I could write
>>> a file that activates the venv then runs the script, then run that
>>> file, but that seems messy. Is there a better way?
>>
>> How do you do that?
>
> How? Open a file and write the commands I need then invoke that.
>
>> It sounds messy but not wrong...
>>
>> I would activate the venv and then run my Python script. In the Python
>> script you can call another python script in a subprocess like this:
>>
>> import sys
>> import subprocess
>>
>> # https://docs.python.org/3/library/subprocess.html#popen-constructor
>> proc = subprocess.Popen([sys.executable, "/path/to/an/otherscript.py"])
>>
>> # https://docs.python.org/3/library/subprocess.html#popen-objects
>> # Do your process communication/handling... proc.communicate(),
>> # proc.wait(), proc.terminate(), proc.kill() etc.
>>
>> Is this the answer you are looking for?
>>
>> Detailed docs: https://docs.python.org/3/library/subprocess.html
>
> I know how to use Popen. What I was missing was running the script
> using sys.executable. Thanks.

A nice feature of using sys.executable is that you automatically use the
same Python installation as your invoking program is running with. On a
system that has several different Python installations, that's a very
good thing.

[Mats Wichmann]

On 10/21/23 07:01, Larry Martell via Python-list wrote:
> I have a python script, and from that I want to run another script in
> a subprocess in a venv. What is the best way to do that? I could write
> a file that activates the venv then runs the script, then run that
> file, but that seems messy. Is there a better way?

You don't need to "activate" a virtualenv. The activation script does
some helpful things along the way (setup and cleanup) but none of them
are required. The most important thing it does is basically:

VIRTUAL_ENV='path-where-you-put-the-virtualenv'
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

and that's really only so that commands that belong to that virtualenv
(python, pip, and things where you installed a package in the venv wich
creates an "executable" in bin/) are in a directory first in your search
path. As long as you deal with necessary paths yourself, you're fine
without activating. So as mentioned elsewhere, just use the path to the
virtualenv's Python and you're good to go.