Accessing pip from inside a python program

[Arthur Goldberg]

Hi

We're building utilities to create portable test environments on a continuous integration and delivery platform (CircleCI). We want our packages to run on Python 2 and 3 (2.7 and 3.5, currently). We've developed build utilities that creates these environments, runs tests, and then saves the results. 

We use pip to create environments from requirements.txt files in the packages. But I'm finding it tricky to use pip and obtain good error handling. For example, this approach

        with open('requirements.txt', 'r') as file:

            reqs = [line.rstrip() for line in file.readlines()]

            pip.main(['install'] + reqs)

raises exceptions for lines in requirements.txt  that just contain comments.

Is there a recommended way to use pip, especially pip install, from within a program and obtain good error handling?

Thanks

Arthur

[Paul Moore]

On 13 October 2016 at 15:27, Arthur Goldberg <artgo...@gmail.com> wrote:
> Is there a recommended way to use pip, especially pip install, from within a
> program and obtain good error handling?

Nothing supported, sorry. The only supported interface is the command
line, so you can use subprocess.Popen to run pip in a subprocess. But
the pip API is for internal use only. (In particular, I'm not
surprised there's not a good interface for trapping errors from pip
APIs, it's not a usage we particularly consider). We're not against
*having* a supported API, it's just that no-one has put in the work to
define such a thing (and error handling would be an obvious thing we'd
need to consider :-)) or document and implement it (and create tests
for it) yet.

I assume, by the way, that you are aware of the risks of installing
new packages into site-packages from within a Python process? Most of
the time you should be OK, but Python does cache information within
the import system, and you do run the risk of getting stale data.
That's not a pip-specific issue, though, so it's something you should
be aware of regardless of how you call pip.

Paul

[Nathaniel Smith]

I think the only supported option is

subprocess.run(["pip", "install", "-r", "requirements.txt"])

though I suppose you could try passing that to pip.main as well.

-n

--
Nathaniel J. Smith -- https://vorpus.org

[Arthur Goldberg]

Thanks Paul and Nathaniel. It would be nice if some people built a pip API; unfortunately, I don't have the time to contribute.

Arthur