Notes on using vertualenvs and pinned versions

[Jim Fulton]

Yesterday, I had to wrestle with a buildout that had pinned versions and that was to be installed using a virtualenv.  The issue was that the versions of buildout and setuptools (and other setuptools dependencies) were bound to conflict with the project's pins.

I ended up dealing with this as follows:

- Created a venv that was not the buildout directory.

- Install buildout in the venv.

- Use that buildout to bootstrap the project buildout.

- Run the project buildout which "upgraded" (downgraded :)) itself to the pinned versions.

I think this is a workable scheme, although a bit more work may be needed to make it work reliably. (In particular, I think the upgrade code needs to consider all of buildout's transitive requirements. https://github.com/buildout/buildout/issues/336)

I also was defeated by a setuptools bug: 

  https://github.com/pypa/setuptools/issues/997

that I had to work around in a rather hackish way.  I made a setuptools PR with a fix.

An alternative approach I considered was to provide a requirements.txt file to control the versions in the versions used in the venv.  This would have made version pinning more complicated because versions would be pinned via 2 separate mechanisms.  I considered making buildout understand requirements.txt, but that would have been too deep of a rabbit hole, at least at the time, and would not address all of the issues.

Jim

--

Jim Fulton
http://jimfulton.info

[Jim Fulton]

I was just reminded of another issue I wanted to mention.

Extensions are loaded before upgrading and thus before upgrading can conform to pins, but extensions are still held to pins.  The project I was using has mr.developer as an extension and I had to disable it while upgrading.

Jim

[Dylan Jay]

Ive used this before as a reliable bootstrap.py free bootstrap method.

``` virtualenv . - bin/pip install --upgrade pip zc.buildout
bin/buildout -c travis.cfg annotate | tee annotate.txt | grep -E 'setuptools= |zc.buildout= ' | sed 's/= /==/' > requirements.txt
cat annotate.txt
cat requirements.txt - bin/pip install --upgrade -r requirements.txt
bin/buildout -qNc travis.cfg
```

What would be nice is if buildout can be used generate a requirements.txt so it would be

```
virtualenv .
pip install zc.buildout
pip install `bin/buildout requirements`
bin/buildout
```

--
You received this message because you are subscribed to the Google Groups "Buildout Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to buildout-develop...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jim Fulton]

On Sat, Mar 25, 2017 at 2:42 AM, Dylan Jay <dj...@pretaweb.com> wrote:

Ive used this before as a reliable bootstrap.py free bootstrap method.

``` virtualenv . - bin/pip install --upgrade pip zc.buildout
bin/buildout -c travis.cfg annotate | tee annotate.txt | grep -E 'setuptools= |zc.buildout= ' | sed 's/= /==/' > requirements.txt
cat annotate.txt
cat requirements.txt - bin/pip install --upgrade -r requirements.txt
bin/buildout -qNc travis.cfg
```

What would be nice is if buildout can be used generate a requirements.txt so it would be

```
virtualenv .
pip install zc.buildout
pip install `bin/buildout requirements`
bin/buildout
```

Where would the data for the hypothetical requirements command come from?

What would the output contain?

Jim

[Dylan Jay]

It is just doing an annotate to exact any pinned zc.buildout and setuptools pins. 

So really its a more an 'initrequiments'.

Same as the first code just without the need for sed. 

Jim