Installing Trac and Trac plugins in wheel format

75 views
Skip to first unread message

RjOllos

unread,
Feb 21, 2017, 6:44:51 PM2/21/17
to Trac Development
I've been experimenting how to install Trac and plugins in wheel format using pip, and encountering various nuances. It appears that the egg cache is not needed when installing wheels, which would simplify the configuration (1).

I've found no issues if a wheel is available on PyPI. However, it gets more complex when installing from a tarball/zip or installing from a repository. I haven't found a single pip command that will force install of a wheel when installing from a tarball/zip or repository. The workaround I described in (2) is to first create the wheels, then install the wheels.

It gets more complex if the installation requirements are more complex. For example, I want to install Genshi 0.6 with Trac 1.2 to workaround #11184 (3). There are two complications here:
* There is no wheel for Genshi 0.6 on PyPI
* If steps other than those described below are followed, it's easy to end up with Genshi 0.7.
* It's easy to end up with Genshi compiled without speedups (I still haven't solved this for the case of using requirements.txt)

Create virtualenv:

~/tmp$virtualenv pve
New python executable in /
Users/rjollos/tmp/pve/bin/python2.7
Also creating executable in /Users/rjollos/tmp/pve/bin/python
Installing setuptools, pip, wheel...pidone.
(pve) ~/tmp$pip install -U pip setuptools wheel
Collecting pip
  Using cached pip-9.0.1-py2.py3-none-any.whl
Collecting setuptools
  Using cached setuptools-34.2.0-py2.py3-none-any.whl
Collecting wheel
  Using cached wheel-0.29.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools)
  Using cached packaging-16.8-py2.py3-none-any.whl
Collecting six>=1.6.0 (from setuptools)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools)
  Using cached appdirs-1.4.0-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools)
  Using cached pyparsing-2.1.10-py2.py3-none-any.whl
Installing collected packages: pip, six, pyparsing, packaging, appdirs, setuptools, wheel
  Found existing installation: pip 8.0.2
    Uninstalling pip-8.0.2:
      Successfully uninstalled pip-8.0.2
  Found existing installation: setuptools 19.6.2
    Uninstalling setuptools-19.6.2:
      Successfully uninstalled setuptools-19.6.2
  Found existing installation: wheel 0.26.0
    Uninstalling wheel-0.26.0:
      Successfully uninstalled wheel-0.26.0
Successfully installed appdirs-1.4.0 packaging-16.8 pip-9.0.1 pyparsing-2.1.10 setuptools-34.2.0 six-1.10.0 wheel-0.29.0
(pve) ~/
tmp$python --version
Python 2.7.13

Create wheels from packages specified in requirements.txt:

(pve) ~/tmp$cat requirements.txt
genshi==0.6
trac
(pve) ~/
tmp$pip wheel --no-cache-dir --wheel-dir wheels -r requirements.txt
Collecting genshi==0.6 (from -r requirements.txt (line 1))
 
Downloading Genshi-0.6.tar.gz (433kB)
   
100% |████████████████████████████████| 440kB 1.7MB/s
Collecting trac (from -r requirements.txt (line 2))
 
Downloading Trac-1.2-py2-none-any.whl (3.8MB)
   
100% |████████████████████████████████| 3.8MB 1.5MB/s
 
Saved ./wheels/Trac-1.2-py2-none-any.whl
Collecting setuptools>=0.6 (from trac->-r requirements.txt (line 2))
 
Downloading setuptools-34.2.0-py2.py3-none-any.whl (389kB)
   
100% |████████████████████████████████| 399kB 1.8MB/s
 
Saved ./wheels/setuptools-34.2.0-py2.py3-none-any.whl
Collecting six>=1.6.0 (from setuptools>=0.6->trac->-r requirements.txt (line 2))
 
Downloading six-1.10.0-py2.py3-none-any.whl
 
Saved ./wheels/six-1.10.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools>=0.6->trac->-r requirements.txt (line 2))
 
Downloading packaging-16.8-py2.py3-none-any.whl
 
Saved ./wheels/packaging-16.8-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools>=0.6->trac->-r requirements.txt (line 2))
 
Downloading appdirs-1.4.0-py2.py3-none-any.whl
 
Saved ./wheels/appdirs-1.4.0-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools>=0.6->trac->-r requirements.txt (line 2))
 
Downloading pyparsing-2.1.10-py2.py3-none-any.whl (56kB)
   
100% |████████████████████████████████| 61kB 2.3MB/s
 
Saved ./wheels/pyparsing-2.1.10-py2.py3-none-any.whl
Skipping trac, due to already being wheel.
Skipping setuptools, due to already being wheel.
Skipping six, due to already being wheel.
Skipping packaging, due to already being wheel.
Skipping appdirs, due to already being wheel.
Skipping pyparsing, due to already being wheel.
Building wheels for collected packages: genshi
 
Running setup.py bdist_wheel for genshi ... done
 
Stored in directory: /Users/rjollos/tmp/wheels
Successfully built genshi
(pve) ~/tmp$ls wheels/
Genshi-0.6-py2-none-any.whl pyparsing-2.1.10-py2.py3-none-any.whl
Trac-1.2-py2-none-any.whl setuptools-34.2.0-py2.py3-none-any.whl
appdirs
-1.4.0-py2.py3-none-any.whl six-1.10.0-py2.py3-none-any.whl
packaging
-16.8-py2.py3-none-any.whl

I tried to install with "wheel install", but tracd and trac-admin don't get installed. Oddly, there is no output from "wheel install".

(pve) ~/tmp$pip freeze
appdirs==1.4.0
packaging==16.8
pyparsing==2.1.10
six==1.10.0
(pve) ~/
tmp$wheel install --wheel-dir wheels -r requirements.txt
(pve) ~/tmp$pip freeze
appdirs
==1.4.0
Genshi==0.6
packaging
==16.8
pyparsing
==2.1.10
six
==1.10.0
Trac==1.2(pve)
(pve) ~/tmp$ls -d pve/lib/python2.7/site-packages/*.egg-info
ls: pve/lib/python2.7/site-packages/*.egg-info: No such file or directory
(pve) ~/tmp$ls -d pve/lib/python2.7/site-packages/*.dist-info
pve/lib/python2.7/site-packages/Genshi-0.6.dist-info
pve/lib/python2.7/site-packages/Trac-1.2.dist-info
pve/lib/python2.7/site-packages/appdirs-1.4.0.dist-info
pve/lib/python2.7/site-packages/packaging-16.8.dist-info
pve/lib/python2.7/site-packages/pip-9.0.1.dist-info
pve/lib/python2.7/site-packages/pyparsing-2.1.10.dist-info
pve/lib/python2.7/site-packages/setuptools-34.2.0.dist-info
pve/lib/python2.7/site-packages/six-1.10.0.dist-info
pve/lib/python2.7/site-packages/wheel-0.30.0a0.dist-info
(pve) ~/tmp$trac-admin trac initenv
-bash: trac-admin: command not found
(pve) ~/tmp$ls pve/bin/
activate easy_install-2.7 python-config
activate.csh pip python2
activate.fish pip2 python2.7
activate_this.py pip2.7 wheel
easy_install python

"pip install" does work, by pointing to the wheels dir:

(pve) ~/tmp$pip install wheels/*.whl
Processing ./wheels/Genshi-0.6-py2-none-any.whl
Processing ./wheels/Trac-1.2-py2-none-any.whl
Requirement already satisfied: appdirs==1.4.0 from file:///Users/rjollos/tmp/wheels/appdirs-1.4.0-py2.py3-none-any.whl in ./pve/lib/python2.7/site-packages
Requirement already satisfied: packaging==16.8 from file:///Users/rjollos/tmp/wheels/packaging-16.8-py2.py3-none-any.whl in ./pve/lib/python2.7/site-packages
Requirement already satisfied: pyparsing==2.1.10 from file:///Users/rjollos/tmp/wheels/pyparsing-2.1.10-py2.py3-none-any.whl in ./pve/lib/python2.7/site-packages
Requirement already satisfied: setuptools==34.2.0 from file:///Users/rjollos/tmp/wheels/setuptools-34.2.0-py2.py3-none-any.whl in ./pve/lib/python2.7/site-packages
Requirement already satisfied: six==1.10.0 from file:///Users/rjollos/tmp/wheels/six-1.10.0-py2.py3-none-any.whl in ./pve/lib/python2.7/site-packages
Installing collected packages: Genshi, Trac
Successfully installed Genshi-0.6 Trac-1.2


The key points seem to be:
* Use a requirements.txt if you need to pin installation versions. It's easy to end up with Genshi 0.7 by running "pip wheel .. genshi==0.6" followed by "pip wheel ... trac". The latter command will also download Genshi 0.7 and build a wheel, unless "--no-deps" is used.
* Use "pip wheel" followed by "pip install" to avoid getting eggs when the wheel isn't available on PyPI, or when installing from a repository.

With these steps I was able to get all the packages installed to trac-hacks.org in the wheel format, and the somewhat complex requirements.txt:

pytz
babel
docutils
psycopg2
pygments
httplib2
oauth2
pillow
mollom
spambayes
dnspython
genshi
==0.6
trac
svn
+https://trac-hacks.org/svn/changelogmacro/trunk/
svn
+https://trac-hacks.org/svn/footnotemacro/trunk/
svn
+https://trac-hacks.org/svn/svnauthzadminplugin/1.0/
svn
+https://trac-hacks.org/svn/accountmanagerplugin/trunk/
svn
+https://trac-hacks.org/svn/acronymsplugin/0.11/
svn
+https://trac-hacks.org/svn/autowikifyplugin/trunk/
svn
+https://trac-hacks.org/svn/extracturlplugin/0.11/
svn
+https://trac-hacks.org/svn/fullblogplugin/0.11/
svn
+https://trac-hacks.org/svn/includemacro/trunk/
svn
+https://trac-hacks.org/svn/traciniadminpanelplugin/trunk/
svn
+https://trac-hacks.org/svn/newsflashmacro/0.11/
svn
+https://trac-hacks.org/svn/pollmacro/trunk/
svn
+https://trac-hacks.org/svn/serversideredirectplugin/1.0/
svn
+https://svn.edgewall.org/repos/trac/plugins/1.2/spam-filter/
svn
+https://trac-hacks.org/svn/tagsplugin/trunk/
svn
+https://trac-hacks.org/svn/tocmacro/0.11/
svn
+https://trac-hacks.org/svn/voteplugin/trunk/
svn
+https://trac-hacks.org/svn/wikiextrasplugin/trunk/
svn
+https://trac-hacks.org/svn/xmlrpcplugin/trunk/
svn
+https://trac-hacks.org/svn/wikiautocompleteplugin/trunk/
svn
+https://trac-hacks.org/svn/trachacksplugin/trunk/

Please let me know if you have any other tips, or advice on accomplishing (2).

I'm thinking it would be good to create a TracInstallVirtualEnv recipe that describes how to install to a virtual environment using pip and wheels.

- Ryan


Thomas Moschny

unread,
Feb 22, 2017, 6:55:58 AM2/22/17
to trac...@googlegroups.com
Using your requirements.txt, things worked well here:

# virtualenv t
# source t/bin/activate
# pip install -U pip setuptools wheel
# pip wheel -w wheels -r requirements.txt
# pip wheel -w wheels --global-option=--with-speedups genshi==0.6
# rm -vf wheels/Genshi-0.6-py2-none-any.whl
# pip install wheels/*.whl

In theory, one should be able to add global options to the
requirements.txt file:

...
genshi==0.6 --global-option=--with-speedups
...

but in my tests that didn't work - it is honored by "pip install", but
ignored by "pip wheel". Also, it has unwanted global effects, see
https://github.com/pypa/pip/issues/4118.


The more general question is, whether it is really an issue if
packages are installed from a wheel or as egg. Using this recipe (and
your unmodified requirements.txt) to directly installing packages

# virtualenv t
# source t/bin/activate
# pip install -U pip setuptools wheel
# pip install --global-option=--with-speedups genshi==0.6
# pip install -r requirements.txt

will not put any zipped eggs in t/lib/python2.7/site-packages, so I
guess in that case also no egg cache will be used?

- Thomas

RjOllos

unread,
Feb 22, 2017, 12:26:32 PM2/22/17
to Trac Development


On Wednesday, February 22, 2017 at 3:55:58 AM UTC-8, Thomas Moschny wrote:
Using your requirements.txt, things worked well here:

# virtualenv t
# source t/bin/activate
# pip install -U pip setuptools wheel
# pip wheel -w wheels -r requirements.txt
# pip wheel -w wheels --global-option=--with-speedups genshi==0.6
# rm -vf wheels/Genshi-0.6-py2-none-any.whl
# pip install wheels/*.whl

In theory, one should be able to add global options to the
requirements.txt file:

...
genshi==0.6 --global-option=--with-speedups
...

but in my tests that didn't work - it is honored by "pip install", but
ignored by "pip wheel". Also, it has unwanted global effects, see
https://github.com/pypa/pip/issues/4118.

Thanks, I was seeing the same behavior, but was unsure why. If we could make a whl available on PyPI that would avoid the need for using the --global-option flag.
 
The more general question is, whether it is really an issue if
packages are installed from a wheel or as egg. Using this recipe (and
your unmodified requirements.txt) to directly installing packages

# virtualenv t
# source t/bin/activate
# pip install -U pip setuptools wheel
# pip install --global-option=--with-speedups genshi==0.6
# pip install -r requirements.txt

will not put any zipped eggs in t/lib/python2.7/site-packages, so I
guess in that case also no egg cache will be used?

I'm also assuming no egg cache is necessary.

- Ryan
 

RjOllos

unread,
Feb 28, 2017, 4:20:11 PM2/28/17
to Trac Development


On Tuesday, February 21, 2017 at 3:44:51 PM UTC-8, RjOllos wrote:
I tried to install with "wheel install", but tracd and trac-admin don't get installed. Oddly, there is no output from "wheel install".

Console scripts are installed with "wheel install-scripts" (1), or using "pip install" rather "wheel install".


(pve) ~/tmp$pip wheel --no-cache-dir --wheel-dir wheels -r requirements.txt
(pve) ~/tmp$wheel install --wheel-dir wheels -r requirements.txt
(pve) ~/tmp$ls pve/bin/trac*
ls: pve/bin/trac*: No such file or directory
(pve) ~/tmp$wheel install-scripts trac
(pve) ~/tmp$ls pve/bin/trac*
pve/bin/trac-admin    pve/bin/tracd

Anyway, I just wanted to understand that the behavior is by design. I suspect we'd always use "pip install" to install wheels.

- Ryan

(1) http://wheel.readthedocs.io/en/latest/#setuptools-scripts-handling
Reply all
Reply to author
Forward
0 new messages