tradadmin: install as zipped egg, or unpacked egg

7 views
Skip to first unread message

rupert....@gmail.com

unread,
Jul 1, 2008, 6:10:16 AM7/1/08
to Trac Development
hi,

when doing a hardware change we noted that tracadmin gets installed as
unzipped egg, contrary to most other eggs and we have to set access
permissions on subdirectories because of this.

why is this, and how could be install it as zipped egg?

# easy_install http://svn.edgewall.com/repos/trac/sandbox/webadmin/
Downloading http://svn.edgewall.com/repos/trac/sandbox/webadmin/
Doing subversion checkout from http://svn.edgewall.com/repos/trac/sandbox/webadmin/
to /tmp/easy_install-Kuru-D/webadmin
Processing webadmin
Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-Kuru-D/
webadmin/egg-dist-tmp-_IP0Tz
zip_safe flag not set; analyzing archive contents...
webadmin.plugin: module references __file__

kr, rupert.

Alec Thomas

unread,
Jul 1, 2008, 6:16:17 AM7/1/08
to trac...@googlegroups.com
This occurs because Trac's static content and templates are shipped
within the egg. They must be accessible for normal file access and
thus the egg must be unpacked, or cracked open, if you like.

2008/7/1 rupert....@gmail.com <rupert....@gmail.com>:

--
And the next thing you know, you're at the zoo, shaving a yak,
all so you can wax your car.

Jani Tiainen

unread,
Jul 1, 2008, 6:47:00 AM7/1/08
to trac...@googlegroups.com
Alec Thomas kirjoitti:

> This occurs because Trac's static content and templates are shipped
> within the egg. They must be accessible for normal file access and
> thus the egg must be unpacked, or cracked open, if you like.

I don't get it.

Wasn't setuptools meant for using simple .egg files (at least most of my
plugins are in zipfiles, even they contain shared data) and in runtime
setuptools extracts plugins (and staticdata) to some temporary directory
(egg-cache).

So there is something incorrect in above statement.

Actually now that I look my plugins only EGG that is installed as
extracted is Trac itself for some reason...

Altough that shouldn't be necessary since to get static resources out -
user is required to run "deploy" command to extract static data from .egg.

Which makes me wonder how system can cope with several trac-admin/tracd
scripts...

--
Jani Tiainen

osimons

unread,
Jul 1, 2008, 7:09:49 AM7/1/08
to Trac Development

On Jul 1, 12:47 pm, Jani Tiainen <rede...@gmail.com> wrote:
> Alec Thomas kirjoitti:
>
> > This occurs because Trac's static content and templates are shipped
> > within the egg. They must be accessible for normal file access and
> > thus the egg must be unpacked, or cracked open, if you like.
>
> I don't get it.
>
> Wasn't setuptools meant for using simple .egg files (at least most of my
> plugins are in zipfiles, even they contain shared data) and in runtime
> setuptools extracts plugins (and staticdata) to some temporary directory
> (egg-cache).
>
> So there is something incorrect in above statement.
>
> Actually now that I look my plugins only EGG that is installed as
> extracted is Trac itself for some reason...

Trac setup.py uses the zip_safe = False flag to ask setuptools to
extract when installing.

>
> Altough that shouldn't be necessary since to get static resources out -
> user is required to run "deploy" command to extract static data from .egg.

Templates are also static resources that need to be accessed - they
don't get 'deployed'.

>
> Which makes me wonder how system can cope with several trac-admin/tracd
> scripts...

Quite simple actually - there can only be one... The scripts are very
simple though, and just loads everything from the active egg (as seen
in site-packages/easy_install.pth).

>
> --
> Jani Tiainen


:::simon

https://www.coderesort.com

Jani Tiainen

unread,
Jul 1, 2008, 7:36:16 AM7/1/08
to trac...@googlegroups.com
osimons kirjoitti:

>
> On Jul 1, 12:47 pm, Jani Tiainen <rede...@gmail.com> wrote:
>> Alec Thomas kirjoitti:
>>
>>> This occurs because Trac's static content and templates are shipped
>>> within the egg. They must be accessible for normal file access and
>>> thus the egg must be unpacked, or cracked open, if you like.
>> I don't get it.
>>
>> Wasn't setuptools meant for using simple .egg files (at least most of my
>> plugins are in zipfiles, even they contain shared data) and in runtime
>> setuptools extracts plugins (and staticdata) to some temporary directory
>> (egg-cache).
>>
>> So there is something incorrect in above statement.
>>
>> Actually now that I look my plugins only EGG that is installed as
>> extracted is Trac itself for some reason...
>
> Trac setup.py uses the zip_safe = False flag to ask setuptools to
> extract when installing.
>
>> Altough that shouldn't be necessary since to get static resources out -
>> user is required to run "deploy" command to extract static data from .egg.
>
> Templates are also static resources that need to be accessed - they
> don't get 'deployed'.

How they get accessed from zipped eggs?

Like Timing and Estimation that I've installed, it's single egg file, it
contains at least one template and still it works, so there must be way
better explanation. Maybe Trac does something for plugins that are
beyond of capabilities of setuptools so plugins don't need to be extracted?

>> Which makes me wonder how system can cope with several trac-admin/tracd
>> scripts...
>
> Quite simple actually - there can only be one... The scripts are very
> simple though, and just loads everything from the active egg (as seen
> in site-packages/easy_install.pth).

But how you spesify which trac to run... Let's think that I've two
versions in my system: 0.11-stable to run production and 0.12-dev now
how I can tell which one to run?

Since this new "deploy" was meant for side-by-side installation of
several versions of Trac...

Quoting Noah about that one:
"
In short: we cannot have a single global data folder like that anymore
since to be compliant with the spirit of setuptools we need to allow
things like multiple versions installed concurrently. In fact we
cannot install anything outside the egg. trac-admin deploy will
extract/generate the content you need, and since it is manual,
hopefully you won't overwrite one version's data with another :)
"

Deploy really doesn't extract templates, which actually were part of old
"/usr/share/trac" tree.

--
Jani Tiainen

osimons

unread,
Jul 1, 2008, 7:55:15 AM7/1/08
to Trac Development

On Jul 1, 1:36 pm, Jani Tiainen <rede...@gmail.com> wrote:
> osimons kirjoitti:
>
> > On Jul 1, 12:47 pm, Jani Tiainen <rede...@gmail.com> wrote:
> >> Alec Thomas kirjoitti:
>
> >>> This occurs because Trac's static content and templates are shipped
> >>> within the egg. They must be accessible for normal file access and
> >>> thus the egg must be unpacked, or cracked open, if you like.
> >> I don't get it.
>
> >> Wasn't setuptools meant for using simple .egg files (at least most of my
> >> plugins are in zipfiles, even they contain shared data) and in runtime
> >> setuptools extracts plugins (and staticdata) to some temporary directory
> >> (egg-cache).
>
> >> So there is something incorrect in above statement.
>
> >> Actually now that I look my plugins only EGG that is installed as
> >> extracted is Trac itself for some reason...
>
> > Trac setup.py uses the zip_safe = False flag to ask setuptools to
> > extract when installing.
>
> >> Altough that shouldn't be necessary since to get static resources out -
> >> user is required to run "deploy" command to extract static data from .egg.
>
> > Templates are also static resources that need to be accessed - they
> > don't get 'deployed'.
>
> How they get accessed from zipped eggs?

I am quite sure they need to be unzipped. Try opening a python
interpreter, and enter (not knowing what the timing and estimation
namespace actually is - you need to find that):

python -c "import timeest; print timeest.__file__"

>
> Like Timing and Estimation that I've installed, it's single egg file, it
> contains at least one template and still it works, so there must be way
> better explanation. Maybe Trac does something for plugins that are
> beyond of capabilities of setuptools so plugins don't need to be extracted?
>
> >> Which makes me wonder how system can cope with several trac-admin/tracd
> >> scripts...
>
> > Quite simple actually - there can only be one... The scripts are very
> > simple though, and just loads everything from the active egg (as seen
> > in site-packages/easy_install.pth).
>
> But how you spesify which trac to run... Let's think that I've two
> versions in my system: 0.11-stable to run production and 0.12-dev now
> how I can tell which one to run?
>
> Since this new "deploy" was meant for side-by-side installation of
> several versions of Trac...
>
> Quoting Noah about that one:
> "
> In short: we cannot have a single global data folder like that anymore
> since to be compliant with the spirit of setuptools we need to allow
> things like multiple versions installed concurrently. In fact we
> cannot install anything outside the egg. trac-admin deploy will
> extract/generate the content you need, and since it is manual,
> hopefully you won't overwrite one version's data with another :)
> "
>
> Deploy really doesn't extract templates, which actually were part of old
> "/usr/share/trac" tree.
>

Right, but I think you are confusing two different things. Noah really
talks about having two or more Python installations. You cannot have
more than one active version inside one Python installation at a time.
However, my machine has several (osx default, macports 2.4 and 2.5,
python.org version) all at differnet locations. In addition I have
dozens of virtualenv setups for various purposes - Trac + various
other development and testing.

If all my Tracs were to look inside /usr/share for templates and
config, this would be messy beyond belief... With setuptools and 0.11
it is selfcontained - either in the egg or in the environment, with
the option that each environment can point to a global template_dir,
config and plugins_dir located wherever I want. And, also the option
to deploy to chosen location - although the deploy is not necessary as
Trac will run fine fetching the resources from the egg (it just makes
it faster for web servers to access static files than calling Python
and Trac).


:::simon

https://www.coderesort.com

Noah Kantrowitz

unread,
Jul 1, 2008, 8:18:56 AM7/1/08
to trac...@googlegroups.com
On Jul 1, 2008, at 6:47 AM, Jani Tiainen wrote:

>
> Alec Thomas kirjoitti:
>> This occurs because Trac's static content and templates are shipped
>> within the egg. They must be accessible for normal file access and
>> thus the egg must be unpacked, or cracked open, if you like.
>
> I don't get it.
>
> Wasn't setuptools meant for using simple .egg files (at least most
> of my
> plugins are in zipfiles, even they contain shared data) and in runtime
> setuptools extracts plugins (and staticdata) to some temporary
> directory
> (egg-cache).

Yes, however this is 1) slower and 2) more likely to result in the
famous PYTHON_EGG_CACHE errors. It is tricky to make sure the egg
cache variable gets set correctly, as seen by the current issues with
zipped eggs and Genshi, so this just side steps the problem for now.
If you aren't using mod_python you could probably remove
zip_safe=False with no ill effects.

>
>
> So there is something incorrect in above statement.
>
> Actually now that I look my plugins only EGG that is installed as
> extracted is Trac itself for some reason...
>
> Altough that shouldn't be necessary since to get static resources
> out -
> user is required to run "deploy" command to extract static data
> from .egg.
>
> Which makes me wonder how system can cope with several trac-admin/
> tracd
> scripts...

As long as you used differing exec prefixes (so the scripts won't
overwrite each other) it will work fine.

--Noah

rupert....@gmail.com

unread,
Jul 1, 2008, 8:42:17 AM7/1/08
to Trac Development
we also do in apache conf:
SetEnv PYTHON_EGG_CACHE /tmp/cache/trac-eggs

some of the eggs get extracted there, like accountmanager. webadmin,
TracPermRedirect, and trac itself are extracted in python site-
packages. some of the eggs are not extracted at all, like
TracHTTPAuth.

is the python egg cache for the eggs having the zip_safe flag set, and
which need unpacking anyway?

rupert.
> https://www.coderesort.com- Hide quoted text -
>
> - Show quoted text -- Hide quoted text -
>
> - Show quoted text -

osimons

unread,
Jul 1, 2008, 8:55:23 AM7/1/08
to Trac Development


On Jul 1, 2:42 pm, "rupert.thur...@gmail.com"
<rupert.thur...@gmail.com> wrote:
> we also do in apache conf:
> SetEnv PYTHON_EGG_CACHE /tmp/cache/trac-eggs
>
> some of the eggs get extracted there, like accountmanager. webadmin,
> TracPermRedirect, and trac itself are extracted in python site-
> packages. some of the eggs are not extracted at all, like
> TracHTTPAuth.
>
> is the python egg cache for the eggs having the zip_safe flag set, and
> which need unpacking anyway?

Python has support for reading code files from zip files, so as long
as the setup compiles and packages compiled versions (.pyc) Python
does not need anything else. Basically Python needs to extract to
provide access for non-code resources (like static content and
templates), or to load compiled C libraries and modules (like .so/.dll
files).

:::simon

https://www.coderesort.com

Noah Kantrowitz

unread,
Jul 1, 2008, 9:10:09 AM7/1/08
to trac...@googlegroups.com

Actually pkg_resources supports reading the content files directly
from the zip, we force it to extract to cache by asking for a path to
the package_data file instead of just getting the content. The reason
not all plugins show up in the cache is generally only templates and
htdocs content is stored in this manner, so a plugin with neither will
operate directly from the egg.

--Noah

Reply all
Reply to author
Forward
0 new messages