Setuptools 5.4 performance

[Peter Suter]

When I upgraded setuptools to 5.4.x, Trac became completely unresponsive
(loading a simple page takes 30 seconds instead of less than 1).
(On Windows, Python 2.6 or 2.7, Trac trunk tracd. The problem only
manifested itself when Trac was installed as an egg, not in "setup.py
develop" mode.)

Downgrading setuptools (to 5.3 or below) fixed the problem.

[1] points to [2] and mentionsthe PKG_RESOURCES_CACHE_ZIP_MANIFESTS
environment variable. Setting that environment variable with 5.4 also
enables reasonable performance. Although it sounds like this enables a
new experimental feature. So why is it required to get back performance
similar to 5.3?

Am I missing something? (Should we report a bug? Warn users against this
version?)

[1] https://pypi.python.org/pypi/setuptools#id3
[2] https://bitbucket.org/pypa/setuptools/issue/154

[Jun Omae]

I think that is a setuptools issue in 5.4. It seems the issue has been
introduced in [10cc90d9b828] and [2d13c675f84c] of setuptools.

After setuptools 5.4, the zipinfo property of ZipProvider class reads
egg file each time. Before 5.3, __init__ method of ZipProvider class
reads egg file and the result will be stored in its "zipinfo" instance
variable.

The following patch would fix it.

--- pkg_resources.py.orig 2014-08-09 22:06:34.877375000 +0900
+++ pkg_resources.py 2014-08-09 22:06:37.533625000 +0900
@@ -1636,7 +1636,11 @@

@property
def zipinfo(self):
- return self._zip_manifests.load(self.loader.archive)
+ try:
+ return self._zipinfo
+ except AttributeError:
+ self._zipinfo = self._zip_manifests.load(self.loader.archive)
+ return self._zipinfo

def get_resource_filename(self, manager, resource_name):
if not self.egg_name:

--
Jun Omae <jun...@gmail.com> (大前 潤)

[Peter Suter]

On 09.08.2014 15:23, Jun Omae wrote:
> I think that is a setuptools issue in 5.4. It seems the issue has been
> introduced in [10cc90d9b828] and [2d13c675f84c] of setuptools.
>
> After setuptools 5.4, the zipinfo property of ZipProvider class reads
> egg file each time. Before 5.3, __init__ method of ZipProvider class
> reads egg file and the result will be stored in its "zipinfo" instance
> variable.
>
> The following patch would fix it.
>
> --- pkg_resources.py.orig 2014-08-09 22:06:34.877375000 +0900
> +++ pkg_resources.py 2014-08-09 22:06:37.533625000 +0900
> @@ -1636,7 +1636,11 @@
>
> @property
> def zipinfo(self):
> - return self._zip_manifests.load(self.loader.archive)
> + try:
> + return self._zipinfo
> + except AttributeError:
> + self._zipinfo = self._zip_manifests.load(self.loader.archive)
> + return self._zipinfo
>
> def get_resource_filename(self, manager, resource_name):
> if not self.egg_name:
>

Thank you for investigating! Have you sent the patch to setuptools
developers?

[Jun Omae]

I just posted a new issue with the patch at
https://bitbucket.org/pypa/setuptools/issue/240.

[RjOllos]

In http://trac.edgewall.org/ticket/11694#comment:13 we are discussing how to document the defect and solution for setuptools 5.4 - 5.6.