mystifying xmlrpclib performance problem...
Skip Montanaro
I'm stumped trying to figure out why Python runs xmlrpclib marshalling and
unmarshalling so much faster on one machine than another, when everything
suggests the results should be similar on the two machines. Here's a simple
script I've been using to test things:
import xmlrpclib
import time
def test(obj, func, n):
t = time.clock()
for i in range(n):
x = func(obj)
return time.clock()-t
def test_dumps(obj, n):
return test((obj,), xmlrpclib.dumps, n)
def test_loads(obj, n):
s = xmlrpclib.dumps((obj,))
return test(s, xmlrpclib.loads, n)
obj = ['MusicEntry',
{'email': 'lalcorn@ultranet', 'time': '7:30pm', 'tickets': '',
'program': '', 'state': 'MA', 'start': '2002-01-26', 'venueurl': '',
'country': '', 'performers': ['An Evening with Karen Savoca'],
'addressid': 7283, 'name': '', 'zip': '', 'city': 'Sudbury',
'info': 'Reservations required. Please call (978)443-3253 or e-mail Laurie at lal...@ultranet.com.',
'merchandise': [], 'event': '', 'keywords': ['.zyx.41'],
'submit_time': '2001-08-28', 'key': 325629,
'active': 1, 'end': '2002-01-26', 'price': '$17', 'address3': '',
'address1': '', 'venue': 'Fox Run House Concerts',
'address2': '', 'update_time': '2001-09-22:19:28:44'}]
n = 1000
print "%.3g dumps per second" % (n/test_dumps(obj, n))
n /= 2
print "%.3g loads per second" % (n/test_loads(obj, n))
Here are the machine parameters and software I am working with:
Machine 1 (Dell laptop) Machine 2 (Dell XPS)
CPU P-III 450MHz, 256k cache P-III 550MHz, 512k cache
Memory 128MB 256MB
OS Mandrake 8.0 Mandrake 7.2
Bogomips 897.84 1094.45
Python 2.1.1 2.1.1
GCC 3.0.1 3.0.1
OPT flags -O6 -mcpu=pentiumpro -O6 -mcpu=pentiumpro
pystone.py 5400/5850 (-O) 5950/6410 (-O)
xmlrpclib 1.0b3 1.0b3
sgmlop 000528 000528
Unfortunately, when I run the testxmlrpc.py script, Machine 2 runs
substantially slower than Machine 1:
dumps 407/sec 222/sec
loads 104/sec 89.5/sec
I verified manually that xmlrpclib.getparser() returns an SgmlopParser
instance on both machines. The test is small enough that it should run
completely in memory. (Even if it didn't, Machine 1's disk subsystem is
likely to be lower performance than Machine 2's.)
The relative performance of the malloc packages installed on the two
machines should have been exposed by the pystone runs, if in fact there was
a difference. Just the same, I built a version of GNU malloc (circa 1995) I
had laying around and relinked the Python executables on both machines with
it. I compiled gmalloc.c with the same optimization flags as I used to
build the Python interpreters:
Machine 1 (Dell laptop) Machine 2 (Dell XPS)
pystone.py 5750/6210 (-O) 6490/6990 (-O)
Other than demonstrating that the malloc packages were apparently not the
cause of the discrepancy and that you should link Python with a heavily
optimized version of malloc, I saw no relative change on xmlrpclib's
dump/load speed:
dumps 478/sec 264/sec
loads 134/sec 114/sec
Any ideas what might be causing the performance drop on the supposedly
faster machine?
Thanks,
--
Skip Montanaro (sk...@pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/
Skip Montanaro
I'm stumped trying to figure out why Python runs xmlrpclib marshalling
and unmarshalling so much faster on one machine than another, when
everything suggests the results should be similar on the two machines.
I finally figured out the problem. I tested the version number of xmlrpclib
from an interactive Python shell:
>>> import xmlrpclib
>>> print xmlrpclib.__version__
1.0b3
However, that xmlrpclib lives in a directory which is only available when
running interactively. If I check the version (on Machine 1) without using
an interactive shell I see a different version:
% python -c 'import xmlrpclib ; print xmlrpclib.__version__'
0.9.8
On the "should-have-been-faster" machine I was getting the 1.0b3 version of
xmlrpclib. It appears some changes were made to xmlrpclib between 0.9.8 and
1.0b3 that slow it down considerably. I'll continue to use 0.9.8 for the
time being.