Mathematica 7.0 slow on OS X
Matthias Gottschalk
I just tried Mathematica 7.0
In comparison to Mathematica 6.0.3, 7.0 seems to be by
factor of 2 slower under OS X.
When I run the "Mathematica Benchmark Test Mathematica 6.0" from
Karl UnterKofler I get for
Mathematica 6.0.3: Benchmark = 0.97038
and for
Mathematica 7.0: Benchmark = 0.471266
The numbers obtained by Mathematica#s own
BenchmarkReport[] are comparable, however: 1.88 vs. 1.91.
If 7.0 is really so much slower on a Mac as indicated by
the UnterKofler test, then is hardly good news.
So, what is your experience?
Matt
(numbers were generated on a MacBook Pro 2.2 GHz Intel
Core 2 Duo, OS X 10.5.5)
magma
There should be an improvement at least there. Or not?
jsk...@gmail.com
wrote:
I am having similar results with Mathematica 7 on Windows XP (HP
laptop, AMD Athlon 64 2.2GHz, 1.25 GB RAM)
Version 6.0.1: Benchmark = .649
Version 7.0.0: Benchmark = .326
The main culprit is the next-to-last test (Binomial[m, k]) which
jumps from 25.4s to 376.7s. Since the first 10 tests show an
improvement and 4 of the last 5 are slower, I suspected a memory
problem. However, free memory never drops below 300MB during the test,
and there is essentially no disk activity, so it's not a lack of
physical memory but probably an issue with Mathematica itself.
Also, the results with MathematicaMark seem dismal, lower than
several computers which should be slower. I do not currently have
Version 6 installed to compare it to, though.
John Keith
peter
I'm a premier member, but then I'm in Britain so maybe that's why I'll
be last to get a copy.
regards
Peter
2008/11/24 Matthias Gottschalk <mg...@gfz-potsdam.de>:
Xerxes
>
> The main culprit is the next-to-last test (Binomial[m, k]) which
> jumps from 25.4s to 376.7s.
I confirm this behavior (under Windows XP). However, this Binomial
test does not appear to be doing anything. For values of k>5, the
Binomial objects will not resolve without application of FunctionExpand.
The test is simply generating a sum of ten million unexpanded
Binomial terms. Why that should be a factor of 10 slower under 7
is an interesting question, but it's certainly not the kind of task
anybody would want to perform in a real calculation.
If I cut the 10^7 binomials down to a manageable 10^3 and actually
expand them, I find that 7 improves on the performance of 6 by
about 3%.
Saul Cohen
peter
P. Lindsay
=== System Information ===
Machine Name: plindsay-macbook
System: Mac OS X x86 (32-bit)
Date: November 25, 2008
Mathematica Version: 7.0.0
Benchmark Result: 1.06
=== MathematicaMark7 System Comparison ===
2.40 GHz Intel Core2 Q6660 3.91
Microsoft Windows (64-bit)
2.33 GHz Intel Xeon E5410 3.78
Microsoft Windows (64-bit)
2.50 GHz Intel Xeon E5420 3.75
Linux x86 (64-bit)
3.0 GHz Intel Xeon 5160 2.84
Microsoft Windows (32-bit)
2.66 GHz Intel Xeon 2.14
Apple Mac OS 10.4.8 (64-bit)
1.6 GHz Intel Xeon 5310 1.88
CentOS Linux 4.4 (32-bit)
1.4 GHz Itanium 2 1.78
Red Hat Linux AS 3.0 (64-bit)
2.4 GHz Opteron 250 1.24
Sun Solaris 10 (64-bit)
2.5 GHz G5 1.22
Apple Mac OS 10.4.8 (32-bit)
plindsay-macbook 1.06
Mac OS X x86 (32-bit)
2.4 GHz Pentium 4 1.00
Microsoft Windows XP (32-bit)
(Faster systems give larger numbers)
=== MathematicaMark7 Detailed Timings ===
Total Test 1 Test 2 Test 3 Test 4
Test 5 Test 6 Test 7 Test 8 Test 9 Test 10 Test 11 Test 12
Test 13 Test 14 Test 15
2.40 GHz Intel Core2 Q6660
Microsoft Windows (64-bit) 22.1 1.11 0.50 0.73 1.59
1.59 0.31 0.70 2.82 1.30 1.68 2.81 2.23
1.77 1.33 1.60
2.33 GHz Intel Xeon E5410
Microsoft Windows (64-bit) 22.8 1.09 0.52 0.76 1.65
3.01 0.31 0.70 2.84 1.06 1.64 2.79 1.83
1.75 1.23 1.59
2.50 GHz Intel Xeon E5420
Linux x86 (64-bit) 23.0 1.21 0.57 0.80 1.45
4.30 0.43 0.84 2.27 0.66 1.91 2.34 1.73
1.52 1.41 1.56
3.0 GHz Intel Xeon 5160
Microsoft Windows (32-bit) 30.4 2.08 0.67 1.09 2.70
4.03 0.50 0.89 3.17 2.00 2.16 2.41 2.08
1.50 2.59 2.52
2.66 GHz Intel Xeon
Apple Mac OS 10.4.8 (64-bit) 40.3 1.75 0.44 1.85 3.38
11.70 0.30 0.52 3.62 2.07 2.39 2.41 1.56
1.71 3.66 2.91
1.6 GHz Intel Xeon 5310
CentOS Linux 4.4 (32-bit) 45.8 2.01 1.20 0.93 4.42
8.32 0.88 1.54 5.12 1.61 3.14 3.91 2.73
3.37 3.49 3.12
1.4 GHz Itanium 2
Red Hat Linux AS 3.0 (64-bit) 48.6 3.82 0.83 1.39 1.92
2.80 0.33 0.76 7.44 3.29 4.84 7.11 4.90
2.90 2.88 3.34
2.4 GHz Opteron 250
Sun Solaris 10 (64-bit) 69.3 3.92 0.60 3.68 5.79
15.30 0.41 0.90 7.62 4.32 3.31 4.27 3.52
5.68 4.53 5.45
2.5 GHz G5
Apple Mac OS 10.4.8 (32-bit) 70.6 3.92 1.25 3.88 4.53
18.10 0.81 1.43 6.39 2.29 4.96 5.83 4.64
2.72 6.02 3.89
plindsay-macbook
Mac OS X x86 (32-bit) 81.2 3.53 1.19 3.37 5.93
14.40 0.92 1.46 6.30 11.40 3.43 4.61 3.96
3.15 8.27 9.27
2.4 GHz Pentium 4
Microsoft Windows XP (32-bit) 86.2 5.05 1.23 3.00 8.84
7.50 0.92 1.63 9.17 8.78 5.39 6.17 5.67
4.51 9.42 8.90
(Timings are CPU time in seconds)
2008/11/25 magma <made...@gmail.com>:
Szabolcs
> Does the Mathematica 7.0 own benchmarkreport take advantage of parallel computing?
> There should be an improvement at least there. Or not?
Mathematica's built-in benchmark changes with every version. It is
not suitable for comparing Mathematica versions. (I believe that it
is meant for comparing different computers running the same version.)
Devendra Kapadia
> On Nov 24, 4:06 am, "Matthias Gottschalk" <mg...@gfz-potsdam.de>
> wrote:
> I am having similar results with Mathematica 7 on Windows XP (HP
> laptop, AMD Athlon 64 2.2GHz, 1.25 GB RAM)
>
> Version 6.0.1: Benchmark = .649
> Version 7.0.0: Benchmark = .326
>
> The main culprit is the next-to-last test (Binomial[m, k]) which
> jumps from 25.4s to 376.7s. Since the first 10 tests show an
> improvement and 4 of the last 5 are slower, I suspected a memory
> problem. However, free memory never drops below 300MB during the test,
> and there is essentially no disk activity, so it's not a lack of
> physical memory but probably an issue with Mathematica itself.
>
> Also, the results with MathematicaMark seem dismal, lower than
> several computers which should be slower. I do not currently have
> Version 6 installed to compare it to, though.
>
> John Keith
>
Hello Matthias and John,
Thank you for reporting the problem with the Sum example used
in the benchmark test.
This example indeed evaluates much slower in Mathematica 7 than
it did in Mathematica 6. The slowdown appears to be caused by an
autoevaluation of Gamma while simplifying the answer given by
the new algorithm for Sum.
It may be noted that the result given by Mathematica 7 is significantly
smaller in size (ByteCount=416) when compared with that given by
Mathematica 6 (ByteCount=27264032).
Also, the test example uses an upper limit 10^7 for the summation.
The autoevaluation issue does not affect higher limits such as 10^8.
Thus, the timing for the test example is expected to be similar to that
in the following example (which uses an upper limit of 10^8).
===================
In[1]:= Sum[Binomial[m, k], {k, 0, 10^8}] // Timing // InputForm
Out[1]//InputForm=
{0.372024, 2^m -
Binomial[m, 100000001]*Hypergeometric2F1[100000001 - m, 1,
100000002, -1]}
===================
We are working on a fix for the problem with the time taken by
the test example.
Thank you once again for reporting the problem. I apologize
for the confusion caused by it.
Sincerely,
Devendra Kapadia.
Wolfram Research, Inc.
Bill Rowe
>Does the Mathematica 7.0 own benchmarkreport take advantage of
>parallel computing? There should be an improvement at least there.
>Or not?
Yes. That is looking at the code in Benchmark.m, there are
points where the code is set to use parallel processing when
possible. But this is also true for the code in Version 6. At
the momement, I don't have a setup that allows me to see what
effect this has.
When I run version 6 and version 7 of the built in benchmark, I
get insignificant differences for all of the tests except 5
(elementary functions), 12 (polynomial expansion) and 15
(solving a linear system). My criteria for significance was the
difference in timing between version 6 and 7 was larger than the
difference between two subsequent runs of version 6.
For polynomial expansion, the code in version 6 and 7 appears
identical. Version 7 takes ~30% (3.3 sec) longer on my machine.
For the other two tests, the code is not the same. In version 6
of the code there is a function (RandomArray) defined for
generating random values. In version 7 of the code, random
values are generated by calling RandomReal etc directly. For
test 5, version 6 of took ~33% (11.1 sec) longer while for test
15 version 7 took ~5% (1.2 sec) longer.
It is tempting to ascribe the differences in these last tests to
the difference in code. But looking at the definition of
RandomArray it is apparent that it simply calls RandomReal etc.
So, I would expect this to add a very small overhead to the
timing that would be much smaller than the differences reported.
Likely the difference in the last test isn't significant. Had I
run version 6 more times, I suspect I would have seen more
variation and the difference between version 7 and 6 would be
within variation seen in version 6 numbers.
Marc Heusser
wrote:
> you're luck - at least you've got a copy.
> I'm a premier member, but then I'm in Britain so maybe that's why I'll
> be last to get a copy.
I got a download link as a Swiss student two days ago, strange that you
have not yet got one.
Marc
> 2008/11/24 Matthias Gottschalk <mg...@gfz-potsdam.de>:
> >
> > I just tried Mathematica 7.0
> >
> >
> > In comparison to Mathematica 6.0.3, 7.0 seems to be by
> > factor of 2 slower under OS X.
> > When I run the "Mathematica Benchmark Test Mathematica 6.0" from
> > Karl UnterKofler I get for
> >
> >
> > Mathematica 6.0.3: Benchmark = 0.97038
> > and for
> > Mathematica 7.0: Benchmark = 0.471266
Same here - on a slightly faster MacBook Pro, same relation.
--
remove bye and from mercial to get valid e-mail
<http://www.heusser.com>
AnttiPenttilä
> On 11/25/08 at 7:18 AM, made...@gmail.com (magma) wrote:
>
>> Does the Mathematica 7.0 own benchmarkreport take advantage of
>> parallel computing? There should be an improvement at least there.
>> Or not?
>
> Yes. That is looking at the code in Benchmark.m, there are
> points where the code is set to use parallel processing when
> possible. But this is also true for the code in Version 6. At
> the momement, I don't have a setup that allows me to see what
> effect this has.
With version 6 the parallel computing happened inside some numerical
matrix library functions (provided by Intel). In version 7 the parallel
computing can, in addition to that, use several Mathematica kernels in a
totally new way.
With Mathematica benchmarking in my platform, however, the use of
parallel subkernels with Math 7.0 seems to be very bad idea:
Needs[ "Benchmarking`"]
Kernels[]
{}
(* Only one kernel *)
Benchmark[]
..."System" -> "Microsoft Windows (64-bit)",
"BenchmarkName" -> "MathematicaMark7", "FullVersionNumber" -> "7.0.0",
"Date" -> "November 26, 2008", "BenchmarkResult" -> 1.699,
"TotalTime" -> 50.72...
LaunchKernels[]
KernelObject[1, "local"], KernelObject[2, "local"]}
(* two parallel subkernels *)
Benchmark[]
...{"MachineName" -> "2-node homogeneous cluster",
"System" -> "Windows-x86-64", "BenchmarkName" -> "MathematicaMark7",
"FullVersionNumber" -> "7.0.0", "Date" -> "November 26, 2008",
"BenchmarkResult" -> 1.97, "TotalTime" -> 262.494}
So, TotalTime went from 51 secs to 262 secs. Not very nice example of
parallel advantages.
Antti