Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Replace, ReplaceAll and If time performace comparition
108 views
Skip to first unread message
Murta
Jun 24, 2012, 4:24:52 AM6/24/12
to
HI all
I was working in some simulations with random numbers and get this example of performance comparition.
randomList = RandomInteger[{-100, 100}, 10 10^6];
(randomList /. (x_ /; x < 0 -> 0)); // AbsoluteTiming
{5.747133, Null}
Replace[randomList, (x_ /; x < 0 -> 0), 1]; // AbsoluteTiming
{4.758984, Null}
(If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming
{0.572200, Null}
I personally prefer work with patterns because they are more compact and functional.
Someone knows why patter is one magnitude order slow?? There is some trick to make it faster?
tks
Murta
I was working in some simulations with random numbers and get this example of performance comparition.
randomList = RandomInteger[{-100, 100}, 10 10^6];
(randomList /. (x_ /; x < 0 -> 0)); // AbsoluteTiming
{5.747133, Null}
Replace[randomList, (x_ /; x < 0 -> 0), 1]; // AbsoluteTiming
{4.758984, Null}
(If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming
{0.572200, Null}
I personally prefer work with patterns because they are more compact and functional.
Someone knows why patter is one magnitude order slow?? There is some trick to make it faster?
tks
Murta
Andrzej Kozlowski
Jun 25, 2012, 4:01:35 AM6/25/12
to
In[1]:= randomList=RandomInteger[{-100,100},10 10^6];
In[2]:= (a=If[#<0,0,#]&/@randomList);//AbsoluteTiming
Out[2]= {0.874206,Null}
In[3]:= (b=Clip[randomList,{0,Infinity},{0,1}]);//AbsoluteTiming
Out[3]= {0.064867,Null}
In[4]:= a==b
Out[4]= True
There is no way to speed up pattern matching to give comparable performance.
Andrzej Kozlowski
Bill Rowe
Jun 25, 2012, 4:02:36 AM6/25/12
to
On 6/24/12 at 4:26 AM, rodrig...@gmail.com (Murta) wrote:
>HI all I was working in some simulations with random numbers and get
>this example of performance comparition.
>randomList = RandomInteger[{-100, 100}, 10 10^6];
>(randomList /. (x_ /; x < 0 -> 0)); // AbsoluteTiming {5.747133,
>Null}
>Replace[randomList, (x_ /; x < 0 -> 0), 1]; // AbsoluteTiming
>{4.758984, Null}
>(If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming {0.572200,
>Null}
>I personally prefer work with patterns because they are more compact
>and functional. Someone knows why patter is one magnitude order
>slow?? There is some trick to make it faster?
I don't know how to make the pattern matching faster. But I do
>HI all I was working in some simulations with random numbers and get
>this example of performance comparition.
>randomList = RandomInteger[{-100, 100}, 10 10^6];
>(randomList /. (x_ /; x < 0 -> 0)); // AbsoluteTiming {5.747133,
>Null}
>Replace[randomList, (x_ /; x < 0 -> 0), 1]; // AbsoluteTiming
>{4.758984, Null}
>(If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming {0.572200,
>Null}
>I personally prefer work with patterns because they are more compact
>and functional. Someone knows why patter is one magnitude order
>slow?? There is some trick to make it faster?
know how to create the list faster. That is:
In[1]:= randomList = RandomInteger[{-100, 100}, 10 10^6];
In[2]:= (a = If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming
Out[2]= {0.597679,Null}
In[3]:= (b = Clip[randomList, {0, 100}]); // AbsoluteTiming
Out[3]= {0.086329,Null}
In[4]:= a == b
Out[4]= True
And it seems to me the syntax of Clip makes it very apparent
what the code does.
Bob Hanlon
Jun 25, 2012, 4:03:37 AM6/25/12
to
With a pattern you get a slight improvement with
(randomList /. (_?Negative -> 0))
However, Clip is much faster
Clip[randomList, {0, Infinity}]
Bob Hanlon
On Sun, Jun 24, 2012 at 4:26 AM, Murta <rodrig...@gmail.com> wrote:
> HI all
> I was working in some simulations with random numbers and get this example of performance comparition.
>
> randomList = RandomInteger[{-100, 100}, 10 10^6];
>
> (randomList /. (x_ /; x < 0 -> 0)); // AbsoluteTiming
> {5.747133, Null}
>
> Replace[randomList, (x_ /; x < 0 -> 0), 1]; // AbsoluteTiming
> {4.758984, Null}
>
> (If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming
> {0.572200, Null}
>
> I personally prefer work with patterns because they are more compact and functional.
> Someone knows why patter is one magnitude order slow?? There is some trick to make it faster?
>
> tks
> Murta
>
--
Bob Hanlon
(randomList /. (_?Negative -> 0))
However, Clip is much faster
Clip[randomList, {0, Infinity}]
Bob Hanlon
On Sun, Jun 24, 2012 at 4:26 AM, Murta <rodrig...@gmail.com> wrote:
> HI all
> I was working in some simulations with random numbers and get this example of performance comparition.
>
> randomList = RandomInteger[{-100, 100}, 10 10^6];
>
> (randomList /. (x_ /; x < 0 -> 0)); // AbsoluteTiming
> {5.747133, Null}
>
> Replace[randomList, (x_ /; x < 0 -> 0), 1]; // AbsoluteTiming
> {4.758984, Null}
>
> (If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming
> {0.572200, Null}
>
> I personally prefer work with patterns because they are more compact and functional.
> Someone knows why patter is one magnitude order slow?? There is some trick to make it faster?
>
> Murta
>
--
Bob Hanlon
Christoph Lhotka
Jun 27, 2012, 4:06:03 AM6/27/12
to
Hello,
I played around to see if it is possible to top the performance goal
of Clip:
WARNING: SET THE LENGTH OF THE LIST randomList TO A SMALLER VALUE,
IF YOU ARE NOT SURE ABOUT HOW MUCH MEMORY YOU CAN ADDRESS AND
WANT TO AVOID CRASHING YOUR PC.
(on my computer it was 10*^9 for function cf2)
cf0 = Compile[{{list, _Integer, 1}},
Module[{clist = list, cur},
Do[
clist[[i]] = If[(cur = list[[i]]) > 0, cur, 0],
{i, 1, Length[list]}];
clist
],
CompilationTarget -> "C"
];
cf1 = Compile[{{list, _Integer, 1}},
Module[{clist = list, cur},
Do[
clist[[i]] = If[(cur = list[[i]]) > 0, cur, 0],
{i, 1, Length[list]}];
clist
],
CompilationTarget -> "C",
RuntimeOptions -> "Quality"
];
cf2 = Compile[{{list, _Integer, 1}},
Module[{clist = list, cur},
Do[
clist[[i]] = If[(cur = list[[i]]) > 0, cur, 0],
{i, 1, Length[list]}];
clist
],
CompilationTarget -> "C",
RuntimeOptions -> "Speed"
];
randomList = RandomInteger[{-100, 100}, 10 10^7];
a = cf2[randomList]; // AbsoluteTiming
{0.756555, Null}
b = Clip[randomList, {0, Infinity}, {0, 1}]; // AbsoluteTiming
{0.713735, Null}
c = cf1[randomList]; // AbsoluteTiming
{0.501313, Null}
d = cf0[randomList]; // AbsoluteTiming
{0.491033, Null}
a == b == c == d
True
As you can see Clip can still be topped by a factor 1.45 by using
procedural and compiled code
(for me the most interesting enhancements in Mathematica 8).
What I found surprising is that cf2 ("Speed") is slower than cf1
("Quality") and cf0. I also
did not increase the performance when parallelizing (use
Parallelization->True). By the way,
the ranking between cf1, cf0 is just an incidence . They are of the same
performance (on my PC).
Can anyone top the result?
Best,
Christoph
On 06/25/2012 10:01 AM, Andrzej Kozlowski wrote:
I played around to see if it is possible to top the performance goal
of Clip:
WARNING: SET THE LENGTH OF THE LIST randomList TO A SMALLER VALUE,
IF YOU ARE NOT SURE ABOUT HOW MUCH MEMORY YOU CAN ADDRESS AND
WANT TO AVOID CRASHING YOUR PC.
(on my computer it was 10*^9 for function cf2)
cf0 = Compile[{{list, _Integer, 1}},
Module[{clist = list, cur},
Do[
clist[[i]] = If[(cur = list[[i]]) > 0, cur, 0],
{i, 1, Length[list]}];
clist
],
CompilationTarget -> "C"
];
cf1 = Compile[{{list, _Integer, 1}},
Module[{clist = list, cur},
Do[
clist[[i]] = If[(cur = list[[i]]) > 0, cur, 0],
{i, 1, Length[list]}];
clist
],
CompilationTarget -> "C",
RuntimeOptions -> "Quality"
];
cf2 = Compile[{{list, _Integer, 1}},
Module[{clist = list, cur},
Do[
clist[[i]] = If[(cur = list[[i]]) > 0, cur, 0],
{i, 1, Length[list]}];
clist
],
CompilationTarget -> "C",
RuntimeOptions -> "Speed"
];
randomList = RandomInteger[{-100, 100}, 10 10^7];
a = cf2[randomList]; // AbsoluteTiming
{0.756555, Null}
b = Clip[randomList, {0, Infinity}, {0, 1}]; // AbsoluteTiming
{0.713735, Null}
c = cf1[randomList]; // AbsoluteTiming
{0.501313, Null}
d = cf0[randomList]; // AbsoluteTiming
{0.491033, Null}
a == b == c == d
True
As you can see Clip can still be topped by a factor 1.45 by using
procedural and compiled code
(for me the most interesting enhancements in Mathematica 8).
What I found surprising is that cf2 ("Speed") is slower than cf1
("Quality") and cf0. I also
did not increase the performance when parallelizing (use
Parallelization->True). By the way,
the ranking between cf1, cf0 is just an incidence . They are of the same
performance (on my PC).
Can anyone top the result?
Best,
Christoph
On 06/25/2012 10:01 AM, Andrzej Kozlowski wrote:
> On 24 Jun 2012, at 10:26, Murta wrote:
>
>> HI all
>> I was working in some simulations with random numbers and get this example of performance comparition.
>>
>> randomList = RandomInteger[{-100, 100}, 10 10^6];
>>
>> (randomList /. (x_ /; x< 0 -> 0)); // AbsoluteTiming
>> {5.747133, Null}
>>
>> Replace[randomList, (x_ /; x< 0 -> 0), 1]; // AbsoluteTiming
>> {4.758984, Null}
>>
>> (If[#< 0, 0, #]& /@ randomList); // AbsoluteTiming
>> {0.572200, Null}
>>
>> I personally prefer work with patterns because they are more compact and functional.
>> Someone knows why patter is one magnitude order slow?? There is some trick to make it faster?
>>
>> tks
>> Murta
>>
>
>> HI all
>> I was working in some simulations with random numbers and get this example of performance comparition.
>>
>> randomList = RandomInteger[{-100, 100}, 10 10^6];
>>
>> (randomList /. (x_ /; x< 0 -> 0)); // AbsoluteTiming
>> {5.747133, Null}
>>
>> Replace[randomList, (x_ /; x< 0 -> 0), 1]; // AbsoluteTiming
>> {4.758984, Null}
>>
>> (If[#< 0, 0, #]& /@ randomList); // AbsoluteTiming
>> {0.572200, Null}
>>
>> I personally prefer work with patterns because they are more compact and functional.
>> Someone knows why patter is one magnitude order slow?? There is some trick to make it faster?
>>
>> tks
>> Murta
>>
awnl
Jun 28, 2012, 4:04:00 AM6/28/12
to
Hi,
> Can anyone top the result?
contradicting to my announcement, I tried anyway and found this to be
considerably faster on my machine:
cf3 = Compile[{{list, _Integer, 1}},
Map[If[# > 0, #, 0] &, list],
CompilationTarget -> "C"
]
hth,
albert
> Can anyone top the result?
considerably faster on my machine:
cf3 = Compile[{{list, _Integer, 1}},
Map[If[# > 0, #, 0] &, list],
CompilationTarget -> "C"
]
hth,
albert
awnl
Jun 28, 2012, 4:00:26 AM6/28/12
to
Hi,
> As you can see Clip can still be topped by a factor 1.45 by using
> procedural and compiled code
> (for me the most interesting enhancements in Mathematica 8).
that seems to be somewhat dependent on your setup (hardware, OS,
compiler ...). For me the compiled versions are slightly slower than
Clip (with MS compiler on Windows 7 64bit). As the code is so simple I
wouldn't expect that a compiler would have much chance to do good or bad
optimizing it, but still, probably your compiler is doing a better job
than mine :-).
> What I found surprising is that cf2 ("Speed") is slower than cf1
> ("Quality") and cf0. I also
> did not increase the performance when parallelizing (use
> Parallelization->True).
that is strange indeed, I see the same thing. I have seen large
differences in other cases, but probably that's because most of those
options are irrelevant for the code that's run (no danger of
underflow/overflow etc.). For the parallelization: I think this will
only have an effect if you make your compiled function Listable and the
function "threads over list arguments". Rewriting your function that way
(avoiding the Do-loop) turns out to be a performance disaster, though :-)
> By the way,
> the ranking between cf1, cf0 is just an incidence . They are of the same
> performance (on my PC).
>
> Can anyone top the result?
not me, I won't even try. And I won't buy a better compiler to proof a
point :-)
albert
> As you can see Clip can still be topped by a factor 1.45 by using
> procedural and compiled code
> (for me the most interesting enhancements in Mathematica 8).
compiler ...). For me the compiled versions are slightly slower than
Clip (with MS compiler on Windows 7 64bit). As the code is so simple I
wouldn't expect that a compiler would have much chance to do good or bad
optimizing it, but still, probably your compiler is doing a better job
than mine :-).
> What I found surprising is that cf2 ("Speed") is slower than cf1
> ("Quality") and cf0. I also
> did not increase the performance when parallelizing (use
> Parallelization->True).
differences in other cases, but probably that's because most of those
options are irrelevant for the code that's run (no danger of
underflow/overflow etc.). For the parallelization: I think this will
only have an effect if you make your compiled function Listable and the
function "threads over list arguments". Rewriting your function that way
(avoiding the Do-loop) turns out to be a performance disaster, though :-)
> By the way,
> the ranking between cf1, cf0 is just an incidence . They are of the same
> performance (on my PC).
>
> Can anyone top the result?
point :-)
albert
Christoph Lhotka
Jun 29, 2012, 4:48:04 AM6/29/12
to
Hi,
congratulations!
It strikes the performance of Clip on my machine by a factor of 2
(instead of
a factor of 1.45 as published in my previous post).
Does it mean, that Compile + Map + raw functions is a good combination
to speed up codes?
Question: Can anyone explain, why RuntimeOptions->"Speed" does not
improve the performance?
Thanks for your code,
Christoph
On 06/28/2012 10:02 AM, awnl wrote:
> Hi,
congratulations!
It strikes the performance of Clip on my machine by a factor of 2
(instead of
a factor of 1.45 as published in my previous post).
Does it mean, that Compile + Map + raw functions is a good combination
to speed up codes?
Question: Can anyone explain, why RuntimeOptions->"Speed" does not
improve the performance?
Thanks for your code,
Christoph
On 06/28/2012 10:02 AM, awnl wrote:
> Hi,
>
>> Can anyone top the result?
>> Can anyone top the result?
awnl
Jun 30, 2012, 5:16:48 AM6/30/12
to
Hi,
> Does it mean, that Compile + Map + raw functions is a good combination
> to speed up codes?
It looks like yes. What you should have in mind is that Compile does
transform Mathematica code to something else (with CompilationTarget ->
"C" to C). The speedup depends a lot on how well optimized the generated
code is. I think the general rule is that constructs which are less
general and easier to analyze will be easier to optimize and thus have a
greater chance to be translated to something efficient.
As we don't know which transformation rules the code generator uses, it
will always need some experimenting, though.
hth,
albert
> Does it mean, that Compile + Map + raw functions is a good combination
> to speed up codes?
transform Mathematica code to something else (with CompilationTarget ->
"C" to C). The speedup depends a lot on how well optimized the generated
code is. I think the general rule is that constructs which are less
general and easier to analyze will be easier to optimize and thus have a
greater chance to be translated to something efficient.
As we don't know which transformation rules the code generator uses, it
will always need some experimenting, though.
hth,
albert
Murta
Jul 11, 2012, 2:19:02 AM7/11/12
to
Amazing and clear example!...
Now I have a lot of tests to do with my old functions.
I didn't know that I could use compile for lists...
Tks a lot!
Now I have a lot of tests to do with my old functions.
I didn't know that I could use compile for lists...
Tks a lot!
Murta
Jul 11, 2012, 2:19:32 AM7/11/12
to
Hi Albert,
I get this message when I tried to teste your suggestion when using " CompilationTarget -> "C" ":
"A C compiler cannot be found on your system. Please consult the documentation to learn how to set up suitable compilers"
I'm using Max OS X 10.7.4
Some clue?
Tks in advance
On Thursday, June 28, 2012 5:04:00 AM UTC-3, awnl wrote:
> Hi,
>
> > Can anyone top the result?
> CompilationTarget -> "C"
> ]
>
> hth,
>
> albert
I get this message when I tried to teste your suggestion when using " CompilationTarget -> "C" ":
"A C compiler cannot be found on your system. Please consult the documentation to learn how to set up suitable compilers"
I'm using Max OS X 10.7.4
Some clue?
Tks in advance
On Thursday, June 28, 2012 5:04:00 AM UTC-3, awnl wrote:
> Hi,
>
> > Can anyone top the result?
>
> contradicting to my announcement, I tried anyway and found this to be
> considerably faster on my machine:
>
> cf3 = Compile[{{list, _Integer, 1}},
> Map[If[# > 0, #, 0] &, list],
> contradicting to my announcement, I tried anyway and found this to be
> considerably faster on my machine:
>
> cf3 = Compile[{{list, _Integer, 1}},
> CompilationTarget -> "C"
> ]
>
> hth,
>
> albert
George Woodrow III
Jul 11, 2012, 6:23:22 PM7/11/12
to
There may be other choices, but you can get Xcode (free) from the App store. Once it is installed, Mathematica will use the XCode compiler.
george
george
Dana DeLouis
Jul 14, 2012, 1:31:39 AM7/14/12
to
> (If[# < 0, 0, #] & /@ randomList); // AbsoluteTiming
> {0.572200, Null}
> ... There is some trick to make it faster?
> {0.572200, Null}
Hi. You've got a great solution with Clip, but just to add 2 cents..
In other programming languages, sometimes using Max or Min instead of an IF statement is nice.
randomList=RandomInteger[{-100,100},10 10^6];
t1=(If[#<0,0,#]&/@randomList);//AbsoluteTiming
{0.729998,Null}
t2=(Max[#,0]&/@randomList);//AbsoluteTiming
{0.454711,Null}
t1==t2
True
In the old days before Clip, one way was to use two IF statements.
One neat technique was to replace the two IF statements with Median.
v=RandomInteger[{-100,100},20];
// New
Clip[v,{0,1}]
{0,0,1,1,1,1,0,1,0,1,0,1,0,0,1,1,1,0,0,0}
// Old days
Median[{0,#,1}]& /@v
{0,0,1,1,1,1,0,1,0,1,0,1,0,0,1,1,1,0,0,0}
%%==%
True
If one were stuck without access to a Median function, then one way without IF statements...
Min[Max[0,#],1]& /@ v
{0,0,1,1,1,1,0,1,0,1,0,1,0,0,1,1,1,0,0,0}
= = = = = = = = = =
Dana DeLouis
Mac & Math 8
= = = = = = = = = =
0 new messages