Advice summing list
Jeannot
I need some advice for summing numbers... ok maybe it is not too
complicated but I'll ask anyway ;-)
I want to sum a list of numbers, usually I use the "trick" (really a
trick?) of joining the list with "+" and then expr the results:
set L [list 1 25 21]
set Sum [expr [join $L "+"]]
Usually it works fine but know I am dealing with a 3000 lists of more
than 10000 elements and seems to be quite slow (ok slow is not
objective). Do you have any advice of doing this quicker? better?
quicker and better ;-)
Thanks for suggestions!
Jeannot
suchenwi
> I want to sum a list of numbers, usually I use the "trick" (really a
> trick?) of joining the list with "+" and then expr the results:
>
> set L [list 1 25 21]
> set Sum [expr [join $L "+"]]
>
> Usually it works fine but know I am dealing with a 3000 lists of more
> than 10000 elements and seems to be quite slow (ok slow is not
> objective). Do you have any advice of doing this quicker? better?
> quicker and better ;-)
The fastest way is the orthodox one:
proc sum list {
set res 0
foreach el $list {set res [expr {$res + $el}]}
set res
}
77 % sum {1 2 3 4}
10
Why the [join] solution scales badly is discussed on http://wiki.tcl.tk/9585
"Elegance vs. performance".
Jeannot
% for {set i 0} {$i<1000} {incr i} {lappend L_1000 $i}
% for {set i 0} {$i<5000} {incr i} {lappend L_5000 $i}
% for {set i 0} {$i<10000} {incr i} {lappend L_10000 $i}
% time {puts [expr [join $L_1000 "+"]]} 1
499500
7159 microseconds per iteration
% time {puts [expr [join $L_5000 "+"]]} 1
12997000
338612 microseconds per iteration
% time {puts [expr [join $L_10000 "+"]]} 1
49995000
938232 microseconds per iteration
% time {set Sum 0;foreach e $L_1000 {set Sum [expr $Sum+$e]};puts
$Sum} 1
499500
7908 microseconds per iteration
% time {set Sum 0;foreach e $L_5000 {set Sum [expr $Sum+$e]};puts
$Sum} 1
12997000
46809 microseconds per iteration
% time {set Sum 0;foreach e $L_10000 {set Sum [expr $Sum+$e]};puts
$Sum} 1
49995000
78817 microseconds per iteration
It seems that for 1000 elements it's the same but for more not
Still, if you have any better solution...
Jeannot
Jeannot
> "Elegance vs. performance".
Thanks I will check the wiki... maybe I should have done this
earlier ...
Uwe Klein
if at global scope you may want to wrap that
into a proc.
proc SumA L {
set Sum [expr [join $L "+"]]
return $Sum
}
And is this any faster? ( integer only )
set Sum 0
foreach l $L { incr Sum $l }
proc SumB L {
set Sum 0
foreach l $L { incr Sum $l }
return $Sum
}
uwe
Jeannot
It should be integer only. But I have not tried yet the version with
incr yet. I'll do this
Jeannot
Arjen Markus
Be sure to brace all your [expr]essions!
So:
proc sumlist {list} {
set r 0
foreach v $list {
set r [expr {$r+$v}]
}
return $r
}
Otherwise the [expr] command needs to parse the expression
each time.
If you need more functionality to deal with large sets of
numerical data, the NAP extension might be of interest too.
(http://wiki.tcl.tk/4015)
Regards,
Arjen
Uwe Klein
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
I will always brace my expressions!
............
I will always brace my expressions!
Jeannot
Thanks for this!!
Jeannot
Arjen Markus
Make that:
I will (1-eps)*always brace my expressions!
(There is a non-vanishing set of situations
where bracing does not work: any situation for
instance where you want the operation to vary,
but in most cases It Is A Good Thing :))
Regards,
Arjen
Glenn Jackman
> On 10 apr, 11:43, "Jeannot" <jean.mul...@igbmc.u-strasbg.fr> wrote:
> > I've made a proc now.
> > It should be integer only. But I have not tried yet the version with
> > incr yet. I'll do this
> > Jeannot
>
> Be sure to brace all your [expr]essions!
Be sure to brace almost all your [expr]essions:
% set L {1 2 3 4}
1 2 3 4
% set sum1 [expr [join $L +]]
10
% set sum2 [expr {[join $L +]}]
1+2+3+4
Always brace your expressions except when you shouldn't.
--
Glenn Jackman
"You can only be young once. But you can always be immature." -- Dave Barry
Don Porter
>> set L [list 1 25 21]
>> set Sum [expr [join $L "+"]]
> Why the [join] solution scales badly is discussed on http://wiki.tcl.tk/9585
> "Elegance vs. performance".
Please note that the quadratic peformance of the expr parser has been
corrected in Tcl 8.5.
--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
Donald Arseneau
> On 10 avr, 11:17, "suchenwi" <richard.suchenwirth-
> > The fastest way is the orthodox one:
Hmmm... Some time ago I found the tricky way was the fastest,
for modest sized lists, even if the list elements had already
been used as numbers (thus, the list [join] method involved a
shimmering penalty).
I dare predict that the fastest method is to chunk the list into
managable pieces, calculate subtotals by [join], and the grand
total by [incr].
--
DA
Peter De Rijk
- avoid shimmering (going back and forth between your data being strings
and numbers)
- brace your expressions where possible (see first)
- use a proc
- code it in C (lmath_sum is a C coded version in the Extral package)
~ $ tclsh
% package require Extral
proc sum {list} {
set Sum 0
foreach e $list {set Sum [expr {$Sum+$e}]}
return $Sum
}
for {set i 0} {$i<1000} {incr i} {lappend L_1000 $i}
time {expr [join $L_1000 "+"]} 10
time {set Sum 0;foreach e $L_1000 {set Sum [expr $Sum+$e]}} 10
time {set Sum 0;foreach e $L_1000 {set Sum [expr {$Sum+$e}]}} 10
time {sum $L_1000} 10
time {lmath_sum $L_1000} 10
2.0
% % % 8218.6 microseconds per iteration
% 7753.8 microseconds per iteration
% 2245.6 microseconds per iteration
% 415.2 microseconds per iteration
% 48.3 microseconds per iteration
% % for {set i 0} {$i<10000} {incr i} {lappend L_10000 $i}
time {expr [join $L_10000 "+"]} 10
time {set Sum 0;foreach e $L_10000 {set Sum [expr {$Sum+$e}]}} 10
time {sum $L_10000} 10
time {lmath_sum $L_10000} 10
% 1354362.7 microseconds per iteration
% 20489.2 microseconds per iteration
% 3146.0 microseconds per iteration
% 427.1 microseconds per iteration
%
~ $ tclsh
% package require Extral
proc sum {list} {
set Sum 0
foreach e $list {set Sum [expr {$Sum+$e}]}
return $Sum
}
for {set i 0} {$i<1000} {incr i} {lappend L_1000 $i}
puts "join: [time {expr [join $L_1000 "+"]} 10]
puts "unbraced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr $Sum+$e]}}
10]
puts "braced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr {$Sum+$e}]}}
10]
puts "proc: [time {sum $L_1000} 10]
puts "C: [time {lmath_sum $L_1000} 10]
2.0
% % % extra characters after close-quote
% extra characters after close-quote
% puts "join: [time {expr [join $L_1000 "+"]} 10]"
puts "unbraced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr $Sum+$e]}}
10]"
puts "braced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr {$Sum+$e}]}}
10]"
puts "proc: [time {sum $L_1000} 10]"
puts "extra characters after close-quote
% C: [time {lmath_sum $L_1000} 10]"
unbraced: 7566.1 microseconds per iteration
% braced: 3638.9 microseconds per iteration
% proc: 325.5 microseconds per iteration
% C: 40.0 microseconds per iteration
%
~ $ tclsh
% package require Extral
proc sum {list} {
set Sum 0
foreach e $list {set Sum [expr {$Sum+$e}]}
return $Sum
}
for {set i 0} {$i<1000} {incr i} {lappend L_1000 $i}
puts 1000
puts "join: [time {expr [join $L_1000 "+"]} 10]"
puts "unbraced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr $Sum+$e]}}
10]"
puts "braced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr
{$Sum+$e}]}} 10]"
puts "proc: [time {sum $L_1000} 10]"
puts "C: [time {lmath_sum $L_1000} 10]"
for {set i 0} {$i<10000} {incr i} {lappend L_10000 $i}
puts 10000
puts "join: [time {expr [join $L_1000 "+"]} 10]"
puts "unbraced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr $Sum+$e]}}
10]"
puts "braced: [time {set Sum 0;foreach e $L_1000 {set Sum [expr
{$Sum+$e}]}} 10]"
puts "proc: [time {sum $L_1000} 10]"
puts "C: [time {lmath_sum $L_1000} 10]"
2.0
% % % 1000
% join: 6852.4 microseconds per iteration
% unbraced: 7310.9 microseconds per iteration
% braced: 2037.8 microseconds per iteration
% proc: 314.9 microseconds per iteration
% C: 40.2 microseconds per iteration
% % 10000
% join: 6873.5 microseconds per iteration
% unbraced: 7050.2 microseconds per iteration
% braced: 2069.0 microseconds per iteration
% proc: 315.7 microseconds per iteration
% C: 39.5 microseconds per iteration
Jeannot wrote:
--
Dr Peter De Rijk E-mail: Peter....@ua.ac.be
Applied Molecular Genomics Group Tel. +32 3 265 10 08
VIB - Department of Molecular Genetics Fax. +32 3 265 10 12
University of Antwerp - CDE http://www.molgen.ua.ac.be/