Tablelist sortbycolumn message

[Cesare Bavazzano]

Hello,

Sorting data by columns brings tablelist to get stuck for about 8" on my PC (*) before showing results.
Table data is large (~10000 rows by 7 columns).
I wonder if there is a way to show a 'wait-message" while the sort takes place.
Of course any other solution to warn users is appreciated.

Thanks,
Cesare

(*) Windows 7 Enterprise (64-bits OS); Quad-Core AMD Opteron Processor 2378 2.40 GHz; 4.00 GB RAM.

[nemethi]

There is currently no built-in way for displaying a wait-message while
sorting by a column. Such an option or subcommand would surely make
sense. On the other hand, I wonder why sorting about 10000 items takes
so long in your application. Are you using some comparison command as
the value of the -sortcommand column configuration option (with
-sortmode command)?

--
Csaba Nemethi http://www.nemethi.de mailto:csaba....@t-online.de

[Cesare Bavazzano]

Hello Nemethi,

No, I didn't make use of the -sortcommand option.
Meanwhile I could try my application on a faster PC (2.80GHz, 16GB RAM) and it took just one second or two to sort the whole table. Possibly the solution for slowness is a new hardware rather than tuning your (fantastic) widget.

Thanks!
Cesare

[Rich]

Cesare Bavazzano <cesare.b...@gmail.com> wrote:
> Hello Nemethi,

> No, I didn't make use of the -sortcommand option.

How _did_ you perform the sort then?

> Meanwhile I could try my application on a faster PC (2.80GHz, 16GB
> RAM) and it took just one second or two to sort the whole table.
> Possibly the solution for slowness is a new hardware rather than
> tuning your (fantastic) widget.

Except that even an "AMD Opteron Processor 2378 2.40 GHz; 4.00 GB RAM"
should sort 10,000 items much faster than 8 seconds.

Note:

$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Celeron(R) CPU 2.00GHz
stepping : 7
cpu MHz : 1992.058
cache size : 128 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
pbe up
pebs bts
bogomips : 3984.11
clflush size : 64
cache_alignment : 128
address sizes : 36 bits physical, 32 bits virtual
power management:

This is a Pentium4 Celeron running at 2Ghz - i.e., it is anchient
compared to your 'slower' operating CPU.

$ cat sort-test
#!/usr/bin/tclsh

for {set i 0} {$i < 10000} {incr i} {
set temp [list]
for {set j 0} {$j < 7} {incr j} {
lappend temp [expr {rand()}]
}
lappend data $temp
}

# temp is now a list of 10,000 elements, each element being a list of 7
# random values from rand()

# now, sort temp:

puts "-ascii [time {lsort -ascii -index 1 $data}]"
puts "-dictionary [time {lsort -dictionary -index 1 $data}]"
puts "-real [time {lsort -real -index 1 $data}]"

puts "-ascii [time {lsort -ascii -index 1 $data}]"
puts "-dictionary [time {lsort -dictionary -index 1 $data}]"
puts "-real [time {lsort -real -index 1 $data}]"

The above generates a 'random' set of data, then times sorting it,
twice to check for any 'warmup' effects.

Results:

$ ./sort-test
-ascii 558799 microseconds per iteration
-dictionary 50199 microseconds per iteration
-real 12513 microseconds per iteration
-ascii 21358 microseconds per iteration
-dictionary 52076 microseconds per iteration
-real 13300 microseconds per iteration

Note, the slowest, on a Pentium4 Celeron 2Ghz is .558 seconds (one half
second). On the second pass, the slowest time is .052 seconds (five
one hundreths of a second).

Taking eight seconds on your significantly faster CPU means something
else is wrong somewhere.

[Rich]

Rich <ri...@example.invalid> wrote:
> Cesare Bavazzano <cesare.b...@gmail.com> wrote:
> > Hello Nemethi,

> > No, I didn't make use of the -sortcommand option.

> How _did_ you perform the sort then?

> > Meanwhile I could try my application on a faster PC (2.80GHz, 16GB
> > RAM) and it took just one second or two to sort the whole table.
> > Possibly the solution for slowness is a new hardware rather than
> > tuning your (fantastic) widget.

Same test code as my previous followup, only running on this CPU:

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 30
model name : Intel(R) Core(TM) i7 CPU Q 840 @ 1.87GHz
stepping : 5
microcode : 0x3
cpu MHz : 1199.000
cache size : 8192 KB

Results in this timing:

$ ./sort-test
-ascii 7513 microseconds per iteration
-dictionary 14561 microseconds per iteration
-real 2548 microseconds per iteration
-ascii 2988 microseconds per iteration
-dictionary 13245 microseconds per iteration
-real 2031 microseconds per iteration

Worst case above is 0.14 seconds, which is slightly more than one one
hundreth of a second.

The timings above are more in keeping with what you should be seeing
for sort times.

Another timing result:

processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 44
model name : Mobile AMD Sempron(tm) Processor 3000+
stepping : 2
cpu MHz : 800.000
cache size : 128 KB

$ ./sort-test
-ascii 323190 microseconds per iteration
-dictionary 51405 microseconds per iteration
-real 7975 microseconds per iteration
-ascii 12503 microseconds per iteration
-dictionary 49904 microseconds per iteration
-real 8323 microseconds per iteration

Worst time, three tenths of a second.

[Cesare Bavazzano]

Yes, There might be something wrong in either the code or in my PC ...
Extract:

------
option add *Tablelist.showSeparators 1
option add *Tablelist.labelCommand tablelist::sortByColumn
tablelist::addBWidgetComboBox
tablelist::addBWidgetEntry

set tbl $lframe.tbl
set vsb $lframe.vsb

tablelist::tablelist $tbl \
-columns {\
0 "No." left
0 "Part Number" right
0 "PWB Footprint" right
0 "DFM Type" right
0 "status" center
0 "Owner" right
0 "Date" right} \
-setgrid no -yscrollcommand [list $vsb set] -width 0 -height 0 \
-font TkTextFont -instanttoggle 1 -selecttype cell -showarrow 1 \
-editselectedonly 1 -editstartcommand DfmtypeMask::editStartCmd \
-editendcommand DfmtypeMask::editEndCmd

$tbl configure -spacing 0 -tight 1

$tbl columnconfigure 0 -name nr -editable no -sortmode integer
$tbl columnconfigure 1 -name partnumber -editable yes -editwindow Entry \
-sortmode dictionary -font $CommonDlg::fontnormal
$tbl columnconfigure 2 -name pwbfootprint -editable yes -editwindow Entry \
-sortmode dictionary -font $CommonDlg::fontnormal
$tbl columnconfigure 3 -name dfmtype -editable yes -editwindow ComboBox \
-sortmode dictionary -font $CommonDlg::fontnormal
$tbl columnconfigure 4 -name status -editable no -sortmode dictionary \
-formatcommand CommonDlg::emptyStr
$tbl columnconfigure 5 -name owner -editable no -editwindow Entry \
-sortmode dictionary -font $CommonDlg::fontnormal
$tbl columnconfigure 6 -name date -editable no -editwindow Entry \
-formatcommand DfmtypeMask::stdDate -sortmode dictionary -font $CommonDlg::fontnormal

scrollbar $vsb -orient vertical -command [list $tbl yview]

pack $vsb -side right -fill both
pack $tbl -side top -expand yes -fill both
------
Cesare

[nemethi]

At first glance I can't see anything obvious in your script that could
cause that poor sorting performance. There are, however, a few aspects
that would be worth examining:

1. Dictionary sorting is, in general, significantly slower than the
default alphabetic one (thanks to Rich for his valuable contribution).
It would be interesting to compare the times needed for the sortings
with -sortmode ascii|dictionary|integer in the case of your application.

2. 5 of the 7 columns have -font $CommonDlg::fontnormal. It would be
better to set this option at widget level and use a different font for
the other two columns. I am, however, not quite sure whether this has a
significant impact on the performance.

If you have any new timing results (by trying the above hints), do not
hesitate to report them here.

Good luck!

Csaba