[PATCH] v4 recycle kernel cache via weak pointer

[Qian Yun]

Added support for clisp, cmucl, ecl, sbcl.

Benchmark (execution time and memory footprint) for mapleok.input:
(after patch vs before patch)

====
)time long
)storage long
systemCommand "read src/input/mapleok.input"
====

sbcl:
10.2s 7.5GB vs 14.4s 13.5GB

cmucl:
17.4s 5.3GB vs 24.9s 8.3GB

clisp:
132s 7.7GB vs 222s 13.7GB

ecl:
46s vs 79s


I've added debugging facility, so that you can test its effects
without compiling twice:

by using "setExpandCacheThreshold(6000)$SCACHE Kernel EXPR INT"
you effectively disable recycle on kernel cache.


I'll try to find more benchmarks as well.

BTW, lispworks and clozurecl doesn't have weak pointers,
but lispworks has weak hashtable and clozurecl has weak array.
They can be implemented, but a bit more complicated.

- Qian

[Qian Yun]

Benchmark on integ.input (replace the 2 "testcase" with
"testcaseNoClear"):

Kernels drop from 4461 to 831 with patch.

(before vs after)

clisp:
78.6s (18.0s GC) 4.4GB vs 70.1s (16.8s GC) 4.2GB

sbcl:
8.9s (0.3s GC) 7.5GB vs 8.0s (0.6s GC) 7.3GB

cmucl:
18.4s (2.9s GC) 8.1GB vs 14.2 (2.5s GC) 4.8GB

ecl:
27.7s vs 23.7s

- Qian

[Qian Yun]

Simplified the code a bit more. See attachment.

- Qian

[Qian Yun]

Fixed 2 bugs:

1. potential dangling weak pointer left in COMPACT_WEAK_ARRAY.

2. search could insert new kernels, which could trigger compact
process, so use new function 'get_index' to handle this.

- Qian

[Qian Yun]

Another benchmark, regarding "Slow formal derivatives",
for 'src/input/derham.input',

sbcl (before vs after):

n:=8, kernel count drop from 8866 to 187

5.5s (0.04s GC) 2.18GB
2.6s (0.33s GC) 0.69GB

n:=9, kernel count drop from 9484 to 236

9.6s (0.05s GC) 4.32GB
4.1s (0.43s GC) 1.28GB

n:=10, kernel count drop from 12976 to 292

16.3s (0.14s GC) 7.98GB
6.2s (0.52s GC) 2.23GB

BTW, I'm testing on reusing dummy variables names instead using
new symbols every time, this should further decrease the number
of new kernels.

- Qian