fast reader developed for the Ring

[Mark Tarver]

The Ring uses a lot of large data structures, specifically a transitivity table of 2.7Mb which takes many minutes to generate as well as thousands of lines of object code.  This data is stored in the cloud and read when the Ring is loaded, but I've found a bottleneck in the read function preventing the data being assimilated as fast as I'd like.  This bottleneck gives polynomial performance in the worst cases.  I'm replacing it by a reader which gives linear performance and this reader will make its way into SP.  It will likely change the game wrt loading very large databases into SP.  

Here are a couple of tests

The current OS reader.

(53-) (set *s* (open "fdg.txt" in))

#<FD-STREAM for "file C:\\Users\\User\\Google Drive\\SP Rebuild 5\\Shen\\Tarver\\fdg.txt" {1006C82713}>

(54-) (time (do (read (value *s*)) ok))

run time: 78.14100646972656 secs

ok

(55-) (close (value *s*))

[]

The prototype SP reader.

(56-) (set *s* (open "fdg.txt" in))

#<FD-STREAM for "file C:\\Users\\User\\Google Drive\\SP Rebuild 5\\Shen\\Tarver\\fdg.txt" {10054CC063}>

(57-) (time (do (sp-read (value *s*)) ok))

run time: 0.608978271484375 secs

ok

(58-) (close (value *s*)) 

This has meant a slight detour for me but the long-term benefits for SP users is well worth it.  Thanks to all for their support.

I'm meeting Professor David Hogg next month wrt OS funding to see if there is cope for proper funding for the project via the university.

Mark

[Mark Tarver]

Improved time.

(9-) (set *s* (open "fdg.txt" in))

#<FD-STREAM for "file C:\\Users\\User\\Google Drive\\SP Rebuild 5\\Shen\\Tarver\\fdg.txt" {100420C2C3}>

(10-) (time (do (read (value *s*)) ok))

run time: 0.37399983406066895 secs

ok

(11-) (close (value *s*))