Hi Racketer
I'm trying to load a log file which size is 600MB, then I found the program exhausted 3 GB resident memory just for load all the content of it via (port->lines...). I do have enough RAM but it looks like some thing went wrong here, and I do need to load them all into RAM for my use case so (read-line ...) doesn't help me.
Any comment would be preciated.
Here is may programe:
#!/usr/bin/racket
#lang racket
; Load input as list of lines
(define (input-load-lines file-name)
(if (file-exists? file-name)
(let* ([input (open-input-file file-name)]
[lines (port->lines input)])
(close-input-port input)
lines)
empty))
; Racket 7.8, compile from source code, (none cs mode)
; 100M of log requires 0.5G runtime memory
; 300M of log requires 1.5G runtime memory
; 600M of log requires 3.0G runtime memory
;
; Reference
; racket/collects/racket/port.rkt :106
; racket/collects/racket/private/portlines.rkt :11
(define input (input-load-lines "main.log"))
; 214M(VIRT)/101M(RSS) without open any file
(let loop()
(sleep 5) ; Waiting here so that I can check it via top/ps
(set! input empty) ; event not help with this line
(loop))
Thanks
Hong