Understanding DartVM memory usage

[Alexey Knyazev]

My app have to continuously (24/7) download some data from network and write it sometimes to disk (or forward somewhere).

After ~55 mins of running (DartVM 1.13.0-dev.5.0 on linux_x64)

# cat /proc/9558/status
Name:   dart
State:  S (sleeping)
Tgid:   9558
Pid:    9558
PPid:   3645
TracerPid:      0
Uid:    1000    1000    1000    1000
Gid:    1000    1000    1000    1000
FDSize: 256
Groups: 27 33 1000
VmPeak:   639644 kB
VmSize:   633508 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:     95300 kB
VmRSS:     95224 kB
VmData:   606016 kB
VmStk:       136 kB
VmExe:      9332 kB
VmLib:      3396 kB
VmPTE:       940 kB
VmSwap:        0 kB

At the same time, 'allocation profile' page in Observatory shows

New Generation
used 721.4KB of 32.0MB
external 40.1KB
collections 310
average collection time 0.20 ms
cumulative collection time 0.06 secs
average time between collections 10346.97 ms

Old Generation
used 8.1MB of 9.6MB
external 0B
collections 5
average collection time 6.87 ms
cumulative collection time 0.03 secs
average time between collections 642628.15 ms

Is it OK for VmSize to grow that big?

[Alexey Knyazev]

Looks like stable branch (DartVM 1.12.1 on linux_x64) manages to keep memory consumption more stable (same app, same inbound network data source, almost same execution time ~55 min):

# cat /proc/28389/status

Name:   dart
State:  S (sleeping)

Tgid:   28389
Pid:    28389
PPid:   28229

TracerPid:      0
Uid:    1000    1000    1000    1000
Gid:    1000    1000    1000    1000
FDSize: 256

Groups: 33 1000
VmPeak:   334736 kB
VmSize:   333892 kB

VmLck:         0 kB
VmPin:         0 kB

VmHWM:    205524 kB
VmRSS:    187836 kB
VmData:   304556 kB
VmStk:       136 kB
VmExe:     11356 kB
VmLib:      3324 kB
VmPTE:       520 kB
VmSwap:        0 kB

Observatory:

New Generation
used 28.7MB of 32.0MB
external 1.7MB
collections 62
average collection time 0.36 ms
cumulative collection time 0.02 secs
average time between collections 62460.11 ms

Old Generation
used 16.7MB of 17.4MB
external 16B
collections 4
average collection time 5.86 ms
cumulative collection time 0.02 secs
average time between collections 968131.72 ms

суббота, 3 октября 2015 г., 16:45:31 UTC+4 пользователь Alexey Knyazev написал:

[William Hesse]

It is possible that there may be a small memory leak in the new SecureSocket code.  We fixed two large memory leaks.  Does your network download use HTTPS?  I'll check for more memory leaks.  Does the memory usage keep increasing on your process, as time passes?

Some I/O code allocates and uses memory on the C++ heap, as well as the DART heap, so that is how the process can use more memory than the Dart heap would suggest.

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[Alexey Knyazev]

My application uses TCP Socket (no TLS, no HTTP) for networking and File.openWrite().addStream() for disk I/O.

As I've found yesterday, the main reason for the process memory to grow that fast was my active usage of Observatory. VmSize gets notably bigger with almost each click on "Refresh" button on "Allocation profile" page.

But nevertheless, with Observatory disabled DartVM (1.12.1 on linux_x64) has almost doubled VmSize (was about 150M, now 270M) and almost tripled VmRSS (was 30M, now 100M) after running 9+ hrs.

I understand that these numbers don't matter a lot because virtual memory is not the same as physical and Dart can allocate a lot while never really use that much. I just want to be sure that dart process wont crash with OOM after several days. 

понедельник, 5 октября 2015 г., 10:33:50 UTC+4 пользователь William Hesse написал: