Memory leak

[feraudet.pro...@feraudet.com]

Hi all,

I have a memory leak with the script :

function sliceobj (obj, start, end) {

        var ret = {};

        for (i in obj) {

                if (parseInt(i) >= start && parseInt(i) <= end) {

                        ret[i] = obj[i];

                }

        }

        return ret;

}

function export_datas() {

        console.log('Saving datas ...');

        var datas0 = JSON.stringify(sliceobj(datas, 0, 559));

        fs.writeFileSync('/run/shm/rawdatas_0.json', datas0);

        mv('/run/shm/rawdatas_0.json', datasfiles+'rawdatas_0.json', function(err) {

                if(err) { console.log(err); }

                console.log('Writing datas[0,559] done.');

        });

        delete datas0;

        var datas1 = JSON.stringify(sliceobj(datas, 600, 1159));

        fs.writeFileSync('/run/shm/rawdatas_1.json', datas1);

        mv('/run/shm/rawdatas_1.json', datasfiles+'rawdatas_1.json', function(err) {

                if(err) { console.log(err); }

                console.log('Writing datas[600,1159] done.');

        });

        delete datas1;

        ....

}

setInterval(function () {

        export_datas();

}, 60000);

datas is a huge object (more than 300MB) like :

{

  "1651":[

               {"ts":1436194266832,"ip":"10.242.8.27","sender":"yyy...@ggg.net","recipient":"eee...@uuu.com","instance":"2bc8.559a95da.c82e5.0","country":"FR","recipient_count":"0","size":"0","ll":[48.8539,2.604],"city":"Champs-sur-marne","mails_count":1,"countries_count":1,"ip_count":1,"countries_list":["FR"],"ip2_count":1},

               {"ts":1436194266832,"ip":"10.242.8.27","sender":"yyy...@ggg.net","recipient":"eee...@uuu.com","instance":"2bc8.559a95da.c82e5.0","country":"FR","recipient_count":"0","size":"0","ll":[48.8539,2.604],"city":"Champs-sur-marne","mails_count":1,"countries_count":1,"ip_count":1,"countries_list":["FR"],"ip2_count":1},

                ....

             ],

  "1652":[

               {"ts":1436194266832,"ip":"10.242.8.27","sender":"yyy...@ggg.net","recipient":"eee...@uuu.com","instance":"2bc8.559a95da.c82e5.0","country":"FR","recipient_count":"0","size":"0","ll":[48.8539,2.604],"city":"Champs-sur-marne","mails_count":1,"countries_count":1,"ip_count":1,"countries_list":["FR"],"ip2_count":1},

               {"ts":1436194266832,"ip":"10.242.8.27","sender":"yyy...@ggg.net","recipient":"eee...@uuu.com","instance":"2bc8.559a95da.c82e5.0","country":"FR","recipient_count":"0","size":"0","ll":[48.8539,2.604],"city":"Champs-sur-marne","mails_count":1,"countries_count":1,"ip_count":1,"countries_list":["FR"],"ip2_count":1},

                ....

             ],

             ....

}

So I need to slice it to do not reach th max variable length.

Each call of export_datas() cause memory usage to increase by 1GB

I use Node v0.12.4 on ubuntu 12.04

Have you an idea ?

Regards,

Cyril 

[Ryan Graham]

Here's a few ideas..

1. Is your app actually crashing due to lack of memory? If not, it is possible that it is not actually leaking memory but V8 just isn't running low enough to bother doing any garbage collection. This behaviour is often mistaken for a memory leak.

2. `delete datas0`, `delete datas1`, etc. probably aren't doing what you think they are (they're doing nothing except evaluating to false). Are you just trying to free the memory used by the giant string? If so, just set the variable to null.

3. Where is the giant `datas` object defined and what are you doing to it outside of the interval? You aren't clearing any of it in the code you've shown, so depending on what the rest of your app is doing, it may just be getting larger and larger..

~Ryan

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/e9affcdb-34ee-4e7e-975f-ba89f6401eac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.