How to free memory after require(JSON) ?

[Валерий Дубчак]

Node version 8.5.0

There are 100 JSON files of 8 mb each.

I read them in the node:

var files = fs.readdirSync ('./ path /');

files.forEach (filename => {

var data = require ('./ path /' + filename);

data = null;

});

after reading all the files, the program takes 1.2 GB.

Question: why is not the memory freed after data = null; ?

I try to do:

delete require.cache ['./ path /' + filename];

global.gc ();

But it did not help.

[Murukesh Sadasivan]

Can you do a simple fs.read instead of require('...')? 

require('...') is used for loading source files and might do some internal caching to prevent repeated loading. 

Cheers

Murukesh

[Joshua Holbrook]

I suspect that your caching theory is a likely culprit. One approach is to sidestep the cache and use fs.readFileSync + JSON.parse. That kind of thing is pretty easy to hand-roll. Give it a shot and combine with your data = null dereferencing strategy and see if it helps?

--Josh

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscribe@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/8b208af4-22bb-43e8-8c50-5bf0369d1b0a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrey]

change this to delete require.cache[require.resolve('./ path /' + filename)];

keys are resolved full paths to a module

[Валерий Дубчак]

Change on it:
var jsonfile = require('jsonfile')
var fs = require('fs')

var files = fs.readdirSync('./path/'); 

files.forEach(filename => {
var data = jsonfile.readFileSync('./path/' + filename);
data = null;
});

It is help.

среда, 20 сентября 2017 г., 12:36:10 UTC+9 пользователь Валерий Дубчак написал: