encoding while parsing - polish symbols

104 views
Skip to first unread message

Kamil Lewandowski

unread,
Feb 6, 2017, 3:29:07 PM2/6/17
to nodejs
Hi 
I'm new on developing in node and i'm stuck with parsing csv file with polish symbols. Example using fast-csv with encoding settings utf16le / utf8 /base64 with "łódź" in source:
[ '齤਍' ],
[ '��d�' ],
[ 's/Nknw0K' ]


how should I load file represents texts in Central European and Eastern European languages ??


Path i try with fast-csv package

var fs = require('fs');
var csv = require("fast-csv");

var stream = fs.createReadStream("test2.csv", {
    encoding: "utf8",
    header: "true",
    rowDelimiter:';'
}
);

var csvStream = csv()
    .on("data", function(data){
         console.log(data);
    })
    .on("end", function(){
         console.log("done");
    });

stream.pipe(csvStream);

Matt

unread,
Feb 6, 2017, 10:47:02 PM2/6/17
to nod...@googlegroups.com
You'll need Iconv to convert from whatever encoding the file is in (most likely "ISO 8859-2") to UTF-8 first.

Something like this instead of the last line above:

var Iconv = require('iconv').Iconv;
var iconv = new Iconv('latin2', 'utf-8');
stream.pipe(iconv).pipe(csvStream);


--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscribe@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/71854533-bc26-45d0-8217-18a5bc85d541%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages