How to do a PostgreSQL COPY with Node.JS?

[Victor Hooi]

Hi,

PostgreSQL offers a COPY command, to move data efficient between the database and files.

So basically, you can bulk-export a SQL command into a local CSV command.

I come from the Python world, and there's the psycopg2 library, which offers copy_to() method to dump a SQL command straight into a local file-like object:

http://initd.org/psycopg/docs/usage.html#using-copy-to-and-copy-from

I'm new to Node.js, and I'm wondering if there's something similar for Node.JS - perhaps that can stream the resulting CSV file back?

I had a look at the node-postgres library:

https://github.com/brianc/node-postgres

and from what I can tell, it seems to operate on individual rows for a query result.

Is there an efficient way of bulk dumping a PostgreSQL table into Node.JS?

Cheers,

Victor

[greelgorke]

my 10 secs on that: https://github.com/brianc/node-postgres/wiki/Client#wiki-bulk-load

[Victor Hooi]

Hi,

I saw the author also has this repo:

https://github.com/brianc/node-pg-copy-streams

I'm curious if anybody has experience with it, or how it compares to the node-postgres Client that greelgorke mentioned?

Cheers,

Victor

[Peter Rust]

The "brianc/node-postgres" library is the main pg library for node, but the author, BrianC, is concerned about it growing too bloated and has been working recently to spin off non-core functionality into 3rd party modules. node-pg-copy-streams is one of these:

Instead of adding a bunch more code to the already bloated node-postgres I am trying to make the internals extensible and work on adding edge-case features as 3rd party modules. This is one of those.

I don't have personal experience with either of these -- we just use child_process.spawn() to run the pg copy command directly (which is probably similar to what these do under the hood). I'm not sure if we tried the copy command in the "node-postgres" library and had trouble with it or if we didn't know about it or it wasn't available at the time.

-- peter