Citus columnar copy command issue in different sessions

[zohaib khan]

Hi,

I observed that copy command to columnar database worked fine when its used in the same session but it crashed PG database when I opened a new session and used same copy command to push data into columnar table. 

These sequences of command works fine in single database connection. 

CREATE FOREIGN TABLE your_cstore_table

(

your_fields

)

SERVER cstore_server

OPTIONS(compression 'pglz');

copy your_cstore_table from file;

The moment I execute command "copy your_cstore_table from file;" in another session, I get this error message.

server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
You are currently not connected to a database.

Can someone please advise if I am missing something ?

[Balazs Gunics]

Client side copy (psql \copy ) works fine for me. I had 20+ running at the same time.

Maybe try using that?

--
You received this message because you are subscribed to the Google Groups "cstore users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cstore-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cstore-users/4b03680a-86f5-4a85-9502-2b249fe4abd7%40googlegroups.com.

[zohaib khan]

I tried it with psql command and it returned same error.  

 psql -h hostname -d database -c "\COPY TABLE FROM 'file.trans.dat' delimiter '|' CSV HEADER QUOTE '\"' ESCAPE '\'; "
Password for user xxxx:

server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

On Friday, January 17, 2020 at 4:00:07 PM UTC, Balazs Gunics wrote:

Client side copy (psql \copy ) works fine for me. I had 20+ running at the same time.

Maybe try using that?

On Fri, 17 Jan 2020, 16:36 zohaib khan, <kzo...@gmail.com> wrote:

Hi,

I observed that copy command to columnar database worked fine when its used in the same session but it crashed PG database when I opened a new session and used same copy command to push data into columnar table. 

These sequences of command works fine in single database connection. 

CREATE FOREIGN TABLE your_cstore_table

(

your_fields

)

SERVER cstore_server

OPTIONS(compression 'pglz');

copy your_cstore_table from file;

The moment I execute command "copy your_cstore_table from file;" in another session, I get this error message.

server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
You are currently not connected to a database.

Can someone please advise if I am missing something ?

--
You received this message because you are subscribed to the Google Groups "cstore users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cstore...@googlegroups.com.

[Balazs Gunics]

Look into the server logs. Check dmesg for oomkiller maybe. You need to find why the server crash. Do you have enough memory allocated to postgres?

To unsubscribe from this group and stop receiving emails from it, send an email to cstore-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cstore-users/af1abbaf-098f-44ed-9d20-aa4bffea45df%40googlegroups.com.

[Murat Tuncer]

when you copy into the same table only one operation is allowed to continue, others wait until the first one completes. 

On Friday, January 17, 2020 at 7:00:07 PM UTC+3, Balazs Gunics wrote:

Client side copy (psql \copy ) works fine for me. I had 20+ running at the same time.

Maybe try using that?

On Fri, 17 Jan 2020, 16:36 zohaib khan, <kzo...@gmail.com> wrote:

Hi,

I observed that copy command to columnar database worked fine when its used in the same session but it crashed PG database when I opened a new session and used same copy command to push data into columnar table. 

These sequences of command works fine in single database connection. 

CREATE FOREIGN TABLE your_cstore_table

(

your_fields

)

SERVER cstore_server

OPTIONS(compression 'pglz');

copy your_cstore_table from file;

The moment I execute command "copy your_cstore_table from file;" in another session, I get this error message.

server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
You are currently not connected to a database.

Can someone please advise if I am missing something ?

--
You received this message because you are subscribed to the Google Groups "cstore users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cstore...@googlegroups.com.

[zohaib khan]

Server has enough memory and number of cores to cope with load. As I said earlier, it crashes only when I perform copy to table in another session.  I can create columnar table and copy one or multiple inserts sequentially in a session without any issue. If I try copy in separate session into same table later, it causes database to crash.

here's error available in dmesg

[193492.975625] postgres[17477]: segfault at 48 ip 00007f14d8ba51c4 sp 00007fff0e8308a0 error 4 in cstore_fdw.so[7f14d8b9c000+10000]

To view this discussion on the web visit https://groups.google.com/d/msgid/cstore-users/af1abbaf-098f-44ed-9d20-aa4bffea45df%40googlegroups.com.

[Murat Tuncer]

could you also try attaching a debugger to backend so that we can pinpoint exactly where it crashed ?

To view this discussion on the web visit https://groups.google.com/d/msgid/cstore-users/af1abbaf-098f-44ed-9d20-aa4bffea45df%40googlegroups.com.

[zohaib khan]

Here's PG log.

2020-01-17 11:45:48.686 EST [17432] DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2020-01-17 11:45:48.686 EST [17432] HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2020-01-17 11:45:48.687 EST [6015] LOG:  all server processes terminated; reinitializing
2020-01-17 11:45:48.701 EST [17478] LOG:  database system was interrupted; last known up at 2020-01-17 11:45:02 EST
2020-01-17 11:45:48.732 EST [17478] LOG:  database system was not properly shut down; automatic recovery in progress
2020-01-17 11:45:48.738 EST [17478] LOG:  redo starts at 8D/BCC62CB0
2020-01-17 11:45:48.738 EST [17478] LOG:  invalid record length at 8D/BCC62CE8: wanted 24, got 0
2020-01-17 11:45:48.738 EST [17478] LOG:  redo done at 8D/BCC62CB0

[zohaib khan]

Is it possible to enable some logging in postgres to get required information ?

[zohaib khan]

please find strace log attached.