Neo4j: Ver 3.0.4 delete and re-insert label corruption

[kmc...@absolute-performance.com]

I deleted the nodes from a label:

  match(n:clients) delete n;

When I attempted to re-load from CSV:

using periodic commit 10000

LOAD CSV WITH HEADERS FROM 'file:/export/warehouse/tmp/clients_08-31-2016_02-03.txt' as row FIELDTERMINATOR '\t' with row as row where row.ClientsID is not null

merge (x:clients {ClientsID:row.ClientsID})

on create set x+=row on match set x=row

return count(*);

I get the error:

104 ms

WARNING: Node with id 122046120

I take it this is a node re-use error.

But the label is now corrupted. There are no node properties when I attempt to access the nodes.

Trying to delete all nodes:

 match (n:clients) delete n;

produces the result:

+--------------------------------------------+

| No data returned, and nothing was changed. |

+--------------------------------------------+

6 ms

A count shows there are 29 rows. How do I remove the rows or then entire label? 

 

 

[Michael Hunger]

This sounds very unsual.

Where does the count show 29 ?

Do you have a constraint on clients(ClientsID) ?

please note that both labels and properties are case-sensitive

Are you sure that your row.ClientsID is unique ?

Can you share the full error, it misses the second part that explains the duplicate.

I presume the duplicate is on another constrained field.

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kevin McGinn]

The count of 29 is via the command:

      match(n:clients) return count(*);

The query

      match(n:clients) return n;

returns an empty set.

ClientsID had a unique constraint defined on. I dropped it with the intent of re-creating in hopes it would help correct the problem.

Currently, I can not restart neo4j. I shut down neo4j and removed the indexes to force an index rebuild. At restart attempt, the neo4j.log file contains the error:

Caused by: java.lang.IllegalStateException: Index entered FAILED state while recovery waited for it to be fully populated

If I can figure out what this means and fix this error, I will regen the clients data error and get more information to you. As an aside, I have 66G of data; 13G of indexes but have only 36G of ram.

Kevin P. McGinn, PMP

Absolute Performance Inc.

12303 Airport Way, Suite 100

Broomfield, CO 80021

cell: (303) 305-9844

kmc...@absolute-performance.com


NON-DISCLOSURE NOTICE:  This communication including any and all attachments is for the intended recipient(s) only and may contain confidential and privileged information.  If you are not the intended recipient of this communication, any disclosure, copying further distribution or use of this communication is prohibited.  If you received this communication in error, please contact the sender and delete/destroy all copies of this communication immediately.

On Fri, Sep 2, 2016 at 3:44 AM, 'Michael Hunger' via Neo4j <ne...@googlegroups.com> wrote:

This sounds very unsual.

Where does the count show 29 ?

Do you have a constraint on clients(ClientsID) ?

please note that both labels and properties are case-sensitive

Are you sure that your row.ClientsID is unique ?

Can you share the full error, it misses the second part that explains the duplicate.

I presume the duplicate is on another constrained field.

Am 01.09.2016 um 22:05 schrieb kmcginn@absolute-performance.com:

I deleted the nodes from a label:

  match(n:clients) delete n;

When I attempted to re-load from CSV:

using periodic commit 10000

LOAD CSV WITH HEADERS FROM 'file:/export/warehouse/tmp/clients_08-31-2016_02-03.txt' as row FIELDTERMINATOR '\t' with row as row where row.ClientsID is not null

merge (x:clients {ClientsID:row.ClientsID})

on create set x+=row on match set x=row

return count(*);

I get the error:

104 ms

WARNING: Node with id 122046120

I take it this is a node re-use error.

But the label is now corrupted. There are no node properties when I attempt to access the nodes.

Trying to delete all nodes:

 match (n:clients) delete n;

produces the result:

+--------------------------------------------+

| No data returned, and nothing was changed. |

+--------------------------------------------+

6 ms

A count shows there are 29 rows. How do I remove the rows or then entire label? 

 

 

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/gwurFd4JGhM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+unsubscribe@googlegroups.com.

[Michael Hunger]

That sounds really unusual.

Can you stop the server and delete $NEO4J_HOME/data/databases/graph.db/schema/label

and restart, then the label index should be rebuilt.

also please share the result of the "schema" command.

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

[Kevin McGinn]

I deleted the index and attempted to restart. After an hour, I received an out of memory error. I changed the heap size from 4G to 8G and retried. After 45 min. I received the error:

Caused by: java.lang.IllegalStateException: Index entered FAILED state while recovery waited for it to be fully populated

        at org.neo4j.kernel.impl.api.index.IndexingService.awaitOnline(IndexingService.java:339)

        at org.neo4j.kernel.impl.api.index.IndexingService.start(IndexingService.java:319)

I am not sure if there are any options other than to start from scratch which is high undesirable.

Kevin P. McGinn, PMP

Absolute Performance Inc.

12303 Airport Way, Suite 100

Broomfield, CO 80021

cell: (303) 305-9844

kmc...@absolute-performance.com


NON-DISCLOSURE NOTICE:  This communication including any and all attachments is for the intended recipient(s) only and may contain confidential and privileged information.  If you are not the intended recipient of this communication, any disclosure, copying further distribution or use of this communication is prohibited.  If you received this communication in error, please contact the sender and delete/destroy all copies of this communication immediately.

On Fri, Sep 2, 2016 at 6:33 AM, 'Michael Hunger' via Neo4j <ne...@googlegroups.com> wrote:

That sounds really unusual.

Can you stop the server and delete $NEO4J_HOME/data/databases/graph.db/schema/label

and restart, then the label index should be rebuilt.

also please share the result of the "schema" command.

Michael

Am 02.09.2016 um 14:28 schrieb Kevin McGinn <kmcginn@absolute-performance.com>:

The count of 29 is via the command:

      match(n:clients) return count(*);

The query

      match(n:clients) return n;

returns an empty set.

ClientsID had a unique constraint defined on. I dropped it with the intent of re-creating in hopes it would help correct the problem.

Currently, I can not restart neo4j. I shut down neo4j and removed the indexes to force an index rebuild. At restart attempt, the neo4j.log file contains the error:

Caused by: java.lang.IllegalStateException: Index entered FAILED state while recovery waited for it to be fully populated

If I can figure out what this means and fix this error, I will regen the clients data error and get more information to you. As an aside, I have 66G of data; 13G of indexes but have only 36G of ram.

Kevin P. McGinn, PMP

Absolute Performance Inc.

12303 Airport Way, Suite 100

Broomfield, CO 80021

cell: (303) 305-9844

kmcginn@absolute-performance.com

[Mattias Persson]

Failures to populate index should put the actual error in <db>/schema/index/lucene/<indexid>/failure-message or similar. Can you see what's inside that file?

On Friday, September 2, 2016 at 5:31:24 PM UTC+2, Kevin McGinn wrote:

I deleted the index and attempted to restart. After an hour, I received an out of memory error. I changed the heap size from 4G to 8G and retried. After 45 min. I received the error:

Caused by: java.lang.IllegalStateException: Index entered FAILED state while recovery waited for it to be fully populated

        at org.neo4j.kernel.impl.api.index.IndexingService.awaitOnline(IndexingService.java:339)

        at org.neo4j.kernel.impl.api.index.IndexingService.start(IndexingService.java:319)

I am not sure if there are any options other than to start from scratch which is high undesirable.

Kevin P. McGinn, PMP

Absolute Performance Inc.

12303 Airport Way, Suite 100

Broomfield, CO 80021

cell: (303) 305-9844

kmc...@absolute-performance.com


NON-DISCLOSURE NOTICE:  This communication including any and all attachments is for the intended recipient(s) only and may contain confidential and privileged information.  If you are not the intended recipient of this communication, any disclosure, copying further distribution or use of this communication is prohibited.  If you received this communication in error, please contact the sender and delete/destroy all copies of this communication immediately.

On Fri, Sep 2, 2016 at 6:33 AM, 'Michael Hunger' via Neo4j <ne...@googlegroups.com> wrote:

That sounds really unusual.

Can you stop the server and delete $NEO4J_HOME/data/databases/graph.db/schema/label

and restart, then the label index should be rebuilt.

also please share the result of the "schema" command.

Michael

Am 02.09.2016 um 14:28 schrieb Kevin McGinn <kmc...@absolute-performance.com>:

The count of 29 is via the command:

      match(n:clients) return count(*);

The query

      match(n:clients) return n;

returns an empty set.

ClientsID had a unique constraint defined on. I dropped it with the intent of re-creating in hopes it would help correct the problem.

Currently, I can not restart neo4j. I shut down neo4j and removed the indexes to force an index rebuild. At restart attempt, the neo4j.log file contains the error:

Caused by: java.lang.IllegalStateException: Index entered FAILED state while recovery waited for it to be fully populated

If I can figure out what this means and fix this error, I will regen the clients data error and get more information to you. As an aside, I have 66G of data; 13G of indexes but have only 36G of ram.

Kevin P. McGinn, PMP

Absolute Performance Inc.

12303 Airport Way, Suite 100

Broomfield, CO 80021

cell: (303) 305-9844

kmc...@absolute-performance.com


NON-DISCLOSURE NOTICE:  This communication including any and all attachments is for the intended recipient(s) only and may contain confidential and privileged information.  If you are not the intended recipient of this communication, any disclosure, copying further distribution or use of this communication is prohibited.  If you received this communication in error, please contact the sender and delete/destroy all copies of this communication immediately.

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/gwurFd4JGhM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/gwurFd4JGhM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

[Kevin McGinn]

I appreciate you responding to my issue. Unfortunately, do to time constraints I rebuilt the database before I was able to do as you suggested. As an aside, I have found that many issues are resolved when I shut down neo4j, remove the <database>/schema/label dir and allow it to rebuild on startup. The one issue that I have found that is not recoverable is if disk space is exhausted during an insert/update operation.

Kevin P. McGinn, PMP

Absolute Performance Inc.

12303 Airport Way, Suite 100

Broomfield, CO 80021

cell: (303) 305-9844

kmc...@absolute-performance.com


NON-DISCLOSURE NOTICE:  This communication including any and all attachments is for the intended recipient(s) only and may contain confidential and privileged information.  If you are not the intended recipient of this communication, any disclosure, copying further distribution or use of this communication is prohibited.  If you received this communication in error, please contact the sender and delete/destroy all copies of this communication immediately.

To unsubscribe from this group and all its topics, send an email to neo4j+unsubscribe@googlegroups.com.