errors on delete - Neo4j

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion errors on delete
Received-SPF: pass (google.com: domain of yaron...@gmail.com designates 209.85.216.179 as permitted sender) client-ip=209.85.216.179;
MIME-Version: 1.0
In-Reply-To: <CACODHWN=kia78_Z=vTGPLvub2hPmW7zLhBYGDowHpND_kQQ...@mail.gmail.com>
References: <37aa1a34-1c74-4fbb-87a4-1ceca873a6ef@googlegroups.com>
	<CACODHWN=kia78_Z=vTGPLvub2hPmW7zLhBYGDowHpND_kQQ...@mail.gmail.com>
Date: Sun, 24 Jun 2012 01:44:34 +0300
Message-ID: <CAFLNJNYp7V_frtMJD5Ee4M7aqSyPT2-kTqtdSS3x-MiV8iA...@mail.gmail.com>
Subject: Re: [Neo4j] errors on delete
From: Yaron Naveh <yaron...@gmail.com>
To: neo4j@googlegroups.com
Content-Type: multipart/alternative; boundary=485b397dd04124df4304c32b7eba

--485b397dd04124df4304c32b7eba
Content-Type: text/plain; charset=UTF-8

Thanks

Actually I have a node X, and multiple nodes a..z that are linked to it. I
am deleting all links. Since all links are different (except their end
point X) I'm not sure how deleting one link should lock another?

what does it mean to do it on the server side?
not sure how how to handle it externally - if I have multiple clients on
different machines I cannot synchronize them. I am expecting neo4j to lock
everything of interest while deleting a link.

On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis <florent.em...@gmail.com>wrote:

> (Answering this with very partial knowledge, I use this as an occasion to
> learn stuff about Neo4J)
> In 2009, similar issues arose (
> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
> The advice given then was to manage the locking mechanism by hand.
> Looking at your trace, I'd say that your data is like this:
>
> (Node X) -[Rel 8253]-(Node 1841)
>
> You are currently trying to delete node X.
> Sometime before, you ran an operation (a delete I guess?) on node 1841
> To be able to perform this, Relationship 8253 has to be locked (it will be
> deleted, since per
> http://docs.neo4j.org/chunked/milestone/transactions-delete.html  all
> properties and relationships of a node are deleted upon delete of the node
> itself)
> When you do it in synch mode:
> You: Delete Node 1841
> You: Wait
> Server: Delete confirmed
> You: Delete Node X
>
> Works, obviously
> When you do it in asynch mode:
> What you are attempting to do is:
> You:Delete Node 1841
> Server:Locks Node 1841 and all its relationships
> You:Delete Node X
> Server:Attemps to lock 1841 and all its relationships: fails on rel 8253
> Server:Boom, Err 500 on delete Node X
> Server:(probably) Delete Node 1841 confirmed
>
> Either you can manage the lock on the server side like suggested in the
> 2009 thread, or I think you'll have to manage it externally.
>
> Can you add some sort of management in your delete calls to the server,
> partionning it to avoid collisions? Something akin to this:
> [0...33][34...66][67...99]
> (this example assumes that relationships are all between direct neighbours
> of course...)
> 3 threads run in parallel.
> Thread A starts at 0 stops at 33
> Thread B starts at 66 stops at 34
> Thread B starts at 99 stops at 67
>
> This should in theory avoid collisions if we assume delete operation
> duration to be constant for all nodes...?
> (if you want to be extra careful, add a dampenning at the end of the
> partition: as you get closer to the boundary, wait a little bit between
> calls...it will give extra time to the neighbouring thread to move further
> away....)
>
> Please remember I'm probably as new to Neo4J as you, so take all this with
> a grain of salt :-)
>
> Florent
> 2012/6/23 Yaron Naveh <yaron...@gmail.com>
>
>> Hi
>>
>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
>>
>> I need to delete 100 nodes. When I do this synchronously there is no
>> problem (e.g. I wait for one delete to finish before calling the next one).
>> When I delete them in an async manner (many rest requests sent
>> concurrently) I intermittently get a few failures. I see this coming back
>> in the http response once per each error:
>>
>> HTTP/1.1 500 Transaction(15744)[STATUS_ACTIVE,Resources=1] can't wait on
>> resource RWLock[Relationship[8253]] since =>
>> Transaction(15744)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]-
>> RWLock[Node[1841]] <-[:WAITING_FOR]-
>> Transaction(15744)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]-
>> RWLock[Relationship[8253]]
>> Content-Length: 0
>> Server: Jetty(6.1.25)
>>
>> I believe neo4j uses keep-alive to reuse the same connection but this
>> does not seem relevant. I have verified my code is (logically) correct,
>> e.g. I do not delete an already deleted node and etc.
>>
>> Any idea?
>>
>> Thanks,
>> Yaron
>>
>
>


-- 

I'm on Twitter (@YaronNaveh <http://twitter.com/#!/YaronNaveh>)

--485b397dd04124df4304c32b7eba
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks<div><br></div><div>Actually I have a node X, and mu=
ltiple nodes a..z that are linked to it. I am deleting all links. Since all=
 links are different (except their end point X) I&#39;m not sure how deleti=
ng one link should lock another?</div>
<div><br></div><div>what does it mean to do it on the server side?</div><di=
v>not sure how how to handle it externally - if I have multiple clients on =
different machines I cannot=C2=A0synchronize=C2=A0them. I am expecting neo4=
j to lock everything of interest while deleting a link.</div>
<div><br><div class=3D"gmail_quote">On Sat, Jun 23, 2012 at 10:52 PM, Flore=
nt Empis <span dir=3D"ltr">&lt;<a href=3D"mailto:florent.em...@gmail.com" t=
arget=3D"_blank">florent.em...@gmail.com</a>&gt;</span> wrote:<br><blockquo=
te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so=
lid;padding-left:1ex">
(Answering this with very partial knowledge, I use this as an occasion to l=
earn stuff about Neo4J)<div>In 2009, similar issues arose (
<a href=3D"http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html" =
target=3D"_blank">http://www.mail-archive.com/u...@lists.neo4j.org/msg01870=
.html</a>=C2=A0)</div><div>The advice given then was to manage the locking =
mechanism by hand.</div>

<div>Looking at your trace, I&#39;d say that your data is like this:</div><=
div><br></div><div>(Node X) -[Rel 8253]-(Node 1841)<br><br>You are currentl=
y trying to delete node X.</div><div>Sometime before, you ran an operation =
(a delete I guess?) on node 1841<br>

To be able to perform this, Relationship 8253 has to be locked (it will be =
deleted, since per=C2=A0
<a href=3D"http://docs.neo4j.org/chunked/milestone/transactions-delete.html=
" target=3D"_blank">http://docs.neo4j.org/chunked/milestone/transactions-de=
lete.html</a>=C2=A0 all properties and relationships of a node are deleted =
upon delete of the node itself)</div>

<div>When you do it in synch mode:</div><div>You: Delete Node 1841</div><di=
v>You: Wait</div><div>Server: Delete confirmed</div><div>You: Delete Node X=
</div><div><br></div><div>Works, obviously</div><div>When you do it in asyn=
ch mode:</div>

<div>What you are attempting to do is:</div><div>You:Delete Node 1841</div>=
<div>Server:Locks Node 1841 and all its relationships</div><div>You:Delete =
Node X</div><div>Server:Attemps to lock 1841 and all its relationships: fai=
ls on rel 8253</div>

<div>Server:Boom, Err 500 on delete Node X</div><div>Server:(probably) Dele=
te Node 1841 confirmed</div><div><br></div><div>Either you can manage the l=
ock on the server side like suggested in the 2009 thread, or I think you&#3=
9;ll have to manage it externally.</div>

<div><br></div><div>Can you add some sort of management in your delete call=
s to the server, partionning it to avoid collisions? Something akin to this=
:</div><div>[0...33][34...66][67...99]</div><div>(this example assumes that=
 relationships are all between direct neighbours of course...)</div>

<div>3 threads run in parallel.</div><div>Thread A starts at 0 stops at 33<=
/div><div>Thread B starts at 66 stops at 34</div><div>Thread B starts at 99=
 stops at 67</div><div><br></div><div>This should in theory avoid collision=
s if we assume delete operation duration to be constant for all nodes...?</=
div>

<div>(if you want to be extra careful, add a dampenning at the end of the p=
artition: as you get closer to the boundary, wait a little bit between call=
s...it will give extra time to the neighbouring thread to move further away=
....)</div>

<div><br></div><div>Please remember I&#39;m probably as new to Neo4J as you=
, so take all this with a grain of salt :-)</div><span class=3D"HOEnZb"><fo=
nt color=3D"#888888"><div><br></div><div>Florent</div></font></span><div cl=
ass=3D"HOEnZb">
<div class=3D"h5"><div><div class=3D"gmail_quote">2012/6/23 Yaron Naveh <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:yaron...@gmail.com" target=3D"_blank">=
yaron...@gmail.com</a>&gt;</span><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div>Hi</div><div><br></div><div>I&#39;m usi=
ng neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).</div><div=
>

<br></div><div>I need to delete 100 nodes. When I do this=C2=A0synchronousl=
y=C2=A0there is no problem (e.g. I wait for one delete to finish before cal=
ling the next one).</div><div>When I delete them in an async manner (many r=
est requests sent concurrently) I=C2=A0intermittently=C2=A0get a few failur=
es. I see this coming back in the http response once per each error:</div>

<div><br></div><div>HTTP/1.1 500 Transaction(15744)[STATUS_ACTIVE,Resources=
=3D1] can&#39;t wait on resource RWLock[Relationship[8253]] since =3D&gt; T=
ransaction(15744)[STATUS_ACTIVE,Resources=3D1] &lt;-[:HELD_BY]- RWLock[Node=
[1841]] &lt;-[:WAITING_FOR]- Transaction(15744)[STATUS_ACTIVE,Resources=3D1=
] &lt;-[:HELD_BY]- RWLock[Relationship[8253]]</div>

<div>Content-Length: 0</div><div>Server: Jetty(6.1.25)</div><div><br></div>=
<div>I=C2=A0believe=C2=A0neo4j uses keep-alive to reuse the same connection=
 but this does not seem relevant. I have verified my code is (logically) co=
rrect, e.g. I do not delete an already deleted node and etc.</div>

<div><br></div><div>Any idea?</div><div><br></div><div>Thanks,</div><div>Ya=
ron</div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>=
<div dir=3D"ltr"><p style=3D"text-align:left;direction:ltr">I&#39;m on Twit=
ter (<a href=3D"http://twitter.com/#!/YaronNaveh" target=3D"_blank">@YaronN=
aveh</a>)</p>
</div><br>
</div></div>

--485b397dd04124df4304c32b7eba--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
