I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
I need to delete 100 nodes. When I do this synchronously there is no problem (e.g. I wait for one delete to finish before calling the next one). When I delete them in an async manner (many rest requests sent concurrently) I intermittently get a few failures. I see this coming back in the http response once per each error:
I believe neo4j uses keep-alive to reuse the same connection but this does not seem relevant. I have verified my code is (logically) correct, e.g. I do not delete an already deleted node and etc.
(Answering this with very partial knowledge, I use this as an occasion to
learn stuff about Neo4J)
In 2009, similar issues arose (
http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
The advice given then was to manage the locking mechanism by hand.
Looking at your trace, I'd say that your data is like this:
(Node X) -[Rel 8253]-(Node 1841)
You are currently trying to delete node X.
Sometime before, you ran an operation (a delete I guess?) on node 1841
To be able to perform this, Relationship 8253 has to be locked (it will be
deleted, since per
http://docs.neo4j.org/chunked/milestone/transactions-delete.html all
properties and relationships of a node are deleted upon delete of the node
itself)
When you do it in synch mode:
You: Delete Node 1841
You: Wait
Server: Delete confirmed
You: Delete Node X
Works, obviously
When you do it in asynch mode:
What you are attempting to do is:
You:Delete Node 1841
Server:Locks Node 1841 and all its relationships
You:Delete Node X
Server:Attemps to lock 1841 and all its relationships: fails on rel 8253
Server:Boom, Err 500 on delete Node X
Server:(probably) Delete Node 1841 confirmed
Either you can manage the lock on the server side like suggested in the
2009 thread, or I think you'll have to manage it externally.
Can you add some sort of management in your delete calls to the server,
partionning it to avoid collisions? Something akin to this:
[0...33][34...66][67...99]
(this example assumes that relationships are all between direct neighbours
of course...)
3 threads run in parallel.
Thread A starts at 0 stops at 33
Thread B starts at 66 stops at 34
Thread B starts at 99 stops at 67
This should in theory avoid collisions if we assume delete operation
duration to be constant for all nodes...?
(if you want to be extra careful, add a dampenning at the end of the
partition: as you get closer to the boundary, wait a little bit between
calls...it will give extra time to the neighbouring thread to move further
away....)
Please remember I'm probably as new to Neo4J as you, so take all this with
a grain of salt :-)
> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
> I need to delete 100 nodes. When I do this synchronously there is no
> problem (e.g. I wait for one delete to finish before calling the next one).
> When I delete them in an async manner (many rest requests sent
> concurrently) I intermittently get a few failures. I see this coming back
> in the http response once per each error:
> I believe neo4j uses keep-alive to reuse the same connection but this does
> not seem relevant. I have verified my code is (logically) correct, e.g. I
> do not delete an already deleted node and etc.
Actually I have a node X, and multiple nodes a..z that are linked to it. I
am deleting all links. Since all links are different (except their end
point X) I'm not sure how deleting one link should lock another?
what does it mean to do it on the server side?
not sure how how to handle it externally - if I have multiple clients on
different machines I cannot synchronize them. I am expecting neo4j to lock
everything of interest while deleting a link.
On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis <florent.em...@gmail.com>wrote:
> (Answering this with very partial knowledge, I use this as an occasion to
> learn stuff about Neo4J)
> In 2009, similar issues arose (
> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
> The advice given then was to manage the locking mechanism by hand.
> Looking at your trace, I'd say that your data is like this:
> (Node X) -[Rel 8253]-(Node 1841)
> You are currently trying to delete node X.
> Sometime before, you ran an operation (a delete I guess?) on node 1841
> To be able to perform this, Relationship 8253 has to be locked (it will be
> deleted, since per
> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all
> properties and relationships of a node are deleted upon delete of the node
> itself)
> When you do it in synch mode:
> You: Delete Node 1841
> You: Wait
> Server: Delete confirmed
> You: Delete Node X
> Works, obviously
> When you do it in asynch mode:
> What you are attempting to do is:
> You:Delete Node 1841
> Server:Locks Node 1841 and all its relationships
> You:Delete Node X
> Server:Attemps to lock 1841 and all its relationships: fails on rel 8253
> Server:Boom, Err 500 on delete Node X
> Server:(probably) Delete Node 1841 confirmed
> Either you can manage the lock on the server side like suggested in the
> 2009 thread, or I think you'll have to manage it externally.
> Can you add some sort of management in your delete calls to the server,
> partionning it to avoid collisions? Something akin to this:
> [0...33][34...66][67...99]
> (this example assumes that relationships are all between direct neighbours
> of course...)
> 3 threads run in parallel.
> Thread A starts at 0 stops at 33
> Thread B starts at 66 stops at 34
> Thread B starts at 99 stops at 67
> This should in theory avoid collisions if we assume delete operation
> duration to be constant for all nodes...?
> (if you want to be extra careful, add a dampenning at the end of the
> partition: as you get closer to the boundary, wait a little bit between
> calls...it will give extra time to the neighbouring thread to move further
> away....)
> Please remember I'm probably as new to Neo4J as you, so take all this with
> a grain of salt :-)
>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
>> I need to delete 100 nodes. When I do this synchronously there is no
>> problem (e.g. I wait for one delete to finish before calling the next one).
>> When I delete them in an async manner (many rest requests sent
>> concurrently) I intermittently get a few failures. I see this coming back
>> in the http response once per each error:
>> I believe neo4j uses keep-alive to reuse the same connection but this
>> does not seem relevant. I have verified my code is (logically) correct,
>> e.g. I do not delete an already deleted node and etc.
> Actually I have a node X, and multiple nodes a..z that are linked to it. I am deleting all links. Since all links are different (except their end point X) I'm not sure how deleting one link should lock another?
> what does it mean to do it on the server side?
> not sure how how to handle it externally - if I have multiple clients on different machines I cannot synchronize them. I am expecting neo4j to lock everything of interest while deleting a link.
> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis <florent.em...@gmail.com> wrote:
> (Answering this with very partial knowledge, I use this as an occasion to learn stuff about Neo4J)
> In 2009, similar issues arose ( http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
> The advice given then was to manage the locking mechanism by hand.
> Looking at your trace, I'd say that your data is like this:
> (Node X) -[Rel 8253]-(Node 1841)
> You are currently trying to delete node X.
> Sometime before, you ran an operation (a delete I guess?) on node 1841
> To be able to perform this, Relationship 8253 has to be locked (it will be deleted, since per http://docs.neo4j.org/chunked/milestone/transactions-delete.html all properties and relationships of a node are deleted upon delete of the node itself)
> When you do it in synch mode:
> You: Delete Node 1841
> You: Wait
> Server: Delete confirmed
> You: Delete Node X
> Works, obviously
> When you do it in asynch mode:
> What you are attempting to do is:
> You:Delete Node 1841
> Server:Locks Node 1841 and all its relationships
> You:Delete Node X
> Server:Attemps to lock 1841 and all its relationships: fails on rel 8253
> Server:Boom, Err 500 on delete Node X
> Server:(probably) Delete Node 1841 confirmed
> Either you can manage the lock on the server side like suggested in the 2009 thread, or I think you'll have to manage it externally.
> Can you add some sort of management in your delete calls to the server, partionning it to avoid collisions? Something akin to this:
> [0...33][34...66][67...99]
> (this example assumes that relationships are all between direct neighbours of course...)
> 3 threads run in parallel.
> Thread A starts at 0 stops at 33
> Thread B starts at 66 stops at 34
> Thread B starts at 99 stops at 67
> This should in theory avoid collisions if we assume delete operation duration to be constant for all nodes...?
> (if you want to be extra careful, add a dampenning at the end of the partition: as you get closer to the boundary, wait a little bit between calls...it will give extra time to the neighbouring thread to move further away....)
> Please remember I'm probably as new to Neo4J as you, so take all this with a grain of salt :-)
> Florent
> 2012/6/23 Yaron Naveh <yaron...@gmail.com>
> Hi
> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
> I need to delete 100 nodes. When I do this synchronously there is no problem (e.g. I wait for one delete to finish before calling the next one).
> When I delete them in an async manner (many rest requests sent concurrently) I intermittently get a few failures. I see this coming back in the http response once per each error:
> I believe neo4j uses keep-alive to reuse the same connection but this does not seem relevant. I have verified my code is (logically) correct, e.g. I do not delete an already deleted node and etc.
I still don't get the problem. I have node A with links to different nodes.
I delete the links. There should be no deadlock in my scenario. Possibly A
will be locked for a while by one delete, by why don't other deletes wait
for the lock to be released?
Also can you give me a link for batch transactions?
> Actually I have a node X, and multiple nodes a..z that are linked to it. I
> am deleting all links. Since all links are different (except their end
> point X) I'm not sure how deleting one link should lock another?
> what does it mean to do it on the server side?
> not sure how how to handle it externally - if I have multiple clients on
> different machines I cannot synchronize them. I am expecting neo4j to lock
> everything of interest while deleting a link.
> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis <florent.em...@gmail.com>wrote:
>> (Answering this with very partial knowledge, I use this as an occasion to
>> learn stuff about Neo4J)
>> In 2009, similar issues arose (
>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
>> The advice given then was to manage the locking mechanism by hand.
>> Looking at your trace, I'd say that your data is like this:
>> (Node X) -[Rel 8253]-(Node 1841)
>> You are currently trying to delete node X.
>> Sometime before, you ran an operation (a delete I guess?) on node 1841
>> To be able to perform this, Relationship 8253 has to be locked (it will
>> be deleted, since per
>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all
>> properties and relationships of a node are deleted upon delete of the node
>> itself)
>> When you do it in synch mode:
>> You: Delete Node 1841
>> You: Wait
>> Server: Delete confirmed
>> You: Delete Node X
>> Works, obviously
>> When you do it in asynch mode:
>> What you are attempting to do is:
>> You:Delete Node 1841
>> Server:Locks Node 1841 and all its relationships
>> You:Delete Node X
>> Server:Attemps to lock 1841 and all its relationships: fails on rel 8253
>> Server:Boom, Err 500 on delete Node X
>> Server:(probably) Delete Node 1841 confirmed
>> Either you can manage the lock on the server side like suggested in the
>> 2009 thread, or I think you'll have to manage it externally.
>> Can you add some sort of management in your delete calls to the server,
>> partionning it to avoid collisions? Something akin to this:
>> [0...33][34...66][67...99]
>> (this example assumes that relationships are all between direct
>> neighbours of course...)
>> 3 threads run in parallel.
>> Thread A starts at 0 stops at 33
>> Thread B starts at 66 stops at 34
>> Thread B starts at 99 stops at 67
>> This should in theory avoid collisions if we assume delete operation
>> duration to be constant for all nodes...?
>> (if you want to be extra careful, add a dampenning at the end of the
>> partition: as you get closer to the boundary, wait a little bit between
>> calls...it will give extra time to the neighbouring thread to move further
>> away....)
>> Please remember I'm probably as new to Neo4J as you, so take all this
>> with a grain of salt :-)
>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
>>> I need to delete 100 nodes. When I do this synchronously there is no
>>> problem (e.g. I wait for one delete to finish before calling the next one).
>>> When I delete them in an async manner (many rest requests sent
>>> concurrently) I intermittently get a few failures. I see this coming back
>>> in the http response once per each error:
>>> I believe neo4j uses keep-alive to reuse the same connection but this
>>> does not seem relevant. I have verified my code is (logically) correct,
>>> e.g. I do not delete an already deleted node and etc.
On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> wrote:
> Thanks
> I still don't get the problem. I have node A with links to different nodes.
> I delete the links. There should be no deadlock in my scenario. Possibly A
> will be locked for a while by one delete, by why don't other deletes wait
> for the lock to be released?
> Also can you give me a link for batch transactions?
> On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
> <michael.hun...@neotechnology.com> wrote:
>> Deleting a relationship currently locks both nodes.
>> What about deleting them all in a batch transaction?
>> Actually I have a node X, and multiple nodes a..z that are linked to it. I
>> am deleting all links. Since all links are different (except their end point
>> X) I'm not sure how deleting one link should lock another?
>> what does it mean to do it on the server side?
>> not sure how how to handle it externally - if I have multiple clients on
>> different machines I cannot synchronize them. I am expecting neo4j to lock
>> everything of interest while deleting a link.
>> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis <florent.em...@gmail.com>
>> wrote:
>>> (Answering this with very partial knowledge, I use this as an occasion to
>>> learn stuff about Neo4J)
>>> In 2009, similar issues arose (
>>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
>>> The advice given then was to manage the locking mechanism by hand.
>>> Looking at your trace, I'd say that your data is like this:
>>> (Node X) -[Rel 8253]-(Node 1841)
>>> You are currently trying to delete node X.
>>> Sometime before, you ran an operation (a delete I guess?) on node 1841
>>> To be able to perform this, Relationship 8253 has to be locked (it will
>>> be deleted, since per
>>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all
>>> properties and relationships of a node are deleted upon delete of the node
>>> itself)
>>> When you do it in synch mode:
>>> You: Delete Node 1841
>>> You: Wait
>>> Server: Delete confirmed
>>> You: Delete Node X
>>> Works, obviously
>>> When you do it in asynch mode:
>>> What you are attempting to do is:
>>> You:Delete Node 1841
>>> Server:Locks Node 1841 and all its relationships
>>> You:Delete Node X
>>> Server:Attemps to lock 1841 and all its relationships: fails on rel 8253
>>> Server:Boom, Err 500 on delete Node X
>>> Server:(probably) Delete Node 1841 confirmed
>>> Either you can manage the lock on the server side like suggested in the
>>> 2009 thread, or I think you'll have to manage it externally.
>>> Can you add some sort of management in your delete calls to the server,
>>> partionning it to avoid collisions? Something akin to this:
>>> [0...33][34...66][67...99]
>>> (this example assumes that relationships are all between direct
>>> neighbours of course...)
>>> 3 threads run in parallel.
>>> Thread A starts at 0 stops at 33
>>> Thread B starts at 66 stops at 34
>>> Thread B starts at 99 stops at 67
>>> This should in theory avoid collisions if we assume delete operation
>>> duration to be constant for all nodes...?
>>> (if you want to be extra careful, add a dampenning at the end of the
>>> partition: as you get closer to the boundary, wait a little bit between
>>> calls...it will give extra time to the neighbouring thread to move further
>>> away....)
>>> Please remember I'm probably as new to Neo4J as you, so take all this
>>> with a grain of salt :-)
>>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of neo4j).
>>>> I need to delete 100 nodes. When I do this synchronously there is no
>>>> problem (e.g. I wait for one delete to finish before calling the next one).
>>>> When I delete them in an async manner (many rest requests sent
>>>> concurrently) I intermittently get a few failures. I see this coming back in
>>>> the http response once per each error:
>>>> I believe neo4j uses keep-alive to reuse the same connection but this
>>>> does not seem relevant. I have verified my code is (logically) correct, e.g.
>>>> I do not delete an already deleted node and etc.
I'm currently using a workaround - doing everything in a sync manner. but
I'm still not sure why the async version fails, this is my main worry now.
I wouldn't expect batch to be required here - deleting multiple
relationships of a single node.
> If you can write, you can code - @coderdojomalmo
> If you can sketch, you can use a graph database - @neo4j
> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> wrote:
> > Thanks
> > I still don't get the problem. I have node A with links to different
> nodes.
> > I delete the links. There should be no deadlock in my scenario. Possibly
> A
> > will be locked for a while by one delete, by why don't other deletes wait
> > for the lock to be released?
> > Also can you give me a link for batch transactions?
> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
> > <michael.hun...@neotechnology.com> wrote:
> >> Deleting a relationship currently locks both nodes.
> >> What about deleting them all in a batch transaction?
> >> Actually I have a node X, and multiple nodes a..z that are linked to
> it. I
> >> am deleting all links. Since all links are different (except their end
> point
> >> X) I'm not sure how deleting one link should lock another?
> >> what does it mean to do it on the server side?
> >> not sure how how to handle it externally - if I have multiple clients on
> >> different machines I cannot synchronize them. I am expecting neo4j to
> lock
> >> everything of interest while deleting a link.
> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis <
> florent.em...@gmail.com>
> >> wrote:
> >>> (Answering this with very partial knowledge, I use this as an occasion
> to
> >>> learn stuff about Neo4J)
> >>> In 2009, similar issues arose (
> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
> >>> The advice given then was to manage the locking mechanism by hand.
> >>> Looking at your trace, I'd say that your data is like this:
> >>> (Node X) -[Rel 8253]-(Node 1841)
> >>> You are currently trying to delete node X.
> >>> Sometime before, you ran an operation (a delete I guess?) on node 1841
> >>> To be able to perform this, Relationship 8253 has to be locked (it will
> >>> be deleted, since per
> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all
> >>> properties and relationships of a node are deleted upon delete of the
> node
> >>> itself)
> >>> When you do it in synch mode:
> >>> You: Delete Node 1841
> >>> You: Wait
> >>> Server: Delete confirmed
> >>> You: Delete Node X
> >>> Works, obviously
> >>> When you do it in asynch mode:
> >>> What you are attempting to do is:
> >>> You:Delete Node 1841
> >>> Server:Locks Node 1841 and all its relationships
> >>> You:Delete Node X
> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel
> 8253
> >>> Server:Boom, Err 500 on delete Node X
> >>> Server:(probably) Delete Node 1841 confirmed
> >>> Either you can manage the lock on the server side like suggested in the
> >>> 2009 thread, or I think you'll have to manage it externally.
> >>> Can you add some sort of management in your delete calls to the server,
> >>> partionning it to avoid collisions? Something akin to this:
> >>> [0...33][34...66][67...99]
> >>> (this example assumes that relationships are all between direct
> >>> neighbours of course...)
> >>> 3 threads run in parallel.
> >>> Thread A starts at 0 stops at 33
> >>> Thread B starts at 66 stops at 34
> >>> Thread B starts at 99 stops at 67
> >>> This should in theory avoid collisions if we assume delete operation
> >>> duration to be constant for all nodes...?
> >>> (if you want to be extra careful, add a dampenning at the end of the
> >>> partition: as you get closer to the boundary, wait a little bit between
> >>> calls...it will give extra time to the neighbouring thread to move
> further
> >>> away....)
> >>> Please remember I'm probably as new to Neo4J as you, so take all this
> >>> with a grain of salt :-)
> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of
> neo4j).
> >>>> I need to delete 100 nodes. When I do this synchronously there is no
> >>>> problem (e.g. I wait for one delete to finish before calling the next
> one).
> >>>> When I delete them in an async manner (many rest requests sent
> >>>> concurrently) I intermittently get a few failures. I see this coming
> back in
> >>>> the http response once per each error:
> >>>> I believe neo4j uses keep-alive to reuse the same connection but this
> >>>> does not seem relevant. I have verified my code is (logically)
> correct, e.g.
> >>>> I do not delete an already deleted node and etc.
On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote:
> Thanks
> I'm currently using a workaround - doing everything in a sync manner. but
> I'm still not sure why the async version fails, this is my main worry now. I
> wouldn't expect batch to be required here - deleting multiple relationships
> of a single node.
> On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer
> <peter.neuba...@neotechnology.com> wrote:
>> If you can write, you can code - @coderdojomalmo
>> If you can sketch, you can use a graph database - @neo4j
>> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> wrote:
>> > Thanks
>> > I still don't get the problem. I have node A with links to different
>> > nodes.
>> > I delete the links. There should be no deadlock in my scenario. Possibly
>> > A
>> > will be locked for a while by one delete, by why don't other deletes
>> > wait
>> > for the lock to be released?
>> > Also can you give me a link for batch transactions?
>> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
>> > <michael.hun...@neotechnology.com> wrote:
>> >> Deleting a relationship currently locks both nodes.
>> >> What about deleting them all in a batch transaction?
>> >> Actually I have a node X, and multiple nodes a..z that are linked to
>> >> it. I
>> >> am deleting all links. Since all links are different (except their end
>> >> point
>> >> X) I'm not sure how deleting one link should lock another?
>> >> what does it mean to do it on the server side?
>> >> not sure how how to handle it externally - if I have multiple clients
>> >> on
>> >> different machines I cannot synchronize them. I am expecting neo4j to
>> >> lock
>> >> everything of interest while deleting a link.
>> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis
>> >> <florent.em...@gmail.com>
>> >> wrote:
>> >>> (Answering this with very partial knowledge, I use this as an occasion
>> >>> to
>> >>> learn stuff about Neo4J)
>> >>> In 2009, similar issues arose (
>> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
>> >>> The advice given then was to manage the locking mechanism by hand.
>> >>> Looking at your trace, I'd say that your data is like this:
>> >>> (Node X) -[Rel 8253]-(Node 1841)
>> >>> You are currently trying to delete node X.
>> >>> Sometime before, you ran an operation (a delete I guess?) on node 1841
>> >>> To be able to perform this, Relationship 8253 has to be locked (it
>> >>> will
>> >>> be deleted, since per
>> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all
>> >>> properties and relationships of a node are deleted upon delete of the
>> >>> node
>> >>> itself)
>> >>> When you do it in synch mode:
>> >>> You: Delete Node 1841
>> >>> You: Wait
>> >>> Server: Delete confirmed
>> >>> You: Delete Node X
>> >>> Works, obviously
>> >>> When you do it in asynch mode:
>> >>> What you are attempting to do is:
>> >>> You:Delete Node 1841
>> >>> Server:Locks Node 1841 and all its relationships
>> >>> You:Delete Node X
>> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel
>> >>> 8253
>> >>> Server:Boom, Err 500 on delete Node X
>> >>> Server:(probably) Delete Node 1841 confirmed
>> >>> Either you can manage the lock on the server side like suggested in
>> >>> the
>> >>> 2009 thread, or I think you'll have to manage it externally.
>> >>> Can you add some sort of management in your delete calls to the
>> >>> server,
>> >>> partionning it to avoid collisions? Something akin to this:
>> >>> [0...33][34...66][67...99]
>> >>> (this example assumes that relationships are all between direct
>> >>> neighbours of course...)
>> >>> 3 threads run in parallel.
>> >>> Thread A starts at 0 stops at 33
>> >>> Thread B starts at 66 stops at 34
>> >>> Thread B starts at 99 stops at 67
>> >>> This should in theory avoid collisions if we assume delete operation
>> >>> duration to be constant for all nodes...?
>> >>> (if you want to be extra careful, add a dampenning at the end of the
>> >>> partition: as you get closer to the boundary, wait a little bit
>> >>> between
>> >>> calls...it will give extra time to the neighbouring thread to move
>> >>> further
>> >>> away....)
>> >>> Please remember I'm probably as new to Neo4J as you, so take all this
>> >>> with a grain of salt :-)
>> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of
>> >>>> neo4j).
>> >>>> I need to delete 100 nodes. When I do this synchronously there is no
>> >>>> problem (e.g. I wait for one delete to finish before calling the next
>> >>>> one).
>> >>>> When I delete them in an async manner (many rest requests sent
>> >>>> concurrently) I intermittently get a few failures. I see this coming
>> >>>> back in
>> >>>> the http response once per each error:
>> >>>> I believe neo4j uses keep-alive to reuse the same connection but this
>> >>>> does not seem relevant. I have verified my code is (logically)
>> >>>> correct, e.g.
>> >>>> I do not delete an already deleted node and etc.
The REST API seems to have serious issues with concurrency - the new mutating Cypher stuff does appear to address this but you'll basically be doing everything the Cypher way. Which, mind you, isn't necessarily a bad thing; I am finding Cypher to be very intuitive and powerful... excepting the inconvenient pseudo-JSON format for sending property data :)
> If you can write, you can code - @coderdojomalmo > If you can sketch, you can use a graph database - @neo4j
> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: > > Thanks
> > I'm currently using a workaround - doing everything in a sync manner. > but > > I'm still not sure why the async version fails, this is my main worry > now. I > > wouldn't expect batch to be required here - deleting multiple > relationships > > of a single node.
> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer > > <peter.neuba...@neotechnology.com> wrote:
> >> If you can write, you can code - @coderdojomalmo > >> If you can sketch, you can use a graph database - @neo4j
> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> > wrote: > >> > Thanks
> >> > I still don't get the problem. I have node A with links to different > >> > nodes. > >> > I delete the links. There should be no deadlock in my scenario. > Possibly > >> > A > >> > will be locked for a while by one delete, by why don't other deletes > >> > wait > >> > for the lock to be released?
> >> > Also can you give me a link for batch transactions?
> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger > >> > <michael.hun...@neotechnology.com> wrote:
> >> >> Deleting a relationship currently locks both nodes.
> >> >> What about deleting them all in a batch transaction?
> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
> >> >> Thanks
> >> >> Actually I have a node X, and multiple nodes a..z that are linked to > >> >> it. I > >> >> am deleting all links. Since all links are different (except their > end > >> >> point > >> >> X) I'm not sure how deleting one link should lock another?
> >> >> what does it mean to do it on the server side? > >> >> not sure how how to handle it externally - if I have multiple > clients > >> >> on > >> >> different machines I cannot synchronize them. I am expecting neo4j > to > >> >> lock > >> >> everything of interest while deleting a link.
> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis > >> >> <florent.em...@gmail.com> > >> >> wrote:
> >> >>> (Answering this with very partial knowledge, I use this as an > occasion > >> >>> to > >> >>> learn stuff about Neo4J) > >> >>> In 2009, similar issues arose ( > >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) > >> >>> The advice given then was to manage the locking mechanism by hand. > >> >>> Looking at your trace, I'd say that your data is like this:
> >> >>> (Node X) -[Rel 8253]-(Node 1841)
> >> >>> You are currently trying to delete node X. > >> >>> Sometime before, you ran an operation (a delete I guess?) on node > 1841 > >> >>> To be able to perform this, Relationship 8253 has to be locked (it > >> >>> will > >> >>> be deleted, since per > >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all > >> >>> properties and relationships of a node are deleted upon delete of > the > >> >>> node > >> >>> itself) > >> >>> When you do it in synch mode: > >> >>> You: Delete Node 1841 > >> >>> You: Wait > >> >>> Server: Delete confirmed > >> >>> You: Delete Node X
> >> >>> Works, obviously > >> >>> When you do it in asynch mode: > >> >>> What you are attempting to do is: > >> >>> You:Delete Node 1841 > >> >>> Server:Locks Node 1841 and all its relationships > >> >>> You:Delete Node X > >> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel > >> >>> 8253 > >> >>> Server:Boom, Err 500 on delete Node X > >> >>> Server:(probably) Delete Node 1841 confirmed
> >> >>> Either you can manage the lock on the server side like suggested in > >> >>> the > >> >>> 2009 thread, or I think you'll have to manage it externally.
> >> >>> Can you add some sort of management in your delete calls to the > >> >>> server, > >> >>> partionning it to avoid collisions? Something akin to this: > >> >>> [0...33][34...66][67...99] > >> >>> (this example assumes that relationships are all between direct > >> >>> neighbours of course...) > >> >>> 3 threads run in parallel. > >> >>> Thread A starts at 0 stops at 33 > >> >>> Thread B starts at 66 stops at 34 > >> >>> Thread B starts at 99 stops at 67
> >> >>> This should in theory avoid collisions if we assume delete > operation > >> >>> duration to be constant for all nodes...? > >> >>> (if you want to be extra careful, add a dampenning at the end of > the > >> >>> partition: as you get closer to the boundary, wait a little bit > >> >>> between > >> >>> calls...it will give extra time to the neighbouring thread to move > >> >>> further > >> >>> away....)
> >> >>> Please remember I'm probably as new to Neo4J as you, so take all > this > >> >>> with a grain of salt :-)
> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of > >> >>>> neo4j).
> >> >>>> I need to delete 100 nodes. When I do this synchronously there is > no > >> >>>> problem (e.g. I wait for one delete to finish before calling the > next > >> >>>> one). > >> >>>> When I delete them in an async manner (many rest requests sent > >> >>>> concurrently) I intermittently get a few failures. I see this > coming > >> >>>> back in > >> >>>> the http response once per each error:
> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but > this > >> >>>> does not seem relevant. I have verified my code is (logically) > >> >>>> correct, e.g. > >> >>>> I do not delete an already deleted node and etc.
> >> >>>> Any idea?
> >> >>>> Thanks, > >> >>>> Yaron
> >> >> --
> >> >> I'm on Twitter (@YaronNaveh)
> >> > --
> >> > I'm on Twitter (@YaronNaveh)
> > --
> > I'm on Twitter (@YaronNaveh)
On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
> Could you just try Cypher to delete your relationships?
> If you can write, you can code - @coderdojomalmo > If you can sketch, you can use a graph database - @neo4j
> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: > > Thanks
> > I'm currently using a workaround - doing everything in a sync manner. > but > > I'm still not sure why the async version fails, this is my main worry > now. I > > wouldn't expect batch to be required here - deleting multiple > relationships > > of a single node.
> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer > > <peter.neuba...@neotechnology.com> wrote:
> >> If you can write, you can code - @coderdojomalmo > >> If you can sketch, you can use a graph database - @neo4j
> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> > wrote: > >> > Thanks
> >> > I still don't get the problem. I have node A with links to different > >> > nodes. > >> > I delete the links. There should be no deadlock in my scenario. > Possibly > >> > A > >> > will be locked for a while by one delete, by why don't other deletes > >> > wait > >> > for the lock to be released?
#1 Each REST call is an individual tx and individual thread, except in cypher and batch operations which group multiple operations.
#2 if you want to send properties as parameters (which you should anyway) then you can post real json as params, see
> The REST API seems to have serious issues with concurrency - the new mutating Cypher stuff does appear to address this but you'll basically be doing everything the Cypher way. Which, mind you, isn't necessarily a bad thing; I am finding Cypher to be very intuitive and powerful... excepting the inconvenient pseudo-JSON format for sending property data :)
> If you can write, you can code - @coderdojomalmo > If you can sketch, you can use a graph database - @neo4j
> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: > > Thanks
> > I'm currently using a workaround - doing everything in a sync manner. but > > I'm still not sure why the async version fails, this is my main worry now. I > > wouldn't expect batch to be required here - deleting multiple relationships > > of a single node.
> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer > > <peter.neuba...@neotechnology.com> wrote:
> >> If you can write, you can code - @coderdojomalmo > >> If you can sketch, you can use a graph database - @neo4j
> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> wrote: > >> > Thanks
> >> > I still don't get the problem. I have node A with links to different > >> > nodes. > >> > I delete the links. There should be no deadlock in my scenario. Possibly > >> > A > >> > will be locked for a while by one delete, by why don't other deletes > >> > wait > >> > for the lock to be released?
> >> > Also can you give me a link for batch transactions?
> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger > >> > <michael.hun...@neotechnology.com> wrote:
> >> >> Deleting a relationship currently locks both nodes.
> >> >> What about deleting them all in a batch transaction?
> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
> >> >> Thanks
> >> >> Actually I have a node X, and multiple nodes a..z that are linked to > >> >> it. I > >> >> am deleting all links. Since all links are different (except their end > >> >> point > >> >> X) I'm not sure how deleting one link should lock another?
> >> >> what does it mean to do it on the server side? > >> >> not sure how how to handle it externally - if I have multiple clients > >> >> on > >> >> different machines I cannot synchronize them. I am expecting neo4j to > >> >> lock > >> >> everything of interest while deleting a link.
> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis > >> >> <florent.em...@gmail.com> > >> >> wrote:
> >> >>> (Answering this with very partial knowledge, I use this as an occasion > >> >>> to > >> >>> learn stuff about Neo4J) > >> >>> In 2009, similar issues arose ( > >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) > >> >>> The advice given then was to manage the locking mechanism by hand. > >> >>> Looking at your trace, I'd say that your data is like this:
> >> >>> (Node X) -[Rel 8253]-(Node 1841)
> >> >>> You are currently trying to delete node X. > >> >>> Sometime before, you ran an operation (a delete I guess?) on node 1841 > >> >>> To be able to perform this, Relationship 8253 has to be locked (it > >> >>> will > >> >>> be deleted, since per > >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all > >> >>> properties and relationships of a node are deleted upon delete of the > >> >>> node > >> >>> itself) > >> >>> When you do it in synch mode: > >> >>> You: Delete Node 1841 > >> >>> You: Wait > >> >>> Server: Delete confirmed > >> >>> You: Delete Node X
> >> >>> Works, obviously > >> >>> When you do it in asynch mode: > >> >>> What you are attempting to do is: > >> >>> You:Delete Node 1841 > >> >>> Server:Locks Node 1841 and all its relationships > >> >>> You:Delete Node X > >> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel > >> >>> 8253 > >> >>> Server:Boom, Err 500 on delete Node X > >> >>> Server:(probably) Delete Node 1841 confirmed
> >> >>> Either you can manage the lock on the server side like suggested in > >> >>> the > >> >>> 2009 thread, or I think you'll have to manage it externally.
> >> >>> Can you add some sort of management in your delete calls to the > >> >>> server, > >> >>> partionning it to avoid collisions? Something akin to this: > >> >>> [0...33][34...66][67...99] > >> >>> (this example assumes that relationships are all between direct > >> >>> neighbours of course...) > >> >>> 3 threads run in parallel. > >> >>> Thread A starts at 0 stops at 33 > >> >>> Thread B starts at 66 stops at 34 > >> >>> Thread B starts at 99 stops at 67
> >> >>> This should in theory avoid collisions if we assume delete operation > >> >>> duration to be constant for all nodes...? > >> >>> (if you want to be extra careful, add a dampenning at the end of the > >> >>> partition: as you get closer to the boundary, wait a little bit > >> >>> between > >> >>> calls...it will give extra time to the neighbouring thread to move > >> >>> further > >> >>> away....)
> >> >>> Please remember I'm probably as new to Neo4J as you, so take all this > >> >>> with a grain of salt :-)
> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of > >> >>>> neo4j).
> >> >>>> I need to delete 100 nodes. When I do this synchronously there is no > >> >>>> problem (e.g. I wait for one delete to finish before calling the next > >> >>>> one). > >> >>>> When I delete them in an async manner (many rest requests sent > >> >>>> concurrently) I intermittently get a few failures. I see this coming > >> >>>> back in > >> >>>> the http response once per each error:
> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but this > >> >>>> does not seem relevant. I have verified my code is (logically) > >> >>>> correct, e.g. > >> >>>> I do not delete an already deleted node and etc.
> If you can write, you can code - @coderdojomalmo > If you can sketch, you can use a graph database - @neo4j
> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: > > Thanks
> > I'm currently using a workaround - doing everything in a sync manner. but > > I'm still not sure why the async version fails, this is my main worry now. I > > wouldn't expect batch to be required here - deleting multiple relationships > > of a single node.
> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer > > <peter.neuba...@neotechnology.com> wrote:
> The REST API seems to have serious issues with concurrency - the new
> mutating Cypher stuff does appear to address this but you'll basically be
> doing everything the Cypher way. Which, mind you, isn't necessarily a bad
> thing; I am finding Cypher to be very intuitive and powerful... excepting
> the inconvenient pseudo-JSON format for sending property data :)
> On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
>> If you can write, you can code - @coderdojomalmo
>> If you can sketch, you can use a graph database - @neo4j
>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote:
>> > Thanks
>> > I'm currently using a workaround - doing everything in a sync manner.
>> but
>> > I'm still not sure why the async version fails, this is my main worry
>> now. I
>> > wouldn't expect batch to be required here - deleting multiple
>> relationships
>> > of a single node.
>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer
>> > <peter.neubauer@neotechnology.**com <peter.neuba...@neotechnology.com>>
>> wrote:
>> >> If you can write, you can code - @coderdojomalmo
>> >> If you can sketch, you can use a graph database - @neo4j
>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com>
>> wrote:
>> >> > Thanks
>> >> > I still don't get the problem. I have node A with links to different
>> >> > nodes.
>> >> > I delete the links. There should be no deadlock in my scenario.
>> Possibly
>> >> > A
>> >> > will be locked for a while by one delete, by why don't other deletes
>> >> > wait
>> >> > for the lock to be released?
>> >> > Also can you give me a link for batch transactions?
>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
>> >> > <michael.hunger@neotechnology.**com<michael.hun...@neotechnology.com>>
>> wrote:
>> >> >> Deleting a relationship currently locks both nodes.
>> >> >> What about deleting them all in a batch transaction?
>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>> >> >> Thanks
>> >> >> Actually I have a node X, and multiple nodes a..z that are linked
>> to
>> >> >> it. I
>> >> >> am deleting all links. Since all links are different (except their
>> end
>> >> >> point
>> >> >> X) I'm not sure how deleting one link should lock another?
>> >> >> what does it mean to do it on the server side?
>> >> >> not sure how how to handle it externally - if I have multiple
>> clients
>> >> >> on
>> >> >> different machines I cannot synchronize them. I am expecting neo4j
>> to
>> >> >> lock
>> >> >> everything of interest while deleting a link.
>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis
>> >> >> <florent.em...@gmail.com>
>> >> >> wrote:
>> >> >>> (Answering this with very partial knowledge, I use this as an
>> occasion
>> >> >>> to
>> >> >>> learn stuff about Neo4J)
>> >> >>> In 2009, similar issues arose (
>> >> >>> http://www.mail-archive.com/**u...@lists.neo4j.org/msg01870.**html<http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html>)
>> >> >>> The advice given then was to manage the locking mechanism by hand.
>> >> >>> Looking at your trace, I'd say that your data is like this:
>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>> >> >>> You are currently trying to delete node X.
>> >> >>> Sometime before, you ran an operation (a delete I guess?) on node
>> 1841
>> >> >>> To be able to perform this, Relationship 8253 has to be locked (it
>> >> >>> will
>> >> >>> be deleted, since per
>> >> >>> http://docs.neo4j.org/chunked/**milestone/transactions-delete.** >> html <http://docs.neo4j.org/chunked/milestone/transactions-delete.html> all
>> >> >>> properties and relationships of a node are deleted upon delete of
>> the
>> >> >>> node
>> >> >>> itself)
>> >> >>> When you do it in synch mode:
>> >> >>> You: Delete Node 1841
>> >> >>> You: Wait
>> >> >>> Server: Delete confirmed
>> >> >>> You: Delete Node X
>> >> >>> Works, obviously
>> >> >>> When you do it in asynch mode:
>> >> >>> What you are attempting to do is:
>> >> >>> You:Delete Node 1841
>> >> >>> Server:Locks Node 1841 and all its relationships
>> >> >>> You:Delete Node X
>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on
>> rel
>> >> >>> 8253
>> >> >>> Server:Boom, Err 500 on delete Node X
>> >> >>> Server:(probably) Delete Node 1841 confirmed
>> >> >>> Either you can manage the lock on the server side like suggested
>> in
>> >> >>> the
>> >> >>> 2009 thread, or I think you'll have to manage it externally.
>> >> >>> Can you add some sort of management in your delete calls to the
>> >> >>> server,
>> >> >>> partionning it to avoid collisions? Something akin to this:
>> >> >>> [0...33][34...66][67...99]
>> >> >>> (this example assumes that relationships are all between direct
>> >> >>> neighbours of course...)
>> >> >>> 3 threads run in parallel.
>> >> >>> Thread A starts at 0 stops at 33
>> >> >>> Thread B starts at 66 stops at 34
>> >> >>> Thread B starts at 99 stops at 67
>> >> >>> This should in theory avoid collisions if we assume delete
>> operation
>> >> >>> duration to be constant for all nodes...?
>> >> >>> (if you want to be extra careful, add a dampenning at the end of
>> the
>> >> >>> partition: as you get closer to the boundary, wait a little bit
>> >> >>> between
>> >> >>> calls...it will give extra time to the neighbouring thread to move
>> >> >>> further
>> >> >>> away....)
>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all
>> this
>> >> >>> with a grain of salt :-)
>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of
>> >> >>>> neo4j).
>> >> >>>> I need to delete 100 nodes. When I do this synchronously there is
>> no
>> >> >>>> problem (e.g. I wait for one delete to finish before calling the
>> next
>> >> >>>> one).
>> >> >>>> When I delete them in an async manner (many rest requests sent
>> >> >>>> concurrently) I intermittently get a few failures. I see this
>> coming
>> >> >>>> back in
>> >> >>>> the http response once per each error:
>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but
>> this
>> >> >>>> does not seem relevant. I have verified my code is (logically)
>> >> >>>> correct, e.g.
>> >> >>>> I do not delete an already deleted node and etc.
>> >> >>>> Any idea?
>> >> >>>> Thanks,
>> >> >>>> Yaron
>> >> >> --
>> >> >> I'm on Twitter (@YaronNaveh)
>> >> > --
>> >> > I'm on Twitter (@YaronNaveh)
>> > --
>> > I'm on Twitter (@YaronNaveh)
> On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
>> If you can write, you can code - @coderdojomalmo
>> If you can sketch, you can use a graph database - @neo4j
>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote:
>> > Thanks
>> > I'm currently using a workaround - doing everything in a sync manner.
>> but
>> > I'm still not sure why the async version fails, this is my main worry
>> now. I
Hey guys, thanks for getting back to me - #2 looks like it solves my JSON issue, I will definitely be checking that out.
Back to the original topic though: the new mutating Cypher syntax I realize only makes it easier to group multiple operations into the same transaction - there still seem to be concurrency problems if one tries to simultaneously delete two distinct relationships to the same node with a Cypher call each. I get back something like:
Transaction(197)[STATUS_ACTIVE,Resources=1] can't wait on resource RWLock[Relationship[120]] since => Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- RWLock[Node[60]] <-[:WAITING_FOR]- Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- RWLock[Relationship[120]]
If Neo4j can detect and prevent a RWLock ahead of time, can't it simply queue my Cypher transactions FIFO style? I can't think of an easy way to handle this situation in a scalable fashion from my application (a webserver).
On Thursday, July 5, 2012 1:10:08 PM UTC-4, Michael Hunger wrote:
> #1 Each REST call is an individual tx and individual thread, except in > cypher and batch operations which group multiple operations. > #2 if you want to send properties as parameters (which you should anyway) > then you can post real json as params, see
> The REST API seems to have serious issues with concurrency - the new > mutating Cypher stuff does appear to address this but you'll basically be > doing everything the Cypher way. Which, mind you, isn't necessarily a bad > thing; I am finding Cypher to be very intuitive and powerful... excepting > the inconvenient pseudo-JSON format for sending property data :)
> On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
>> Could you just try Cypher to delete your relationships?
>> If you can write, you can code - @coderdojomalmo >> If you can sketch, you can use a graph database - @neo4j
>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: >> > Thanks
>> > I'm currently using a workaround - doing everything in a sync manner. >> but >> > I'm still not sure why the async version fails, this is my main worry >> now. I >> > wouldn't expect batch to be required here - deleting multiple >> relationships >> > of a single node.
>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer >> > <peter.neuba...@neotechnology.com> wrote:
>> >> If you can write, you can code - @coderdojomalmo >> >> If you can sketch, you can use a graph database - @neo4j
>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> >> wrote: >> >> > Thanks
>> >> > I still don't get the problem. I have node A with links to different >> >> > nodes. >> >> > I delete the links. There should be no deadlock in my scenario. >> Possibly >> >> > A >> >> > will be locked for a while by one delete, by why don't other deletes >> >> > wait >> >> > for the lock to be released?
>> >> > Also can you give me a link for batch transactions?
>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger >> >> > <michael.hun...@neotechnology.com> wrote:
>> >> >> Deleting a relationship currently locks both nodes.
>> >> >> What about deleting them all in a batch transaction?
>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>> >> >> Thanks
>> >> >> Actually I have a node X, and multiple nodes a..z that are linked >> to >> >> >> it. I >> >> >> am deleting all links. Since all links are different (except their >> end >> >> >> point >> >> >> X) I'm not sure how deleting one link should lock another?
>> >> >> what does it mean to do it on the server side? >> >> >> not sure how how to handle it externally - if I have multiple >> clients >> >> >> on >> >> >> different machines I cannot synchronize them. I am expecting neo4j >> to >> >> >> lock >> >> >> everything of interest while deleting a link.
>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis >> >> >> <florent.em...@gmail.com> >> >> >> wrote:
>> >> >>> (Answering this with very partial knowledge, I use this as an >> occasion >> >> >>> to >> >> >>> learn stuff about Neo4J) >> >> >>> In 2009, similar issues arose ( >> >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) >> >> >>> The advice given then was to manage the locking mechanism by hand. >> >> >>> Looking at your trace, I'd say that your data is like this:
>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>> >> >>> You are currently trying to delete node X. >> >> >>> Sometime before, you ran an operation (a delete I guess?) on node >> 1841 >> >> >>> To be able to perform this, Relationship 8253 has to be locked (it >> >> >>> will >> >> >>> be deleted, since per >> >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all >> >> >>> properties and relationships of a node are deleted upon delete of >> the >> >> >>> node >> >> >>> itself) >> >> >>> When you do it in synch mode: >> >> >>> You: Delete Node 1841 >> >> >>> You: Wait >> >> >>> Server: Delete confirmed >> >> >>> You: Delete Node X
>> >> >>> Works, obviously >> >> >>> When you do it in asynch mode: >> >> >>> What you are attempting to do is: >> >> >>> You:Delete Node 1841 >> >> >>> Server:Locks Node 1841 and all its relationships >> >> >>> You:Delete Node X >> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on >> rel >> >> >>> 8253 >> >> >>> Server:Boom, Err 500 on delete Node X >> >> >>> Server:(probably) Delete Node 1841 confirmed
>> >> >>> Either you can manage the lock on the server side like suggested >> in >> >> >>> the >> >> >>> 2009 thread, or I think you'll have to manage it externally.
>> >> >>> Can you add some sort of management in your delete calls to the >> >> >>> server, >> >> >>> partionning it to avoid collisions? Something akin to this: >> >> >>> [0...33][34...66][67...99] >> >> >>> (this example assumes that relationships are all between direct >> >> >>> neighbours of course...) >> >> >>> 3 threads run in parallel. >> >> >>> Thread A starts at 0 stops at 33 >> >> >>> Thread B starts at 66 stops at 34 >> >> >>> Thread B starts at 99 stops at 67
>> >> >>> This should in theory avoid collisions if we assume delete >> operation >> >> >>> duration to be constant for all nodes...? >> >> >>> (if you want to be extra careful, add a dampenning at the end of >> the >> >> >>> partition: as you get closer to the boundary, wait a little bit >> >> >>> between >> >> >>> calls...it will give extra time to the neighbouring thread to move >> >> >>> further >> >> >>> away....)
>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all >> this >> >> >>> with a grain of salt :-)
>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of >> >> >>>> neo4j).
>> >> >>>> I need to delete 100 nodes. When I do this synchronously there is >> no >> >> >>>> problem (e.g. I wait for one delete to finish before calling the >> next >> >> >>>> one). >> >> >>>> When I delete them in an async manner (many rest requests sent >> >> >>>> concurrently) I intermittently get a few failures. I see this >> coming >> >> >>>> back in >> >> >>>> the http response once per each error:
>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but >> this >> >> >>>> does not seem relevant. I have verified my code is (logically) >> >> >>>> correct, e.g. >> >> >>>> I do not delete an already deleted node and etc.
>> >> >>>> Any idea?
>> >> >>>> Thanks, >> >> >>>> Yaron
>> >> >> --
>> >> >> I'm on Twitter (@YaronNaveh)
>> >> > --
>> >> > I'm on Twitter (@YaronNaveh)
>> > --
>> > I'm on Twitter (@YaronNaveh)
> On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
>> Could you just try Cypher to delete your relationships?
On Thursday, July 5, 2012 6:56:01 PM UTC-4, Jesse wrote:
> Hey guys, thanks for getting back to me - #2 looks like it solves my JSON > issue, I will definitely be checking that out.
> Back to the original topic though: the new mutating Cypher syntax I > realize only makes it easier to group multiple operations into the same > transaction - there still seem to be concurrency problems if one tries to > simultaneously delete two distinct relationships to the same node with a > Cypher call each. I get back something like:
> Transaction(197)[STATUS_ACTIVE,Resources=1] can't wait on resource > RWLock[Relationship[120]] since => > Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- RWLock[Node[60]] > <-[:WAITING_FOR]- Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- > RWLock[Relationship[120]]
> If Neo4j can detect and prevent a RWLock ahead of time, can't it simply > queue my Cypher transactions FIFO style? I can't think of an easy way to > handle this situation in a scalable fashion from my application (a > webserver).
> Jesse
> On Thursday, July 5, 2012 1:10:08 PM UTC-4, Michael Hunger wrote:
>> #1 Each REST call is an individual tx and individual thread, except in >> cypher and batch operations which group multiple operations. >> #2 if you want to send properties as parameters (which you should anyway) >> then you can post real json as params, see
>> The REST API seems to have serious issues with concurrency - the new >> mutating Cypher stuff does appear to address this but you'll basically be >> doing everything the Cypher way. Which, mind you, isn't necessarily a bad >> thing; I am finding Cypher to be very intuitive and powerful... excepting >> the inconvenient pseudo-JSON format for sending property data :)
>> On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
>>> Could you just try Cypher to delete your relationships?
>>> If you can write, you can code - @coderdojomalmo >>> If you can sketch, you can use a graph database - @neo4j
>>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: >>> > Thanks
>>> > I'm currently using a workaround - doing everything in a sync manner. >>> but >>> > I'm still not sure why the async version fails, this is my main worry >>> now. I >>> > wouldn't expect batch to be required here - deleting multiple >>> relationships >>> > of a single node.
>>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer >>> > <peter.neuba...@neotechnology.com> wrote:
>>> >> If you can write, you can code - @coderdojomalmo >>> >> If you can sketch, you can use a graph database - @neo4j
>>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> >>> wrote: >>> >> > Thanks
>>> >> > I still don't get the problem. I have node A with links to >>> different >>> >> > nodes. >>> >> > I delete the links. There should be no deadlock in my scenario. >>> Possibly >>> >> > A >>> >> > will be locked for a while by one delete, by why don't other >>> deletes >>> >> > wait >>> >> > for the lock to be released?
>>> >> > Also can you give me a link for batch transactions?
>>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger >>> >> > <michael.hun...@neotechnology.com> wrote:
>>> >> >> Deleting a relationship currently locks both nodes.
>>> >> >> What about deleting them all in a batch transaction?
>>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>>> >> >> Thanks
>>> >> >> Actually I have a node X, and multiple nodes a..z that are linked >>> to >>> >> >> it. I >>> >> >> am deleting all links. Since all links are different (except their >>> end >>> >> >> point >>> >> >> X) I'm not sure how deleting one link should lock another?
>>> >> >> what does it mean to do it on the server side? >>> >> >> not sure how how to handle it externally - if I have multiple >>> clients >>> >> >> on >>> >> >> different machines I cannot synchronize them. I am expecting neo4j >>> to >>> >> >> lock >>> >> >> everything of interest while deleting a link.
>>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis >>> >> >> <florent.em...@gmail.com> >>> >> >> wrote:
>>> >> >>> (Answering this with very partial knowledge, I use this as an >>> occasion >>> >> >>> to >>> >> >>> learn stuff about Neo4J) >>> >> >>> In 2009, similar issues arose ( >>> >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) >>> >> >>> The advice given then was to manage the locking mechanism by >>> hand. >>> >> >>> Looking at your trace, I'd say that your data is like this:
>>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>>> >> >>> You are currently trying to delete node X. >>> >> >>> Sometime before, you ran an operation (a delete I guess?) on node >>> 1841 >>> >> >>> To be able to perform this, Relationship 8253 has to be locked >>> (it >>> >> >>> will >>> >> >>> be deleted, since per >>> >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all >>> >> >>> properties and relationships of a node are deleted upon delete of >>> the >>> >> >>> node >>> >> >>> itself) >>> >> >>> When you do it in synch mode: >>> >> >>> You: Delete Node 1841 >>> >> >>> You: Wait >>> >> >>> Server: Delete confirmed >>> >> >>> You: Delete Node X
>>> >> >>> Works, obviously >>> >> >>> When you do it in asynch mode: >>> >> >>> What you are attempting to do is: >>> >> >>> You:Delete Node 1841 >>> >> >>> Server:Locks Node 1841 and all its relationships >>> >> >>> You:Delete Node X >>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on >>> rel >>> >> >>> 8253 >>> >> >>> Server:Boom, Err 500 on delete Node X >>> >> >>> Server:(probably) Delete Node 1841 confirmed
>>> >> >>> Either you can manage the lock on the server side like suggested >>> in >>> >> >>> the >>> >> >>> 2009 thread, or I think you'll have to manage it externally.
>>> >> >>> Can you add some sort of management in your delete calls to the >>> >> >>> server, >>> >> >>> partionning it to avoid collisions? Something akin to this: >>> >> >>> [0...33][34...66][67...99] >>> >> >>> (this example assumes that relationships are all between direct >>> >> >>> neighbours of course...) >>> >> >>> 3 threads run in parallel. >>> >> >>> Thread A starts at 0 stops at 33 >>> >> >>> Thread B starts at 66 stops at 34 >>> >> >>> Thread B starts at 99 stops at 67
>>> >> >>> This should in theory avoid collisions if we assume delete >>> operation >>> >> >>> duration to be constant for all nodes...? >>> >> >>> (if you want to be extra careful, add a dampenning at the end of >>> the >>> >> >>> partition: as you get closer to the boundary, wait a little bit >>> >> >>> between >>> >> >>> calls...it will give extra time to the neighbouring thread to >>> move >>> >> >>> further >>> >> >>> away....)
>>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all >>> this >>> >> >>> with a grain of salt :-)
>>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of >>> >> >>>> neo4j).
>>> >> >>>> I need to delete 100 nodes. When I do this synchronously there >>> is no >>> >> >>>> problem (e.g. I wait for one delete to finish before calling the >>> next >>> >> >>>> one). >>> >> >>>> When I delete them in an async manner (many rest requests sent >>> >> >>>> concurrently) I intermittently get a few failures. I see this >>> coming >>> >> >>>> back in >>> >> >>>> the http response once per each error:
>>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but >>> this >>> >> >>>> does not seem relevant. I have verified my code is (logically) >>> >> >>>> correct, e.g. >>> >> >>>> I do not delete an already deleted node and etc.
It detects and abort deadlocks, so usually you would retry the operation.
The deadlock detection is happening after the fact, to precompute all that would not be feasible as the tx-system would have to know a lot about the semantics of all your operations. And usually it is seldom enough that a simple
retry is a good solution.
> Hey guys, thanks for getting back to me - #2 looks like it solves my JSON issue, I will definitely be checking that out.
> Back to the original topic though: the new mutating Cypher syntax I realize only makes it easier to group multiple operations into the same transaction - there still seem to be concurrency problems if one tries to simultaneously delete two distinct relationships to the same node with a Cypher call each. I get back something like:
> Transaction(197)[STATUS_ACTIVE,Resources=1] can't wait on resource RWLock[Relationship[120]] since => Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- RWLock[Node[60]] <-[:WAITING_FOR]- Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- RWLock[Relationship[120]]
> If Neo4j can detect and prevent a RWLock ahead of time, can't it simply queue my Cypher transactions FIFO style? I can't think of an easy way to handle this situation in a scalable fashion from my application (a webserver).
> Jesse
> On Thursday, July 5, 2012 1:10:08 PM UTC-4, Michael Hunger wrote:
> #1 Each REST call is an individual tx and individual thread, except in cypher and batch operations which group multiple operations.
> #2 if you want to send properties as parameters (which you should anyway) then you can post real json as params, see
>> The REST API seems to have serious issues with concurrency - the new mutating Cypher stuff does appear to address this but you'll basically be doing everything the Cypher way. Which, mind you, isn't necessarily a bad thing; I am finding Cypher to be very intuitive and powerful... excepting the inconvenient pseudo-JSON format for sending property data :)
>> If you can write, you can code - @coderdojomalmo >> If you can sketch, you can use a graph database - @neo4j
>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: >> > Thanks
>> > I'm currently using a workaround - doing everything in a sync manner. but >> > I'm still not sure why the async version fails, this is my main worry now. I >> > wouldn't expect batch to be required here - deleting multiple relationships >> > of a single node.
>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer >> > <peter.neuba...@neotechnology.com> wrote:
>> >> If you can write, you can code - @coderdojomalmo >> >> If you can sketch, you can use a graph database - @neo4j
>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> wrote: >> >> > Thanks
>> >> > I still don't get the problem. I have node A with links to different >> >> > nodes. >> >> > I delete the links. There should be no deadlock in my scenario. Possibly >> >> > A >> >> > will be locked for a while by one delete, by why don't other deletes >> >> > wait >> >> > for the lock to be released?
>> >> > Also can you give me a link for batch transactions?
>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger >> >> > <michael.hun...@neotechnology.com> wrote:
>> >> >> Deleting a relationship currently locks both nodes.
>> >> >> What about deleting them all in a batch transaction?
>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>> >> >> Thanks
>> >> >> Actually I have a node X, and multiple nodes a..z that are linked to >> >> >> it. I >> >> >> am deleting all links. Since all links are different (except their end >> >> >> point >> >> >> X) I'm not sure how deleting one link should lock another?
>> >> >> what does it mean to do it on the server side? >> >> >> not sure how how to handle it externally - if I have multiple clients >> >> >> on >> >> >> different machines I cannot synchronize them. I am expecting neo4j to >> >> >> lock >> >> >> everything of interest while deleting a link.
>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis >> >> >> <florent.em...@gmail.com> >> >> >> wrote:
>> >> >>> (Answering this with very partial knowledge, I use this as an occasion >> >> >>> to >> >> >>> learn stuff about Neo4J) >> >> >>> In 2009, similar issues arose ( >> >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) >> >> >>> The advice given then was to manage the locking mechanism by hand. >> >> >>> Looking at your trace, I'd say that your data is like this:
>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>> >> >>> You are currently trying to delete node X. >> >> >>> Sometime before, you ran an operation (a delete I guess?) on node 1841 >> >> >>> To be able to perform this, Relationship 8253 has to be locked (it >> >> >>> will >> >> >>> be deleted, since per >> >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all >> >> >>> properties and relationships of a node are deleted upon delete of the >> >> >>> node >> >> >>> itself) >> >> >>> When you do it in synch mode: >> >> >>> You: Delete Node 1841 >> >> >>> You: Wait >> >> >>> Server: Delete confirmed >> >> >>> You: Delete Node X
>> >> >>> Works, obviously >> >> >>> When you do it in asynch mode: >> >> >>> What you are attempting to do is: >> >> >>> You:Delete Node 1841 >> >> >>> Server:Locks Node 1841 and all its relationships >> >> >>> You:Delete Node X >> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel >> >> >>> 8253 >> >> >>> Server:Boom, Err 500 on delete Node X >> >> >>> Server:(probably) Delete Node 1841 confirmed
>> >> >>> Either you can manage the lock on the server side like suggested in >> >> >>> the >> >> >>> 2009 thread, or I think you'll have to manage it externally.
>> >> >>> Can you add some sort of management in your delete calls to the >> >> >>> server, >> >> >>> partionning it to avoid collisions? Something akin to this: >> >> >>> [0...33][34...66][67...99] >> >> >>> (this example assumes that relationships are all between direct >> >> >>> neighbours of course...) >> >> >>> 3 threads run in parallel. >> >> >>> Thread A starts at 0 stops at 33 >> >> >>> Thread B starts at 66 stops at 34 >> >> >>> Thread B starts at 99 stops at 67
>> >> >>> This should in theory avoid collisions if we assume delete operation >> >> >>> duration to be constant for all nodes...? >> >> >>> (if you want to be extra careful, add a dampenning at the end of the >> >> >>> partition: as you get closer to the boundary, wait a little bit >> >> >>> between >> >> >>> calls...it will give extra time to the neighbouring thread to move >> >> >>> further >> >> >>> away....)
>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all this >> >> >>> with a grain of salt :-)
>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of >> >> >>>> neo4j).
>> >> >>>> I need to delete 100 nodes. When I do this synchronously there is no >> >> >>>> problem (e.g. I wait for one delete to finish before calling the next >> >> >>>> one). >> >> >>>> When I delete them in an async manner (many rest requests sent >> >> >>>> concurrently) I intermittently get a few failures. I see this coming >> >> >>>> back in >> >> >>>> the http response once per each error:
>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but this >> >> >>>> does not seem relevant. I have verified my code is (logically) >> >> >>>> correct, e.g. >> >> >>>> I do not delete an already deleted node and etc.
Ok. Thanks again for getting back to me so quick. I've just tested passing JSON as a param and auto-retrying when I see the RWLock error - seems to solve both problems. Cheers!
On Thursday, July 5, 2012 7:10:05 PM UTC-4, Michael Hunger wrote:
> It detects and abort deadlocks, so usually you would retry the operation.
> The deadlock detection is happening after the fact, to precompute all that > would not be feasible as the tx-system would have to know a lot about the > semantics of all your operations. And usually it is seldom enough that a > simple > retry is a good solution.
> Michael
> Am 06.07.2012 um 00:56 schrieb Jesse:
> Hey guys, thanks for getting back to me - #2 looks like it solves my JSON > issue, I will definitely be checking that out.
> Back to the original topic though: the new mutating Cypher syntax I > realize only makes it easier to group multiple operations into the same > transaction - there still seem to be concurrency problems if one tries to > simultaneously delete two distinct relationships to the same node with a > Cypher call each. I get back something like:
> Transaction(197)[STATUS_ACTIVE,Resources=1] can't wait on resource > RWLock[Relationship[120]] since => > Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- RWLock[Node[60]] > <-[:WAITING_FOR]- Transaction(197)[STATUS_ACTIVE,Resources=1] <-[:HELD_BY]- > RWLock[Relationship[120]]
> If Neo4j can detect and prevent a RWLock ahead of time, can't it simply > queue my Cypher transactions FIFO style? I can't think of an easy way to > handle this situation in a scalable fashion from my application (a > webserver).
> Jesse
> On Thursday, July 5, 2012 1:10:08 PM UTC-4, Michael Hunger wrote:
>> #1 Each REST call is an individual tx and individual thread, except in >> cypher and batch operations which group multiple operations. >> #2 if you want to send properties as parameters (which you should anyway) >> then you can post real json as params, see
>> The REST API seems to have serious issues with concurrency - the new >> mutating Cypher stuff does appear to address this but you'll basically be >> doing everything the Cypher way. Which, mind you, isn't necessarily a bad >> thing; I am finding Cypher to be very intuitive and powerful... excepting >> the inconvenient pseudo-JSON format for sending property data :)
>> On Thursday, July 5, 2012 11:21:46 AM UTC-4, Peter Neubauer wrote:
>>> Could you just try Cypher to delete your relationships?
>>> If you can write, you can code - @coderdojomalmo >>> If you can sketch, you can use a graph database - @neo4j
>>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: >>> > Thanks
>>> > I'm currently using a workaround - doing everything in a sync manner. >>> but >>> > I'm still not sure why the async version fails, this is my main worry >>> now. I >>> > wouldn't expect batch to be required here - deleting multiple >>> relationships >>> > of a single node.
>>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer >>> > <peter.neuba...@neotechnology.com> wrote:
>>> >> If you can write, you can code - @coderdojomalmo >>> >> If you can sketch, you can use a graph database - @neo4j
>>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> >>> wrote: >>> >> > Thanks
>>> >> > I still don't get the problem. I have node A with links to >>> different >>> >> > nodes. >>> >> > I delete the links. There should be no deadlock in my scenario. >>> Possibly >>> >> > A >>> >> > will be locked for a while by one delete, by why don't other >>> deletes >>> >> > wait >>> >> > for the lock to be released?
>>> >> > Also can you give me a link for batch transactions?
>>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger >>> >> > <michael.hun...@neotechnology.com> wrote:
>>> >> >> Deleting a relationship currently locks both nodes.
>>> >> >> What about deleting them all in a batch transaction?
>>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>>> >> >> Thanks
>>> >> >> Actually I have a node X, and multiple nodes a..z that are linked >>> to >>> >> >> it. I >>> >> >> am deleting all links. Since all links are different (except their >>> end >>> >> >> point >>> >> >> X) I'm not sure how deleting one link should lock another?
>>> >> >> what does it mean to do it on the server side? >>> >> >> not sure how how to handle it externally - if I have multiple >>> clients >>> >> >> on >>> >> >> different machines I cannot synchronize them. I am expecting neo4j >>> to >>> >> >> lock >>> >> >> everything of interest while deleting a link.
>>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis >>> >> >> <florent.em...@gmail.com> >>> >> >> wrote:
>>> >> >>> (Answering this with very partial knowledge, I use this as an >>> occasion >>> >> >>> to >>> >> >>> learn stuff about Neo4J) >>> >> >>> In 2009, similar issues arose ( >>> >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) >>> >> >>> The advice given then was to manage the locking mechanism by >>> hand. >>> >> >>> Looking at your trace, I'd say that your data is like this:
>>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>>> >> >>> You are currently trying to delete node X. >>> >> >>> Sometime before, you ran an operation (a delete I guess?) on node >>> 1841 >>> >> >>> To be able to perform this, Relationship 8253 has to be locked >>> (it >>> >> >>> will >>> >> >>> be deleted, since per >>> >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all >>> >> >>> properties and relationships of a node are deleted upon delete of >>> the >>> >> >>> node >>> >> >>> itself) >>> >> >>> When you do it in synch mode: >>> >> >>> You: Delete Node 1841 >>> >> >>> You: Wait >>> >> >>> Server: Delete confirmed >>> >> >>> You: Delete Node X
>>> >> >>> Works, obviously >>> >> >>> When you do it in asynch mode: >>> >> >>> What you are attempting to do is: >>> >> >>> You:Delete Node 1841 >>> >> >>> Server:Locks Node 1841 and all its relationships >>> >> >>> You:Delete Node X >>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on >>> rel >>> >> >>> 8253 >>> >> >>> Server:Boom, Err 500 on delete Node X >>> >> >>> Server:(probably) Delete Node 1841 confirmed
>>> >> >>> Either you can manage the lock on the server side like suggested >>> in >>> >> >>> the >>> >> >>> 2009 thread, or I think you'll have to manage it externally.
>>> >> >>> Can you add some sort of management in your delete calls to the >>> >> >>> server, >>> >> >>> partionning it to avoid collisions? Something akin to this: >>> >> >>> [0...33][34...66][67...99] >>> >> >>> (this example assumes that relationships are all between direct >>> >> >>> neighbours of course...) >>> >> >>> 3 threads run in parallel. >>> >> >>> Thread A starts at 0 stops at 33 >>> >> >>> Thread B starts at 66 stops at 34 >>> >> >>> Thread B starts at 99 stops at 67
>>> >> >>> This should in theory avoid collisions if we assume delete >>> operation >>> >> >>> duration to be constant for all nodes...? >>> >> >>> (if you want to be extra careful, add a dampenning at the end of >>> the >>> >> >>> partition: as you get closer to the boundary, wait a little bit >>> >> >>> between >>> >> >>> calls...it will give extra time to the neighbouring thread to >>> move >>> >> >>> further >>> >> >>> away....)
>>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all >>> this >>> >> >>> with a grain of salt :-)
>>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of >>> >> >>>> neo4j).
>>> >> >>>> I need to delete 100 nodes. When I do this synchronously there >>> is no >>> >> >>>> problem (e.g. I wait for one delete to finish before calling the >>> next >>> >> >>>> one). >>> >> >>>> When I delete them in an async manner (many rest requests sent >>> >> >>>> concurrently) I intermittently get a few failures. I see this >>> coming >>> >> >>>> back in >>> >> >>>> the http response once per each error:
I tried cyper delete, but it deletes all the connected nodes and edges, but i need to delete connected nodes based on their properties (eg : delete connected node only if it has value "x" on a certain property and which is connected to parent node through the edge "relx")
I used server plugins to achieve the above behavior.
How good is this approach, any comments would be helpful.
> If you can write, you can code - @coderdojomalmo > If you can sketch, you can use a graph database - @neo4j
> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote: > > Thanks
> > I'm currently using a workaround - doing everything in a sync manner. > but > > I'm still not sure why the async version fails, this is my main worry > now. I > > wouldn't expect batch to be required here - deleting multiple > relationships > > of a single node.
> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer > > <peter.neuba...@neotechnology.com> wrote:
> >> If you can write, you can code - @coderdojomalmo > >> If you can sketch, you can use a graph database - @neo4j
> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com> > wrote: > >> > Thanks
> >> > I still don't get the problem. I have node A with links to different > >> > nodes. > >> > I delete the links. There should be no deadlock in my scenario. > Possibly > >> > A > >> > will be locked for a while by one delete, by why don't other deletes > >> > wait > >> > for the lock to be released?
> >> > Also can you give me a link for batch transactions?
> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger > >> > <michael.hun...@neotechnology.com> wrote:
> >> >> Deleting a relationship currently locks both nodes.
> >> >> What about deleting them all in a batch transaction?
> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
> >> >> Thanks
> >> >> Actually I have a node X, and multiple nodes a..z that are linked to > >> >> it. I > >> >> am deleting all links. Since all links are different (except their > end > >> >> point > >> >> X) I'm not sure how deleting one link should lock another?
> >> >> what does it mean to do it on the server side? > >> >> not sure how how to handle it externally - if I have multiple > clients > >> >> on > >> >> different machines I cannot synchronize them. I am expecting neo4j > to > >> >> lock > >> >> everything of interest while deleting a link.
> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis > >> >> <florent.em...@gmail.com> > >> >> wrote:
> >> >>> (Answering this with very partial knowledge, I use this as an > occasion > >> >>> to > >> >>> learn stuff about Neo4J) > >> >>> In 2009, similar issues arose ( > >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) > >> >>> The advice given then was to manage the locking mechanism by hand. > >> >>> Looking at your trace, I'd say that your data is like this:
> >> >>> (Node X) -[Rel 8253]-(Node 1841)
> >> >>> You are currently trying to delete node X. > >> >>> Sometime before, you ran an operation (a delete I guess?) on node > 1841 > >> >>> To be able to perform this, Relationship 8253 has to be locked (it > >> >>> will > >> >>> be deleted, since per > >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all > >> >>> properties and relationships of a node are deleted upon delete of > the > >> >>> node > >> >>> itself) > >> >>> When you do it in synch mode: > >> >>> You: Delete Node 1841 > >> >>> You: Wait > >> >>> Server: Delete confirmed > >> >>> You: Delete Node X
> >> >>> Works, obviously > >> >>> When you do it in asynch mode: > >> >>> What you are attempting to do is: > >> >>> You:Delete Node 1841 > >> >>> Server:Locks Node 1841 and all its relationships > >> >>> You:Delete Node X > >> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel > >> >>> 8253 > >> >>> Server:Boom, Err 500 on delete Node X > >> >>> Server:(probably) Delete Node 1841 confirmed
> >> >>> Either you can manage the lock on the server side like suggested in > >> >>> the > >> >>> 2009 thread, or I think you'll have to manage it externally.
> >> >>> Can you add some sort of management in your delete calls to the > >> >>> server, > >> >>> partionning it to avoid collisions? Something akin to this: > >> >>> [0...33][34...66][67...99] > >> >>> (this example assumes that relationships are all between direct > >> >>> neighbours of course...) > >> >>> 3 threads run in parallel. > >> >>> Thread A starts at 0 stops at 33 > >> >>> Thread B starts at 66 stops at 34 > >> >>> Thread B starts at 99 stops at 67
> >> >>> This should in theory avoid collisions if we assume delete > operation > >> >>> duration to be constant for all nodes...? > >> >>> (if you want to be extra careful, add a dampenning at the end of > the > >> >>> partition: as you get closer to the boundary, wait a little bit > >> >>> between > >> >>> calls...it will give extra time to the neighbouring thread to move > >> >>> further > >> >>> away....)
> >> >>> Please remember I'm probably as new to Neo4J as you, so take all > this > >> >>> with a grain of salt :-)
> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of > >> >>>> neo4j).
> >> >>>> I need to delete 100 nodes. When I do this synchronously there is > no > >> >>>> problem (e.g. I wait for one delete to finish before calling the > next > >> >>>> one). > >> >>>> When I delete them in an async manner (many rest requests sent > >> >>>> concurrently) I intermittently get a few failures. I see this > coming > >> >>>> back in > >> >>>> the http response once per each error:
> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but > this > >> >>>> does not seem relevant. I have verified my code is (logically) > >> >>>> correct, e.g. > >> >>>> I do not delete an already deleted node and etc.
Hi there,
If you can find and return the nodes you are interested in deleting with a
Cypher query then you should be able to delete them too. Do you have any
example graph and what to delete?
/peter
Send from mobile.
On Jul 7, 2012 7:26 PM, "Shireesh" <ashirees...@gmail.com> wrote:
> I tried cyper delete, but it deletes all the connected nodes and edges,
> but i need to delete connected nodes based on their properties (eg : delete
> connected node only if it has value "x" on a certain property and which is
> connected to parent node through the edge "relx")
> I used server plugins to achieve the above behavior.
> How good is this approach, any comments would be helpful.
> Thanks,
> Shireesh.
> On Thursday, 5 July 2012 20:51:46 UTC+5:30, Peter Neubauer wrote:
>> If you can write, you can code - @coderdojomalmo
>> If you can sketch, you can use a graph database - @neo4j
>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote:
>> > Thanks
>> > I'm currently using a workaround - doing everything in a sync manner.
>> but
>> > I'm still not sure why the async version fails, this is my main worry
>> now. I
>> > wouldn't expect batch to be required here - deleting multiple
>> relationships
>> > of a single node.
>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer
>> > <peter.neubauer@neotechnology.**com <peter.neuba...@neotechnology.com>>
>> wrote:
>> >> If you can write, you can code - @coderdojomalmo
>> >> If you can sketch, you can use a graph database - @neo4j
>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com>
>> wrote:
>> >> > Thanks
>> >> > I still don't get the problem. I have node A with links to different
>> >> > nodes.
>> >> > I delete the links. There should be no deadlock in my scenario.
>> Possibly
>> >> > A
>> >> > will be locked for a while by one delete, by why don't other deletes
>> >> > wait
>> >> > for the lock to be released?
>> >> > Also can you give me a link for batch transactions?
>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
>> >> > <michael.hunger@neotechnology.**com<michael.hun...@neotechnology.com>>
>> wrote:
>> >> >> Deleting a relationship currently locks both nodes.
>> >> >> What about deleting them all in a batch transaction?
>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>> >> >> Thanks
>> >> >> Actually I have a node X, and multiple nodes a..z that are linked
>> to
>> >> >> it. I
>> >> >> am deleting all links. Since all links are different (except their
>> end
>> >> >> point
>> >> >> X) I'm not sure how deleting one link should lock another?
>> >> >> what does it mean to do it on the server side?
>> >> >> not sure how how to handle it externally - if I have multiple
>> clients
>> >> >> on
>> >> >> different machines I cannot synchronize them. I am expecting neo4j
>> to
>> >> >> lock
>> >> >> everything of interest while deleting a link.
>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis
>> >> >> <florent.em...@gmail.com>
>> >> >> wrote:
>> >> >>> (Answering this with very partial knowledge, I use this as an
>> occasion
>> >> >>> to
>> >> >>> learn stuff about Neo4J)
>> >> >>> In 2009, similar issues arose (
>> >> >>> http://www.mail-archive.com/**u...@lists.neo4j.org/msg01870.**html<http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html>)
>> >> >>> The advice given then was to manage the locking mechanism by hand.
>> >> >>> Looking at your trace, I'd say that your data is like this:
>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>> >> >>> You are currently trying to delete node X.
>> >> >>> Sometime before, you ran an operation (a delete I guess?) on node
>> 1841
>> >> >>> To be able to perform this, Relationship 8253 has to be locked (it
>> >> >>> will
>> >> >>> be deleted, since per
>> >> >>> http://docs.neo4j.org/chunked/**milestone/transactions-delete.** >> html <http://docs.neo4j.org/chunked/milestone/transactions-delete.html> all
>> >> >>> properties and relationships of a node are deleted upon delete of
>> the
>> >> >>> node
>> >> >>> itself)
>> >> >>> When you do it in synch mode:
>> >> >>> You: Delete Node 1841
>> >> >>> You: Wait
>> >> >>> Server: Delete confirmed
>> >> >>> You: Delete Node X
>> >> >>> Works, obviously
>> >> >>> When you do it in asynch mode:
>> >> >>> What you are attempting to do is:
>> >> >>> You:Delete Node 1841
>> >> >>> Server:Locks Node 1841 and all its relationships
>> >> >>> You:Delete Node X
>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on
>> rel
>> >> >>> 8253
>> >> >>> Server:Boom, Err 500 on delete Node X
>> >> >>> Server:(probably) Delete Node 1841 confirmed
>> >> >>> Either you can manage the lock on the server side like suggested
>> in
>> >> >>> the
>> >> >>> 2009 thread, or I think you'll have to manage it externally.
>> >> >>> Can you add some sort of management in your delete calls to the
>> >> >>> server,
>> >> >>> partionning it to avoid collisions? Something akin to this:
>> >> >>> [0...33][34...66][67...99]
>> >> >>> (this example assumes that relationships are all between direct
>> >> >>> neighbours of course...)
>> >> >>> 3 threads run in parallel.
>> >> >>> Thread A starts at 0 stops at 33
>> >> >>> Thread B starts at 66 stops at 34
>> >> >>> Thread B starts at 99 stops at 67
>> >> >>> This should in theory avoid collisions if we assume delete
>> operation
>> >> >>> duration to be constant for all nodes...?
>> >> >>> (if you want to be extra careful, add a dampenning at the end of
>> the
>> >> >>> partition: as you get closer to the boundary, wait a little bit
>> >> >>> between
>> >> >>> calls...it will give extra time to the neighbouring thread to move
>> >> >>> further
>> >> >>> away....)
>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all
>> this
>> >> >>> with a grain of salt :-)
>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of
>> >> >>>> neo4j).
>> >> >>>> I need to delete 100 nodes. When I do this synchronously there is
>> no
>> >> >>>> problem (e.g. I wait for one delete to finish before calling the
>> next
>> >> >>>> one).
>> >> >>>> When I delete them in an async manner (many rest requests sent
>> >> >>>> concurrently) I intermittently get a few failures. I see this
>> coming
>> >> >>>> back in
>> >> >>>> the http response once per each error:
>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but
>> this
>> >> >>>> does not seem relevant. I have verified my code is (logically)
>> >> >>>> correct, e.g.
>> >> >>>> I do not delete an already deleted node and etc.
On Sat, Jul 7, 2012 at 9:44 PM, Peter Neubauer <neubauer.pe...@gmail.com> wrote:
> Hi there,
> If you can find and return the nodes you are interested in deleting with a
> Cypher query then you should be able to delete them too. Do you have any
> example graph and what to delete?
> /peter
> Send from mobile.
> On Jul 7, 2012 7:26 PM, "Shireesh" <ashirees...@gmail.com> wrote:
>> I tried cyper delete, but it deletes all the connected nodes and edges,
>> but i need to delete connected nodes based on their properties (eg : delete
>> connected node only if it has value "x" on a certain property and which is
>> connected to parent node through the edge "relx")
>> I used server plugins to achieve the above behavior.
>> How good is this approach, any comments would be helpful.
>> Thanks,
>> Shireesh.
>> On Thursday, 5 July 2012 20:51:46 UTC+5:30, Peter Neubauer wrote:
>>> Could you just try Cypher to delete your relationships?
>>> If you can write, you can code - @coderdojomalmo
>>> If you can sketch, you can use a graph database - @neo4j
>>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote:
>>> > Thanks
>>> > I'm currently using a workaround - doing everything in a sync manner.
>>> > but
>>> > I'm still not sure why the async version fails, this is my main worry
>>> > now. I
>>> > wouldn't expect batch to be required here - deleting multiple
>>> > relationships
>>> > of a single node.
>>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer
>>> > <peter.neuba...@neotechnology.com> wrote:
>>> >> If you can write, you can code - @coderdojomalmo
>>> >> If you can sketch, you can use a graph database - @neo4j
>>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com>
>>> >> wrote:
>>> >> > Thanks
>>> >> > I still don't get the problem. I have node A with links to different
>>> >> > nodes.
>>> >> > I delete the links. There should be no deadlock in my scenario.
>>> >> > Possibly
>>> >> > A
>>> >> > will be locked for a while by one delete, by why don't other deletes
>>> >> > wait
>>> >> > for the lock to be released?
>>> >> > Also can you give me a link for batch transactions?
>>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
>>> >> > <michael.hun...@neotechnology.com> wrote:
>>> >> >> Deleting a relationship currently locks both nodes.
>>> >> >> What about deleting them all in a batch transaction?
>>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>>> >> >> Thanks
>>> >> >> Actually I have a node X, and multiple nodes a..z that are linked
>>> >> >> to
>>> >> >> it. I
>>> >> >> am deleting all links. Since all links are different (except their
>>> >> >> end
>>> >> >> point
>>> >> >> X) I'm not sure how deleting one link should lock another?
>>> >> >> what does it mean to do it on the server side?
>>> >> >> not sure how how to handle it externally - if I have multiple
>>> >> >> clients
>>> >> >> on
>>> >> >> different machines I cannot synchronize them. I am expecting neo4j
>>> >> >> to
>>> >> >> lock
>>> >> >> everything of interest while deleting a link.
>>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis
>>> >> >> <florent.em...@gmail.com>
>>> >> >> wrote:
>>> >> >>> (Answering this with very partial knowledge, I use this as an
>>> >> >>> occasion
>>> >> >>> to
>>> >> >>> learn stuff about Neo4J)
>>> >> >>> In 2009, similar issues arose (
>>> >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html )
>>> >> >>> The advice given then was to manage the locking mechanism by hand.
>>> >> >>> Looking at your trace, I'd say that your data is like this:
>>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>>> >> >>> You are currently trying to delete node X.
>>> >> >>> Sometime before, you ran an operation (a delete I guess?) on node
>>> >> >>> 1841
>>> >> >>> To be able to perform this, Relationship 8253 has to be locked (it
>>> >> >>> will
>>> >> >>> be deleted, since per
>>> >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html >>> >> >>> all
>>> >> >>> properties and relationships of a node are deleted upon delete of
>>> >> >>> the
>>> >> >>> node
>>> >> >>> itself)
>>> >> >>> When you do it in synch mode:
>>> >> >>> You: Delete Node 1841
>>> >> >>> You: Wait
>>> >> >>> Server: Delete confirmed
>>> >> >>> You: Delete Node X
>>> >> >>> Works, obviously
>>> >> >>> When you do it in asynch mode:
>>> >> >>> What you are attempting to do is:
>>> >> >>> You:Delete Node 1841
>>> >> >>> Server:Locks Node 1841 and all its relationships
>>> >> >>> You:Delete Node X
>>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on
>>> >> >>> rel
>>> >> >>> 8253
>>> >> >>> Server:Boom, Err 500 on delete Node X
>>> >> >>> Server:(probably) Delete Node 1841 confirmed
>>> >> >>> Either you can manage the lock on the server side like suggested
>>> >> >>> in
>>> >> >>> the
>>> >> >>> 2009 thread, or I think you'll have to manage it externally.
>>> >> >>> Can you add some sort of management in your delete calls to the
>>> >> >>> server,
>>> >> >>> partionning it to avoid collisions? Something akin to this:
>>> >> >>> [0...33][34...66][67...99]
>>> >> >>> (this example assumes that relationships are all between direct
>>> >> >>> neighbours of course...)
>>> >> >>> 3 threads run in parallel.
>>> >> >>> Thread A starts at 0 stops at 33
>>> >> >>> Thread B starts at 66 stops at 34
>>> >> >>> Thread B starts at 99 stops at 67
>>> >> >>> This should in theory avoid collisions if we assume delete
>>> >> >>> operation
>>> >> >>> duration to be constant for all nodes...?
>>> >> >>> (if you want to be extra careful, add a dampenning at the end of
>>> >> >>> the
>>> >> >>> partition: as you get closer to the boundary, wait a little bit
>>> >> >>> between
>>> >> >>> calls...it will give extra time to the neighbouring thread to move
>>> >> >>> further
>>> >> >>> away....)
>>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all
>>> >> >>> this
>>> >> >>> with a grain of salt :-)
>>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of
>>> >> >>>> neo4j).
>>> >> >>>> I need to delete 100 nodes. When I do this synchronously there is
>>> >> >>>> no
>>> >> >>>> problem (e.g. I wait for one delete to finish before calling the
>>> >> >>>> next
>>> >> >>>> one).
>>> >> >>>> When I delete them in an async manner (many rest requests sent
>>> >> >>>> concurrently) I intermittently get a few failures. I see this
>>> >> >>>> coming
>>> >> >>>> back in
>>> >> >>>> the http response once per each error:
>>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but
>>> >> >>>> this
>>> >> >>>> does not seem relevant. I have verified my code is (logically)
>>> >> >>>> correct, e.g.
>>> >> >>>> I do not delete an already deleted node and etc.
A --> [produces] X1 [consumes] <-- B
A --> [produces] X2
A --> [consumes] X3 [produces] <-- B
A --> [uses] R1 [uses] <-- B
A --> [uses] R2
X2 -->[has] L1
Now delete [A] needs to delete in the following way,
-- delete all those nodes which A produces only is no other node consumes
it.[eg X2]
-- delete all nodes which A uses, if no other node uses it. [eg : R2]
-- delete the grand child nodes connected to the child node [L1]
so based on above conditions if i delete A,
nodes to be deleted : A,X2,L1,R2
Thanks,
Shireesh
On Sun, Jul 8, 2012 at 1:14 AM, Peter Neubauer <neubauer.pe...@gmail.com>wrote:
> Hi there,
> If you can find and return the nodes you are interested in deleting with a
> Cypher query then you should be able to delete them too. Do you have any
> example graph and what to delete?
> /peter
> Send from mobile.
> On Jul 7, 2012 7:26 PM, "Shireesh" <ashirees...@gmail.com> wrote:
>> I tried cyper delete, but it deletes all the connected nodes and edges,
>> but i need to delete connected nodes based on their properties (eg : delete
>> connected node only if it has value "x" on a certain property and which is
>> connected to parent node through the edge "relx")
>> I used server plugins to achieve the above behavior.
>> How good is this approach, any comments would be helpful.
>> Thanks,
>> Shireesh.
>> On Thursday, 5 July 2012 20:51:46 UTC+5:30, Peter Neubauer wrote:
>>> If you can write, you can code - @coderdojomalmo
>>> If you can sketch, you can use a graph database - @neo4j
>>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com> wrote:
>>> > Thanks
>>> > I'm currently using a workaround - doing everything in a sync manner.
>>> but
>>> > I'm still not sure why the async version fails, this is my main worry
>>> now. I
>>> > wouldn't expect batch to be required here - deleting multiple
>>> relationships
>>> > of a single node.
>>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer
>>> > <peter.neubauer@neotechnology.**com <peter.neuba...@neotechnology.com>>
>>> wrote:
>>> >> If you can write, you can code - @coderdojomalmo
>>> >> If you can sketch, you can use a graph database - @neo4j
>>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com>
>>> wrote:
>>> >> > Thanks
>>> >> > I still don't get the problem. I have node A with links to
>>> different
>>> >> > nodes.
>>> >> > I delete the links. There should be no deadlock in my scenario.
>>> Possibly
>>> >> > A
>>> >> > will be locked for a while by one delete, by why don't other
>>> deletes
>>> >> > wait
>>> >> > for the lock to be released?
>>> >> > Also can you give me a link for batch transactions?
>>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
>>> >> > <michael.hunger@neotechnology.**com<michael.hun...@neotechnology.com>>
>>> wrote:
>>> >> >> Deleting a relationship currently locks both nodes.
>>> >> >> What about deleting them all in a batch transaction?
>>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
>>> >> >> Thanks
>>> >> >> Actually I have a node X, and multiple nodes a..z that are linked
>>> to
>>> >> >> it. I
>>> >> >> am deleting all links. Since all links are different (except their
>>> end
>>> >> >> point
>>> >> >> X) I'm not sure how deleting one link should lock another?
>>> >> >> what does it mean to do it on the server side?
>>> >> >> not sure how how to handle it externally - if I have multiple
>>> clients
>>> >> >> on
>>> >> >> different machines I cannot synchronize them. I am expecting neo4j
>>> to
>>> >> >> lock
>>> >> >> everything of interest while deleting a link.
>>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis
>>> >> >> <florent.em...@gmail.com>
>>> >> >> wrote:
>>> >> >>> (Answering this with very partial knowledge, I use this as an
>>> occasion
>>> >> >>> to
>>> >> >>> learn stuff about Neo4J)
>>> >> >>> In 2009, similar issues arose (
>>> >> >>> http://www.mail-archive.com/**u...@lists.neo4j.org/msg01870.**
>>> html <http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html> )
>>> >> >>> The advice given then was to manage the locking mechanism by
>>> hand.
>>> >> >>> Looking at your trace, I'd say that your data is like this:
>>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
>>> >> >>> You are currently trying to delete node X.
>>> >> >>> Sometime before, you ran an operation (a delete I guess?) on node
>>> 1841
>>> >> >>> To be able to perform this, Relationship 8253 has to be locked
>>> (it
>>> >> >>> will
>>> >> >>> be deleted, since per
>>> >> >>> http://docs.neo4j.org/chunked/**milestone/transactions-delete.** >>> html <http://docs.neo4j.org/chunked/milestone/transactions-delete.html> all
>>> >> >>> properties and relationships of a node are deleted upon delete of
>>> the
>>> >> >>> node
>>> >> >>> itself)
>>> >> >>> When you do it in synch mode:
>>> >> >>> You: Delete Node 1841
>>> >> >>> You: Wait
>>> >> >>> Server: Delete confirmed
>>> >> >>> You: Delete Node X
>>> >> >>> Works, obviously
>>> >> >>> When you do it in asynch mode:
>>> >> >>> What you are attempting to do is:
>>> >> >>> You:Delete Node 1841
>>> >> >>> Server:Locks Node 1841 and all its relationships
>>> >> >>> You:Delete Node X
>>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on
>>> rel
>>> >> >>> 8253
>>> >> >>> Server:Boom, Err 500 on delete Node X
>>> >> >>> Server:(probably) Delete Node 1841 confirmed
>>> >> >>> Either you can manage the lock on the server side like suggested
>>> in
>>> >> >>> the
>>> >> >>> 2009 thread, or I think you'll have to manage it externally.
>>> >> >>> Can you add some sort of management in your delete calls to the
>>> >> >>> server,
>>> >> >>> partionning it to avoid collisions? Something akin to this:
>>> >> >>> [0...33][34...66][67...99]
>>> >> >>> (this example assumes that relationships are all between direct
>>> >> >>> neighbours of course...)
>>> >> >>> 3 threads run in parallel.
>>> >> >>> Thread A starts at 0 stops at 33
>>> >> >>> Thread B starts at 66 stops at 34
>>> >> >>> Thread B starts at 99 stops at 67
>>> >> >>> This should in theory avoid collisions if we assume delete
>>> operation
>>> >> >>> duration to be constant for all nodes...?
>>> >> >>> (if you want to be extra careful, add a dampenning at the end of
>>> the
>>> >> >>> partition: as you get closer to the boundary, wait a little bit
>>> >> >>> between
>>> >> >>> calls...it will give extra time to the neighbouring thread to
>>> move
>>> >> >>> further
>>> >> >>> away....)
>>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all
>>> this
>>> >> >>> with a grain of salt :-)
>>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of
>>> >> >>>> neo4j).
>>> >> >>>> I need to delete 100 nodes. When I do this synchronously there
>>> is no
>>> >> >>>> problem (e.g. I wait for one delete to finish before calling the
>>> next
>>> >> >>>> one).
>>> >> >>>> When I delete them in an async manner (many rest requests sent
>>> >> >>>> concurrently) I intermittently get a few failures. I see this
>>> coming
>>> >> >>>> back in
>>> >> >>>> the http response once per each error:
>>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but
>>> this
>>> >> >>>> does not seem relevant. I have verified my code is (logically)
>>> >> >>>> correct, e.g.
>>> >> >>>> I do not delete an already deleted node and etc.
> start n=...
> match n-[r]-m
> where r.foo = 'bar'
> delete r
> On Sat, Jul 7, 2012 at 9:44 PM, Peter Neubauer <neubauer.pe...@gmail.com>
> wrote:
> > Hi there,
> > If you can find and return the nodes you are interested in deleting with
> a
> > Cypher query then you should be able to delete them too. Do you have any
> > example graph and what to delete?
> >> I tried cyper delete, but it deletes all the connected nodes and edges,
> >> but i need to delete connected nodes based on their properties (eg :
> delete
> >> connected node only if it has value "x" on a certain property and which
> is
> >> connected to parent node through the edge "relx")
> >> I used server plugins to achieve the above behavior.
> >> How good is this approach, any comments would be helpful.
> >> Thanks,
> >> Shireesh.
> >> On Thursday, 5 July 2012 20:51:46 UTC+5:30, Peter Neubauer wrote:
> >>> Could you just try Cypher to delete your relationships?
> >>> If you can write, you can code - @coderdojomalmo
> >>> If you can sketch, you can use a graph database - @neo4j
> >>> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaron...@gmail.com>
> wrote:
> >>> > Thanks
> >>> > I'm currently using a workaround - doing everything in a sync manner.
> >>> > but
> >>> > I'm still not sure why the async version fails, this is my main worry
> >>> > now. I
> >>> > wouldn't expect batch to be required here - deleting multiple
> >>> > relationships
> >>> > of a single node.
> >>> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer
> >>> > <peter.neuba...@neotechnology.com> wrote:
> >>> >> If you can write, you can code - @coderdojomalmo
> >>> >> If you can sketch, you can use a graph database - @neo4j
> >>> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaron...@gmail.com>
> >>> >> wrote:
> >>> >> > Thanks
> >>> >> > I still don't get the problem. I have node A with links to
> different
> >>> >> > nodes.
> >>> >> > I delete the links. There should be no deadlock in my scenario.
> >>> >> > Possibly
> >>> >> > A
> >>> >> > will be locked for a while by one delete, by why don't other
> deletes
> >>> >> > wait
> >>> >> > for the lock to be released?
> >>> >> > Also can you give me a link for batch transactions?
> >>> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger
> >>> >> > <michael.hun...@neotechnology.com> wrote:
> >>> >> >> Deleting a relationship currently locks both nodes.
> >>> >> >> What about deleting them all in a batch transaction?
> >>> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
> >>> >> >> Thanks
> >>> >> >> Actually I have a node X, and multiple nodes a..z that are linked
> >>> >> >> to
> >>> >> >> it. I
> >>> >> >> am deleting all links. Since all links are different (except
> their
> >>> >> >> end
> >>> >> >> point
> >>> >> >> X) I'm not sure how deleting one link should lock another?
> >>> >> >> what does it mean to do it on the server side?
> >>> >> >> not sure how how to handle it externally - if I have multiple
> >>> >> >> clients
> >>> >> >> on
> >>> >> >> different machines I cannot synchronize them. I am expecting
> neo4j
> >>> >> >> to
> >>> >> >> lock
> >>> >> >> everything of interest while deleting a link.
> >>> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis
> >>> >> >> <florent.em...@gmail.com>
> >>> >> >> wrote:
> >>> >> >>> (Answering this with very partial knowledge, I use this as an
> >>> >> >>> occasion
> >>> >> >>> to
> >>> >> >>> learn stuff about Neo4J)
> >>> >> >>> In 2009, similar issues arose (
> >>> >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html)
> >>> >> >>> The advice given then was to manage the locking mechanism by
> hand.
> >>> >> >>> Looking at your trace, I'd say that your data is like this:
> >>> >> >>> (Node X) -[Rel 8253]-(Node 1841)
> >>> >> >>> You are currently trying to delete node X.
> >>> >> >>> Sometime before, you ran an operation (a delete I guess?) on
> node
> >>> >> >>> 1841
> >>> >> >>> To be able to perform this, Relationship 8253 has to be locked
> (it
> >>> >> >>> will
> >>> >> >>> be deleted, since per
> http://docs.neo4j.org/chunked/milestone/transactions-delete.html > >>> >> >>> all
> >>> >> >>> properties and relationships of a node are deleted upon delete
> of
> >>> >> >>> the
> >>> >> >>> node
> >>> >> >>> itself)
> >>> >> >>> When you do it in synch mode:
> >>> >> >>> You: Delete Node 1841
> >>> >> >>> You: Wait
> >>> >> >>> Server: Delete confirmed
> >>> >> >>> You: Delete Node X
> >>> >> >>> Works, obviously
> >>> >> >>> When you do it in asynch mode:
> >>> >> >>> What you are attempting to do is:
> >>> >> >>> You:Delete Node 1841
> >>> >> >>> Server:Locks Node 1841 and all its relationships
> >>> >> >>> You:Delete Node X
> >>> >> >>> Server:Attemps to lock 1841 and all its relationships: fails on
> >>> >> >>> rel
> >>> >> >>> 8253
> >>> >> >>> Server:Boom, Err 500 on delete Node X
> >>> >> >>> Server:(probably) Delete Node 1841 confirmed
> >>> >> >>> Either you can manage the lock on the server side like suggested
> >>> >> >>> in
> >>> >> >>> the
> >>> >> >>> 2009 thread, or I think you'll have to manage it externally.
> >>> >> >>> Can you add some sort of management in your delete calls to the
> >>> >> >>> server,
> >>> >> >>> partionning it to avoid collisions? Something akin to this:
> >>> >> >>> [0...33][34...66][67...99]
> >>> >> >>> (this example assumes that relationships are all between direct
> >>> >> >>> neighbours of course...)
> >>> >> >>> 3 threads run in parallel.
> >>> >> >>> Thread A starts at 0 stops at 33
> >>> >> >>> Thread B starts at 66 stops at 34
> >>> >> >>> Thread B starts at 99 stops at 67
> >>> >> >>> This should in theory avoid collisions if we assume delete
> >>> >> >>> operation
> >>> >> >>> duration to be constant for all nodes...?
> >>> >> >>> (if you want to be extra careful, add a dampenning at the end of
> >>> >> >>> the
> >>> >> >>> partition: as you get closer to the boundary, wait a little bit
> >>> >> >>> between
> >>> >> >>> calls...it will give extra time to the neighbouring thread to
> move
> >>> >> >>> further
> >>> >> >>> away....)
> >>> >> >>> Please remember I'm probably as new to Neo4J as you, so take all
> >>> >> >>> this
> >>> >> >>> with a grain of salt :-)
> >>> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api
> of
> >>> >> >>>> neo4j).
> >>> >> >>>> I need to delete 100 nodes. When I do this synchronously there
> is
> >>> >> >>>> no
> >>> >> >>>> problem (e.g. I wait for one delete to finish before calling
> the
> >>> >> >>>> next
> >>> >> >>>> one).
> >>> >> >>>> When I delete them in an async manner (many rest requests sent
> >>> >> >>>> concurrently) I intermittently get a few failures. I see this
> >>> >> >>>> coming
> >>> >> >>>> back in
> >>> >> >>>> the http response once per each error:
> >>> >> >>>> I believe neo4j uses keep-alive to reuse the same connection
> but
> >>> >> >>>> this
> >>> >> >>>> does not seem relevant. I have verified my code is (logically)
> >>> >> >>>> correct, e.g.
> >>> >> >>>> I do not delete an already deleted node and etc.
I am running into the same problem and will have to use the Cypher method you describe. However I'm also confused as to why deleting multiple relationships concurrently via the REST API produces a deadlock. Could you explain further?
> If you can write, you can code - @coderdojomalmo > If you can sketch, you can use a graph database - @neo4j
> On Thu, Jul 5, 2012 at 5:20 PM, Yaron Naveh <yaro...@gmail.com<javascript:>> > wrote: > > Thanks
> > I'm currently using a workaround - doing everything in a sync manner. > but > > I'm still not sure why the async version fails, this is my main worry > now. I > > wouldn't expect batch to be required here - deleting multiple > relationships > > of a single node.
> > On Thu, Jul 5, 2012 at 11:30 AM, Peter Neubauer > > <peter.n...@neotechnology.com <javascript:>> wrote:
> >> If you can write, you can code - @coderdojomalmo > >> If you can sketch, you can use a graph database - @neo4j
> >> On Sun, Jun 24, 2012 at 11:59 AM, Yaron Naveh <yaro...@gmail.com<javascript:>> > wrote: > >> > Thanks
> >> > I still don't get the problem. I have node A with links to different > >> > nodes. > >> > I delete the links. There should be no deadlock in my scenario. > Possibly > >> > A > >> > will be locked for a while by one delete, by why don't other deletes > >> > wait > >> > for the lock to be released?
> >> > Also can you give me a link for batch transactions?
> >> > On Sun, Jun 24, 2012 at 5:19 AM, Michael Hunger > >> > <michael...@neotechnology.com <javascript:>> wrote:
> >> >> Deleting a relationship currently locks both nodes.
> >> >> What about deleting them all in a batch transaction?
> >> >> Am 24.06.2012 um 00:44 schrieb Yaron Naveh:
> >> >> Thanks
> >> >> Actually I have a node X, and multiple nodes a..z that are linked to > >> >> it. I > >> >> am deleting all links. Since all links are different (except their > end > >> >> point > >> >> X) I'm not sure how deleting one link should lock another?
> >> >> what does it mean to do it on the server side? > >> >> not sure how how to handle it externally - if I have multiple > clients > >> >> on > >> >> different machines I cannot synchronize them. I am expecting neo4j > to > >> >> lock > >> >> everything of interest while deleting a link.
> >> >> On Sat, Jun 23, 2012 at 10:52 PM, Florent Empis > >> >> <floren...@gmail.com <javascript:>> > >> >> wrote:
> >> >>> (Answering this with very partial knowledge, I use this as an > occasion > >> >>> to > >> >>> learn stuff about Neo4J) > >> >>> In 2009, similar issues arose ( > >> >>> http://www.mail-archive.com/u...@lists.neo4j.org/msg01870.html ) > >> >>> The advice given then was to manage the locking mechanism by hand. > >> >>> Looking at your trace, I'd say that your data is like this:
> >> >>> (Node X) -[Rel 8253]-(Node 1841)
> >> >>> You are currently trying to delete node X. > >> >>> Sometime before, you ran an operation (a delete I guess?) on node > 1841 > >> >>> To be able to perform this, Relationship 8253 has to be locked (it > >> >>> will > >> >>> be deleted, since per > >> >>> http://docs.neo4j.org/chunked/milestone/transactions-delete.html all > >> >>> properties and relationships of a node are deleted upon delete of > the > >> >>> node > >> >>> itself) > >> >>> When you do it in synch mode: > >> >>> You: Delete Node 1841 > >> >>> You: Wait > >> >>> Server: Delete confirmed > >> >>> You: Delete Node X
> >> >>> Works, obviously > >> >>> When you do it in asynch mode: > >> >>> What you are attempting to do is: > >> >>> You:Delete Node 1841 > >> >>> Server:Locks Node 1841 and all its relationships > >> >>> You:Delete Node X > >> >>> Server:Attemps to lock 1841 and all its relationships: fails on rel > >> >>> 8253 > >> >>> Server:Boom, Err 500 on delete Node X > >> >>> Server:(probably) Delete Node 1841 confirmed
> >> >>> Either you can manage the lock on the server side like suggested in > >> >>> the > >> >>> 2009 thread, or I think you'll have to manage it externally.
> >> >>> Can you add some sort of management in your delete calls to the > >> >>> server, > >> >>> partionning it to avoid collisions? Something akin to this: > >> >>> [0...33][34...66][67...99] > >> >>> (this example assumes that relationships are all between direct > >> >>> neighbours of course...) > >> >>> 3 threads run in parallel. > >> >>> Thread A starts at 0 stops at 33 > >> >>> Thread B starts at 66 stops at 34 > >> >>> Thread B starts at 99 stops at 67
> >> >>> This should in theory avoid collisions if we assume delete > operation > >> >>> duration to be constant for all nodes...? > >> >>> (if you want to be extra careful, add a dampenning at the end of > the > >> >>> partition: as you get closer to the boundary, wait a little bit > >> >>> between > >> >>> calls...it will give extra time to the neighbouring thread to move > >> >>> further > >> >>> away....)
> >> >>> Please remember I'm probably as new to Neo4J as you, so take all > this > >> >>> with a grain of salt :-)
> >> >>>> I'm using neo4j 1.8 using node-neo4j (which uses the REST api of > >> >>>> neo4j).
> >> >>>> I need to delete 100 nodes. When I do this synchronously there is > no > >> >>>> problem (e.g. I wait for one delete to finish before calling the > next > >> >>>> one). > >> >>>> When I delete them in an async manner (many rest requests sent > >> >>>> concurrently) I intermittently get a few failures. I see this > coming > >> >>>> back in > >> >>>> the http response once per each error:
> >> >>>> I believe neo4j uses keep-alive to reuse the same connection but > this > >> >>>> does not seem relevant. I have verified my code is (logically) > >> >>>> correct, e.g. > >> >>>> I do not delete an already deleted node and etc.
Just discovered this thread -- great stuff. We used to see transaction
errors from time to time too, but they've mostly disappeared that now that
I'm finally migrating our app to mutable Cypher.
(Though I agree 100% that, intuitively, I would expect Neo4j to internally
serialize queries that need access to the same resources. Isn't
transactional support a key selling point of mutable Cypher?)
We obviously shouldn't be retrying every 500 response from Neo4j though (or
should we?). So I wanted to ask how you guys recommend we detect retryable
failures specifically.
Should we be checking the `exception` property in the response for a
whitelisted set of exception names? Which names?
I think it is most sensible to return a different error code as you suggested when a deadlock exception arises.
What do you think?
- 409 Conflict Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
- 423 Locked (WebDAV; RFC 4918) The resource that is being accessed is locked.
The only thing I see as an issue (in general) is when streaming back results from requests, that the deadlock exception (and others) only after the header has already been sent (and parts of the response too) as the execution happens lazily and is ongoing on the server.
> Just discovered this thread -- great stuff. We used to see transaction errors from time to time too, but they've mostly disappeared that now that I'm finally migrating our app to mutable Cypher.
> Still, I just saw one earlier -- a deadlock error message for the first time -- and it's good to know that a simple retry is recommended in this case.
> (Though I agree 100% that, intuitively, I would expect Neo4j to internally serialize queries that need access to the same resources. Isn't transactional support a key selling point of mutable Cypher?)
> We obviously shouldn't be retrying every 500 response from Neo4j though (or should we?). So I wanted to ask how you guys recommend we detect retryable failures specifically.
> Should we be checking the `exception` property in the response for a whitelisted set of exception names? Which names?
Streaming will indeed be tricky. Besides this signaling issue, how should
clients act btw if they got and processed some results back before this
kind of transaction error?
Any example kind of scenarios where a transaction error might invalidate
earlier results? Or are earlier results always okay by definition?
> I think it is most sensible to return a different error code as you
> suggested when a deadlock exception arises.
> What do you think?
> - 409 Conflict Indicates that the request could not be processed because
> of conflict in the request, such as an edit conflict.
> - 423 Locked (WebDAV; RFC 4918) The resource that is being accessed is
> locked.
> The only thing I see as an issue (in general) is when streaming back
> results from requests, that the deadlock exception (and others) only after
> the header has already been sent (and parts of the response too) as the
> execution happens lazily and is ongoing on the server.
> Michael
> Am 28.10.2012 um 01:42 schrieb Aseem Kishore:
> > Just discovered this thread -- great stuff. We used to see transaction
> errors from time to time too, but they've mostly disappeared that now that
> I'm finally migrating our app to mutable Cypher.
> > Still, I just saw one earlier -- a deadlock error message for the first
> time -- and it's good to know that a simple retry is recommended in this
> case.
> > (Though I agree 100% that, intuitively, I would expect Neo4j to
> internally serialize queries that need access to the same resources. Isn't
> transactional support a key selling point of mutable Cypher?)
> > We obviously shouldn't be retrying every 500 response from Neo4j though
> (or should we?). So I wanted to ask how you guys recommend we detect
> retryable failures specifically.
> > Should we be checking the `exception` property in the response for a
> whitelisted set of exception names? Which names?
as the transaction is rolled back the previous elements that originated from a mutating operation are invalid. So earlier results of those are _never_ ok. The transaction that is rolled back spans the whole http request.
Things that originated from a read operation which was not affected by the mutation should still be valid.
I would want to change the results for streaming operations to be more individual records that are returned, so that one or more of those record might be "error-records" reporting failures. This is still TDB though. Much like the twitter streaming API results.
> Streaming will indeed be tricky. Besides this signaling issue, how should clients act btw if they got and processed some results back before this kind of transaction error?
> Any example kind of scenarios where a transaction error might invalidate earlier results? Or are earlier results always okay by definition?
> Aseem
> On Sat, Oct 27, 2012 at 8:45 PM, Michael Hunger <michael.hun...@neotechnology.com> wrote:
> Aseem,
> can you raise an issue about this?
> I think it is most sensible to return a different error code as you suggested when a deadlock exception arises.
> What do you think?
> - 409 Conflict Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
> - 423 Locked (WebDAV; RFC 4918) The resource that is being accessed is locked.
> The only thing I see as an issue (in general) is when streaming back results from requests, that the deadlock exception (and others) only after the header has already been sent (and parts of the response too) as the execution happens lazily and is ongoing on the server.
> Michael
> Am 28.10.2012 um 01:42 schrieb Aseem Kishore:
> > Just discovered this thread -- great stuff. We used to see transaction errors from time to time too, but they've mostly disappeared that now that I'm finally migrating our app to mutable Cypher.
> > Still, I just saw one earlier -- a deadlock error message for the first time -- and it's good to know that a simple retry is recommended in this case.
> > (Though I agree 100% that, intuitively, I would expect Neo4j to internally serialize queries that need access to the same resources. Isn't transactional support a key selling point of mutable Cypher?)
> > We obviously shouldn't be retrying every 500 response from Neo4j though (or should we?). So I wanted to ask how you guys recommend we detect retryable failures specifically.
> > Should we be checking the `exception` property in the response for a whitelisted set of exception names? Which names?
