errors on delete - Neo4j

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion errors on delete
Received-SPF: pass (google.com: domain of neubauer.pe...@gmail.com designates 209.85.215.45 as permitted sender) client-ip=209.85.215.45;
MIME-Version: 1.0
Sender: neubauer.pe...@gmail.com
In-Reply-To: <CAGppaT0LUK2pjpyg4DCmGf8ZyFOy=A7Ze4g_hEgEcUvX82a...@mail.gmail.com>
References: <37aa1a34-1c74-4fbb-87a4-1ceca873a6ef@googlegroups.com>
 <CACODHWN=kia78_Z=vTGPLvub2hPmW7zLhBYGDowHpND_kQQ...@mail.gmail.com>
 <CAFLNJNYp7V_frtMJD5Ee4M7aqSyPT2-kTqtdSS3x-MiV8iA...@mail.gmail.com>
 <54FC159C-C54D-45C9-AA4F-39B4BF8EE...@neotechnology.com> <CAFLNJNa0NXRzD5sjN2PYU8ym+TEscHiYhd-2OR4QAcZZtfg...@mail.gmail.com>
 <CAF59RW78ANM+cXUHoheCKYLAUfB=-brXdd1mJB04mK0Hg3m...@mail.gmail.com>
 <CAFLNJNYwu45DK4PKK11vMWY-BDdSVfHPXUQL8v+L+tnwg-q...@mail.gmail.com>
 <CAF59RW4XHUReVUowERMVPrM=7o_=x5gaFYbscEze+fyshgk...@mail.gmail.com>
 <0b624324-787c-4e93-b115-cc49993f3c8c@googlegroups.com> <CAETHeiiWrEvVYfadWJZo0CjDiXz5vT1DLRR8ssNbZ4FYbCD...@mail.gmail.com>
 <74A09C93-7B2B-4095-95D7-A6F7966FD...@neotechnology.com> <CAETHeig2eLi9YCaNF3cbsm2eGo9fmaBuF=B61KVmJbiTEk5...@mail.gmail.com>
 <202F2DF2-9076-4179-A4A9-1502B5944...@neotechnology.com> <CAHKVbKKZGOL+hv0uOpC64PD_2PKZRa-esP9sFpE4DCLzMJd...@mail.gmail.com>
 <CAGppaT0LUK2pjpyg4DCmGf8ZyFOy=A7Ze4g_hEgEcUvX82a...@mail.gmail.com>
From: Peter Neubauer <peter.neuba...@neotechnology.com>
Date: Sun, 28 Oct 2012 17:33:28 +0100
Message-ID: <CAF59RW5jtNsOdsx+_C3sw=Qd2xSHesCf9i3_Fa415eMTWvk...@mail.gmail.com>
Subject: Re: [Neo4j] errors on delete
To: neo4j@googlegroups.com
Content-Type: multipart/alternative; boundary=e0cb4efe2a100db9c004cd211ee1

--e0cb4efe2a100db9c004cd211ee1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Brian,
I like the retry strategy config for the REST endpoint. Should that be part
of every query as an additional parameter, or a global config in the server
properties you think?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Neo4j 1.8 GA -
http://www.dzone.com/links/neo4j_18_release_fluent_graph_literacy.html


On Sun, Oct 28, 2012 at 4:08 PM, Brian Levine <br...@brianlevine.net> wrote=
:

> A few points.  In any Cypher query which may contain parts that are
> mutating and parts that are not, isn't there always a single response whi=
ch
> depends on the last clause in the query?  For example, if the last clause
> is a RETURN, you'll get a response that is dictated by the RETURN clause.
> If the final clause is a mutating clause, you'll get whatever that clause
> is supposed to return (e.g. number of rows affected by a DELETE). The poi=
nt
> is that the entire query is evaluated before the server begins to return
> the response.  This is true even in the streaming case, correct?  So if a
> transaction error occurs during even the most complex Cypher query, it's =
a
> transaction error for that whole query. And it occurs before the first by=
te
> is returned to the caller.  So it should be possible to immediately
> communicate the deadlock error in the response. Please correct me if I'm
> wrong.
>
> Having said all this, I think there is too much focus on how to map any
> given Neo4j error to an HTTP status code.  Yes, that's supposed to be the
> "RESTful way", but I think that when you're sending queries (which may be
> read queries, mutating querie, or both), a different approach is necessar=
y
> because you're effectively tunneling a command (Cypher query) inside a RE=
ST
> request. In that case, you either need the HTTP status to reflect the
> underlying error as closely as possible or the status code should be a 20=
0
> (since the query was valid and was accepted by the server).  409 is
> probably as close as you can get for a deadlock exception, but I'm still
> not sure it's appropriate as compared to a 200. When an error occurs the
> body should contain a structured message that provides as much informatio=
n
> as possible about what happened and what you can do about it if it's
> correctable.  In other words, as part of the media type you've defined fo=
r
> your REST API, the representation of an error should also be defined.  Th=
e
> actual form of the error result should at least contain well-documented
> Neo4j-specific error codes/strings.
>
> With respect to deadlock errors, IMHO it has become way to easy to
> introduce them. This can happen any time you try to delete 2 or more node=
s
> that share relationships since you must delete the relationships prior to
> deleting the nodes. Deleting a node, locks the node. Deleting a
> relationship locks the relationship and both its Nodes. As Aseem pointed
> out, it seems like Neo4j itself could figure out how to order the
> operations so that the locks are created/released in the correct order.  =
It
> also seems that the server itself could implement a configurable retry
> strategy so that it's not the responsibility of the client of the REST AP=
I
> to retry over the wire when these errors do occur.
>
> -brian
>
>
> On Sun, Oct 28, 2012 at 10:26 AM, Andres Taylor <
> andres.tay...@neotechnology.com> wrote:
>
>> Updating queries are executed eagerly, to avoid this exact problem. If a=
n
>> exception is thrown during execution, no results are returned.
>>
>> Andr=E9s
>> On Oct 28, 2012 2:28 AM, "Michael Hunger" <
>> michael.hun...@neotechnology.com> wrote:
>>
>>> Good points,
>>>
>>> as the transaction is rolled back the previous elements that originated
>>> from a mutating operation are invalid. So earlier results of those are
>>> _never_ ok. The transaction that is rolled back spans the whole http
>>> request.
>>>
>>> Things that originated from a read operation which was not affected by
>>> the mutation should still be valid.
>>>
>>> I would want to change the results for streaming operations to be more
>>> individual records that are returned, so that one or more of those reco=
rd
>>> might be "error-records" reporting failures. This is still TDB though. =
Much
>>> like
>>> the twitter streaming API results.
>>>
>>> Michael
>>>
>>> Am 28.10.2012 um 02:57 schrieb Aseem Kishore:
>>>
>>> Done:
>>>
>>> https://github.com/neo4j/community/issues/950
>>>
>>> Streaming will indeed be tricky. Besides this signaling issue, how
>>> should clients act btw if they got and processed some results back befo=
re
>>> this kind of transaction error?
>>>
>>> Any example kind of scenarios where a transaction error might invalidat=
e
>>> earlier results? Or are earlier results always okay by definition?
>>>
>>> Aseem
>>>
>>> On Sat, Oct 27, 2012 at 8:45 PM, Michael Hunger <
>>> michael.hun...@neotechnology.com> wrote:
>>>
>>>> Aseem,
>>>>
>>>> can you raise an issue about this?
>>>>
>>>> I think it is most sensible to return a different error code as you
>>>> suggested when a deadlock exception arises.
>>>>
>>>> What do you think?
>>>> - 409 Conflict Indicates that the request could not be processed
>>>> because of conflict in the request, such as an edit conflict.
>>>> - 423 Locked (WebDAV; RFC 4918) The resource that is being accessed is
>>>> locked.
>>>>
>>>> The only thing I see as an issue (in general) is when streaming back
>>>> results from requests, that the deadlock exception (and others) only a=
fter
>>>> the header has already been sent (and parts of the response too) as th=
e
>>>> execution happens lazily and is ongoing on the server.
>>>>
>>>> Michael
>>>>
>>>> Am 28.10.2012 um 01:42 schrieb Aseem Kishore:
>>>>
>>>> > Just discovered this thread -- great stuff. We used to see
>>>> transaction errors from time to time too, but they've mostly disappear=
ed
>>>> that now that I'm finally migrating our app to mutable Cypher.
>>>> >
>>>> > Still, I just saw one earlier -- a deadlock error message for the
>>>> first time -- and it's good to know that a simple retry is recommended=
 in
>>>> this case.
>>>> >
>>>> > (Though I agree 100% that, intuitively, I would expect Neo4j to
>>>> internally serialize queries that need access to the same resources. I=
sn't
>>>> transactional support a key selling point of mutable Cypher?)
>>>> >
>>>> > We obviously shouldn't be retrying every 500 response from Neo4j
>>>> though (or should we?). So I wanted to ask how you guys recommend we d=
etect
>>>> retryable failures specifically.
>>>> >
>>>> > Should we be checking the `exception` property in the response for a
>>>> whitelisted set of exception names? Which names?
>>>> >
>>>> > Thanks much!
>>>> >
>>>> > Aseem
>>>> >
>>>> >
>>>> > --
>>>> >
>>>> >
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>>  --
>>>
>>>
>>>
>>  --
>>
>>
>>
>
>  --
>
>
>

--e0cb4efe2a100db9c004cd211ee1
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Brian,<div>I like the retry strategy config for the REST endpoint. Should t=
hat be part of every query as an additional parameter, or a global config i=
n the server properties you think?<br clear=3D"all"><br>Cheers,<br><br>/pet=
er neubauer<br>

<br>G: =A0neubauer.peter<br>S: =A0peter.neubauer<br>P: =A0+46 704 106975<br=
>L: =A0 <a href=3D"http://www.linkedin.com/in/neubauer" target=3D"_blank">h=
ttp://www.linkedin.com/in/neubauer</a><br>T: =A0 @peterneubauer<br><br>Neo4=
j 1.8 GA - <a href=3D"http://www.dzone.com/links/neo4j_18_release_fluent_gr=
aph_literacy.html" target=3D"_blank">http://www.dzone.com/links/neo4j_18_re=
lease_fluent_graph_literacy.html</a><br>


<br><br><div class=3D"gmail_quote">On Sun, Oct 28, 2012 at 4:08 PM, Brian L=
evine <span dir=3D"ltr">&lt;<a href=3D"mailto:br...@brianlevine.net" target=
=3D"_blank">br...@brianlevine.net</a>&gt;</span> wrote:<br><blockquote clas=
s=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pad=
ding-left:1ex">

A few points. =A0In any Cypher query which may contain parts that are mutat=
ing and parts that are not, isn&#39;t there always a single response which =
depends on the last clause in the query? =A0For example, if the last clause=
 is a RETURN, you&#39;ll get a response that is dictated by the RETURN clau=
se. If the final clause is a mutating clause, you&#39;ll get whatever that =
clause is supposed to return (e.g. number of rows affected by a DELETE). Th=
e point is that the entire query is evaluated before the server begins to r=
eturn the response. =A0This is true even in the streaming case, correct? =
=A0So if a transaction error occurs during even the most complex Cypher que=
ry, it&#39;s a transaction error for that whole query. And it occurs before=
 the first byte is returned to the caller. =A0So it should be possible to i=
mmediately communicate the deadlock error in the response.=A0Please correct=
 me if I&#39;m wrong.=A0<div>


<br></div><div>Having said all this, I think there is too much focus on how=
 to map any given Neo4j error to an HTTP status code. =A0Yes, that&#39;s su=
pposed to be the &quot;RESTful way&quot;, but I think that when you&#39;re =
sending queries (which may be read queries, mutating querie, or both), a di=
fferent approach is necessary because you&#39;re effectively tunneling a co=
mmand (Cypher query) inside a REST request. In that case, you either need t=
he HTTP status to reflect the underlying error as closely as possible or th=
e status code should be a 200 (since the query was valid and was accepted b=
y the server). =A0409 is probably as close as you can get for a deadlock ex=
ception, but I&#39;m still not sure it&#39;s appropriate as compared to a 2=
00. When an error occurs the body should contain a structured message that =
provides as much information as possible about what happened and what you c=
an do about it if it&#39;s correctable. =A0In other words, as part of the m=
edia type you&#39;ve defined for your REST API, the representation of an er=
ror should also be defined. =A0The actual form of the error result should a=
t least contain well-documented Neo4j-specific error codes/strings.</div>


<div><br></div><div>With respect to deadlock errors, IMHO it has become way=
 to easy to introduce them. This can happen any time you try to delete 2 or=
 more nodes that share relationships since you must delete the relationship=
s prior to deleting the nodes. Deleting a node, locks the node. Deleting a =
relationship locks the relationship and both its Nodes. As Aseem pointed ou=
t, it seems like Neo4j itself could figure out how to order the operations =
so that the locks are created/released in the correct order. =A0It also see=
ms that the server itself could implement a configurable retry strategy so =
that it&#39;s not the responsibility of the client of the REST API to retry=
 over the wire when these errors do occur.</div>

<span class=3D"HOEnZb"><font color=3D"#888888">
<div><br></div><div>-brian</div></font></span><div><div class=3D"h5"><div><=
br><div><div><div><br><div class=3D"gmail_quote">On Sun, Oct 28, 2012 at 10=
:26 AM, Andres Taylor <span dir=3D"ltr">&lt;<a href=3D"mailto:andres.taylor=
@neotechnology.com" target=3D"_blank">andres.tay...@neotechnology.com</a>&g=
t;</span> wrote:<br>


<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><p>Updating queries are executed eagerly, to=
 avoid this exact problem. If an exception is thrown during execution, no r=
esults are returned.</p>



<p>Andr=E9s </p><div><div>
<div class=3D"gmail_quote">On Oct 28, 2012 2:28 AM, &quot;Michael Hunger&qu=
ot; &lt;<a href=3D"mailto:michael.hun...@neotechnology.com" target=3D"_blan=
k">michael.hun...@neotechnology.com</a>&gt; wrote:<br type=3D"attribution">=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">



<div style=3D"word-wrap:break-word">Good points,<div><br></div><div>as the =
transaction is rolled back the previous elements that originated from a mut=
ating operation are invalid.=A0So earlier results of those are _never_ ok. =
The transaction that is rolled back spans the whole http request.</div>



<div><br></div><div>Things that originated from a read operation which was =
not affected by the mutation should still be valid.</div><div><br></div><di=
v>I would want to change the results for streaming operations to be more in=
dividual records that are returned, so that one or more of those record mig=
ht be &quot;error-records&quot; reporting failures. This is still TDB thoug=
h. Much like=A0</div>



<div>the twitter streaming API results.=A0</div><div><br></div><div>Michael=
</div><div><br><div><div>Am 28.10.2012 um 02:57 schrieb Aseem Kishore:</div=
><br><blockquote type=3D"cite">Done:<div><br></div><div><a href=3D"https://=
github.com/neo4j/community/issues/950" target=3D"_blank">https://github.com=
/neo4j/community/issues/950</a></div>



<div><br></div><div>Streaming will indeed be tricky. Besides this signaling=
 issue, how should clients act btw if they got and processed some results b=
ack before this kind of transaction error?</div>

<div><br></div><div>Any example kind of scenarios where a transaction error=
 might invalidate earlier results? Or are earlier results always okay by de=
finition?</div><div><br></div><div>Aseem<br><br><div class=3D"gmail_quote">





On Sat, Oct 27, 2012 at 8:45 PM, Michael Hunger <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:michael.hun...@neotechnology.com" target=3D"_blank">michael.hu=
n...@neotechnology.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_=
quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1=
ex">





Aseem,<br>
<br>
can you raise an issue about this?<br>
<br>
I think it is most sensible to return a different error code as you suggest=
ed when a deadlock exception arises.<br>
<br>
What do you think?<br>
- 409 Conflict Indicates that the request could not be processed because of=
 conflict in the request, such as an edit conflict.<br>
- 423 Locked (WebDAV; RFC 4918) The resource that is being accessed is lock=
ed.<br>
<br>
The only thing I see as an issue (in general) is when streaming back result=
s from requests, that the deadlock exception (and others) only after the he=
ader has already been sent (and parts of the response too) as the execution=
 happens lazily and is ongoing on the server.<br>






<br>
Michael<br>
<br>
Am 28.10.2012 um 01:42 schrieb Aseem Kishore:<br>
<div><div><br>
&gt; Just discovered this thread -- great stuff. We used to see transaction=
 errors from time to time too, but they&#39;ve mostly disappeared that now =
that I&#39;m finally migrating our app to mutable Cypher.<br>
&gt;<br>
&gt; Still, I just saw one earlier -- a deadlock error message for the firs=
t time -- and it&#39;s good to know that a simple retry is recommended in t=
his case.<br>
&gt;<br>
&gt; (Though I agree 100% that, intuitively, I would expect Neo4j to intern=
ally serialize queries that need access to the same resources. Isn&#39;t tr=
ansactional support a key selling point of mutable Cypher?)<br>
&gt;<br>
&gt; We obviously shouldn&#39;t be retrying every 500 response from Neo4j t=
hough (or should we?). So I wanted to ask how you guys recommend we detect =
retryable failures specifically.<br>
&gt;<br>
&gt; Should we be checking the `exception` property in the response for a w=
hitelisted set of exception names? Which names?<br>
&gt;<br>
&gt; Thanks much!<br>
&gt;<br>
&gt; Aseem<br>
&gt;<br>
&gt;<br>
</div></div>&gt; --<br>
&gt;<br>
&gt;<br>
<br>
--<br>
<br>
<br>
</blockquote></div><br></div><div><br></div>

-- <br>
=A0<br>
=A0<br>
</blockquote></div><br></div></div>

<p></p>

-- <br>
=A0<br>
=A0<br>
</blockquote></div>

<p></p></div></div>

-- <br>
=A0<br>
=A0<br>
</blockquote></div><br></div></div></div></div>

<p></p></div></div>

-- <br>
=A0<br>
=A0<br>
</blockquote></div><br></div>

--e0cb4efe2a100db9c004cd211ee1--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
