[RavenDB] Bulk data operations: UPDATE Users SET IsActive = 0 WHERE LastLogin < '2010-05-10'

[Ayende Rahien]

Raven supports patching: http://www.ravendb.net/documentation/docs-http-api-patch

And while answering a question in the mailing list it occurred to me that it would be pretty simple to extend that support further.

Imagine:

PATCH /indexes/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

[

  { "Type": "set", "Name": "IsActive", "Value": false }

]

Which is the equivalent of:

UPDATE Users

SET IsActive = false

WHERE LastLogin < '2010-05-10'

Would be pretty simple, but might have some problems with regards to consistency.

I think we can solve that by making cutOff date mandatory, then, if the request is stale according to the cutoff, we fail the request.

The client can retry until they are happy about it or time out.

Thoughts?

[Rob Ashton]

I'm all for this sort of behaviour being added in if we can - because
bulk operations *are* useful from time to time.

It's weird the notion of it failing, although I can see why you'd
potentially want it to - Perhaps make it okay to update against a
stale index (another option)? Then we can always re-run the batch
operation if necessary

[Ayende Rahien]

It is certainly a feature that would be pretty unique, CouchDB doesn't do it, and I don't think MongoDB does.

The point of it failing is that the idea of a partially successful update is pretty scary.

You mean it _didn't_ update everything?

On the other hand, people are already used to stuff like that when you have concurrent writes, because read committed and repeatable read explicitly allows for that. 

I know that I fell into that trap before: http://ayende.com/Blog/archive/2007/03/25/Transactions-and-concurrency.aspx

[Jan Benny Thomas]

This looks awesome, we should have something like this, although it is more of a dba feature. Do we need to make cutOff mandatory? Why not give it a strict default value?

 

And when we are at it, why not?

 

DELETE /indexes/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

 

Which is the equivalent of:

 

DELETE Users

WHERE LastLogin < '2010-05-10'

[Ben Hall]

How come you are doing NULL TO <value> ??

[Ayende Rahien]

That is the lucene syntax for range queries.

It means anything up to that value.

[Ayende Rahien]

Pretty scary here, because DELETE would delete the index.

I think that we need another endpoint for this, not /indexes

On Sun, May 23, 2010 at 12:49 AM, Jan Benny Thomas <jan.t...@lyse.net> wrote:

This looks awesome, we should have something like this, although it is more of a dba feature. Do we need to make cutOff mandatory? Why not give it a strict default value?

 

And when we are at it, why not?

 

DELETE /indexes/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

 

Which is the equivalent of:

 

DELETE Users

WHERE LastLogin < '2010-05-10'

 

 

From: rav...@googlegroups.com [mailto:rav...@googlegroups.com] On Behalf Of Ayende Rahien

Sent: 22. mai 2010 23:19
To: ravendb

[Ayende Rahien]

we already have /bulk_docs :-)

[Jan Benny Thomas]

That’s true.

 

But that tells us that we might make another endpoint for the patch too, maybe the same endpoint for the two operations.

 

DELETE /[batchoperation]/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

PATCH /[batchoperation]/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

[

  { "Type": "set", "Name": "IsActive", "Value": false }

]

 

 

[Ayende Rahien]

Yep, I think we can add both to /bulk_docs with no issues.

On Sun, May 23, 2010 at 1:05 AM, Jan Benny Thomas <jan.t...@lyse.net> wrote:

That’s true.

 

But that tells us that we might make another endpoint for the patch too, maybe the same endpoint for the two operations.

 

DELETE /[batchoperation]/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

PATCH /[batchoperation]/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]

[

  { "Type": "set", "Name": "IsActive", "Value": false }

]

 

 

From: rav...@googlegroups.com [mailto:rav...@googlegroups.com] On Behalf Of Ayende Rahien

Sent: 22. mai 2010 23:56
To: rav...@googlegroups.com

[Ayende Rahien]

Implemented, when the test finish running, it will be on github

[Ayende Rahien]

On github.

Any good ideas about how to name this feature?

[nabils]

Nice feature. How does this integrate into the c-sharp client api? Do
you have sample code?

On May 23, 11:42 am, Ayende Rahien <aye...@ayende.com> wrote:
> On github.
> Any good ideas about how to name this feature?
>
>
>
> On Sun, May 23, 2010 at 12:04 PM, Ayende Rahien <aye...@ayende.com> wrote:
> > Implemented, when the test finish running, it will be on github
>
> > On Sun, May 23, 2010 at 11:22 AM, Ayende Rahien <aye...@ayende.com> wrote:
>
> >> Yep, I think we can add both to /bulk_docs with no issues.
>

> >> On Sun, May 23, 2010 at 1:05 AM, Jan Benny Thomas <jan.tho...@lyse.net>wrote:
>
> >>>  That’s true.
>
> >>> But that tells us that we might make another endpoint for the patch too,
> >>> maybe the same endpoint for the two operations.
>
> >>> DELETE /[batchoperation]/UsersByLastLogin?query=LastLogin:[NULL TO
> >>> 20100510]
>
> >>> PATCH /[batchoperation]/UsersByLastLogin?query=LastLogin:[NULL TO
> >>> 20100510]
>
> >>> [
>
> >>>   { "Type": "set", "Name": "IsActive", "Value": false }
>
> >>> ]
>

> >>> *From:* rav...@googlegroups.com [mailto:rav...@googlegroups.com] *On
> >>> Behalf Of *Ayende Rahien
> >>> *Sent:* 22. mai 2010 23:56
> >>> *To:* rav...@googlegroups.com
> >>> *Subject:* Re: [RavenDB] Bulk data operations: UPDATE Users SET IsActive

> >>> = 0 WHERE LastLogin < '2010-05-10'
>
> >>> Pretty scary here, because DELETE would delete the index.
>
> >>> I think that we need another endpoint for this, not /indexes
>

> >>> On Sun, May 23, 2010 at 12:49 AM, Jan Benny Thomas <jan.tho...@lyse.net>

> >>> wrote:
>
> >>> This looks awesome, we should have something like this, although it is
> >>> more of a dba feature. Do we need to make cutOff mandatory? Why not give it
> >>> a strict default value?
>
> >>> And when we are at it, why not?
>
> >>> DELETE /indexes/UsersByLastLogin?query=LastLogin:[NULL TO 20100510]
>
> >>> Which is the equivalent of:
>
> >>> DELETE Users
>
> >>> WHERE LastLogin < '2010-05-10'
>

> >>> *From:* rav...@googlegroups.com [mailto:rav...@googlegroups.com] *On
> >>> Behalf Of *Ayende Rahien
> >>> *Sent:* 22. mai 2010 23:19
> >>> *To:* ravendb
> >>> *Subject:* [RavenDB] Bulk data operations: UPDATE Users SET IsActive = 0

[Barry King]

"Raven sets" as it sounds like a set operation from SQL world.

Sent from my iPhone

[Barry King]

"Raven sets" as it's a set operation in SQL terminolgy.

Sent from my iPhone

[Ayende Rahien]

Bails,

No, not currently, will be soon.

[Ayende Rahien]

Set based operations sounds perfect