Efficiency of normal updates vs. PatchRequest

[Stacey]

I have a situation where I need to update about 120 documents at once, very frequently. The updates are extremely small; Almost trivial. Usually it involves just setting a single property to a new value. But it does have to happen to over a hundred documents, and it has to happen frequently, and it needs to happen very fast. 

I'm not really having trouble doing this operation. The code is very straightforward. But I've noticed that trying to change that many documents all at once seems a bit slow. So, I wrote code to do it with a PatchRequest. From what I can observe, this seems to be much faster.

But I'm curious, is it? Is a PatchRequest a better tool for a niche operation like this? Or would it just be better design to keep it as standard "load document. edit. save document" process? The difference isn't that huge; Maybe a few seconds. I'm just anticipating that it will get slower as the documents get bigger. 

[Mircea Chirea]

Load-Modify-Save will replace the document: it will have to send it twice over the network and might cause concurrency issues.

A patch will modify the document directly in the database, the only thing going over the network is the patch request and there are no concurrency issues (this applies for the scripted patch as well).

Thus a patch is always faster, but it's quite hard to work with, as it's a low level API. However, it was added precisely for your use case: lots and lots of very small, simple updates, where the standard approach of loading and saving documents has too much overhead.

[Stacey Thornton]

Thank you! That’s more or less what I was wanting to confirm. If this was the appropriate use of PatchRequest, or if I had just designed something terrible.  

[Kijana Woodard]

Have you considered ScriptedPatch? Personally I find it easier to work with.

On Tue, Aug 23, 2016 at 3:23 PM, Stacey Thornton <stacey.ci...@gmail.com> wrote:

Thank you! That’s more or less what I was wanting to confirm. If this was the appropriate use of PatchRequest, or if I had just designed something terrible.  

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mircea Chirea]

With this class it's going to be quite easy to construct patches: https://gist.github.com/CMircea/b15a1f1b7347ee160da5b8d508275fbe

For example:

var patch = RavenPatch.For<Document>("documents/1")
                      .Set(d => d.Something, "bla bla")
                      .Unset(d => d.UselessStuff)
                      .Add(d => d.SomeList, 1337)
                      .Build();

session.Advanced.Defer(patch);

// Or execute right away.
store.DatabaseCommands.Batch(new[] { patch });

The patch builder class is not complete, it doesn't cover all patch operations (only set, unset and add), but it should be quite easy to add the rest. It handles nested properties just fine.

Of course you can make a facade over the session and document commands API to simplify and reorganize some of the operations and integrate this better, but you get the point. If you're interested I can give you those as I well.

On Tuesday, August 23, 2016 at 11:31:59 PM UTC+3, Kijana Woodard wrote:

Have you considered ScriptedPatch? Personally I find it easier to work with.

On Tue, Aug 23, 2016 at 3:23 PM, Stacey Thornton <stacey.ci...@gmail.com> wrote:

Thank you! That’s more or less what I was wanting to confirm. If this was the appropriate use of PatchRequest, or if I had just designed something terrible.  

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Stacey Thornton]

Oh wow! I am very interested, yes! I would love to have this bit of code. That would simplify things greatly!

--
You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/OkP8KAaTfqw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.

[Daniel Häfele]

Sounds like a good idea for an improvement in the client api. 

Pull request maybe?

[Mircea Chirea]

I haven't tested it, just pulled it out of one of my code bases, but it should work with the IDocumentSession API. You need JetBrains.Annotations for the attributes.

I looked in my code and I don't have anything nicer than this. I used to implement something nicer:

_session.Patch<Document>("documents/1")
        .Set(d => d.Property, "value")
        // etc
        .Defer();

_session.Patch<Document>("documents/1")
        .Set(d => d.Property, "value")
        // etc
        .Execute();

But I prefer the current one where you build a patch and then defer or execute it from the session, because I usually work with patches to multiple documents. However you can implement the above with a subclass (say RavenSessionDocumentPatch) that you return from the Patch method of the session. That subclass would get the session in the constructor as a parameter, allowing you to call the defer and batch methods without any additional arguments. This will make the code a bit cleaner for patches to a single document, or for patches what you then defer to the session on SaveChanges.

There's also this for the scripted patch, though it doesn't help THAT much: https://gist.github.com/CMircea/620208479b604753d15ddf70863b95c9

You use it like so:

string removeCarFromStationScript = $"var i = this.{nameof(StationDocument.CarIdsQueue)}.indexOf(carId); if (i !== -1) this.{nameof(StationDocument.CarIdsQueue)}.splice(i, 1)";

var script = RavenScript.For(car.CurrentStationId)
                        .Script(removeCarFromStationScript)
                        .Parameters(new { carId = car.Id })
                        .Build();

var carPatch = RavenPatch.For<CarDocument>(car.Id)

                         .Set(c => c.Status, CarStatus.Free)

                         .Unset(c => c.CurrentStationId)

                         .Build();

await _raven.Commands.ExecuteAsync(carPatch, stationScript);

This code removes a car from a station and marks it as free. Note how clean and easy to understand it is. The nice part is that the entire operation is atomic, there are no concurrency issues, as would be the case with load-modify-save.

[Mircea Chirea]

I'll look into it when I have the time, but what I have will require more work to make it complete, I haven't exposed all the functionality of IDocumentSession and IDatabaseCommands.

And of course only for 4.0, as it would be a breaking change; though it can easily be added for older versions as another library and you just use RavenSession (or whatever you want to call it) instead of IDocumentSession in your code, just like I do.

[Mircea Chirea]

This is all the code I have that wraps the RavenDB session and commands API into something a bit nicer and more consistent, and includes the simpe patch builders: https://github.com/CMircea/Raven.Facade

Please feel free to use it under the MIT license, fork it and extend it (it would be great to improve it and submit a pull request for inclusion into 4.0).

You can look at my other posts in the thread for usage examples for the patch and scripted patch builders; the session parts should be self-explanatory.

What things are missing off the top of my head:

• attributes on all methods of RavenSession (and inner classes like commands and advanced)
• most patch types, like remove, insert, increment, etc.
• raw document commands like get, head, put.
• properties from the session like has changes and number of requests
• probably more low-level stuff