Update cascade bundle

[Jesús López]

Hi,

 

I plan to write an update cascade bundle to automatically update denormalized references, by using put triggers. I'm thinking on implementing two approaches to iterate through the documents I need to update:

 

GetDocumentsWithIdStartingWith.  I think this is the best approach when the referenced document collection has less than one hundred or so documents. I think this will cover a lot of cases.

 

Something similar to UpdateByIndex. In this case I will need a mechanism to wait for not stale results. At this time UpdateByIndex does not support this. Where is the piece of code at the server side that waits for not stale results? I can't find It. How can I wait for not stale results in server side code. I will also need to be able to provide aditional filters to UpdateByIndex. The filter will be a function that takes a JsonDocument and retuns a boolean because I need to be sure the document refers to the modified document before uptate it.

 

I would like update cascade bundle to be reliable so I would need to continue to update documents after a shutdown or a crash. How can I be notified when the server starts?

 

 

I'm thinking also to use ETags. Denormaliced references should have an ETag property corresponding to the referenced document ETag. This will allow me to avoid updating documents already up to date. Also, the index I iterate will have the ETag, so I will iterate only on non up to date documents.

 

Thanks.

 

[Oren Eini (Ayende Rahien)]

What is it that you are trying to do?

[Jesús López]

Ayende,

Sorry, I don't understand your question. I want to write a reliable update cascade bundle. That's all.

What I need is:

A way to wait for a particular index to be not stale in server side code.

A way to my own server side code to be notified when server starts.

I'm willing to contrib to ravendb, both core and bundles.

Please, could you help?

Thanks.

 

El miércoles, 1 de agosto de 2012 15:46:31 UTC+2, Oren Eini escribió:

What is it that you are trying to do?

[Oren Eini (Ayende Rahien)]

inline

On Wed, Aug 1, 2012 at 5:43 PM, Jesús López <jesus.lope...@gmail.com> wrote:

Ayende,

 

Sorry, I don't understand your question. I want to write a reliable update cascade bundle. That's all.

 

What I need is:

 

A way to wait for a particular index to be not stale in server side code.

You can't have that. 

You can get notified when an index is done, but there may be more docs there.

 

A way to server my own server side code be notified when server starts.

IStartupTask will do it for you. 

 

I'm willing to contrib to ravendb, both core and bundles.

 

Please, could you help?

Thanks.

 

El miércoles, 1 de agosto de 2012 15:46:31 UTC+2, Oren Eini escribió:

What is it that you are trying to do?

[Jesús López]

Thanks Oren

How can I get notified when an index is done?

How can I get the pending to index documents for a particular index?

I would be happy being notified when the index is done if I also can get the pending documents.

El miércoles, 1 de agosto de 2012 17:20:40 UTC+2, Oren Eini escribió:

inline

[Oren Eini (Ayende Rahien)]

inline

On Wed, Aug 1, 2012 at 8:11 PM, Jesús López <jesus.lope...@gmail.com> wrote:

Thanks Oren

How can I get notified when an index is done?

You write an AbstractIndexUpdateTriggger 

 

How can I get the pending to index documents for a particular index?

You check what is the last etag indexed by an index, then you compare to the last etag in the docs.

You can get all docs by etag.

[Jesús López]

Thanks.

 

I don't think I will use AbstractUpdateIndexTrigger because as I can tell, the only thing I can figure out is when a batch is processed, not when the index is done. To wait for index not to be stale I have written this:

 

public static Task WaitForIndexNotStaleAsync(this DocumentDatabase db, string indexName, DateTime? cutOff, Guid cuttOfEtag, TimeSpan timeout)
{
   
 return Task.Factory.StartNew(() =>
 {
  TimeSpan delayIncrement = TimeSpan.FromMilliseconds(50);
  TimeSpan currentDelay = TimeSpan.FromMilliseconds(100);
  DateTime timeLimit = DateTime.UtcNow.Add(timeout);
  while (db.IsIndexStale(indexName, cutOff, cuttOfEtag))
  {
   if (DateTime.UtcNow > timeLimit)
    throw new TimeoutException("Timeout exceeded waiting for index not to be stale");

   TaskExtensions.Delay(currentDelay).Wait();
   currentDelay = currentDelay.Add(delayIncrement);
  }
 });
}

 

 

I hate to do polling. But the other way is too complex.

 

Do you thing is this a good enough approach?

 

Thanks.

On Thursday, August 2, 2012 6:24:11 AM UTC+2, Oren Eini wrote:

inline

[Oren Eini (Ayende Rahien)]

That won't work in a case where you have continuous updates to the server.
Not unless you specify the cutoffs.

[Jesús López]

The idea is to specify cutoffEtag, and later iterate over pending to index documents.

 

El viernes, 3 de agosto de 2012 17:50:11 UTC+2, Oren Eini escribió:

That won't work in a case where you have continuous updates to the server.
Not unless you specify the cutoffs.

[Jesús López]

The idea is to specify the cutoff etag and later iterate on documents with greater etag than that cutoff etag. 

 

El viernes, 3 de agosto de 2012 17:50:11 UTC+2, Oren Eini escribió:

That won't work in a case where you have continuous updates to the server.
Not unless you specify the cutoffs.

[Oren Eini (Ayende Rahien)]

Note that this won't work with map/reduce indexes.

[Jesús López]

The update cascade bundle will need just map only indexes

 

El sábado, 4 de agosto de 2012 10:06:01 UTC+2, Oren Eini escribió:

Note that this won't work with map/reduce indexes.