Breaking changes in RavenDB 4

[Yogesh Jagota]

Hello,

Is there any information source on the breaking changes made in RavenDB 4?

What I cannot find:

1. WaitForNonStaleResultsAsOfLastWrite.

2. RavenQueryStatistics

3. Etag - ???

4. IDocumentStore.DatabaseCommands

5. IDocumentStore.Conventions.DocumentKeyGenerator

6. IDocumentStore.JsonRequestFactory

7. IDocumentStoreListener & RegisterListener

8. DocumentStore doesn't have Url and DefaultDatabase fields anymore

Any many more....

Some which are mentioned in the blog:

1. Patch commands

PS: There are a lot of namespace changes, so maybe I am unable to find the current namespace. 

Any help will be appreciated.

[Oren Eini (Ayende Rahien)]

We'll have the breaking change list for RC, IIRC.

1) WaitForNonStaleResultsAsOfLastWrite has been replaced with  WaitForNonStaleResultsAsOf(long cutOffEtag);

You can track the last etag yourself, since the global tracking didn't work very well for complex scenarios.

However, a much better alternative is to shift the cost to WaitForIndexesAfterSaveChanges.

2) RavenQueryStatistics -> QueryStatistics

3) Etag -> long

4) DatabaseCommands has been removed, a lot of the functionality exists in operations.

store.Operations.Send(new PatchCollectionOperation("stuffs", new PatchRequest
{
    Script = "this.Key = this.Key + 1;"
})).WaitForCompletion(TimeSpan.FromSeconds(15));

5) DocumentKeyGenerator -> RegisterAsyncIdConvention

6) JsonRequestFactory -> RequestExecuter (and you probably don't need to touch it)

7) IDocumentStoreListener was changed to events.

newSession.Advanced.OnBeforeStore += eventTest2;

 private void eventTest2(object sender, BeforeStoreEventArgs e)
 {
     var user = e.Entity as User;
     if (user != null)
     {
         user.LastName = "ravendb";
     }
 }
8) Url -> Urls, DefaultDatabase -> Database

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kijana Woodard]

I'd be interested to learn more about the back story on "3) Etag -> long".

I don't have an opinion other than I vaguely remember running into an issue working with the Etag type once, but nothing major.

[Yogesh Jagota]

Thanks for answering Oren.

Some more questions:

1. DocumentKeyGenerator works with all entity types. RegisterAsyncIdConvention needs to be used with all entity types seperately? I was doing this in 3.5:

   DocumentStore.Conventions.DocumentKeyGenerator =

   (dbname, commands, entity) => DocumentStore.Conventions.GetTypeTagName(entity.GetType()) + "/";

How to go about this in 4.0?

2. How do we implement IDocumentConversionListener? I used it to load etags in 3.5. Does this happen automatically now, or do we still need to do this manually?

3. Has EnsureDatabaseExists method been removed? And what about StartIndexing & StopIndexing?

4. Is there any way to check server availability in 4.0? 

I was shifting a big project to 4.0. The project is still under development. So many breaking changes without any documentation are making me rethink about jumping on 4.0 so early. Will you recommend me to wait?

I am also curious about the decision behind etag to long. A blog entry on ayende would be a nice read, I guess.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

The issue is that Etag was 128 bytes number that was composed of type, restarts, changes in current run.

That made things like estimating ranges very hard.

We changed that to be a long which means that given two etags we can simply subtract them to get a diff of the number of operations between them.

They are also more readable and give us better space usage internally.

[Oren Eini (Ayende Rahien)]

inline

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sun, Jun 11, 2017 at 7:55 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Thanks for answering Oren.

Some more questions:

1. DocumentKeyGenerator works with all entity types. RegisterAsyncIdConvention needs to be used with all entity types seperately? I was doing this in 3.5:

   DocumentStore.Conventions.DocumentKeyGenerator =

   (dbname, commands, entity) => DocumentStore.Conventions.GetTypeTagName(entity.GetType()) + "/";

How to go about this in 4.0?

In 4.0, this is a pretty bad idea. 

https://github.com/ravendb/book/blob/v4.0/Ch02/Ch02.md

The section about: Identity generation strategy

We have a "users|" mode, which does something similar, but it generates IDs that look like:

 users/0000000000000001332-A

 

2. How do we implement IDocumentConversionListener? I used it to load etags in 3.5. Does this happen automatically now, or do we still need to do this manually?

What is it that you are trying to do?

There is DeserializeEntityFromBlittable which is the new endpoint for that.

 

3. Has EnsureDatabaseExists method been removed? And what about StartIndexing & StopIndexing?

Yes, you need to create the db yourself.

You can write it on your own, though, but it turns out that db settings are quite important and creating it on the fly causes issues.

_store.Admin.Send(new StopIndexingOperation());

_store.Admin.Send(new StartIndexingOperation());

 

4. Is there any way to check server availability in 4.0? 

 

What do you mean by that?

 

I was shifting a big project to 4.0. The project is still under development. So many breaking changes without any documentation are making me rethink about jumping on 4.0 so early. Will you recommend me to wait?

Go ahead and start, we are working on all of that, but it will be something that you'll need to do at some point anyway.

We are working on all those things, including the list of breaking changes, but having people actually talk to us about what they are using is best. 

I am also curious about the decision behind etag to long. A blog entry on ayende would be a nice read, I guess.

I'll write something, it will show up tomorrow.

 

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

I am using IDocumentConversionListener for loading etag values, something like this:

public class DocumentConversionListener : IDocumentConversionListener
{
 ...


 public void AfterConversionToEntity(string key, RavenJObject document, RavenJObject metadata, object entity)
 {
 var entityWithEtag = entity as IHasEtag;
 if (entityWithEtag != null)
 entityWithEtag.Etag = metadata.Value<Guid>("@etag");
 }
}

Is this automatically in 4.0 or should I use DeserializeEntityFromBlittable?

By server availability, I mean, can we somehow check that is the server online or not?

EnsureDatabaseExists is very helpful in my app, but I think I can find a way around it or write my own method. But I really think that it was a nice method to have. A doesdatabaseexists/createdatabase method with fluent db settings will be even better. Just my 2 cents.

And once again, thanks for the replies. Much appreciated. I am moving my app to 4.0 as we speak. 

[Yogesh Jagota]

Some more questions:

1. MaxIndexOutputsPerDocument setting is no longer present. Do we don't need it anymore?

2. SortOptions.Double is gone. Use SortOptions.Numeric?

3. Cannot find SeedIdentityFor & NextIdentityFor.

4. DeleteCommandData now requires the etag (long?) along with the id in the constructor. Will null work? What does etag do here?

Also, I can't seem to figure out how to do something like this:

new PatchCommandData
{
 Key = Id,
 Patches = new[]
 {
 new PatchRequest
 {
 Type = PatchCommandType.Modify,
 Name = nameof(Items),
 Position = index,
 Nested = new[]
 {
 new PatchRequest { Type = PatchCommandType.Set, Name = "HasBeenPosted", Value = new RavenJValue(true) },
 }
 }
 }
};

Something along these lines:

repo.Session.Advanced.Patch(this, l => l.Items, i => i[index].HasBeenPosted = true);

[Grisha Kotler]

1. Don't need it anymore. If there will be more than 1024 results per document, you'll see a warning in the studio.

2. Yes

3. I don't think that we've implemented the client side commands for that.

http://issues.hibernatingrhinos.com/issue/RavenDB-7449

4. null will work.

We'll compare the current document etag with the provided etag (if not null). The command will fail in case of a mismatch.

You can do it using the new patching API:

session.Advanced.Patch<Item, bool>(_docId, u => u[index].HasBeenPosted , true);

See:

https://github.com/ravendb/ravendb/blob/v4.0/test/FastTests/Client/FirstClassPatch.cs#L61

Hibernating Rhinos Ltd  

Grisha Kotler l RavenDB Core Team Developer l Mobile: +972-54-586-8647

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

RavenDB paving the way to "Data Made Simple" http://ravendb.net/

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Thanks for the replies.

I tried this already:

session.Advanced.Patch<Item, bool>(_docId, u => u[index].HasBeenPosted , true);

My test is failing. As my full project cannot be completely build because of the changes as of now, will check when all issues are ironed out.

[Yogesh Jagota]

session.Advanced.Patch<Item, bool>(_docId, u => u.Items[index].HasBeenPosted , true);

not 

session.Advanced.Patch<Item, bool>(_docId, u => u[index].HasBeenPosted , true);

[Oren Eini (Ayende Rahien)]

We'll handle this in 4.0 automatically, see: http://issues.hibernatingrhinos.com/issue/RavenDB-7445

However, right now, customize using DeserializeEntityFromBlittable

Server online is a problem, because we usually operate in cluster in 4.0

You can make a request, such as trying to load a on existing document.

If you get a null result, that means the cluster is online.

If you get an error, then not.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Oren Eini (Ayende Rahien)]

1) Yes, we changed the way we are doing things. Now we still have perf issue if you have very large index items, but they will only impact that particular index, not globally.

Given that, it make more sense to just do what the index wants and not put a limit.

2) Yes

3) You can add null.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Oren Eini (Ayende Rahien)]

This should work, can you end a failing test case?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

I will post the test as soon as my projects builds. Just need to bypass usage of SeedIdentityFor & NextIdentityFor somehow. 

[Yogesh Jagota]

Some help with DeserializeEntityFromBlittable will be nice. 

[Oren Eini (Ayende Rahien)]

What do you need?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Never work with Blittable format before. Also I don't know the api. 

As I mentioned before, I want to implement this from 3.5 in 4.0 using DeserializeEntityFromBlittable, but don't know how to:

[Oren Eini (Ayende Rahien)]

store.Conventions.DeserializeEntityFromBlittable = (type, doc) =>
{
    var entity = new JsonNetBlittableEntitySerializer(store.Conventions).EntityFromJsonStream(type, doc);
    if (entity is IHasEtag hasEtag)
    {
        if (doc.TryGet(Constants.Documents.Metadata.Key, out BlittableJsonReaderObject metadata) &&
            doc.TryGet(Constants.Documents.Metadata.Etag, out long etag))
        {
            hasEtag.Etag = etag;
        }
    }
    return entity;
};

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Cannot find JsonNetBlittableEntitySerializer. Which assembly contains this class?

[Oren Eini (Ayende Rahien)]

Sorry, use:

var old = store.Conventions.DeserializeEntityFromBlittable;

store.Conventions.DeserializeEntityFromBlittable = (type, doc) =>
{

    var entity = old(type, doc);

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Thanks a lot Oren. 

Some more issues I found:

1. System.Runtime.Serialization.Primitives should be a dependency.

2. Raven-Entity-Name has been renamed to @collection in document metadata. This causes failure of all my indexes where I was using:

xyz = MetadataFor(x).Value<string>("Raven-Entity-Name")

Goes without saying, this fixes it:

xyz = MetadataFor(x).Value<string>("@collection")

But still is a change which should go in breaking changes.

[Oren Eini (Ayende Rahien)]

Thanks, I updated both

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

How can I implement something like DoesDatabaseExists and CreateDatabase methods? DoesDatabaseExists I cannot figure out at all. 

CreateDatabase I tried something like this:

var dbRecord = new DatabaseRecord("TestDatabase")

      {

      Settings = new Dictionary<string, string>()

      };

store.Admin.Server.Send(new CreateDatabaseOperation(dbRecord));

This throws a System.Security.VerificationException:Operation could destabilize the runtime. I checked settings.json,  Raven/AnonymousUserAccessMode is set to Admin.

[Oren Eini (Ayende Rahien)]

What?!

That most certainly shouldn't be able to do that. 

Can you send us a full repro?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Oren Eini (Ayende Rahien)]

You can use GetDatabaseTopologyOperation to check if the db exists

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

"That most certainly shouldn't be able to do that."

I didn't get that. It should not be able to create the database? So the exception is intentional?

[Oren Eini (Ayende Rahien)]

No, you shouldn't get the verification exception

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Thu, Jun 15, 2017 at 8:48 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

"That most certainly shouldn't be able to do that."

I didn't get that. It should not be able to create the database? So the exception is intentional?

[Yogesh Jagota]

Strangely, GetDatabaseTopologyOperation is also throwing the same exception. I am making a separate console app to try to reproduce the issue. 

System.Security.VerificationException occurred

  HResult=-2146233075

  Message=Operation could destabilize the runtime.

  Source=Anonymously Hosted DynamicMethods Assembly

  StackTrace:

       at lambda_method(Closure , BlittableJsonReaderObject )

       at Sparrow.Json.JsonDeserializationBase.ToObject[T](BlittableJsonReaderObject json, String name, Func`2 converter)

       at lambda_method(Closure , BlittableJsonReaderObject )

       at Raven.Client.Server.Commands.GetClusterTopologyCommand.SetResponse(BlittableJsonReaderObject response, Boolean fromCache)

       at Raven.Client.Http.RavenCommand`1.<ProcessResponse>d__27.MoveNext()

    --- End of stack trace from previous location where exception was thrown ---

       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

       at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)

       at Raven.Client.Http.RequestExecutor.<ExecuteAsync>d__42`1.MoveNext()

    --- End of stack trace from previous location where exception was thrown ---

       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

       at Raven.Client.Http.ClusterRequestExecutor.<UpdateTopologyAsync>d__6.MoveNext()

    --- End of stack trace from previous location where exception was thrown ---

       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

       at Raven.Client.Http.RequestExecutor.<FirstTopologyUpdate>d__39.MoveNext()

    --- End of stack trace from previous location where exception was thrown ---

       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

       at Raven.Client.Util.AsyncHelpers.RunSync[T](Func`1 task)

       at Raven.Client.Server.Operations.ServerOperationExecutor.Send[TResult](IServerOperation`1 operation)

       at NovusERPTests.DatabaseTests.PostVoucherBalancesAssertion() in D:\Short Utilities\Novus\NovusERP\Tests\NovusERPTests\DatabaseTests.cs:line 36

  InnerException: 

On Thursday, June 15, 2017 at 11:21:18 AM UTC+5:30, Oren Eini wrote:

No, you shouldn't get the verification exception

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Thu, Jun 15, 2017 at 8:48 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

"That most certainly shouldn't be able to do that."

I didn't get that. It should not be able to create the database? So the exception is intentional?

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

Are you running this on .NET core or the full runtime?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Full. 4.6.

[Oren Eini (Ayende Rahien)]

Hm... see if you can send a small repro, it would be very helpful

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Should I attach the nuget package folder too?

[Oren Eini (Ayende Rahien)]

No need, probably

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Sorry for the stupid question. Nuget restore.

Here is the attached console app which is throwing the same exception.

[ar...@ayende.com]

Hi Yogesh,

We were able to reproduce VerificationException using the console app you attached and fixed it in just released RavenDB-4.0.0-beta-40015

Regards,
Arek

[Yogesh Jagota]

Thanks. It is working now. Although there is another issue. Sometimes documentStore.Admin.Server.Send just hangs. I cannot reproduce this as it is pretty random. 

Secondly, I am getting a ThrowAlreadyDisposedException with some operations:

   {"Cannot access a disposed object.\r\nObject name: 'This ArenaMemoryAllocator is already disposed'."}

   at Sparrow.Json.ArenaMemoryAllocator.ThrowAlreadyDisposedException()

   at Sparrow.Json.BlittableWriter`1.ResetAndRenew()

   at Sparrow.Json.BlittableJsonDocumentBuilder.Renew(String debugTag, UsageMode mode)

   at Sparrow.Json.JsonOperationContext.ReadObjectInternal(Object builder, String documentId, UsageMode mode, IBlittableDocumentModifier modifier)

   at Raven.Client.Documents.Session.InMemoryDocumentSessionOperations.StoreEntityInUnitOfWork(String id, Object entity, Nullable`1 etag, DynamicJsonValue metadata, ConcurrencyCheckMode forceConcurrencyCheck)

   at Raven.Client.Documents.Session.InMemoryDocumentSessionOperations.StoreInternal(Object entity, Nullable`1 etag, String id, ConcurrencyCheckMode forceConcurrencyCheck)

   at Raven.Client.Documents.Session.InMemoryDocumentSessionOperations.Store(Object entity, Nullable`1 etag, String id)

   at NovusERP.Data.Repositories.RavenDB.DocumentRepository.Store[TEntity](TEntity item) in D:\Short Utilities\Novus\NovusERP\NovusERP.Data\Repositories\RavenDB\DocumentRepository.cs:line 90

Trying to reproduce this in a test but cannot. It is not random as it happening only at specific locations.

[Oren Eini (Ayende Rahien)]

Did you dispose of the document store when this happened?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sat, Jun 17, 2017 at 10:53 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Thanks. It is working now. Although there is another issue. Sometimes documentStore.Admin.Server.Send just hangs. I cannot reproduce this as it is pretty random. 

Secondly, I am getting a ThrowAlreadyDisposedException with some operations:

   {"Cannot access a disposed object.\r\nObject name: 'This ArenaMemoryAllocator is already disposed'."}

   at Sparrow.Json.ArenaMemoryAllocator.ThrowAlreadyDisposedException()

   at Sparrow.Json.BlittableWriter`1.ResetAndRenew()

   at Sparrow.Json.BlittableJsonDocumentBuilder.Renew(String debugTag, UsageMode mode)

   at Sparrow.Json.JsonOperationContext.ReadObjectInternal(Object builder, String documentId, UsageMode mode, IBlittableDocumentModifier modifier)

   at Raven.Client.Documents.Session.InMemoryDocumentSessionOperations.StoreEntityInUnitOfWork(String id, Object entity, Nullable`1 etag, DynamicJsonValue metadata, ConcurrencyCheckMode forceConcurrencyCheck)

   at Raven.Client.Documents.Session.InMemoryDocumentSessionOperations.StoreInternal(Object entity, Nullable`1 etag, String id, ConcurrencyCheckMode forceConcurrencyCheck)

   at Raven.Client.Documents.Session.InMemoryDocumentSessionOperations.Store(Object entity, Nullable`1 etag, String id)

   at NovusERP.Data.Repositories.RavenDB.DocumentRepository.Store[TEntity](TEntity item) in D:\Short Utilities\Novus\NovusERP\NovusERP.Data\Repositories\RavenDB\DocumentRepository.cs:line 90

Trying to reproduce this in a test but cannot. It is not random as it happening only at specific locations.

Also found something interesting:

public class ParentObject

{

public string Name { get; set; }

}

public class ChildObject

{

public string Name { get; set; }

public ParentObject Parent { get; set; }

}

var p = new ParentObject { Name = "xyz" };

// In ravendb 3.x, I had to use NextIdentityFor with p here...

// In 4.x this works without NextIdentityFor ...

var c = new ChildObject { Name = "abc", Parent = p };

session.Store(p);

session.Store(c);

session.SaveChanges();

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Yogesh Jagota]

Yes. I am initializing the DocumentStore twice. Once to see if the database exists or not and creating it if it's not (local), and then for a application wide use (global). If I merge the two things the issue goes away.

[Oren Eini (Ayende Rahien)]

Are you possible disposing the document store _while_ this is running?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sat, Jun 17, 2017 at 12:19 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Yes. I am initializing the DocumentStore twice. Once to see if the database exists or not and creating it if it's not (local), and then for a application wide use (global). If I merge the two things the issue goes away.

--

[Oren Eini (Ayende Rahien)]

Or using it after dispose?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sat, Jun 17, 2017 at 12:19 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Yes. I am initializing the DocumentStore twice. Once to see if the database exists or not and creating it if it's not (local), and then for a application wide use (global). If I merge the two things the issue goes away.

--

[Yogesh Jagota]

Reinitialising the document store causes the problem. 

some method:

// local variable

IDocumentStore documentStore = new DocumentStore { Urls = new[] { url }, Database = null };

documentStore.Initialize();

some other method:

// global variable

DocumentStore = new DocumentStore { Urls = new[] { url }, Database = "somedatabase" };

DocumentStore.Initialize();

The first document store get disposed at the end of the method so yes, dispose is the reason.

Also, while trying to write a test program for the above, I found something else. In the attached program, if I do not dispose explicitly, exiting the program causes a ObjectDisposedException. Maybe the two are related?

On Saturday, June 17, 2017 at 2:52:38 PM UTC+5:30, Oren Eini wrote:

Or using it after dispose?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sat, Jun 17, 2017 at 12:19 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Yes. I am initializing the DocumentStore twice. Once to see if the database exists or not and creating it if it's not (local), and then for a application wide use (global). If I merge the two things the issue goes away.

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Yogesh Jagota]

Any word on this? I cannot reproduce the error in a test.

[Yogesh Jagota]

Are you possible disposing the document store _while_ this is running?

No, I am not.

Or using it after dispose?

I am using another instance. Not the one which is disposed.

[ar...@ayende.com]

Hi Yogesh,

thank you for reporting this. I created a ticked for this: http://issues.hibernatingrhinos.com/issue/RavenDB-7560

A.

[Yogesh Jagota]

I am able to reproduce this issue very consistently in my main project, but cannot reproduce it in a test. Can I have access to the PDB files for Sparrow and Raven.Client so that I can debug why is this happening?

[Yogesh Jagota]

I am talking about the ObjectDisposedException thrown by Sparrow.Json.ArenaMemoryAllocator.ThrowAlreadyDisposedException.

[ar...@ayende.com]

We have already fixed that and it'll be included in the next beta. See details http://issues.hibernatingrhinos.com/issue/RavenDB-7560

[Yogesh Jagota]

So the errors are related? The ObjectDisposedException I am talking about is thrown in the Allocate method of ArenaMemoryAllocator method which throws with the message "This ArenaMemoryAllocator is already disposed", and not from the finalizer.

[ar...@ayende.com]

Oh I see you reported two errors here. We fixed the problem which you reported as " In the attached program, if I do not dispose explicitly, exiting the program causes a ObjectDisposedException." and attached ConsoleApplication1.zip

Regarding "This ArenaMemoryAllocator is already disposed" exception, this isn't reproducible using ConsoleApplication1.zip, is it? I've attached Raven.Client and Sparrow dlls from the latest v4.0 branch.

[Yogesh Jagota]

Thanks. But I still cannot debug this properly as your dlls/pdbs are different then the current branch of ravendb-4.0. What I can make out is AllocateOperationContext is returning a JsonOperationContext which has a disposed ArenaMemoryAllocator.

[Yogesh Jagota]

I am attaching exception details of the same error happening at three different locations. Maybe it can help. 

The one thing I am sure about, the ArenaMemoryAllocator  is being disposed by a task/thread asynchronously. Because it only happens after some 2-3 secs into the application. In the attached file, 2nd and 3rd error are from the same ui window. 2nd is when the window opens, and 3rd when it saves changes at close. If I open the window fast enough, I can read the data, and only get the error at close. But if I wait some secs before opening the window, I get the error straight away.

[Yogesh Jagota]

Forgot the attachment. Sorry.

[Arkadiusz Palinski]

DLLs and PDBs I provided are from current 4.0 branch build locally on my machine (note it's ahead of latest beta). I'm also attaching PDBs for Raven.Client and Sparrow DLLs as of 40015 beta.

How do you reproduce this on your end?

--
You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/j1nBCHmBBe0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.

[Arkadiusz Palinski]

Thanks for exception details. Are you able to isolate it somehow?

[Yogesh Jagota]

I am trying to isolate it but cannot. I think I will have to build ravendb locally to trace this, which I am trying to avoid. Can you point me to the source zip of 40015 beta. I cannot seem to find it.

[Arkadiusz Palinski]

Source of 40015 beta:

0) Go to: https://github.com/ravendb/ravendb/tree/b51473c8c4142aa0d6b1e36088b743171e498d1e
1) Clone or download

2) Download ZIP

I can help you in debugging this if needed (my skype: arek.palinski). What you basically need is to grab the stacktrace when ArenaMemoryAllocator is disposed to figure out who is disposing it.
https://github.com/ravendb/ravendb/blob/v4.0/src/Sparrow/Json/ArenaMemoryAllocator.cs#L268

[Yogesh Jagota]

I can now reproduce this error consistently. Attached a test project at:

http://issues.hibernatingrhinos.com/issue/RavenDB-7571

[Oren Eini (Ayende Rahien)]

Thanks, we'll have a solution for it this week

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Fri, Jul 7, 2017 at 12:15 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

I can now reproduce this error consistently. Attached a test project at:

http://issues.hibernatingrhinos.com/issue/RavenDB-7571

--

You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Wallace Turner]

You said June 11

>>We'll have the breaking change list for RC, IIRC.

This thread seems to be the most relevant result for breaking changes in 4; is there another dedicated article?

Also heres another

Indexes.Add(x => x.Address, FieldIndexing.Analyzed);

'FieldIndexing' does not contain a definition for 'Analyzed'

On Sunday, 9 July 2017 18:06:40 UTC+8, Oren Eini wrote:

Thanks, we'll have a solution for it this week

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Fri, Jul 7, 2017 at 12:15 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

I can now reproduce this error consistently. Attached a test project at:

http://issues.hibernatingrhinos.com/issue/RavenDB-7571

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

FieldIndexing.Search is the equivalent to FieldIndexing.Analyzed in 3.5

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Wallace Turner]

what about in raven3.x

store.JsonRequestFactory.RequestTimeout = TimeSpan.FromHours(1);

in raven 4 is this

store.SetRequestsTimeout(TimeSpan.FromHours(1))

?

[Oren Eini (Ayende Rahien)]

The issue for this is tracked here:

http://issues.hibernatingrhinos.com/issue/RavenDB-8788

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[georgiosd]

For anyone visiting this now, there's a typo here (took me a while to figure out)

Should be:

if (document.TryGet(Constants.Documents.Metadata.Key, out BlittableJsonReaderObject metadata)

                        && metadata.TryGet(Constants.Documents.Metadata.ChangeVector, out string changeVector))

On Tuesday, June 13, 2017 at 4:23:23 PM UTC+3, Oren Eini wrote:

Sorry, use:

var old = store.Conventions.DeserializeEntityFromBlittable;

store.Conventions.DeserializeEntityFromBlittable = (type, doc) =>
{
    var entity = old(type, doc);
    if (entity is IHasEtag hasEtag)
    {
        if (doc.TryGet(Constants.Documents.Metadata.Key, out BlittableJsonReaderObject metadata) &&
            doc.TryGet(Constants.Documents.Metadata.Etag, out long etag))
        {
            hasEtag.Etag = etag;
        }
    }
    return entity;
};

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Tue, Jun 13, 2017 at 4:08 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Cannot find JsonNetBlittableEntitySerializer. Which assembly contains this class?

On Tuesday, June 13, 2017 at 6:09:05 PM UTC+5:30, Oren Eini wrote:

store.Conventions.DeserializeEntityFromBlittable = (type, doc) =>
{
    var entity = new JsonNetBlittableEntitySerializer(store.Conventions).EntityFromJsonStream(type, doc);
    if (entity is IHasEtag hasEtag)
    {
        if (doc.TryGet(Constants.Documents.Metadata.Key, out BlittableJsonReaderObject metadata) &&
            doc.TryGet(Constants.Documents.Metadata.Etag, out long etag))
        {
            hasEtag.Etag = etag;
        }
    }
    return entity;
};

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Tue, Jun 13, 2017 at 3:16 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Never work with Blittable format before. Also I don't know the api. 

As I mentioned before, I want to implement this from 3.5 in 4.0 using DeserializeEntityFromBlittable, but don't know how to:

public class DocumentConversionListener : IDocumentConversionListener
{
 ...


 public void AfterConversionToEntity(string key, RavenJObject document, RavenJObject metadata, object entity)
 {
 var entityWithEtag = entity as IHasEtag;
 if (entityWithEtag != null)
 entityWithEtag.Etag = metadata.Value<Guid>("@etag");
 }
}

On Tuesday, June 13, 2017 at 5:12:24 PM UTC+5:30, Oren Eini wrote:

What do you need?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Tue, Jun 13, 2017 at 11:51 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Some help with DeserializeEntityFromBlittable will be nice. 

On Tuesday, June 13, 2017 at 1:31:09 PM UTC+5:30, Yogesh Jagota wrote:

I will post the test as soon as my projects builds. Just need to bypass usage of SeedIdentityFor & NextIdentityFor somehow. 

On Tuesday, June 13, 2017 at 11:34:29 AM UTC+5:30, Oren Eini wrote:

This should work, can you end a failing test case?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Tue, Jun 13, 2017 at 7:37 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

session.Advanced.Patch<Item, bool>(_docId, u => u.Items[index].HasBeenPosted , true);

not 

session.Advanced.Patch<Item, bool>(_docId, u => u[index].HasBeenPosted , true);

On Tuesday, June 13, 2017 at 10:03:09 AM UTC+5:30, Yogesh Jagota wrote:

Thanks for the replies.

I tried this already:

session.Advanced.Patch<Item, bool>(_docId, u => u[index].HasBeenPosted , true);

My test is failing. As my full project cannot be completely build because of the changes as of now, will check when all issues are ironed out.

On Tuesday, June 13, 2017 at 1:19:31 AM UTC+5:30, Grisha Kotler wrote:

1. Don't need it anymore. If there will be more than 1024 results per document, you'll see a warning in the studio.

2. Yes

3. I don't think that we've implemented the client side commands for that.

http://issues.hibernatingrhinos.com/issue/RavenDB-7449

4. null will work.

We'll compare the current document etag with the provided etag (if not null). The command will fail in case of a mismatch.

You can do it using the new patching API:

session.Advanced.Patch<Item, bool>(_docId, u => u[index].HasBeenPosted , true);

See:

https://github.com/ravendb/ravendb/blob/v4.0/test/FastTests/Client/FirstClassPatch.cs#L61

Hibernating Rhinos Ltd  

Grisha Kotler l RavenDB Core Team Developer l Mobile: +972-54-586-8647

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

RavenDB paving the way to "Data Made Simple" http://ravendb.net/

On 12 June 2017 at 22:24, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Some more questions:

1. MaxIndexOutputsPerDocument setting is no longer present. Do we don't need it anymore?

2. SortOptions.Double is gone. Use SortOptions.Numeric?

3. Cannot find SeedIdentityFor & NextIdentityFor.

4. DeleteCommandData now requires the etag (long?) along with the id in the constructor. Will null work? What does etag do here?

Also, I can't seem to figure out how to do something like this:

new PatchCommandData
{
 Key = Id,
 Patches = new[]
 {
 new PatchRequest
 {
 Type = PatchCommandType.Modify,
 Name = nameof(Items),
 Position = index,
 Nested = new[]
 {
 new PatchRequest { Type = PatchCommandType.Set, Name = "HasBeenPosted", Value = new RavenJValue(true) },
 }
 }
 }
};

Something along these lines:

repo.Session.Advanced.Patch(this, l => l.Items, i => i[index].HasBeenPosted = true);

On Monday, June 12, 2017 at 8:54:00 PM UTC+5:30, Yogesh Jagota wrote:

I am using IDocumentConversionListener for loading etag values, something like this:

public class DocumentConversionListener : IDocumentConversionListener
{
 ...


 public void AfterConversionToEntity(string key, RavenJObject document, RavenJObject metadata, object entity)
 {
 var entityWithEtag = entity as IHasEtag;
 if (entityWithEtag != null)
 entityWithEtag.Etag = metadata.Value<Guid>("@etag");
 }
}

Is this automatically in 4.0 or should I use DeserializeEntityFromBlittable?

By server availability, I mean, can we somehow check that is the server online or not?

EnsureDatabaseExists is very helpful in my app, but I think I can find a way around it or write my own method. But I really think that it was a nice method to have. A doesdatabaseexists/createdatabase method with fluent db settings will be even better. Just my 2 cents.

And once again, thanks for the replies. Much appreciated. I am moving my app to 4.0 as we speak. 

On Monday, June 12, 2017 at 1:51:53 PM UTC+5:30, Oren Eini wrote:

inline

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sun, Jun 11, 2017 at 7:55 PM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Thanks for answering Oren.

Some more questions:

1. DocumentKeyGenerator works with all entity types. RegisterAsyncIdConvention needs to be used with all entity types seperately? I was doing this in 3.5:

   DocumentStore.Conventions.DocumentKeyGenerator =

   (dbname, commands, entity) => DocumentStore.Conventions.GetTypeTagName(entity.GetType()) + "/";

How to go about this in 4.0?

In 4.0, this is a pretty bad idea. 

https://github.com/ravendb/book/blob/v4.0/Ch02/Ch02.md

The section about: Identity generation strategy

We have a "users|" mode, which does something similar, but it generates IDs that look like:

 users/0000000000000001332-A

 

2. How do we implement IDocumentConversionListener? I used it to load etags in 3.5. Does this happen automatically now, or do we still need to do this manually?

What is it that you are trying to do?

There is DeserializeEntityFromBlittable which is the new endpoint for that.

 

3. Has EnsureDatabaseExists method been removed? And what about StartIndexing & StopIndexing?

Yes, you need to create the db yourself.

You can write it on your own, though, but it turns out that db settings are quite important and creating it on the fly causes issues.

_store.Admin.Send(new StopIndexingOperation());

_store.Admin.Send(new StartIndexingOperation());

 

4. Is there any way to check server availability in 4.0? 

 

What do you mean by that?

 

I was shifting a big project to 4.0. The project is still under development. So many breaking changes without any documentation are making me rethink about jumping on 4.0 so early. Will you recommend me to wait?

Go ahead and start, we are working on all of that, but it will be something that you'll need to do at some point anyway.

We are working on all those things, including the list of breaking changes, but having people actually talk to us about what they are using is best. 

I am also curious about the decision behind etag to long. A blog entry on ayende would be a nice read, I guess.

I'll write something, it will show up tomorrow.

 

On Sunday, June 11, 2017 at 7:17:02 PM UTC+5:30, Kijana Woodard wrote:

I'd be interested to learn more about the back story on "3) Etag -> long".

I don't have an opinion other than I vaguely remember running into an issue working with the Etag type once, but nothing major.

On Sun, Jun 11, 2017 at 6:15 AM, Oren Eini (Ayende Rahien) <aye...@ayende.com> wrote:

We'll have the breaking change list for RC, IIRC.

1) WaitForNonStaleResultsAsOfLastWrite has been replaced with  WaitForNonStaleResultsAsOf(long cutOffEtag);

You can track the last etag yourself, since the global tracking didn't work very well for complex scenarios.

However, a much better alternative is to shift the cost to WaitForIndexesAfterSaveChanges.

2) RavenQueryStatistics -> QueryStatistics

3) Etag -> long

4) DatabaseCommands has been removed, a lot of the functionality exists in operations.

store.Operations.Send(new PatchCollectionOperation("stuffs", new PatchRequest
{
    Script = "this.Key = this.Key + 1;"
})).WaitForCompletion(TimeSpan.FromSeconds(15));

5) DocumentKeyGenerator -> RegisterAsyncIdConvention

6) JsonRequestFactory -> RequestExecuter (and you probably don't need to touch it)

7) IDocumentStoreListener was changed to events.

newSession.Advanced.OnBeforeStore += eventTest2;

 private void eventTest2(object sender, BeforeStoreEventArgs e)
 {
     var user = e.Entity as User;
     if (user != null)
     {
         user.LastName = "ravendb";
     }
 }
8) Url -> Urls, DefaultDatabase -> Database

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Sun, Jun 11, 2017 at 10:55 AM, Yogesh Jagota <justj...@jagotagroup.com> wrote:

Hello,

Is there any information source on the breaking changes made in RavenDB 4?

What I cannot find:

1. WaitForNonStaleResultsAsOfLastWrite.

2. RavenQueryStatistics

3. Etag - ???

4. IDocumentStore.DatabaseCommands

5. IDocumentStore.Conventions.DocumentKeyGenerator

6. IDocumentStore.JsonRequestFactory

7. IDocumentStoreListener & RegisterListener

8. DocumentStore doesn't have Url and DefaultDatabase fields anymore

Any many more....

Some which are mentioned in the blog:

1. Patch commands

PS: There are a lot of namespace changes, so maybe I am unable to find the current namespace. 

Any help will be appreciated.