JSON reference in ravendb 5
221 views
Skip to first unread message
Bram
Aug 18, 2021, 5:24:35 AM8/18/21
to RavenDB - an awesome database
Hello,
We recently switched over from ravendb 4 to 5 and we wanted to optimize some document by creating json references for some of the object in a document. We got it to mostly work but we ran into some trouble when accessing in the references in an index. I found a post that someone else made with the same problem (https://groups.google.com/g/ravendb/c/toB6ZK1LIbY/m/qJmAXAUjAQAJ), the solution seemed to be something along the lines of don't use it. Are there any plans of supporting it in the future or does anyone know an alternative?
Example:
{
"DocumentReferences": [
{
"$id": "fb1f9c61-e042-49dc-944f-aeeb69e3d9f1",
"$type": "...",
"FirstName": "Bob",
"MiddleName": "",
"LastName": "Last name",
"RefId": "fb1f9c61-e042-49dc-944f-aeeb69e3d9f1"
},
{
"$id": "4b0a3f22-c5df-43e8-b860-f3a6407748d4",
"$type": "...",
"FirstName": "Jan",
"MiddleName": "",
"LastName": "Last name",
"RefId": "4b0a3f22-c5df-43e8-b860-f3a6407748d4"
},
{
"$id": "1bf21433-0cf4-4883-ac15-19ef20029520",
"$type": "...",
"BrandName": "Opel",
"Dealership": ["Dealership1", "Dealership2", "Dealership3"],
"RefId": "1bf21433-0cf4-4883-ac15-19ef20029520"
}
],
"FirstName": "Foo",
"MiddleName": "",
"LastName": "Bar",
"Car": {
"Brand":{
"$ref": "1bf21433-0cf4-4883-ac15-19ef20029520"
},
"Color": "Red"
},
"Contacts": [
{
"$ref": "fb1f9c61-e042-49dc-944f-aeeb69e3d9f1"
},
{
"$ref": "4b0a3f22-c5df-43e8-b860-f3a6407748d4"
}
],
"Family": [
{
"$ref": "4b0a3f22-c5df-43e8-b860-f3a6407748d4"
}
],
"Id": "2d228f94-4182-4c3a-be83-3227588021a8",
"@metadata": {
"@collection": "_People",
"Raven-Clr-Type": "..."
}
}
Igal Merhavia
Aug 18, 2021, 5:51:02 AM8/18/21
to rav...@googlegroups.com
Hi,
Are you using `CustomizeJsonSerializer`?
Can you submit a failing test to demonstrate your issue? https://ravendb.net/docs/article-page/4.2/csharp/start/test-driver
Best regards,
Igal
--
You received this message because you are subscribed to the Google Groups "RavenDB - an awesome database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ravendb/ea85d41f-78eb-4f78-84de-ce640e1ed76bn%40googlegroups.com.
Bram
Aug 18, 2021, 6:52:07 AM8/18/21
to RavenDB - an awesome database
Yes, I'm using CustomizeJsonSerializer. There is an unit test attached to this comment.
Op woensdag 18 augustus 2021 om 11:51:02 UTC+2 schreef ig...@ravendb.net:
Oren Eini (Ayende Rahien)
Aug 23, 2021, 8:48:17 AM8/23/21
to ravendb
This is the output of the document you have:
{
"DocumentReferences": [
{
"$id": "85894ee4-37f3-4be3-a4c5-a706830820b8",
"$type": "Tests.Ravendb.Indexes.JSONReferenceInIndexTest+PersonRef, SlowTests",
"RefId": "85894ee4-37f3-4be3-a4c5-a706830820b8",
"Name": "Test"
}
],
"Name": "Bob",
"Family": [
{
"$ref": "85894ee4-37f3-4be3-a4c5-a706830820b8"
}
],
"@metadata": {
"@collection": "People",
"Raven-Clr-Type": "Tests.Ravendb.Indexes.JSONReferenceInIndexTest+Person, SlowTests"
}
}
"DocumentReferences": [
{
"$id": "85894ee4-37f3-4be3-a4c5-a706830820b8",
"$type": "Tests.Ravendb.Indexes.JSONReferenceInIndexTest+PersonRef, SlowTests",
"RefId": "85894ee4-37f3-4be3-a4c5-a706830820b8",
"Name": "Test"
}
],
"Name": "Bob",
"Family": [
{
"$ref": "85894ee4-37f3-4be3-a4c5-a706830820b8"
}
],
"@metadata": {
"@collection": "People",
"Raven-Clr-Type": "Tests.Ravendb.Indexes.JSONReferenceInIndexTest+Person, SlowTests"
}
}
And your index looks like this:
from person in personsHowever, when we run this on the server side, we are operating on the shape of the server side document.
select new
{
person.Name,
Names = person.Family.Select(e => e.Name),
Query = AsJson(person).Select(x => x.Value),
};
As such , we try to index to Family, but there are no Name properties there.
You can try to write a JS index that would do a lookup for the right values, if you really want to, but that seems odd to do.
What is the scenario that you are trying to resolve?
To view this discussion on the web visit https://groups.google.com/d/msgid/ravendb/86cd06f7-c382-42b1-b34e-7cd7b4f2ddacn%40googlegroups.com.
 | Oren Eini CEO / Hibernating Rhinos LTD
    |
Bram
Aug 23, 2021, 10:41:44 AM8/23/21
to RavenDB - an awesome database
We currently have a system that allows us to keep track of reference between documents by storing a reference object in certain properties within a document. We wanted to optimize it so that we wouldn't have to store duplicate reference objects within the same document and we wanted to have an easy way of updating the references by using a patch instead of loading all the documents the have to be updated and updating them one by one.
Op maandag 23 augustus 2021 om 14:48:17 UTC+2 schreef Oren Eini:
Igal Merhavia
Aug 25, 2021, 3:10:50 AM8/25/21
to rav...@googlegroups.com
Hi,
You can always use dynamic indexes by using a javascript index or create the RQL by yourself.
If you want a strong type index, you can also define an additional class that meets the document's actual structure and use it in the index.
You achieve that you can either define it in a different namespace (or inside a class) or give it a different name and modify DocumentStore.Conventions.FindCollectionName
using Xunit.Abstractions;
using Autofac;
using Newtonsoft.Json.Serialization;
using Raven.Client.Documents.Indexes;
using Raven.Client.Json.Serialization.NewtonsoftJson;
using System;
using System.Collections.Generic;
using System.Linq;
using FastTests.Client;
using Xunit;
namespace FastTests.Client
{
public class TestClass : RavenTestBase
{
public TestClass(ITestOutputHelper output) : base(output)
{
}
[Fact]
public void LoadDocumentWithRefFromIndex()
{
using (var store = GetDocumentStore(new Options
{
ModifyDocumentStore = s => s.Conventions.Serialization = new NewtonsoftJsonSerializationConventions()
{
CustomizeJsonSerializer = serializer =>
{
serializer.ContractResolver = new ContractResolver(serializer.ContractResolver);
serializer.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects;
serializer.ReferenceResolver = new IdReferenceResolver(serializer.ReferenceResolver);
}
}
}))
{
Person_ForSearch abstractIndexCreationTask = new Person_ForSearch();
var definition = abstractIndexCreationTask.CreateIndexDefinition();
var maps = definition.Maps; // Easy way to get RQL
store.ExecuteIndex(abstractIndexCreationTask);
store.ExecuteIndex(new RawRqlIndex());
store.ExecuteIndex(new JavaScriptIndex());
using (var session = store.OpenSession())
{
var documentReferences = new List<IRef>()
{
new PersonRef()
{
RefId = Guid.NewGuid().ToString(),
Name = "Test"
}
};
Person person = new Person
{
Id = Guid.NewGuid().ToString(),
Name = "Bob",
Family = documentReferences.OfType<PersonRef>().ToList(),
DocumentReferences = documentReferences,
};
session.Store(person);
session.SaveChanges();
}
WaitForIndexing(store);
WaitForUserToContinueTheTest(store);
using (var session = store.OpenSession())
{
var documentQuery = session.Advanced.DocumentQuery<Person_ForSearch.Result, Person_ForSearch>()
.Search(e => e.Names, "Test").ToList();
Assert.True(documentQuery.Any());
}
using (var session = store.OpenSession())
{
var documentQuery = session.Advanced.DocumentQuery<Person_ForSearch.Result, RawRqlIndex>()
.Search(e => e.Names, "Test").ToList();
Assert.True(documentQuery.Any());
}
using (var session = store.OpenSession())
{
var documentQuery = session.Advanced.DocumentQuery<Person_ForSearch.Result, JavaScriptIndex>()
.Search(e => e.Names, "Test").ToList();
Assert.True(documentQuery.Any());
}
}
}
public class Person_ForSearch : AbstractIndexCreationTask<AnotherNameSpace.Person, Person_ForSearch.Result>
{
public class Result : Person
{
public string Query { get; set; }
public List<string> Names { get; set; }
}
public Person_ForSearch()
{
Map = persons => from person in persons
select new
{
person.Name,
Names = person.DocumentReferences.Select(e => e.Name),
Query = AsJson(person).Select(x => x.Value),
};
Index(i => i.Query, FieldIndexing.Search);
Index(i => i.Names, FieldIndexing.Search);
Store(i => i.Names, FieldStorage.Yes);
}
}
private class RawRqlIndex : AbstractIndexCreationTask
{
public override IndexDefinition CreateIndexDefinition()
{
return new IndexDefinition
{
Maps = new HashSet<string>
{
@"
from person in docs.People select new
{
Name = person.Name,
Names = person.DocumentReferences.Select(e => e.Name),
Query = AsJson(person).Select(x => x.Value)
}"
}
};
}
}
private class JavaScriptIndex : AbstractJavaScriptIndexCreationTask
{
public override IndexDefinition CreateIndexDefinition()
{
return new IndexDefinition
{
Maps = new HashSet<string>
{
@"
map('People', function (p) {
return {
Name: p.Name,
Names: p.DocumentReferences.map(r => r.Name),
Query: Object.values(p)
}
})"
}
};
}
}
public class Person
{
internal List<IRef> DocumentReferences { get; set; }
public string Id { get; set; }
public string Name { get; set; }
public List<PersonRef> Family { get; set; }
}
public class PersonRef : IRef
{
public string RefId { get; set; }
public string Name { get; set; }
}
public interface IRef
{
public string RefId { get; set; }
}
public class ContractResolver : IContractResolver
{
private readonly IContractResolver _contractResolver;
public ContractResolver(IContractResolver contractResolver)
{
_contractResolver = contractResolver;
}
public JsonContract ResolveContract(Type type)
{
var contract = _contractResolver.ResolveContract(type);
contract.IsReference = contract.IsReference ?? false;
if (type.IsAssignableTo<IRef>())
{
contract.IsReference = true;
}
return contract;
}
}
public class IdReferenceResolver : IReferenceResolver
{
private readonly IReferenceResolver _referenceResolver;
public IdReferenceResolver(IReferenceResolver referenceResolver)
{
_referenceResolver = referenceResolver;
}
public object ResolveReference(object context, string reference)
{
return _referenceResolver.ResolveReference(context, reference);
}
public string GetReference(object context, object value)
{
if (value is IRef)
{
var r = value as IRef;
// We need the original as the default implementation will see them as different and throw an exception!
var original = _referenceResolver.ResolveReference(context, r.RefId.ToString());
if (original == null)
{
_referenceResolver.AddReference(context, r.RefId.ToString(), value);
}
return _referenceResolver.GetReference(context, original ?? value);
}
return _referenceResolver.GetReference(context, value);
}
public bool IsReferenced(object context, object value)
{
if (value is IRef)
{
var r = value as IRef;
var original = _referenceResolver.ResolveReference(context, r.RefId.ToString());
return original != null;
}
return _referenceResolver.IsReferenced(context, value);
}
public void AddReference(object context, string reference, object value)
{
_referenceResolver.AddReference(context, reference, value);
}
}
}
}
namespace AnotherNameSpace
{
public class Person
{
internal List<TestClass.PersonRef> DocumentReferences { get; set; }
public string Id { get; set; }
public string Name { get; set; }
}
}
Best regards,
Igal
To view this discussion on the web visit https://groups.google.com/d/msgid/ravendb/e6b32084-3639-4da0-8685-d55836854aaan%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages