Index as Denormalized View

[Arm]

Hi all,

I was wondering if it is possible to have an index to act as a
denormalized view of the data. I read somewhere that in RavenDB things
should be denormalized at write time (at the document level) as
opposed to with indexes; but I believe this is a very common problem
to want to write data in some form and read it in some other form.
Here is a lame example :)


public class Person
{
public Person()
{
Id = "Persons/";
Cars = new List<Car>();
}

public string Id { get; set; }
public string Name { get; set; }
public string LastName { get; set; }
public List<Car> Cars { get; set; }
}

public class Car
{
public Car()
{
CarId = Guid.NewGuid();
}

public Guid CarId { get; set; }
public string Make { get; set; }
public string Model { get; set; }
public DateTime Year { get; set; }
}

public class PersonCar
{
public string PersonId { get; set; }
public string PersonName { get; set; }
public Guid CarId { get; set; }
public string CarMake { get; set; }
}

public class CarsIndex : AbstractIndexCreationTask<Person,
PersonCar>
{
public CarsIndex()
{
Map = persons =>
from person in persons
from car in person.Cars
select new
{
PersonId = person.Id,
PersonName = person.Name,
car.CarId,
CarMake = car.Make
};
}
}

I want something like CarsIndex index to denormalize the stored data
so I can query the data store like:

var result = session.Query<PersonCar, CarsIndex>();

foreach (var r in result)
Console.WriteLine(
"Person Id: {0}, Person Name: {1}, Car Id: {2}, Car Make:
{3}",
r.PersonId,
r.PersonName,
r.CarId,
r.CarMake);

Is this possible?

[Rob Ashton]

You can use live projections for this kind of thing, that just then
involves a look-up to get the data and that's cheap

I did an over the top example of this here:

http://codeofrob.com/archive/2010/12/16/ravendb-stackoverflow-style-voting-with-live-projections.aspx

You'll notice I want information from across multiple documents and
TransformResults gives that ability to us

[Arm]

Thanks for the response Rob. That post is interesting.

What I need is different though. I do not need reduction or
transformation; but a very simple denormalization. I just need to have
an index (or view if you will) that returns a different type to me. As
mentioned in my example I need CarsIndex to flatten my object graph
(with no aggregation required) so I can query for PersonCar. The
following index from my example does not work: it returns Person
instead of PersonCar!

public class CarsIndex : AbstractIndexCreationTask<Person,
PersonCar>
{
public CarsIndex()
{
Map = persons =>
from person in persons
from car in person.Cars
select new
{
PersonId = person.Id,
PersonName = person.Name,
car.CarId,
CarMake = car.Make
};
}
}

Thanks,
Mehdi

On Dec 19, 1:55 am, Rob Ashton <robash...@CodeOfRob.com> wrote:
> You can use live projections for this kind of thing, that just then
> involves a look-up to get the data and that's cheap
>
> I did an over the top example of this here:
>

> http://codeofrob.com/archive/2010/12/16/ravendb-stackoverflow-style-v...

[Ayende Rahien]

Oh, you can use:

session.Query<Person, CarsIndex>().As<PersonCar>();

[Arm]

Also to make it a bit clearer, if I changed the index and its result
to the following everything works, but that is hacky!

public class PersonCar
{
public string PersonId { get; set; }
public string PersonName { get; set; }
public Guid CarId { get; set; }
public string CarMake { get; set; }

public int Count { get; set; }

}

public class CarsIndex : AbstractIndexCreationTask<Person,
PersonCar>
{
public CarsIndex()
{
Map = persons => from person in persons
from car in person.Cars
select new
{
PersonId = person.Id,
PersonName = person.Name,
car.CarId,

CarMake = car.Make,
Count = 1
};

Reduce = results => from result in results
group result by new {result.PersonId,
result.PersonName, result.CarId, result.CarMake} into g
select new
{
PersonId =
g.Key.PersonId,
PersonName =
g.Key.PersonName,
CarId = g.Key.CarId,
CarMake =
g.Key.CarMake,
Count = g.Sum(p =>
p.Count)
};

}
}

All I did here was to introduce a useless field Count to PersonType
and then did a useless reduce over it (that in some cases could
actually give me different result depending on what I am grouping on).
And now it gives me what I want: a denormalized view of my data where
PersonType is returned by the index.

But I do not need aggregation/reduction. All I need is a denormalized
view! I hope it makes sense.

Thanks.

[Ayende Rahien]

Arm,

The problem is that you need to use SelectFields (projection)

We are using the index as an index (for querying), if you want to do projections from it, it is possible, certainly.

But you have to be explicit about it. 

In addition to that, you have to mark all the fields as Stored

[Arm]

Thanks for the response Ayende.

Using As<PersonCar> I still get the same error:

System.InvalidCastException was unhandled
Message=Unable to cast object of type 'ConsoleApplication1.Person'
to type 'ConsoleApplication1.PersonCar'.
Source=Raven.Client.Lightweight
StackTrace:
at
Raven.Client.Document.InMemoryDocumentSessionOperations.TrackEntity[T]
(String key, JObject document, JObject metadata)
at Raven.Client.Document.DocumentQuery`1.Deserialize(JObject
result)
at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1
collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Raven.Client.Document.DocumentQuery`1.GetEnumerator()
at Raven.Client.Linq.RavenQueryInspector`1.GetEnumerator()
at ConsoleApplication1.Program.Main(String[] args) in D:\TEMP
\ConsoleApplication1\Program.cs:line 43
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly,
String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile,
Evidence assemblySecurity, String[] args)
at
Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object
state)
at System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
ignoreSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException:

[Ayende Rahien]

Are you calling SelectFields with all the fields?

I think that As just passes null there.

[Arm]

Thanks Ayende.

I did not know about SelectFields. Is it the same as fetch on http API
for indexes?

The problem is still there. I am storing all the fields. I tried the
query with all the combinations of As<>, SelectFields, and Query<,>
but I still get the same exception: InvalidCastException: Unable to

cast object of type 'ConsoleApplication1.Person' to type
'ConsoleApplication1.PersonCar'.

You can find the complete source code below. I would really appreciate
if you could help solve this problem. My apologies for the trouble.

using System;
using System.Collections.Generic;
using System.Linq;
using Raven.Client.Client;
using Raven.Client.Indexes;
using Raven.Database.Indexing;

namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
using (var store = new EmbeddableDocumentStore
{ DataDirectory = "Data"})
{
store.Initialize();
store.DocumentDatabase.Configuration.RunInMemory =
true;

IndexCreation.CreateIndexes(typeof(CarsIndex).Assembly, store);

using (var session = store.OpenSession())
{
var john = new Person { Name = "John", LastName =
"Smith" };
john.AddCar("Holden", "Commodore",
DateTime.Parse("2005-10-05"));
john.AddCar("Toyota", "Rav 4",
DateTime.Parse("2010-12-25"));
session.Store(john);

var hellen = new Person { Name = "Hellen",
LastName = "Smith" };
hellen.AddCar("Citroen", "Xantia",
DateTime.Parse("2004-10-05"));
session.Store(hellen);
session.SaveChanges();

foreach (var p in session.Query<Person>())
{
Console.WriteLine("Id: {0}, Name: {1}, Last
Name: {2}", p.Id, p.Name, p.LastName);
foreach (var car in p.Cars)
{
Console.WriteLine("Id: {0}, Make: {1},
Model: {2}", car.CarId, car.Make, car.Model);
}
}

var result = session
.Query<PersonCar, CarsIndex>()
.Customize(q =>
q.SelectFields<PersonCar>("PersonId", "PersonName", "CarId",
"CarMake"));

foreach (var r in result)
Console.WriteLine(

"Person Name: {0}, Car Id: {1}, Car Make:
{2}",
r.PersonName,
r.CarId,
r.CarMake);
}

Console.ReadLine();

}
}
}

public class Car
{
public Car()
{
CarId = Guid.NewGuid();
}

public Guid CarId { get; set; }
public string Make { get; set; }
public string Model { get; set; }
public DateTime Year { get; set; }
}

public class Person
{
public Person()
{
Id = "Persons/";
Cars = new List<Car>();
}

public void AddCar(string make, string model, DateTime year)
{
var car = new Car
{
Make = make,
Model = model,
Year = year
};
Cars.Add(car);

}

public string Id { get; set; }
public string Name { get; set; }
public string LastName { get; set; }
public List<Car> Cars { get; set; }
}

public class PersonCar
{
public string PersonId { get; set; }
public string PersonName { get; set; }
public Guid CarId { get; set; }
public string CarMake { get; set; }
}

public class CarsIndex : AbstractIndexCreationTask<Person,
PersonCar>
{
public CarsIndex()
{
Map = persons => from person in persons
from car in person.Cars
select new
{
PersonId = person.Id,
PersonName = person.Name,
car.CarId,
CarMake = car.Make
};

Stores.Add(p => p.PersonId, FieldStorage.Yes);
Stores.Add(p => p.PersonName, FieldStorage.Yes);
Stores.Add(p => p.CarId, FieldStorage.Yes);
Stores.Add(p => p.CarMake, FieldStorage.Yes);

[Ayende Rahien]

inline

On Mon, Dec 20, 2010 at 1:08 AM, Arm <armkh...@gmail.com> wrote:

Thanks Ayende.

I did not know about SelectFields. Is it the same as fetch on http API
for indexes?

yes

 

The problem is still there. I am storing all the fields. I tried the
query with all the combinations of As<>, SelectFields, and Query<,>
but I still get the same exception: InvalidCastException: Unable to

cast object of type 'ConsoleApplication1.Person' to type
'ConsoleApplication1.PersonCar'.

I'll look at this later today, tomorrow at worst.

[Arm]

Hi Ayende,

Sorry for the late response. Been busy lately.

Did you have a chance to have a look at this issue?

On Dec 20, 4:58 pm, Ayende Rahien <aye...@ayende.com> wrote:
> inline
>

> > > > > > > >         public string Id {...
>
> read more »

[Ayende Rahien]

Yes, this should be fixed now