Understanding multi-map indexes

[Eb]

Hi -

I would like a better understanding of indexes that span multiple documents.  I had read they blog post from Ayende and think I have a basic understanding.

When the documents are specified, are they sort of "union-ed" from a document perspective?  Or better yet, how does grouping over the multiple documents really work?  What should drive the grouping?

I believe this will help me as I'm trying to create an index that spans multiple documents.

Thanks.

Eb

[Oren Eini (Ayende Rahien)]

During the map phase, the results are concatenated, then sent for reduce, if it is there.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eb]

concatenated in the sense, that if I am selecting 3 fields from each document, the result is 3 documents with 6 fields or is its 6 documents with 3 fields?

[Tal Weiss]

All of the maps should generate the same result structure. The result union of the maps is passed to the reduce.

[Eb]

Understood that it should be the same structure.  I suppose that means that the end result is 6 documents with 3 fields each.

The implications then are that the grouping should be based on a field shared between both documents.

How does this account for the fields in the other documents that are not present and are replaced with (string) null?

Thanks.

[Tal Weiss]

I'm sorry, i don't follow your question, can you share your map definitions and classes involved so i could better understand what you are trying to achieve.

--

Hibernating Rhinos Ltd  

Tal Weiss l Core Team Developer l Mobile:+972-54-802-4849

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811l Skype: talweiss1982

RavenDB paving the way to "Data Made Simple" http://ravendb.net/ 

[Oren Eini (Ayende Rahien)]

Yes, the end result would be 6 docs with 3 each.

And the idea is that you'll reduce all results together and get the non null value in the reduce.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Mon, Jul 27, 2015 at 2:30 AM, Eb <ama...@gmail.com> wrote:

[Tommi Hirvonen]

A good tool for debugging and generally understanding what happens under the hood with map-reduce indexes, is the map-reduce visualizer in raven studio. You can see exactly what is created during mapping phase for individual documents and how the reduce phase combines them to final result.

[Eb]

Thanks. Is this available in the older version of Studio?

[Eb]

But the reduce needs to be done on a field present in both documents?

[Oren Eini (Ayende Rahien)]

In 3.0 only

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--

[Oren Eini (Ayende Rahien)]

Yes

[Tal Weiss]

A multi map index is not necessarily a map/reduce index.

On Mon, Jul 27, 2015 at 2:57 PM, Tommi Hirvonen <tommi.h...@gmail.com> wrote:

A good tool for debugging and generally understanding what happens under the hood with map-reduce indexes, is the map-reduce visualizer in raven studio. You can see exactly what is created during mapping phase for individual documents and how the reduce phase combines them to final result.

--

You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

[Eb]

This was very helpful as it confirms a few things for me.

I am still struggling however with filtering the null value out from the documents that do not have the field I'm pulling.  The equivalent of the line below from Ayende's example does not seem to be working for me.

 UserName = g.Select(x => x.UserName).Where(x => x != null).First()

[Yogesh Jagota]

Can you post the index code?

[Oren Eini (Ayende Rahien)]

Or a full failing test.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Kijana Woodard]

Is this in a Reduce? Do you want FirstOrDefault instead of Where?

---

From: Eb
Sent: ‎7/‎28/‎2015 5:20 AM
To: RavenDB - 2nd generation document database
Cc: t...@ayende.com; aye...@ayende.com
Subject: Re: [RavenDB] Understanding multi-map indexes

[Eb]

Thanks for your help.  Index code is below.  I really want to group by GroupId and MemberTitle.

Map

from patient in docs.Patients

where patient.Visits.Length > 0

select new { GroupId = patient.GroupId, MemberTitle=patient.MemberTitle, Count = 0}

Map

from visit in docs.visits

select new { GroupId = visit.GroupId, MemberTitle=(string) null, Count = 1}

Reduce

from result in results

group result by new {result.GroupId} into g

select new {GroupId = g.Key.GroupId, MemberTitle = g.Select(x => x.MemberTitle), Count = g.Sum(x => x.Count)  }

[Kijana Woodard]

I notice there is patient.Visits.Length. Why not use that for Count?

What is patient.MemberTitle? 

So you want the number of visits per patient?

Can patients be in more than 1 group?

For what you have, this should work:

 MemberTitle = g.Select(x => x.MemberTitle).FirstOrDefault(x => x != null)

[Eb]

I'll try it out.  Thanks.

--

blog: http://eikonne.wordpress.com

twitter: @eikonne

[Eb]

So by and large, I'm seeing the grouping work with the field that I'm interested in containing a list of  nulls from document that doesn't have the field and then the actual value from the document that does have the field.

That being said, I am still seeing results where the value only nulls are in the list even though there should be a legitimate value.

For example.

I expect to see something like this all the time

{

  "PatientId": "patients/5511",

  "Address": [

    null,

    "14132 US Hwy 27 N"

  ]

}

but I am seeing a few documents that look like this:

{

  "PatientId": "patients/15150",

  "Address": [

    null,

    null

  ]

}

Any thoughts?

[Oren Eini (Ayende Rahien)]

Please post your full index defintion.

[Eb]

from patient in docs.Patients

where patient.Visits.Count > 0

select new {PatientId = patient.Id, Address = patient.Address1 + " " + patient.Address2 + " " + patient.City + " " + patient.State + " " + patient.Zip}

from visit in docs.Visits

select new {PatientId = visit.PatientId, Address = "Missing"}

from result in results

group result by new {result.PatientId} into g

select new {PatientId = g.Key.PatientId,  Address = g.Select(x => x.Address).FirstOrDefault(x => x != "Missing")}

[Oren Eini (Ayende Rahien)]

select new {PatientId = g.Key.PatientId,  Address = g.Select(x => x.Address).FirstOrDefault(x => x != "Missing") ?? "Missing" }

[Eb]

A bit confused.  Are you saying that's what it should be? Or something else?

Thanks.

[Oren Eini (Ayende Rahien)]

That is what it should be.

The problem is that your FirstOrDefault() returns null if there isn't a value. But you'll accept a null as a value.

[Eb]

Sorry, my example did not match the index I sent which may have led you down a wrong path.

Using the map just provided the result I get is:

{

  "PatientId": "patients/15150",

  "Address": [

    "Missing",

    "Missing"

  ]

}

where patients/15150 does have an Address in the Patients document

[Oren Eini (Ayende Rahien)]

Please create a full failing test