Projecting fields from index

[Ricardo Brandão]

Hi,

I have a some questions regarding ProjectFromIndexFieldsInto. Although I know that all fields are being fetch from the index - an exception is thrown if not by setting "Raven/ImplicitFetchFieldsFromDocumentMode": "Exception" - the response still brings some additional data as depicted in the following image:

Hereupon, my questions are:

1. Why are we including the original documents in the Includes array?
2. What does SerializedOnDisk stands for? Shouldn't it be greater than 0?
   

The index is an AbstractMultiMapIndexCreationTask and all fields are being stored (i.e. StoreAllFields(FieldStorage.Yes));

Best regards,

Ricardo Brandão

[Oren Eini (Ayende Rahien)]

1) I'm assuming that the includes are there because you are specifying includes in the query. Includes always fetch the full document.

2) SerializedOnDisk stands for the size on disk for documents only, not for projections (they aren't stored on disk).

And I think that you are trying to project values that aren't on the index, but on the documents.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ricardo Brandão]

Hi Oren,

Thanks for the quick response.

1) I don't think that there is any include in the query:

GET http://localhost:8080/databases/SocialStack/indexes/Secure/SearchIndex?&query=%28%20Title%3A%28idea%2A%29%5E2%20Content%3A%28idea%2A%29%29%20AND%20%28Language%3Apt%20OR%20Language%3A%5B%5BNULL_VALUE%5D%5D%29&pageSize=25&fetch=Id&fetch=Title&fetch=PropertiesByName&fetch=Content&fetch=Date&fetch=OwnerId&fetch=ContentType&fetch=RegisteredUsersContent&fetch=InternalUsersContent&fetch=SpaceId&fetch=Language&fetch=SecurityDocumentId&tp-_Secure_UserId=users/1&include=Id HTTP/1.1

2) Ok, thanks.

I guess not because I'm storing all the values in the index and (StoreAllFields(FieldStorage.Yes);) I'm forcing an exception when trying to fetch any value that is not stored in the index ("Raven/ImplicitFetchFieldsFromDocumentMode": "Exception").

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

&include=Id 

You are projecting the following fields

Id

Title

PropertiesByName

Content

Date

OwnerId

ContentType

RegisteredUsersContent

InternalUsersContent

SpaceId

Language

SecurityDocumentId

Do you output all of them in the index?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Ricardo Brandão]

I believe so. On one hand the results are all filled up and in the index I use StoreAllFields(FieldStorage.Yes). On the other hand, opening the index with luke shows all the fields:

[Oren Eini (Ayende Rahien)]

What is the index defintion?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Ricardo Brandão]

I've found the problem. I was analyzing a field which I was trying to fetch.

Thanks :)

[Ricardo Brandão]

However I still can't fetch a list of strings

public IEnumerable<string> Tags { get; set; }

from the index. Is there any reason for that? All the other properties are being fetch from the index properly now.

[Oren Eini (Ayende Rahien)]

What does your index looks like?

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Ricardo Brandão]

It seems that the problem appears when we return an empty list in one or more of the maps. A sample can be found here.

[Ricardo Brandão]

Btw, returning null instead solves the problem.

[Stein Arild Hoem]

Related to "Raven/ImplicitFetchFieldsFromDocumentMode": "Exception" - 

I found "Id" to be problematic... as you would not like to include it in the index and query for it - as you usually just use Load(id) - not query Id =...

One would believe fetch=Id would work - without hitting documents ... but if I recall correct - it still has to be among indexed fields and stored in index... not to throw an exception with this database-setting

fetch=__document_id should at least work (whether or not fetching other fields with storage on) - as it is returned in the json response anyway - but that to throws an exception

It would have been nice if (document)Id could be fetched without having to hit the documents... if you just want to know the documentids that matches a query...

Typical Use case: need to know what documents(big documents) are local if harvesting from some other datasource

Stein Hoem 

[Ricardo Brandão]

Hi Stein,

I believe that if you don't want to make Id searchable you can set Index(x => x.Id, FieldIndexing.No); in the index definition.

[Stein Arild Hoem]

Yes - but implicit one would believe it is available without having to store it in the index at all... __document_id is there - that was my point...:

Results: 

[

• {

  • docid: "DarwinCoreProxies/O/L/37212",
  • __document_id: "darwincoreproxies/o/l/37212",
  • @metadata: 

    {

    • Temp-Index-Score: 5.90187073,
    • SerializedSizeOnDisk: 0

    }

  },

--
You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/S4QiDBiYZjM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Vennlig hilsen,

Stein Hoem

[Ricardo Brandão]

But that won't work for map/reduce indexes since the __document_id is from a reduced document mapped from multiple documents.

To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Vennlig hilsen,

Stein Hoem

[Stein Arild Hoem]

Mapreduce indexes might be the case why document id cant be fetched without putting it in the index definition.

My solution so far has been including the document id (as docid) in the index and setting fieldindexing to No. (docid = document.Id)

... but for other than mapreduce indexes - it sounds like it should not have been necessary... as it is returned anyway....

Stein

[Oren Eini (Ayende Rahien)]

Yes, that would do it, when the list is empty, we don't write anything to the index.

Just specify it so it will always have a null if empty

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Oren Eini (Ayende Rahien)]

That actually was how it worked, and we got bug reports. The reason being that on the index, it is lower case, and we have to hit the doc store to get the real case of it.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Ricardo Brandão]

Ok, thanks. 

I'm going to create some helper method to return null if empty because when doing a projection I expect to retrieve all the stored fields from the index.