Partial index sorting problem

[Eric Luo]

Hello List:

I am new to Couchbase.

I have a 2 node Couchbase environment, and I have 20 documents (Articles in my application) stored in a bucket. I believe that there are 5 articles living physically in node 1 and the others are living in node 2 because when I shut down node 1, there are only 15 articles in my application's list.

I have an 'ordered by postdate' list in my application which I get results from a view like this:...emit(doc.postdate)...

Now, for some reason, the list in my application is always incorrect that some of the articles are not in the right position (not ordered by the post date).My only guess is that: node 1 has an index for the 5 articles and node 2 has an index for the other 15, they both order by the postdate field correctly in there own index, but after my application (using a .NET client library) get 5 from node 1 and 15 from node 2, it just 'concat' the two list into a full list 20,that's why I saw the wrong order in my application's article list

And from this manual here (http://docs.couchbase.com/couchbase-manual-2.2/#view-basics) I read the following:

When you query a view and thereby trigger the indexing process, you send that request to a single node in the cluster. This node then distributes the request to all other nodes in the cluster. Depending on the parameter you send in your query, each node will either send the most current partial index at that node, will update the partial index and send it, or send the partial index and update it on disk. Couchbase Server will collect and collate these partial indexes and sent this aggregate result to a client.

so this paragraph tells me that when I quey a view, I send the request to a SINGLE node, then this single node collect the collate the partial indices  and return to me a full list.Now my only question is: the items in each partial index is sure ordered by default( e.g. ordered by post date field), but what happened when the requested node 'collect and collate' these partial indices?, does it 're-order' this new full list by the sorting field or it just 'concat' the partial lists in to a full one without caring the final ordering?

From what I've learned from my application, I think it's the second situation that is the final returned list is NOT properly ordered. So can you tell me I am wrong about this and explain to me?

Thanks a lot in advance and best regards!

Eric Luo

.

[Matt Ingenthron]

Hi Eric,

From: Eric Luo <staywit...@gmail.com>
Reply-To: "couc...@googlegroups.com" <couc...@googlegroups.com>
Date: Wednesday, February 19, 2014 10:03 PM
To: "couc...@googlegroups.com" <couc...@googlegroups.com>
Subject: Partial index sorting problem

so this paragraph tells me that when I quey a view, I send the request to a SINGLE node, then this single node collect the collate the partial indices  and return to me a full list.Now my only question is: the items in each partial index is sure ordered by default( e.g. ordered by post date field), but what happened when the requested node 'collect and collate' these partial indices?, does it 're-order' this new full list by the sorting field or it just 'concat' the partial lists in to a full one without caring the final ordering?

From what I've learned from my application, I think it's the second situation that is the final returned list is NOT properly ordered. So can you tell me I am wrong about this and explain to me?

Indeed, if you're getting a different order there is an issue somewhere.

Generally, the cluster should merge sort the resulting query from all of the nodes so it should always be ordered.

One question though, do you have a reduce() function?  The reason I ask is that the component that does the merge sort may have to re-reduce your results based on any reductions being done.

It might be great if you could file an issue at couchbase.com/issues with any sample data and views.

Thanks,

Matt

-- 

Matt Ingenthron

Couchbase, Inc.

[Eric Luo]

Hi Matt:

Thanks for you quick reply. No I didn't use any reduce() function. 

So you mean that a requested node SHOULD and WILL merge & resort the merged list then send it to the requesting client? So if "there is NOT an issue somewhere" then I will get the correctly ordered result I want? I mean I don't need to reorder the list in my application if things  go all right?

Thanks and regards!

Eric

Stay with me,stay with my heart,honey.....

--
You received this message because you are subscribed to a topic in the Google Groups "Couchbase" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/couchbase/5zFNQKLjclc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to couchbase+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Matt Ingenthron]

From: Eric Luo <staywit...@gmail.com>
Reply-To: "couc...@googlegroups.com" <couc...@googlegroups.com>

Date: Thursday, February 20, 2014 12:01 AM
To: "couc...@googlegroups.com" <couc...@googlegroups.com>
Subject: Re: Partial index sorting problem

So you mean that a requested node SHOULD and WILL merge & resort the merged list then send it to the requesting client? So if "there is NOT an issue somewhere" then I will get the correctly ordered result I want? I mean I don't need to reorder the list in my application if things  go all right?

Correct.  I've not seen this issue, so I'm rather surprised.  If you can get us some more info, that'd be good.

The only other thing I can think of is possibly an error like generating a current datetime when the map function runs.

Can you share the dataset and map function?  Better to do so as an issue.