How to count things fast?

[qw]

Count via reduce functions takes a lot of processor time, I am wondering if it is possible to count quickly elements emited by some view? 

[Jens Alfke]

On Aug 10, 2014, at 8:40 AM, qw <nikit...@gmail.com> wrote:

Count via reduce functions takes a lot of processor time, I am wondering if it is possible to count quickly elements emited by some view? 

There's not currently anything faster than a reduce function.

What platform are you running, and how many rows are there in the view?

—Jens

[qw]

Mac OS X. I have about 50k documents and several views each from 10 to 40k docs. I remember reduce function is calculated in runtime at the moment, right? Of course I could save intermediate values somewhere, but I thought CBL might already have this functionality :) By the way is it safe to execute raw SQL on underlying database? Or it is not the best idea considering coming of forestdb backend?

воскресенье, 10 августа 2014 г., 19:42:38 UTC+3 пользователь Jens Alfke написал:

[qw]

This is how my reduce functions looks like...

https://gist.github.com/nt9/eea164b908096d22e4da

воскресенье, 10 августа 2014 г., 19:42:38 UTC+3 пользователь Jens Alfke написал:

[Jens Alfke]

On Aug 10, 2014, at 10:49 AM, qw <nikit...@gmail.com> wrote:

Mac OS X. I have about 50k documents and several views each from 10 to 40k docs. I remember reduce function is calculated in runtime at the moment, right?

Yes, because we can't use the CouchDB trick of sticking intermediate reduce values into the b-tree nodes. (With ForestDB we might be able to do that, but it would depend on the ForestDB implementors adding the functionality to do so.)

Of course I could save intermediate values somewhere, but I thought CBL might already have this functionality :)

It doesn't currently, although the idea of doing so just came up in the discussion of issue #163.

By the way is it safe to execute raw SQL on underlying database? Or it is not the best idea considering coming of forestdb backend?

It'll work for now but would of course break in the future when we stop using SQLite.

A good compromise might be for you to implement the fix of #163, and as part of that, expose a native API to get the total row-count of a view.

—Jens

[qw]

Thank you, I'll take a look at that issue, would be glad to help this wonderful project (though my knowledge of internal details of CBL is pretty low at the moment)

воскресенье, 10 августа 2014 г., 21:41:49 UTC+3 пользователь Jens Alfke написал: