Object instantiation

[Leonardo Salgado]

Hi, all.

Im new on CQEngine and evaluating it in a high-throughput/low-latency application.

Im testing it in scenario with hundreds of thousands queries per seconds and one issue in my case is the amount of object instantiation. 

Even with optimizations like StandingQueryIndex, there is a lot of instantiation done (and Im not sure if, in most cases, they are really needed).

The main offender is the Iterator (but there are others), according to profiling.

In some cases i run some tests with object pool (switching backingSets and changing some implementations on other classes) to avoid object instantiation and got better results with throughput.

Is there any plan to optimize the code in this way? 

Im available to contribute in this area (just need to be clarified about the roadmap)

Cheers

[Niall Gallagher]

Hi Leonardo,

Do you know the class names of the iterators which are causing the issue?

In general though - yes if you can contribute improvements in the area, it would be very welcome!

Cheers,

Niall

--
-- You received this message because you are subscribed to the "cqengine-discuss" group.
http://groups.google.com/group/cqengine-discuss
---
You received this message because you are subscribed to the Google Groups "cqengine-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cqengine-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Leonardo Salgado]

Hi, Niall.

I dont think its something related to a set of specific classes that are not doing what they were supposed to. But a work that could be done to allow the library a better fit in scenarios like mine (big data set + heavy reading load).

E.g, com.googlecode.cqengine.query.logical.Or#matches creates 2 Iterator to iterate over 2 java.util.List (super.getSimpleQueries() and super.getLogicalQueries()). These two objects are created (on the worst case, with no or-matching) unnecessarily, and they could be saved with a for-loop and with no changes at all on the library structure. 

There are some other cases, more "invasive". But i believe they could bring a lot of benefits. Queries with StandingQueryIndex, for example, would be really "standing". :D

Forgot to mention on the first email, btw:

Excelent job, this library. 

You received this message because you are subscribed to a topic in the Google Groups "cqengine-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cqengine-discuss/zuFjO5wPLrI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cqengine-discu...@googlegroups.com.

[Niall Gallagher]

Hi Leonardo,

Thanks! 

That sounds like a good idea. If you could supply patches to eliminate unnecessary iterators/object allocations, then I'd welcome them. Let's start with the less invasive cases I guess!