Caching...

[Lewis Henderson]

I am evaluating both CQEngine and Hazelcast to provide a fast, dynamic user interface for an application.

What I have so far :-

1) Spring Framework

2) Hazelcast/EHCache, used as a cache of rapidly changing data. (Maps)

3) CQEngine to query these Maps using a small number of queries but requiring 2 or 3 joins.

Questions :-

1) Does CQEngine use any third party caching?

2) Once built, are the queries thread safe?

3) In the future, I *may* require 'user filters'. Are the queries serializable?

.. more to come as I roll my sleeves up ;-)

Cheers

[Niall Gallagher]

Sounds interesting.

Are the maps primary_key -> what could be in a column?

--
-- You received this message because you are subscribed to the "cqengine-discuss" group.
http://groups.google.com/group/cqengine-discuss
---
You received this message because you are subscribed to the Google Groups "cqengine-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cqengine-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[John Smith]

hazelcast has its own built in querying and indexing and works in map reduce fashion too. Why would you use cq engine on top of hazelcast?

[Winarto Zhao]

I've been able to make CQEngine to work with hazelcast to provide a distributed CQEngine query. However I'm not sure if it is the most effective way. We can discuss if you're interested

[John Smith]

I take my previous comment back. This could be interesting. So you use CQ queries/indexes to read through a Hazlecast map? You are not converting CQ queries/indexes to Hazelcast?

Hazelcast was one grid product where I found the querying to be slow. So if you saying you implemented CQ on top of Hazlecats this could be a game changer.How do you distribute the CQ query to the other nodes?

[Winarto Zhao]

Hi John,

Yes, I've been able to enhance CQEngine to work on top of Hazelcast. CQEngine provides the super fast query mechanism while Hazelcast provides the data distribution mechanism. So each library is performing what they do best.

The way I achieve it is by injecting Hazelcast's distributed set into CQEngine's IndexedCollection so that CQEngine works with a variant of Set that the data is distributed but has transparent Java's Set interface. Then I also hooked the CQEngine's QueryEngineInternal into Hazelcast's ItemListener so that it'll be notified when there is item added/removed from the distributed Set and call the QueryEngineInternal's notify methods so that the same reindexing process is kicked in all nodes.

This way I'm able to start up tens or even hundreds of CQEngine nodes and have the data distributed evenly across nodes.

I hope I explain it clear enough. However I'm interested to know if the threadstarter has managed to solve his problem and willing to share his solution.

Cheers

[Winarto Zhao]

So to answer John's questions, no I didn't convert CQEngine's query into Hazelcast query.

[Lewis Henderson]

All,

I am willing to share what I have.

I will need to dig back into it to see what there is!

I have had to do 'paid' work to do, so I left it as it was...

Cheers

Lewis Henderson

 

Director

CobraFlow Limited

T:01748 850045 

M:0788 7788 436

Skype:CobraFlow

www.cobraflow.com

 

Cheers

--
-- You received this message because you are subscribed to the "cqengine-discuss" group.
http://groups.google.com/group/cqengine-discuss
---

You received this message because you are subscribed to a topic in the Google Groups "cqengine-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cqengine-discuss/F6P_HfLEhnc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cqengine-discu...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lewis Henderson]

All,

I am not sure that my previous post made it to this forum...

I have uploaded two ZIP files to google drive containing the work that I did on this. I am afraid that I have done nothing more since my last post.

I am now looking at Infinispan instead of Hazelcast.

Here are the links. I will have to remove them sometime in the future if I get close to my allowance!!!!

https://drive.google.com/file/d/0B0Er-vBgkWN1N3NZTVdRUjlrVVU/edit?usp=sharing

https://drive.google.com/file/d/0B0Er-vBgkWN1V3pnZkhMdW1oX3c/edit?usp=sharing

Kind regards

To unsubscribe from this group and all its topics, send an email to cqengine-discuss+unsubscribe@googlegroups.com.

[Niall Gallagher]

Thanks Lewis. I have a copy of that now. (Not sure why it wouldn't have reached the forum before, there is no filter set up AFAIK.)

I don't have a lot of free time at the moment either (when the sun is shining outside :). But I'll take a closer look when I get a chance when the weather is not so good.

Also if you get time again in future your help will be much appreciated again on this feature.

All the best,

Niall

You received this message because you are subscribed to the Google Groups "cqengine-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cqengine-discu...@googlegroups.com.