Hazelcast vs Spark: Do they both aim to solve the same problems? If not, how do they differ in the value they deliver?

[solo...@gmail.com]

I have a project where I need to integrate data in real-time and manage the results in-memory. The aim is to provide a place where fresh, integrated information can be retrieved by and presented to end users quickly.

I looked at a data virtualization approach, but from what I've read, that is more about providing a unified abstraction through which to access a variety of datastores (correct me if I'm wrong).

So I'm looking at either Hazelcast or Spark as a potential place to park post-processed data to serve up, and I'm trying to first understand the particular problems they're trying to solve and which one would be better suited to my project.

Any insight, opinion, or experience is appreciated.

Sol

[Christoph Engelbert]

Hi Sol,

what Spark are you talking about?

Spark MiniWeb Framework - http://www.sparkjava.com/

Apache Spark - http://spark.incubator.apache.org/

Ignite Spark - http://www.igniterealtime.org/projects/index.jsp

My guess would be Apache Spark, am I right?

Chris

[solo...@gmail.com]

Hey Chris:

Yes, Apache Spark.

Sol

[Christoph Engelbert]

To be honest I haven't had a llok at it but I will do it until this weekend. Hopefully I'll can give you a good answer :)

[alaa...@gmail.com]

I have the same question, but I think the difference is that Hazelcast has all data replicated across all servers , in Hadoop/Spark world the data has the divide and conquer  elements in them, so you have a sub-set of data distribute across the servers. 

Let me know what did you find out ...

[Ali Gurbuz]

Hi,

"Hazelcast has all data replicated across all servers" is wrong, Hazelcast divide the data into partitions and those partitions are distributed to the cluster. Please take a look at 'How the Data is Partitioned in Hazelcast'

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/a329957a-0df6-43dd-9c8e-5cc6a951238c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Ali Gurbuz
Core Team Lead
Mahir İz Cad. No:35, Altunizade, İstanbul
a...@hazelcast.com 
+90 507 857 7815
@aligurbuz