hazelcast vs terracotta: benchmark?

[Andrei Pozolotin]

Hello;

can you please let me know if there is a benchmark
that compares hazelcast vs terracotta?

thank you

Andrei.

[Talip Ozturk]

> can you please let me know if there is a benchmark
> that compares hazelcast vs terracotta?

I don't know any benchmark; you should not trust the benchmark even if
there is one. I would recommend doing your own benchmarking with your
own requirements. Take a critical part of your application and develop
a prototype for both platforms, run it on a big enough cluster and
see. Also note that running 3 node cluster and 30 node cluster is
quite different. You might be surprised with results. Do no assume
things.

-talip

[Ashish]

Agree with Talip.

I have used both, and each use case is different. I plan to use
Terracotta as well as Hazelcast for different needs.
Right now am more heavily testing hazelcast for some of the needs.
Having worked at Terracotta, I know the options
to tune it. trying out the same for Hazelcast.

Everything will matter, from your network settings to hardware
settings, and to the GC settings.
My recommendation is same as Talip, try out PoC with your needs in
real like environment.

Good Luck and please do share your data :)

thanks
ashish

> --
> You received this message because you are subscribed to the Google Groups "Hazelcast" group.
> To post to this group, send email to haze...@googlegroups.com.
> To unsubscribe from this group, send email to hazelcast+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/hazelcast?hl=en.
>
>

--
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

[Andrei Pozolotin]

Ashish:

thanks for sharing your opinion; it seems you know enough about both
TC & HZ
to actually make educated choices; yes, I understand that every case
is different,
nevertheless, can you please share why did you choose TC for one case
and HZ for the other? your logic could be reused, you know :-)

thank you

Andrei

> > For more options, visit this group athttp://groups.google.com/group/hazelcast?hl=en.

[Ioannis Canellos]

Halo Anrei,

I have run some Hazelcast vs Terracota test for a very specific scenario I had in work.

To give you a slight idea of the scenario:

"I wanted a clustered key value store , in order to store correlation information, for correlating outbound requests with responses. Each node was sending requests to a remote system and the remote system was sending asynchronously responses randomly to any of my nodes". I know, it sounds very specialized but that was my use case.

Anyway, I found Hazelcast to be perform way better than Terracota and I am very happy with it.   

--
Ioannis Canellos

 http://iocanel.blogspot.com

Apache Karaf Committer & PMC

Apache ServiceMix  Committer

[Andrei Pozolotin]

Ioannis, hi!

thank you very much for sharing;

is your Hazelcast vs Terracota code sill visible somewhere on the web?

Andrei

On Apr 30, 12:22 pm, Ioannis Canellos <ioca...@gmail.com> wrote:
> Halo Anrei,
>
> I have run some Hazelcast vs Terracota test for a very specific scenario I
> had in work.
> To give you a slight idea of the scenario:
>
> "I wanted a clustered key value store , in order to store correlation
> information, for correlating outbound requests with responses. Each node was
> sending requests to a remote system and the remote system was sending
> asynchronously responses randomly to any of my nodes". I know, it sounds
> very specialized but that was my use case.
>
> Anyway, I found Hazelcast to be perform way better than Terracota and I am
> very happy with it.
>
> --

> *Ioannis Canellos*
> *
>  http://iocanel.blogspot.com
>
> Apache Karaf <http://karaf.apache.org/> Committer & PMC
> Apache ServiceMix <http://servicemix.apache.org/>  Committer
> *

[nijee taylor]

Hi Ioannis , I dont want to go into extensive details but I can tell you from experience, Terracota does not perform as well as Hazelcast. one reason?

the master node Mst, communicates with slave nodes Slv

Mst - may have 10 active Slv I/O requests each at say 1000ps

with Terracota's Master/slave architecture, all requests are broker'd through that master. If your application need's local reference to object's they must be referenced via master external to JVM's native codeset. Also, the code with HZ's interface is 10x easier with a simpler implementation method.

[Andrei Pozolotin]

Fuad:

I am trying to replicate your demo:
http://cloud.dzone.com/articles/running-hazelcast-100-node

using jenkins
http://jenkins-ci.org/

with this setup:
http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelcast-ec2/doc-inf/how-to.txt

and this test:
http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelcast-ec2/src/test/java/case_01/TestMainSample.java

problem: 2 node setup starts up ok, but nodes do not see each other;
here are logs from both nodes:
http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelcast-ec2/src/test/java/case_01/log-01.txt
http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelcast-ec2/src/test/java/case_01/log-02.txt

same test works fine on local workstation;

in other words, on local I see
######## node 1
Apr 30, 2011 7:28:50 PM com.hazelcast.cluster.ClusterManager
INFO: [dev]
Members [2] {
Member [192.168.123.1:5701] this
Member [192.168.123.1:5702]
}
######## node 2
Apr 30, 2011 7:28:50 PM com.hazelcast.cluster.ClusterManager
INFO: [dev]
Members [2] {
Member [192.168.123.1:5701]
Member [192.168.123.1:5702] this
}
######## end

and on amazon I see:
######## node 1
May 1, 2011 12:16:52 AM com.hazelcast.impl.Node
INFO: [dev]
Members [1] {
Member [10.46.57.116:5701] this
}
######## node 2
May 1, 2011 12:16:56 AM com.hazelcast.impl.Node
INFO: [dev]
Members [1] {
Member [10.202.59.146:5701] this
}
######## end

am I missing something obvious?

BTW, there are no instanceCreated/instanceDestroyed events in either
case.

thank you;

Andrei.

On Apr 29, 4:06 pm, Andrei Pozolotin <andrei.pozolo...@gmail.com>
wrote:

[Fuad Malikov]

HI Andrei,

Can I see your hazelcast.xml? How you set the IP's. Also did you open the 5701 port on security group?

Fuad

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To post to this group, send email to haze...@googlegroups.com.
To unsubscribe from this group, send email to hazelcast+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/hazelcast?hl=en.

--
Fuad Malikov
Managing Partner, Hazelcast
http://www.hazelcast.com
+90.538.378.9777
http://www.linkedin.com/in/malikov
http://twitter.com/fuadm

[Ioannis Canellos]

is your Hazelcast vs Terracota code sill visible somewhere on the web?

No its not! As I've already said, it was a test highly tuned to a specific scenario that we used on top of servicemix, so its not something that's easy to share. But if I find some spare time I will try to simplify it and blog about it.

@Nijee: Hi Nijee, indeed you are right on this.   

 

--
Ioannis Canellos

 http://iocanel.blogspot.com

Apache Karaf Committer & PMC

Apache ServiceMix  Committer

[Andrei Pozolotin]

Fuad:

1) thanks for looking on this;

2) re: "Can I see your hazelcast.xml?"
http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelcast-ec2/src/test/resources/hazelcast.xml

3) re: "How you set the IP's"
I dont; I expect hz multicast discovery to use internal amazon
10.*.*.* ip automatically;

4) re: "did you open the 5701 port on security group"
the default security group is open all-to-all for all new instances
that inherit this group; I did not open it explicitly;

Andrei.

On May 1, 2:28 am, Fuad Malikov <f...@hazelcast.com> wrote:
> HI Andrei,
>
> Can I see your hazelcast.xml? How you set the IP's. Also did you open the
> 5701 port on security group?
>
> Fuad
>

> On Sun, May 1, 2011 at 3:39 AM, Andrei Pozolotin <andrei.pozolo...@gmail.com

>
>
>
> > wrote:
> > Fuad:
>
> > I am trying to replicate your demo:
> >http://cloud.dzone.com/articles/running-hazelcast-100-node
>
> > using jenkins
> >http://jenkins-ci.org/
>
> > with this setup:
>

> >http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelc...
>
> > and this test:
>
> >http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelc...

>
> > problem: 2 node setup starts up ok, but nodes do not see each other;
> > here are logs from both nodes:
>

> >http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelc...
>
> >http://code.google.com/p/carrotgarden/source/browse/trunk/test-hazelc...

[Andrei Pozolotin]

Ioannis:
actually you "specific scenario" sounds very close to one of my use
cases; this is a small world!
it comes in the context of CEP http://esper.codehaus.org/
Andrei.

On May 1, 2:44 am, Ioannis Canellos <ioca...@gmail.com> wrote:
> > is your Hazelcast vs Terracota code sill visible somewhere on the web?
>
> No its not! As I've already said, it was a test highly tuned to a specific
> scenario that we used on top of servicemix, so its not something that's easy
> to share. But if I find some spare time I will try to simplify it and blog
> about it.
>
> @Nijee: Hi Nijee, indeed you are right on this.
>
> --

> *Ioannis Canellos*
> *
>  http://iocanel.blogspot.com
>

> Apache Karaf <http://karaf.apache.org/> Committer & PMC
> Apache ServiceMix <http://servicemix.apache.org/>  Committer
> *

[Andrei Pozolotin]

@Ashish, @Ioannis, @Vladimir, @Nijee:

1) you guys consistently report that hz is faster then tc
and make several valid points why logically this should be so;

2) however, tc by default runs with bdb persistence, and hz by default
runs in memory only;
dont you think as soon as persistence is introduced in hz, it will
become as slow as tc?

3) can you please come back with your benchmarks and share:
did you use any hz persistence, and if so, what kind of?

thanks again;

Andrei.

On Apr 29, 4:06 pm, Andrei Pozolotin <andrei.pozolo...@gmail.com>
wrote:

[Fuad Malikov]

I see now. Multicast is disabled on Amazon servers. That's why you have to make instances join with TCP/IP.  When you have couple of nodes, you can write them by hand. But when there are many nodes it becomes impossible. People asked us to implement join process over Amazon S3. We will implement this but it not available yet.

On the article "100 node Amazon cluster" I have described an automatic way of making any number of instances to join via TCP/IP. I can send you the code if you like. But a faster and easier way can be

1. Take an elastic IP
2. Create an hazelcast xml with TCP/IP join and put the elastic IP into TCP/IP list.
3. Create 1 instance, set the elastic IP to the instance.
4. Then start the rest of the instances.

Fuad

Fuad Malikov

[nijee taylor]

 I can't give you dead specific details of benchmark because of it's proximity to our production code and scheme but I can tell you this without graphs and going into to many details. 

with terracotta I have been able to parse a 200mb xml file with 53,000,000 entries like this: "<e><color>red</color><description>Lorem ipsum dolor sit amet, consectetur adipisicing elit</description><title>test-1<title><e> " 

with terracotta's architecture, I was only able to directly reference all XmlEntry objects via a seemingly synchronous pathway from master to slave which means slaves have direct global reference to only the object aware cluster; that said my rough numbers were in the range of only 2000 entries per second because of all the transactions being broker'd through the master ... add more masters you say ... my response? that's just stupid..

Hz's API integration, allowed me to distribute tasks like a poker dealer to all members irreligious to whom is responsible for brokering or executing, seeing as it's multicast and all initial packets are directed to our firewall's global multicast zone; with that said my rough numbers went into the 85,000 entries per second because not all data is addressed at once and not all HZ members need to know which data is sent if it's not configured to, my maps can send gigs of data to 1/4 members, if the data is sent to one member the other's shouldn't care unless explicitly designated to.

 this may be too vague for you but our application's data distribution throughout our solution was trivial enough for me to debug and locate the performance loss directly.

 if anyone need's further detail I can contact donald trump and ask him to do an extensive investigation on the matter... ;-)

but seriously, feel free to contact me for more info on the build..

[Jack Strohm]

I didn't think multicast would work in Amazon EC2. Am I wrong about this?

Sent from my iPhone

[Talip Ozturk]

You are right. Multicast doesn't work in EC2.

http://twitter.com/oztalip

[Timothy Peierls]

Also see my post about doing node discovery via Amazon S3 for Elastic Beanstalk:

https://groups.google.com/d/topic/hazelcast/FAzFzJjSnJI/discussion

--tim

[Andrei Pozolotin]

Tim:
this is pretty cool idea, thanks; do you by chance have code shared
somewhere?
Andrei.

[Timothy Peierls]

I didn't, but I just now put something in http://pastebin.com/EUrgQAVp about which I make no claims of suitability, advisability, or correctness whatsoever. It's there purely to illustrate the idea. If it's helpful, great; if not, don't blame me.

--tim

[Ashish]

On Sat, Apr 30, 2011 at 9:08 PM, Andrei Pozolotin
<andrei.p...@gmail.com> wrote:
> Ashish:
>
> thanks for sharing your opinion; it seems you know enough about both
> TC & HZ
> to actually make educated choices; yes, I understand that every case
> is different,
> nevertheless, can you please share why did you choose TC for one case
> and HZ for the other? your logic could be reused, you know :-)
>
> thank you
>
> Andrei

Sorry, was on vacation.. just saw the post..

I am using TC for write-behind to a DB. And it works like a charm.

Using Hazelcast, I want to create a IMDG to store huge data that need
not be HA. As of this writing, I am still turning the knobs on
hazelcast :)

thanks
ashish

[Ashish]

On Sun, May 1, 2011 at 10:54 PM, Andrei Pozolotin
<andrei.p...@gmail.com> wrote:
> @Ashish, @Ioannis, @Vladimir, @Nijee:
>
> 1) you guys consistently report that hz is faster then tc
> and make several valid points why logically this should be so;

I am yet to conclude on this. In TC world, striping is not free :(
So either compare one TC Server vs one HZ server and see.
Also, you can see a lot of configuration setting in TC world which can
make a real difference to the TPS.

>
> 2) however, tc by default runs with bdb persistence, and hz by default
> runs in memory only;
> dont you think as soon as persistence is introduced in hz, it will
> become as slow as tc?

in TC persistence config, you can use temp-swap mode for an
apple-apple comparison.
Yes, as soon as you need Reliability of data, you need to pay some price :(
When using TC, always ensure to watch TC dev-console (found in bin
dir). IT tells you a lot about the cluster.

>
> 3) can you please come back with your benchmarks and share:
> did you use any hz persistence, and if so, what kind of?

I would still say, you should use your benchmark.
BTW, TC has an ehcacheperf code
(http://svn.terracotta.org/svn/forge/projects/ehcacheperf/trunk) that
you can use for benchmarking.
Need to write a simple extender for Hz :)

HTH

>
> thanks again;
>
> Andrei.
>
> On Apr 29, 4:06 pm, Andrei Pozolotin <andrei.pozolo...@gmail.com>
> wrote:
>> Hello;
>>
>> can you please let me know if there is a benchmark
>> that compares hazelcast vs terracotta?
>>
>> thank you
>>
>> Andrei.
>

> --
> You received this message because you are subscribed to the Google Groups "Hazelcast" group.
> To post to this group, send email to haze...@googlegroups.com.
> To unsubscribe from this group, send email to hazelcast+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/hazelcast?hl=en.
>
>

--

[saket....@gmail.com]

Hi Ashish,

I also have to benchmark hazelcast & gridgain for IMDGs performance. But not finding any suitable benchmark frame work. However found RadarGun framework but not sure getting much info pertaining to Scenario configuration or other basic configuration. I did it but didnt get success.

Next picked up YCSB, that also seem to be not fruitful. Any help is much Appreciated!

Regards,

Saket

[saket....@gmail.com]

Fuad, I am using RadarGun for hz benchmark but stuck in Config especially scenarios. Do you have any sample or suggestion or steps for same.

Regards,

Saket