Need Help --- Hazelcast settings for performance tuning in Testing Environment and few basic concepts

rafi md

unread,

Jan 30, 2014, 2:09:28 AM1/30/14

to haze...@googlegroups.com

Hi to All,

I am Rafi, and I am new to this Hazelcast.

I have Read Hazelcast documents and saw the Tutorilas of Hazelcast. Every thing about Hazelcast, is Quite Interesting and Really it has best Features.

Still, we have Many Questions in my mind.

Please don't be mind about my curiosity, since I am new and has no Proper Knowledge about the Hazelcast. Thanks to All.

Please Any One Can Help me to know more about Hazelcast.

1. Please Explain us more about, is the data created by the spring-hazelcast instance, will access by the same spring-hazelcast instance only or it can be access from other instances.[like, instances under the same cluster but started in command console using run.bat]

If we start 2 Hazelcast Instances, Instance A is from console,[using 'run.bat'] and other instance B is from Java 'Spring-Hazelcst' Instance.

Now, From "mancenter Admin console", we can view both Hazelcast Instances are Getting Combined and sharing documents each other.

But,

we need to know, is there any commands to get/put entries in user-defined maps | Queue[which are created using spring-hazelcast]

{we mean, we are not able to access non-default map | queue, but we can access default map | queue using m.get | m.put from console of instance A}.

2. we have 2 Testing Servers with the same configuration, Consider the following configuration settings,

RAM: 40gb

CPU Core: 8cores

OS: Red Hat Linux 64-bit

and

we need to handle tens of Millions of Documents using Hazelcast.

Now, we need to create testing environment, please help us to make this environment so that we can get optimum results.

a. how many Hazelcast Instances[nodes] to start for each server to Perfectly handle this big amount of data.

b. how many replicas to maintain to survive form node failures.

c. how to set 30-50% data should be there in hazelcast instances and reaming data should be store in database[here we are using, mongodb] automatically.

We need the spring-hazelcast settings for above environment.

3.Please any one can explain, We are using spring[mvc 3]-hazelcast API.

we are using spring controller to start hazelcast instance and one more controller to handle data insertion/retrial to/from hazelcast.

But

Some strange things are happening,

If we run our project, when spring container loading itself only 2 hazelcast instance are getting created and this instance is not active in mancenter admin console. Why this is happening?

Here, what we are expecting is, hazelcast instance should start when request is raised to controller and this instance should be active in mancenter admin console until the instance getting killed.

4. Please check the following code and please modify if any thing is wrong. and explain what is exactly.

I have one hazlecast-settings.xml file as following:

....................................................................................................................................................................................................................................

<hz:hazelcast id="instance" >

<hz:config>

<hz:group name="dev" password="dev-pass"/>

<hz:management-center enabled="true" url="http://localhost:8088/mancenter"/>

<hz:network port="5701" port-auto-increment="true">

<hz:join>

<hz:multicast enabled="true" multicast-group="224.2.2.3" multicast-port="54327" />

<hz:tcp-ip enabled="true" >

<hz:members >127.0.0.1</hz:members>

</hz:tcp-ip>

</hz:join>

<hz:interfaces enabled="true" >

<hz:interface>10.10.1.*</hz:interface>

</hz:interfaces>

</hz:network>

</hz:config>

</hz:hazelcast>

<hz:map instance-ref="instance" id="usermap" name="test" />

[please explain why and when we need "hz:multicast" or "hz:tcp-ip" or "hz:interface"

And our ip-address is like: 192.168.*.*

please tell us, is this ip address placed under in "hz:multicast" or "hz:tcp-ip" or "hz:interface" or not? ]

....................................................................................................................................................................................................................................

In Controller, I have the following:

Config hazleConfig = new ClasspathXmlConfig("hazlecast-settings.xml");

HazelcastInstance instance = Hazelcast.newHazelcastInstance(hazleConfig);

Map<Integer, String> mapCustomers = instance.getMap("customers");

mapCustomers.put(1, "Joe");

mapCustomers.put(2, "Ali");

mapCustomers.put(3, "jasmin");

System.out.println("Map Size:" + mapCustomers.size());

Your Reply/Suggestions/help is really helpful for us.

Thanks to All

Regards,

G Md Rafi.

Peter Veentjer

unread,

Jan 30, 2014, 2:18:10 AM1/30/14

to haze...@googlegroups.com

On Thu, Jan 30, 2014 at 9:09 AM, rafi md <me.mr...@gmail.com> wrote:

Hi to All,

I am Rafi, and I am new to this Hazelcast.
I have Read Hazelcast documents and saw the Tutorilas of Hazelcast. Every thing about Hazelcast, is Quite Interesting and Really it has best Features.

Still, we have Many Questions in my mind.
Please don't be mind about my curiosity, since I am new and has no Proper Knowledge about the Hazelcast. Thanks to All.
Please Any One Can Help me to know more about Hazelcast.

1. Please Explain us more about, is the data created by the spring-hazelcast instance, will access by the same spring-hazelcast instance only or it can be access from other instances.[like, instances under the same cluster but started in command console using run.bat]

Data is shared between all Hazelcast instances that form a single cluster. So you could put data in a map from a Spring initiated Hazelcast instance, and you could read it from a non spring one for example.

If we start 2 Hazelcast Instances, Instance A is from console,[using 'run.bat'] and other instance B is from Java 'Spring-Hazelcst' Instance.

Now, From "mancenter Admin console", we can view both Hazelcast Instances are Getting Combined and sharing documents each other.
But,
we need to know, is there any commands to get/put entries in user-defined maps | Queue[which are created using spring-hazelcast]

{we mean, we are not able to access non-default map | queue, but we can access default map | queue using m.get | m.put from console of instance A}.

You should not use the console. It is just a toy without any real value. Imho we should remove it from the core.

2. we have 2 Testing Servers with the same configuration, Consider the following configuration settings,
RAM: 40gb
CPU Core: 8cores

OS: Red Hat Linux 64-bit
and
we need to handle tens of Millions of Documents using Hazelcast.
Now, we need to create testing environment, please help us to make this environment so that we can get optimum results.

a. how many Hazelcast Instances[nodes] to start for each server to Perfectly handle this big amount of data.
b. how many replicas to maintain to survive form node failures.
c. how to set 30-50% data should be there in hazelcast instances and reaming data should be store in database[here we are using, mongodb] automatically.

This is very environment specific and we do not have a one fits all answer for that.

Can you tell a bit more about your documents, how big are they?

We need the spring-hazelcast settings for above environment.

3.Please any one can explain, We are using spring[mvc 3]-hazelcast API.
we are using spring controller to start hazelcast instance and one more controller to handle data insertion/retrial to/from hazelcast.

But
Some strange things are happening,
If we run our project, when spring container loading itself only 2 hazelcast instance are getting created and this instance is not active in mancenter admin console. Why this is happening?

Can you post the logging. I guess that one Hazelcast instance is created without your desired configuration, and something is happening that forced a default configured Hazelcast instance to be started.

Why are you creating 2 hazelcast instances?

Regards,

G Md Rafi.

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/66ae2ba9-df7a-428a-b92a-60e2a74041df%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Christoph Engelbert

unread,

Jan 30, 2014, 2:24:20 AM1/30/14

to haze...@googlegroups.com

see comments inlined

Am Donnerstag, 30. Januar 2014 08:09:28 UTC+1 schrieb rafi md:

Hi to All,

I am Rafi, and I am new to this Hazelcast.
I have Read Hazelcast documents and saw the Tutorilas of Hazelcast. Every thing about Hazelcast, is Quite Interesting and Really it has best Features.
Still, we have Many Questions in my mind.
Please don't be mind about my curiosity, since I am new and has no Proper Knowledge about the Hazelcast. Thanks to All.
Please Any One Can Help me to know more about Hazelcast.

1. Please Explain us more about, is the data created by the spring-hazelcast instance, will access by the same spring-hazelcast instance only or it can be access from other instances.[like, instances under the same cluster but started in command console using run.bat]

If we start 2 Hazelcast Instances, Instance A is from console,[using 'run.bat'] and other instance B is from Java 'Spring-Hazelcst' Instance.
Now, From "mancenter Admin console", we can view both Hazelcast Instances are Getting Combined and sharing documents each other.
But,
we need to know, is there any commands to get/put entries in user-defined maps | Queue[which are created using spring-hazelcast]
{we mean, we are not able to access non-default map | queue, but we can access default map | queue using m.get | m.put from console of instance A}.

Yes you can do that, Hazelcast instances are the same no matter how they are created. You can access / write / update values from all nodes that are part of the SAME cluster (means they are connected and sharing the same group name and password).

2. we have 2 Testing Servers with the same configuration, Consider the following configuration settings,
RAM: 40gb
CPU Core: 8cores
OS: Red Hat Linux 64-bit
and
we need to handle tens of Millions of Documents using Hazelcast.
Now, we need to create testing environment, please help us to make this environment so that we can get optimum results.
a. how many Hazelcast Instances[nodes] to start for each server to Perfectly handle this big amount of data.

It depends mostly on the data size. You have multiple options to handle a bigger amount of data. Either you start multiple nodes on the same machine and making the heap area smaller to prevent big GC events (but in exchange for a bit most latency since there is more socket traffic) or you might want to have a look at Elastic Memory (one of our commercial extensions) which will offer off heap support for Hazelcast. That way your data is stored offside the heap and you are giving a lot less pressure on the GC this way.

b. how many replicas to maintain to survive form node failures.

That mostly depends on the number of nodes you want to be able to lose. If you have a three node cluster and you want to be able to lose at most one node one replica is fine, if you want to be able to lose two nodes you need a replica number of two. Remember replicas needing memory, too :)

c. how to set 30-50% data should be there in hazelcast instances and reaming data should be store in database[here we are using, mongodb] automatically.

You can set eviction policy to 50% memory usage (or a number of entries) and data is evicted to given MapStore implementation.

We need the spring-hazelcast settings for above environment.

3.Please any one can explain, We are using spring[mvc 3]-hazelcast API.
we are using spring controller to start hazelcast instance and one more controller to handle data insertion/retrial to/from hazelcast.
But
Some strange things are happening,
If we run our project, when spring container loading itself only 2 hazelcast instance are getting created and this instance is not active in mancenter admin console. Why this is happening?
Here, what we are expecting is, hazelcast instance should start when request is raised to controller and this instance should be active in mancenter admin console until the instance getting killed.

Please provide your spring config and the controller class you're injecting the hazelcast instance to.

4. Please check the following code and please modify if any thing is wrong. and explain what is exactly.

I have one hazlecast-settings.xml file as following:
....................................................................................................................................................................................................................................
<hz:hazelcast id="instance" >
<hz:config>
<hz:group name="dev" password="dev-pass"/>
<hz:management-center enabled="true" url="http://localhost:8088/mancenter"/>
<hz:network port="5701" port-auto-increment="true">
<hz:join>
<hz:multicast enabled="true" multicast-group="224.2.2.3" multicast-port="54327" /> 
<hz:tcp-ip enabled="true" >
<hz:members >127.0.0.1</hz:members> 
</hz:tcp-ip>
</hz:join>
<hz:interfaces enabled="true" >
<hz:interface>10.10.1.*</hz:interface> 
</hz:interfaces>
</hz:network>
</hz:config>
</hz:hazelcast>
<hz:map instance-ref="instance" id="usermap" name="test" />

[please explain why and when we need "hz:multicast" or "hz:tcp-ip" or "hz:interface"
And our ip-address is like: 192.168.*.*
please tell us, is this ip address placed under in "hz:multicast" or "hz:tcp-ip" or "hz:interface" or not? ]

Multicast is automatic discovery of available nodes so you don't need to use that, just disable it.

With tcp-ip configuration you can set a number of well known hosts, I guess you want to use that.

rafi md

unread,

Jan 30, 2014, 6:28:45 AM1/30/14

to haze...@googlegroups.com

Hi to All,

@Peter Veentjer

Thanks a lot for your Suggestions.

>>Can you tell a bit more about your documents, how big are they?

Our data is like,

we are maintaining users logs, where each record has around 30-50 fields and again each filed contain a long or date or String value.[here string may have max of 100 words]

we will get around 30 million documents per each day.

like this we have to maintain up to 3 months of data.

>>You should not use the console. It is just a toy without any real value.

If we start one hazelcast instance using spring-hazelcsat API, then if we want to insert/update/search the entries to/from this hazelcast data, from other nodes[either spring instznce or console], what is the procedure to follow?

that is, if we start one instance using spring, and then

if we want to add few more nodes to insert/search operations to/from this spring instance, how can we add one more hazelcast instance using spring-hazelcast API and with out spring-hazelcast API and how to access data?

>>Can you post the logging. I guess that one Hazelcast instance is created without your desired configuration, and something is happening that forced a default configured Hazelcast instance to be started.

>>Why are you creating 2 hazelcast instances?

I have attached the Hazelcast-Log file along with this post.

We have found one Technical Bug, which causes to start 2 instances and we have fixed it.

now, we are starting only one instance to do hazelcast indexing/retrieval operations. since our requirement also to start one instance only.

@Christoph Engelbert

Thanks a lot for your Suggestions.

>>Please provide your spring config and the controller class you're injecting the hazelcast instance to.

...............................................hazlecast-settings.xml..........................................................................

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"xmlns:hz="http://www.hazelcast.com/schema/spring"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

http://www.hazelcast.com/schema/spring http://www.hazelcast.com/schema/spring/hazelcast-spring-3.1.xsd">

<hz:hazelcast id="instance" >

<hz:config>

<hz:group name="dev" password="dev-pass"/>

<hz:management-center enabled="true" url="http://localhost:8088/mancenter"/>

<hz:network port="5701" port-auto-increment="true">

<hz:join>

<hz:multicast enabled="true" multicast-group="224.2.2.3" multicast-port="54327" />

<hz:tcp-ip enabled="true" >

<hz:members >127.0.0.1</hz:members>

</hz:tcp-ip>

</hz:join>

<hz:interfaces enabled="true" >

<hz:interface>10.10.1.*</hz:interface>

</hz:interfaces>

</hz:network>

</hz:config>

</hz:hazelcast>

<hz:map instance-ref="instance" id="usermap" name="test" />

.......................................................................................................................................................................

....................................................TestController2.java............................................................................

package test.Controller;

import java.util.Map;

import java.util.Queue;

import org.springframework.stereotype.Controller;

import org.springframework.ui.ModelMap;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.bind.annotation.RequestMethod;

import com.hazelcast.config.ClasspathXmlConfig;

import com.hazelcast.config.Config;

import com.hazelcast.core.Hazelcast;

import com.hazelcast.core.HazelcastInstance;

@Controller

@RequestMapping("/hazle")

public class TestController2 {

@RequestMapping(method = RequestMethod.GET)

public String printWelcome(ModelMap model) {

model.addAttribute("message", "In HazleCast API........!");

System.out.println("<---------hazelcast instance started---------------->");

Config hazleConfig = new ClasspathXmlConfig("hazlecast-settings.xml");

HazelcastInstance instance = Hazelcast.newHazelcastInstance(hazleConfig);

Map<Integer, String> mapCustomers = instance.getMap("customers");

Queue<String> queueCustomers = instance.getQueue("customers");

System.out.println("<---------storing data into hazelcast---------------->");

mapCustomers.put(1, "Joe");

mapCustomers.put(2, "Ali");

mapCustomers.put(3, "Avi");

System.out.println("Map Size:" + mapCustomers.size());

queueCustomers.offer("Tom");

queueCustomers.offer("Mary");

queueCustomers.offer("Jane");

System.out.println("Queue size: " + queueCustomers.size());

System.out.println("<---------storing data into hazelcast---------------->");

System.out.println("Customer with key 1: "+ mapCustomers.get(1));

System.out.println("Customer with key 1: "+ mapCustomers.get(2));

System.out.println("Customer with key 1: "+ mapCustomers.get(3));

System.out.println("First customer: " + queueCustomers.poll());

System.out.println("Queue size: " + queueCustomers.size());

System.out.println("Second customer: "+ queueCustomers.peek());

System.out.println("Queue size: " + queueCustomers.size());

return "Test";

}

.......................................................................................................................................................................

Please Help us how to handle this Spring-Hazelcast Instace Properly.

1. even we start one Hazelcast Insatnce we are not able to start mancenter admin console.

[It throwing "No Data Cluster Error Pop-up Box, which asks web url] we have attached this pop-up error alert dialog box also. Please refer the attachments.

When we start the hazlecast instance form console by server.bat or run.bat then only the mancenter admin console is opening.

Because of this, we first started one instance form console by server.bat and then we started one more instance using spring-hazelcast api.

2. If we start insatnce from spring-hazelcsat, that instance is not active in mancenter admin console. but data is there.[check all the images we have attached with this post].

Please explain what is the mistake we have done in our configuration file and in class file.

please share the configuration settings for hazelcast and java logic for start hazelcast instance.

so that with this single instance only we need to open mancenter admin console and this instance should be in active state untill we kill the instance.

Thanks to All.

Regards,

G Md Rafi.

hazelcast-instance-status.png

map-data-after-killing-node.png

map-data-before-killing-node.png

Hazlecast-Log.txt

hazelcast-singlenode-exception.png

rafi md

unread,

Feb 2, 2014, 11:46:19 PM2/2/14

to haze...@googlegroups.com

Hi to All,

Please any one can tell us, How exactly we need to Handle Hazelcast Instance.

Here is our Situation,

we Have started 10 Hazelcast Instance using Spring-Hazelcast API, As of now, Every thing is Fine.

The Data Collections are Available with those Spring-Hazelcast Instance only.

But, We Have to Access These Data From Out of the Spring-hazelcast Instance for Some Admin Purpose, So we Started One More Hazelcast Insatce from Console using, server.bat.

Even Data is Distributed with this newly Created Node[which is instantiated with server.bat], Please Explain us, How can we Access the data with this newly created node.

Please any one can tell, how to access Spring-hazelcast Instance data, from other nodes.

And Also, Please Help us how to handle this Spring-Hazelcast Instace Properly.

1. even we start one Hazelcast Insatnce we are not able to start mancenter admin console.

[It throwing "No Data Cluster Error Pop-up Box, which asks web url] we have attached this pop-up error alert dialog box in last post. Please refer those attachments.

2. If we start insatnce from spring-hazelcsat, that instance is not active in mancenter admin console. but data is there.

Are our Situation is Correct? If We are Wrong also, Please Let us Know, Please Any One Help us to solve these Problems.

Thanks to All.

Regards,

Rafi.

Peter Veentjer

unread,

Feb 3, 2014, 1:11:09 AM2/3/14

to haze...@googlegroups.com

On Mon, Feb 3, 2014 at 5:46 AM, rafi md <me.mr...@gmail.com> wrote:

Hi to All,

Please any one can tell us, How exactly we need to Handle Hazelcast Instance.

Here is our Situation,

we Have started 10 Hazelcast Instance using Spring-Hazelcast API, As of now, Every thing is Fine.
The Data Collections are Available with those Spring-Hazelcast Instance only.

But, We Have to Access These Data From Out of the Spring-hazelcast Instance for Some Admin Purpose, So we Started One More Hazelcast Insatce from Console using, server.bat.

Why don't you:

- start a client. A member has all kinds of responsibilities like hosting data, or executing logic (e.g executor). So if you only need to read out some management data, a client is a better alternative

- use JMX: Hazelcast provides a lot of jmx beans, so all kinds of data can also be read using JMX.

I would certainly reconsider using a real cluster member to read out management data.

Even Data is Distributed with this newly Created Node[which is instantiated with server.bat], Please Explain us, How can we Access the data with this newly created node.

Imho the server.bat and the other shell scripts should be dropped. I have not seen anying using them in production. You are imho better of writing your own 'main' where you have full control on what happens + which classes are included in your application.

Please any one can tell, how to access Spring-hazelcast Instance data, from other nodes.

And Also, Please Help us how to handle this Spring-Hazelcast Instace Properly.
1. even we start one Hazelcast Insatnce we are not able to start mancenter admin console.

[It throwing "No Data Cluster Error Pop-up Box, which asks web url] we have attached this pop-up error alert dialog box in last post. Please refer those attachments.

Can you post the spring configuration.

2. If we start insatnce from spring-hazelcsat, that instance is not active in mancenter admin console. but data is there.

I guess there is some kind of configuration issue.

Are our Situation is Correct? If We are Wrong also, Please Let us Know, Please Any One Help us to solve these Problems.

Post your configuration + get rid of our server.bat.

Thanks to All.

Regards,
Rafi.

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/38388bbf-b215-4fc7-a4b3-71e564d417bb%40googlegroups.com.

rafi md

unread,

Feb 5, 2014, 6:20:50 AM2/5/14

to haze...@googlegroups.com

Hi to All,

@peter veentjer

Thanks a lot for your Valuable Suggestion.

>>I guess there is some kind of configuration issue.

>>Post your configuration + get rid of our server.bat.

Now, We got some Idea About Hazelcast. please consider the following configuration files.

Our Application is Developed by using Spring MVC 3 and following is the overall architecture to be Noticed to Your Point :

we are trying to integrate Hazelcast with MongoDB in Spring.

Project Root Class path :

|

|------src-[This Directory Contains, All Controller, Model, DTO and Services Classes {under the Packages <test.hazel.mongo.*>}]

| |

| ---hazelcast-config.xml

|

|--WebContent-[ META-INF and Views]

| |

| ---WEB-INF-

| |

| ---lib-[all jar files]

---mvc-dispatcher-servlet.xml

---web.xml

And also,

.................................................hazelcast-config.xml....................................................................

<?xml version="1.0" encoding="UTF-8"?>

<context:component-scan base-package="test.hazel.mongo.*" />

<context:annotation-config />

<hz:hazelcast id="hazelcastInstance" lazy-init="true" > [line no: 5 - 18]

<hz:config>

<hz:group name="dev" password="dev-pass"/>

<hz:management-center enabled="true" url="http://localhost:8088/mancenter"/>

<hz:network port="5701" port-auto-increment="true">

<hz:join>

<hz:multicast enabled="true" multicast-group="224.2.2.3" multicast-port="54327" />

</hz:join>

</hz:network>

<hz:map name="customer">

<hz:map-store enabled="true" implementation="mongomapstore" write-delay-seconds="0" />

</hz:map>

</hz:config>

</hz:hazelcast>

<hz:map instance-ref="hazelcastInstance" id="customer" name="customer" /> [line no: 19] <-----*

<bean id="mongomapstore" class="com.hazelcast.spring.mongodb.MongoMapStore" > [line no: 21 - 23]

</bean>

<mongo:mongo id="mongoRef" host="localhost" port="27017" />

<mongo:db-factory id="mongoDbFactory" dbname="test" mongo-ref="mongoRef" />

<constructor-arg name="typeKey">

</constructor-arg>

</bean>

<constructor-arg name="mongoDbFactory" ref="mongoDbFactory" />

<constructor-arg name="mappingContext" ref="mongoMappingContext" />

</bean>

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> [line no: 37-41]

<constructor-arg name="mongoDbFactory" ref="mongoDbFactory"/>

<constructor-arg name="mongoConverter" ref="mongoConverter"/>

</bean>

/beans>

.....................................................................................................................

.................................................mvc-dispatcher-servlet.xml....................................................................

<?xml version="1.0" encoding="UTF-8"?>

<context:component-scan base-package="test.hazel.mongo.*" />

<context:annotation-config />

<property name="prefix"><value>/Views/</value></property>

</bean>

</beans>

.....................................................................................................................

.................................................web.xml....................................................................

<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" version="3.0">

<display-name>Testing</display-name>

<servlet-name>mvc-dispatcher</servlet-name>

<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>

<load-on-startup>1</load-on-startup>

</servlet>

<servlet-mapping>

<servlet-name>mvc-dispatcher</servlet-name>

<url-pattern>/</url-pattern>

</servlet-mapping>

<context-param>

<param-name>contextConfigLocation</param-name>

<param-value>

/WEB-INF/mvc-dispatcher-servlet.xml

classpath:hazelcast-config.xml

</param-value>

</context-param>

<listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener>

<welcome-file-list> <welcome-file>/Views/Test.jsp</welcome-file> </welcome-file-list>

</web-app>

.................................................................................................................................

..........................................Controller Class.......................................................................................

@org.springframework.stereotype.Controller

@RequestMapping("/hazel")

public class Controller {

@Autowired

UserDAO userDAO;

@RequestMapping(value="/save", method = RequestMethod.GET)

public String printWelcome(ModelMap model) {

userDAO.save(HazelcastFactory.hazelcastInstance);

userDAO.view(HazelcastFactory.hazelcastInstance);

model.addAttribute("message", "In HazleCast........!");

return "Test";

}

.................................................................................................................................

.......................................................UserDAOImpl class ..........................................................................

public class UserDAOImpl implements UserDAO {

@Override

public void save(HazelcastInstance hazelcastInstance) {

Map<Integer, String> mapCustomers = hazelcastInstance.getMap("customer");

Queue<String> queueCustomers = hazelcastInstance.getQueue("customer");

mapCustomers.put(1, "jan");

mapCustomers.put(2, "feb");

mapCustomers.put(3, "mar");

mapCustomers.put(4, "apr");

}

@Override

public void view(HazelcastInstance hazelcastInstance) {

Map<Integer, String> mapCustomers = hazelcastInstance.getMap("customer");

Queue<String> queueCustomers = hazelcastInstance.getQueue("customer");

System.out.println("Customer with key 1: "+ mapCustomers.get(1));

System.out.println("Customer with key 2: "+ mapCustomers.get(2));

System.out.println("Customer with key 3: "+ mapCustomers.get(3));

System.out.println("Customer with key 4: "+ mapCustomers.get(4));

}

.................................................................................................................................

...................................................HazelcastFactory class..............................................................................

public class HazelcastFactory {

public static Config hazelcastConfig = new ClasspathXmlConfig("hazelcast-config.xml");

public static HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(hazelcastConfig);

}

.................................................................................................................................

Please Check the Above Configuration files and class files And let us know, what the mistake we are doing here.

1. We are getting two hazelcast instances[first instance, is at spring context loading and second is when we call the controller].

This is not expected.

2. if we try to load data, in hazelcast all the documents are getting loaded but in mongodb few documents[above 25% of the data] are not loading.

3. If we remove the line number 19 in "hazelcast-config.xml" file

that is: <hz:map instance-ref="hazelcastInstance" id="customer" name="customer" />

only one instance is getting created but this time, in hazelcast all documents are loading, but in mongodb, no documnets are loading.

What we want is,

we need to start only one instance, and this should be use the configurations from the "hazelcast-config.xml" and should load all the data in both hazelcast and mongodb.

Please any one can Suggest us, how to solve our Problem.

rafi md

unread,

Feb 12, 2014, 12:46:27 AM2/12/14

to haze...@googlegroups.com

Hi to All,

Thanks for all your Supporting..! :)

Now, We have solved our issues which we posted in our last post.

here is the detailed information how we solved it.

Our requirement is to create one hazelcast instance and to store documents in mongodb along with hazelcast.

issue was: when we start to execute our application, we got 2 hazelcast instances[one is active and another is inactive, but still all data is present in our hazelcat] and few data[above 25%] are missing in mongodb.

our procedure to fix the above issue:

we have used "singleton" Design Pattern to initiate our hazelcast instance in our Spring MVC application. Now, we are getting only one hazelcast instance[perhaps, we can change the design pattern to get more hazelcast instances, if we need], and this instance active in hazelcast mancenter[admin console]. Now, all the data present in both hazelcast and mongodb.

And we did POC activities, but we got Strange Results, which was never expected.

when compare to local system[laptop] with test box[here, test box has more RAM], local system giving the best performance.

let us explain, what we got.

POC: to test performance of loading and querying the customer bean objects and also storing the same objects in mongodb using map-store.

#on one single laptop[ram: 8gb]:

Case 1:

with one hazelcast insatnce:

for loading 1 million objects it took approximately, 100sec

for querying all 1 million objects it took approximately, 10sec

Case 2:

with two hazelcast instances:

for loading 1 million objects it took approximately, 161sec

for querying all 1 million objects it took approximately, 58sec

#on test box: [192.168.xxx.xxx][with 14gb ram]:

Case 3:

with one hazelcast insatnce:

for loading 1 million objects it took approximately, 660sec

for querying all 1 million objects it took approximately, 42sec

Case 4:

with two hazelcast instances:

for loading 1 million objects it took approximately, 1091sec

for querying all 1 million objects it took approximately, 241sec

test box giving poor performance when compare with local system where local system has low configuration.

Why this is happening?

Please help us,

how to set a good hazelcast environment, so that we can get good performance with minimum data loss.

We know, that hazelcast is working based on the in-momery data grid,

which means all the data is present in the RAM and not storing any where in HDD.

so if in case of any node failure, we need to protect the data from lost, so we need to have multiple hazelcast instances with minmun backup count 1.

But from our POC results, what we get is: If we increase hazelcast instances then the performance for indexing/querying is decreasing.

Is this Correct?

Please suggest us,

1. how many hazelcast instances we need to initiate. [suppose, we have 2 test boxes, each has 14gb ram, 400gb hdd, 8core processor And we need to handle tens of millions of bean objects]

2. how to increase the hazelcast instance/decrease heap memory.[by default we are getting 2gb max memory for each hazelcast instance].

3. is there any Explicit Routing mechanism, so that we can explicitly say hazelcast to store a particular bean object into a particular partition.

[since hazelcast itself is doing this routing for us, but we need to use our own algorithm to send a particular object to a particular partition, so that we can get more performance] And how to change the default partition number.

and

4. If Possible, Please share performance tuning related guidance for hazelcast-mongodb integration with spring mvc.

Thanks to All.

regards,

G Rafi.

Reply all

Reply to author

Forward