What is the best way to benchmark Hazelcast's speed to data retrieval, comparing with direct DB calls / RESTful service calls?

178 views
Skip to first unread message

spooky...@gmail.com

unread,
Aug 23, 2020, 5:29:27 AM8/23/20
to Hazelcast

Background: My team has been maintaining a mix of classic ASP/ASP.NET apps for over a decade. My company recently had a re-org in which teams and managers were either laid off or swapped around. My new manager is pushing for a refactoring of our entire app portfolio over the coming year or two to Java/Spring-boot to meet company infrastructural standards. Most of our apps today perform ETL tasks via sprocs from MSSQL or Oracle SQL databases, with a few connecting to APIs via SOAP and REST respectively.


Goal: As part of this rewrite, I've been asked to help make a decision about how we handle data retrieval for our apps and to determine if there are any opportunities for improvement. I want to try out Hazelcast (server) and "prove" it out-performs traditional data-retrieval methods like direct DB / REST calls.


Question: What would be the best approach for achieving my goal (benchmarking Hazelcast)? My initial idea is creating a proof of concept with Java & Spring to test a.) the average speed of a direct DB call, b.) the average speed of a RESTful API call, and c.) the average speed of retrieval from Hazelcast. I would make sure the record set is consistent across all three tests. Is this the right approach, or am I over/under complicating it? Is there already something built out I can use to demonstrate the same?


(Note, I understand Hazelcast is not meant to replace the original source of record, but provide data caching for quicker retrieval of subsequent calls. I want proof that it is indeed quicker to retrieve from the cache rather than just doing another DB/API call).

Sharath Sahadevan

unread,
Aug 24, 2020, 12:37:35 PM8/24/20
to Hazelcast
Hi ,
   Your proof of concept approach looks good to me.
You can download Hazelcast IMDG here and Jet here.

Here are a few links for samples/guides:


If you need additional support during the POC a solution architect can get on a call with you to educate you and your team about the capabilities of Hazelcast.

There is also the Hazelcast simulator you can use for benchmarking that you can download here.

You can also find a number of our published benchmarks here.

Let us know if you have any further questions.

Thanks,
Sharath

Neil Stevenson

unread,
Aug 26, 2020, 3:59:23 AM8/26/20
to Hazelcast
 If you're using "MapLoader", so the Hazelcast cluster fetches data from the database on demand, be sure to use the "MapLoader.getAllKeys()" or a similar way to bring *all* data you'll use into memory before you start measuring.

 If you get a cache-miss in your test when using a MapLoader, Hazelcast will try to find the data in the database. So the run
time for this operation will be (Hazelcast time + database time) which will obviously be higher than the time for the database
alone, and give a very misleading result.

Neil

Reply all
Reply to author
Forward
0 new messages