java in memory operations vs Redis vs file

[Raghava Mutharaju]

Hello all,

I was curious to know the speed difference when reading/writing items using Java data structures, Redis and regular files. So I put 1 million items into java HashMap and read them. In Redis (using jedis), I used set and get operations with pipeline. The read and write code that I used is at  https://gist.github.com/raghavam/2be48a98cae31c418678

The results are as follows (all are in seconds)

Time taken to write (RAM): 0.183383171

Time taken to write (Redis): 4.688271688

Time taken to write (File): 0.218156046

Time taken to read (RAM): 0.086150164

Time taken to read (Redis): 3.719636865

Time taken to read (File): 2.30486171

The results were surprising to me. Since Redis is in-memory, I did not expect that much of a difference compared to java HashMap and secondly I never imagined that writing/reading to/from files is faster than Redis.

My questions are 

1) Is this a fair comparison w.r.t the code given?

2) If it is, then why is Redis slower than even file writing/reading?

I am Redis 2.8.0-rc4 and Jedis 2.5.1

Thank you.

Regards,

Raghava. 

[Itamar Haber]

My understanding is that the hashmap is managed by the same Java process whereas inter process communication (even if not over the network) is needed to communicate with Redis. I would also venture a guess that the file is cached by the OS to a large extent and that the comparative performance gains using it are also due to it being used in the context of the Java process. If that is indeed the case, it  shouldn't be surprising that accessing resources that are local to the application's process is more performant than communicating with Redis over the network/socket. BTW, on a somewhat similar note, perhaps you should check this un-scientific recent benchmark as well: http://charlesleifer.com/blog/completely-un-scientific-benchmarks-of-some-embedded-databases-with-python.

That said, Redis offers a lot more functionality than a flat file or a hashmap. On top of that, while some applications can get by with only their local resources, if your app will need scaling/distribution you'd be better off using a remote dictionary server.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--

Itamar Haber | Chief Developers Advocate
Redis Labs - Enterprise-Class Redis for Developers

Mobile: +1 (415) 688 2443
Mobile (IL): +972 (54) 567 9692
Email: ita...@redislabs.com
Skype: itamar.haber

Blog  |  Twitter  |  LinkedIn

[Raghava Mutharaju]

Hello Itamar,

Thank you for the link and the explanation. It is true that Redis offers more functionality and is most likely be more performant for random access of keys. However, I was just curious to see how the performance is for some simple operations.

Regards,

Raghava.

[Greg Andrews]

Or if the data needs to be shared among different processes.  For example, some processes write data and others read the data.  Especially when the processes are on different servers.

[Tahseen Jamal]

You are comparing a basic hash map with Redis

Have you thought of various aspect, which if you take care of would substantially degrade a Java HashMap compared to Redis

Right now you are just writing to Java HashMap, where as Redis also functions such to take care of Concurrent Write/Read. Because it takes care of that part irrespective of situation, whereas your program in Java doesn't, that is the reason Java looks faster

I too thought so when I wrote Map in Golang. Only to realise later when I had to write complete Mini-Redis, is when I understood there is so many to take care of as a general practice in case of concurrent read / write requests and that then you find that Redis is faster that your code