Point redis memory to ramdisk/swap

[antituhan]

I have been thinking about changes physical memory to SSD (being ramdisk/swap). Is it possible?

[Salvatore Sanfilippo]

Works very poorly as SSD disks are many orders of magnitudes slower
than RAM and Redis is relies on a completely random access model, so
performances will suck hard (unusable more or less).

SSD is more fast disk than slow memory currently, both speed-wise and
semantically.

Salvatore

On Wed, Sep 4, 2013 at 12:20 PM, antituhan <dewan...@xtremenitro.org> wrote:
> I have been thinking about changes physical memory to SSD (being
> ramdisk/swap). Is it possible?
>

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Salvatore 'antirez' Sanfilippo
open source developer - GoPivotal
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter

[Greg Andrews]

Comparing memory, SSD, and HD is difficult, mostly because the time needed to retrieve data from HD is variable.  If the HD heads need to move a distance across the surface of the disk in order to read the data, the time can be relatively long. If they need to move only a short distance, the time is less.  If they don't need to move at all, the time is even less.

Let's take the best-case HD average rotational latency time from this Wikipedia page of 2 milliseconds.  And the typical SSD random access time from this page of 100 microseconds.  And for RAM a time of 5 nanoseconds from pages like this one.  These numbers are so small that they're hard for people to compare them.  What I like to do is scale the shortest one up to a time period that people understand easily, and scale the others by the same amount.

One second is about the time it takes for a software engineer to reach out and pick up a cup of tea (or can of soda) from the desk and bring it to his/her mouth for a sip.  I.e., retrieving or "accessing" the drink.  If accessing RAM is the equivalent of taking that sip of tea, what are SSD and HD accesses like?  I'm going to use a monospace font below so the numbers line up visually:

5 ns is   0.000 000 005 seconds

100 us is 0.000 1       seconds

2 ms is   0.002         seconds

Scale them up so the RAM speed is 1 second:

0.000 000 005  X  200 000 000  =        1.0 second

0.000 1        X  200 000 000  =   20 000.0 seconds

0.002          X  200 000 000  =  400 000.0 seconds

So if retrieving data from RAM is like picking up a cup of tea from your desk, then retrieving it from SSD is like driving 5.5 hours round trip to fetch your cup of tea.  Getting it from HD is a holiday journey of 4.5 days.

As you can see, SSD is a great improvement on HD.  However, it's still nowhere near as fast as RAM.  You could even make the RAM speed 10x worse and the relative time for accessing that cup of tea is still 1 second vs. 33 minutes.

  -Greg

On Wed, Sep 4, 2013 at 3:20 AM, antituhan <dewan...@xtremenitro.org> wrote:

I have been thinking about changes physical memory to SSD (being ramdisk/swap). Is it possible?

--

[antituhan]

Hello, thanks for your reply,

I've kind of this SSD disk https://wiki.archlinux.org/index.php/SSD_Benchmarking#SAMSUNG_830_128GB

From that benchmark, could you calculate how fast is it than RAM?

[antituhan]

Hi Greg,

So clear to me, but is it possible to make real possible benchmark and result? I will post it to this forum if any best method outside swap partition could point the redis to the SSD. From the SAMSUNG benchmark, I have value more than that, 3x or more faster than the benchmark.

[Didier Spezia]

>> From that benchmark, could you calculate how fast is it than RAM?

No, because dd generates only sequential operations whose access pattern

is completely different than what Redis would do. Generally speaking, it is a

bad idea to extrapolate the result of a benchmark to guestimate the result of a

different benchmark.

If you want order of magnitude ideas, GregA's answer is great.

(nice "cup of tea retrieval" analogy, btw).

It you want figures, build you own benchmark with Redis, with your own

workload, and you own hardware.

Regards,

Didier.

[Greg Andrews]

On Wed, Sep 4, 2013 at 12:01 PM, antituhan <dewan...@xtremenitro.org> wrote:

So clear to me, but is it possible to make real possible benchmark and result?

You're asking about the opposite of a ramdisk.  A ramdisk makes RAM look like it's a disk block device.  You would like to make an SSD (a disk block device) look like RAM.  And then test Redis performance when it's using that SSD as RAM.

I suspect that can be done with the Linux kernel, but I don't know how to do it.  The numbers I posted suggest it would be many thousands of times slower than RAM, and that would make it useless for most use cases in today's world.

  -Greg

[antituhan]

Pada Jumat, 06 September 2013 23:48:20 UTC+7, GregA menulis:

I suspect that can be done with the Linux kernel, but I don't know how to do it.  The numbers I posted suggest it would be many thousands of times slower than RAM, and that would make it useless for most use cases in today's world.

  -Greg

Yes right! But is there any aditional configuration to point out redis on disk / ssd ? (eg. tmpfs_path /dev/shm or something else) IMHO

[Aaron Meriwether]

If you use Linux, you can try creating a swap partition on your SSD device and then use "cgroups" to limit the physical memory Redis is allowed to use, forcing most of it into virtual memory.  See here:

http://stackoverflow.com/questions/6882661/limit-physical-memory-per-process

There is also an experimental fork of Redis that attempts to store data on disk natively rather than in RAM, which you can read about here:

https://groups.google.com/d/msg/redis-db/ui4hEVpDxS4/bNnk5CNDu1gJ

You also may want to think about what your actual goals are - there may be better way to reach them:

Are you just curious about Redis architecture?

Do you have a dataset too large to fit in memory?

Are you concerned about losing your data if the box goes down suddenly?

-Aaron