On switching default persistence configuration

[Salvatore Sanfilippo]

Hello all!

A few discussions on Twitter, not of the best technical level
unfortunately, raised what can be a legitimate problem, that is the
persistence default of Redis.

We currently provide both a redis.conf, and a default configuration
(when Redis is running without redis.conf at all), that are configured
with RDB persistence with a few save points presets. This means, for
example, that hitting Ctrl+C after some minutes of hacking may results
into data loss. In general what may happen is that newcomers may not
understand that the default is not tuned for durability. One may say,
well, who runs a database without understanding the tradeoffs and
carefully checking the docs? I hope a few, but maybe the default
config is a "selection bias" big enough that a lot of people use RDB
just because it is the default setting.

Because since 2.4 we have automatic AOF rewrites, so that AOF is not
less "automatic" than RDB, maybe the solution can be to just change
the default to AOF with an fsync policy of everysec. After all it is a
setting that many people are using in production. I would also enable,
additionally, RDB by default, with a single save point every 3600
seconds, so that there is some kind of automatic backup in the default
configuration.

As part of the change the redis.conf file may be modified so that the
comments in the persistence section state the default configuration
settings, point to the redis.io documentation, and so forth.

I wonder what do you think about this. This is not something that's
going to affect most of you, because most people here use redis.conf
that are carefully tuned, but everybody in this list, myself included,
also has a perspective as an user and what a newbie may think and how
he may act.

Is it time for a change?

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter

[Pedro Melo]

+1.

I think AOF with fsync everysec is a better default.

Bye,

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>



--
Pedro Melo
@pedromelo
http://www.simplicidade.org/
http://about.me/melo
xmpp:me...@simplicidade.org
mailto:me...@simplicidade.org

[Dvir Volk]

On place you can take an example from is MySQL.

They have 3 or 4 boilerplate config files for different settings, you can choose from, that are more or less optimized (small db, big db, high memory server, etc)

The default set of configurations selection can be added as an option to the installation script as well.

It can also be included in separate config files where what you need to uncomment is which persistence.conf to include.

[Salvatore Sanfilippo]

Thanks Dvir, makes sense as well to have some dir with a number of
configuration templates, like lru_cache.conf, durability.conf,
... and so forth.

However we still need to pick a default because Redis can run without
a config as well, even in serious setups starting from 2.6, because
you can run it like:

redis-server --port 6380 --slaveof 127.0.0.1 6379 ... and so forth.

So the concept of "default" is one we can't escape! :)

Salvatore

[Dvir Volk]

yeah, I agree. I'm fine with the RDB option, I'm even using it in some production instances that do need need high durability. I like RDB's simplicity and the fact that it never breaks if you do strange stuff like kill -9 your instance, as opposed to AOF. so as a debugging/dev option it's much better IMHO.

I'm guessing the default will not be used in production so it is enough to make the different patterns more accessible and documented. 

[Dvir Volk]

anther idea: you may want to add a very clear warning in the log if persistence is not configured or not configured for durability, just like you do for the vm overcommit.

[Didier Spezia]

Hi,

I would rather favor processing SIGINT just like SIGTERM (i.e. triggering a RDB save).

I think it is a bit too easy to hit CTRL-C while playing with Redis, and realize a

few seconds later that the last operations were meant to be kept.

After all, if people want to terminate Redis instantly without any dump,

they can still use SIGKILL.

AOF with everysec fsync policy is a sensible default IMO.

Best regards,

Didier.

[Matt Keranen]

On Wednesday, August 1, 2012 5:26:37 AM UTC-4, Salvatore Sanfilippo wrote:

One may say, well, who runs a database without understanding the tradeoffs and carefully checking the docs? 

Unfortunately, MongoDB takes a regular amount of criticism for defaulting to performance over durability, because people implement it without reading the docs.

The result is more of an image problem than a technical one, but image is unfortunately an important concept if adoption is a concern.

Correct use is also a concern, and if this change means it prevents some people from shooting themselves in the foot, it may be worth considering.

[Josiah Carlson]

+1 on Didier's point, and +1 on Dvir's point.

Overall, +1 on a change in default default behavior to AOF + dump.

- Josiah

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/EdFIljiF9toJ.

[Jokea]

+1 with changes in redis.conf.

However, I'd like the default config changed to no persistence at all, since
I do most debug/test by simply start redis-server without any parameter, and these
data are useless after the test(or can easily be recreated). Take performance test
for example, if the default config is to enable AOF, then the disk status will have
an impact on the final result.

If anyone wants to store some serious data in redis, he shouldn't/wouldn't just fire up
redis-server without a config file(it's even not running as a daemon).


Regards,
Jokea

[Yiftach Shoolman]

There is one issue here:

If the storage system is by default slow, e.g. AWS EBS, AOF (fsync) can block Redis operation.

So for new users with small dataset that just want to test performance, Redis might look slow, and I'm not sure this is the impression we want to give them.

  

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.

To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

--
Yiftach Shoolman
+972-54-7634621

[Dvir Volk]

Excellent point, which is actually why we hardly use AOF in production.

The forking times are a bad enough issue with AWS, adding AOF to it would be worse.

But OTOH, who installs a server without at least checking their options?

The default when you don't have a config file, IMHO should be considered "debug setting", and as someone here mention, might not need to persist at all!

[Pierre Chapuis]

If the problem is that people do not read the documentation and use default choices, you should provide defaults that cannot possibly work in production (for people who want persistence) IMO, not defaults that have a chance to work in some contexts. No persistence at all by defaults looks good to me.

[Felix Gallo]

There can be no better way to confuse and irritate newcomers and inundate the mailing list with anguished cries than that.  Terrible idea.

+1 to AOF default and a warning message at startup. 

+2 to custom warning messages defined in the conf file, e.g.

warning  "This is a default configuration file!  Do not use this in production.  See the README for details"

warning "This configuration file is for cluster slaves in the test environment for product version 1.45."

F.

On Mon, Aug 6, 2012 at 4:36 AM, Pierre Chapuis <catwell...@catwell.info> wrote:

If the problem is that people do not read the documentation and use default choices, you should provide defaults that cannot possibly work in production (for people who want persistence) IMO, not defaults that have a chance to work in some contexts. No persistence at all by defaults looks good to me.

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.

To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/LfTgTqjIggUJ.