Redis changes my .conf file

[Dinesh Radhakrishnan]

I am starting a redis_server with a .conf file which has information in there saying "slave of x"

After running it for a while, the conf file gets completely overwritten. I can use that .conf file to start again. Is the expectation to keep a master copy and copy it to a temp copy and then restart redis_server with the temp.conf ? thanks.

[Michael Dillon]

If you're using sentinel, it can append configuration to the end of the conf file.  I wouldn't expect the conf to be changed otherwise, but I'm only new to the product.

I'm not sure about cluster mode.

[Jay Rolette]

On Wed, Jun 8, 2016 at 4:00 AM, Michael Dillon <dillon....@gmail.com> wrote:

If you're using sentinel, it can append configuration to the end of the conf file.  I wouldn't expect the conf to be changed otherwise, but I'm only new to the product.

It's an unfortunate design point that Sentinel ends up keeping it's state in the config file. It would be much better for managing updates if that data was broken out into a separate file.

Jay

 

I'm not sure about cluster mode.

On Tuesday, 7 June 2016 18:10:32 UTC+1, Dinesh Radhakrishnan wrote:

I am starting a redis_server with a .conf file which has information in there saying "slave of x"

After running it for a while, the conf file gets completely overwritten. I can use that .conf file to start again. Is the expectation to keep a master copy and copy it to a temp copy and then restart redis_server with the temp.conf ? thanks.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[The Real Bill]

The expectation is that you let Redis manage the configuration file. Redis only rewrites the file when its in-memory config has changed. For example when you run a "config set" command, or when sentinel triggers a failover the config needs to be updated. Redis will do this for you, or you can manually tell it do so. But in either case you don't want to try to manage the file outside of Redis.

[The Real Bill]

For sentinel itself, which is not what the OP posted about, there is very little sentinel configuration that isn't about the pods it is managing. The items you would configure sentinel for other than pods and their state are things you don't dynamically change such as where it listens, PID file location, etc.. Thus in practice it makes no difference as long as you don't try to use a configuration management system such as puppet to manage the config file. Doing this for Redis itself is likewise a bad idea.

Cheers,
Bill

[Jay Rolette]

On Fri, Jun 10, 2016 at 7:36 AM, The Real Bill <ucn...@gmail.com> wrote:

For sentinel itself, which is not what the OP posted about, there is very little sentinel configuration that isn't about the pods it is managing. The items you would configure sentinel for other than pods and their state are things you don't dynamically change such as where it listens, PID file location, etc.. Thus in practice it makes no difference as long as you don't try to use a configuration management system such as puppet to manage the config file. Doing this for Redis itself is likewise a bad idea.

The fact that you can't use a configuration management system (puppet, ansible, etc.) to manage Redis and Sentinel configuration is exactly the reason I had in mind when I said it was a bad design decision.

[Bill Anderson]

It isn't bad, just different. Redis is designed to be manageable directly. File based configuration management systems utterly fail on dynamic configurations such as Sentinel and Redis - they aren't designed to handle that scenario. It isn't the fault of Redis, but the mistake of trying to apply one paradigm's tools to a different paradigm. Just as you don't blame the diesel engine powered car for adding gasoline to it. 

Redis is simply one of a class of tools which is not file based for ongoing dynamic configuration - the file is only used for startup recoverability. Different paradigms exist to handle different scenarios. None of them is inherently bad. File based configuration management tools are intended to solve the problem of file based configurations. When a service doesn't utilize it, it isn't applicable. That doesn't make the design bad, simply different.

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/1JB7OkaaxZo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.

[Jay Rolette]

On Fri, Jun 10, 2016 at 9:53 AM, Bill Anderson <ucn...@gmail.com> wrote:

It isn't bad, just different. Redis is designed to be manageable directly. File based configuration management systems utterly fail on dynamic configurations such as Sentinel and Redis - they aren't designed to handle that scenario. It isn't the fault of Redis, but the mistake of trying to apply one paradigm's tools to a different paradigm. Just as you don't blame the diesel engine powered car for adding gasoline to it. 

Redis is simply one of a class of tools which is not file based for ongoing dynamic configuration - the file is only used for startup recoverability. Different paradigms exist to handle different scenarios. None of them is inherently bad. File based configuration management tools are intended to solve the problem of file based configurations. When a service doesn't utilize it, it isn't applicable. That doesn't make the design bad, simply different.

That's great except you can't actually set everything in Redis dynamically. Just for grins, I went through the first 5 config settings in the default redis.conf file and tried setting them dynamically. Only 1 of the 5 were settable:

127.0.0.1:6379> config set bind 127.0.0.1

(error) ERR Unsupported CONFIG parameter: bind

127.0.0.1:6379> config set protected-mode yes

OK

127.0.0.1:6379> config set port 6380

(error) ERR Unsupported CONFIG parameter: port

127.0.0.1:6379> config set tcp-backlog 128

(error) ERR Unsupported CONFIG parameter: tcp-backlog

127.0.0.1:6379> config set unixsocket /tmp/redis.sock

(error) ERR Unsupported CONFIG parameter: unixsocket 

So I can't use configuration file management tools because Redis and Sentinel rewrite their config files to store state data. I also can't just use scripts to modify configuration dynamically because it's not all settable dynamically.

Tell me again how this wasn't a bad idea?

On Fri, Jun 10, 2016 at 10:43 AM, Jay Rolette <rol...@infinite.io> wrote:

On Fri, Jun 10, 2016 at 7:36 AM, The Real Bill <ucn...@gmail.com> wrote:

For sentinel itself, which is not what the OP posted about, there is very little sentinel configuration that isn't about the pods it is managing. The items you would configure sentinel for other than pods and their state are things you don't dynamically change such as where it listens, PID file location, etc.. Thus in practice it makes no difference as long as you don't try to use a configuration management system such as puppet to manage the config file. Doing this for Redis itself is likewise a bad idea.

The fact that you can't use a configuration management system (puppet, ansible, etc.) to manage Redis and Sentinel configuration is exactly the reason I had in mind when I said it was a bad design decision.

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/1JB7OkaaxZo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

[Bill Anderson]

So in your desire to argue rather than understand you are now arguing that things which Redis can not change dynamically not allowing you to set them dynamically is bad? Redis lacks the ability to alter its port, address, or pidfile, for example. So why should it allow you to "change" them in the config?

If you really want to continuously change what address, port, or socket (or pidfile) via a file based configuration management system, then do it in your startup script. Redis will happily take starting configuration via the command line. It will then write that into the configure file when it writes one of course. But the command line will still take precedence. As you gain experience with dynamic systems you will learn how they operate and that they use a bootstrap process. For many, this is command line arguments or environment variables. Indeed, the of /etc/defaults is in part a means to handle this - dynamic systems aren't exactly a new thing. 

Every commonly used file based configuration management is entirely incapable of managing a dynamic system. None of them know how to do thing slick configure a Redis cluster or manage Redis replication. Nor should they. After all in reality they aren't configuration management but are actually file management systems. This is why they run into a problem when people try to use them to manage dynamic system - because any changes outside of them they don't understand and will blow away. As a file management system this makes sense. As a configuration management system, particularly for dynamic systems, this is a problem. More specifically it is a problem of applying an improper tool for the job. 

You don't get to claim a hammer is a bad screw management system and expect to be taken seriously. And this is exactly what you are doing. You are saying that because you can't use a file management system to manage configuration of a dynamic system it was never designed to do, the system is thus bad. Yet there is nothing wrong with the design - just you trying to apply the wrong tool.

Your lack of understanding of a thing is not the same thing as it being a bad design. From how you're responding you are stuck in a mindset that something you don't understand and that works outside of your comfort zone must be "bad". You have offered nothing other than "I don't like it so it is bad". Thus, this is my last response to you on this matter. Those of us who understand the model and have used it for thousands of complex deployments have had zero problem with it - even if/when we started with your assumption.

Personally, I'd prefer to have Redis support the use of a configuration backing store - where Redis bootstraps with the needed connection information to talk to a backing store such as Consul, Etcd, or Zookeeper (for example) and obtains/stores all configuration information there. But we aren't there yet. 

Cheers,

Bill

[shay rybak]

I faced the same issue and decided on using ansible lineinfile module to set the configurations I needed. that way. if it's a clean run it will create the file with the base config. and if I need to change something I'll replace the line with the regex feature of lineinfile.