Using Exported Resources in nginx loadbalancer conf

[Jon Ward]

Hi all,

I'm attempting to use exported resources to dynamically generate an nginx config file. What I'd like to do if have all my web servers export their hostname inside a file resource, and have these collected by my loadbalancer and used in the config file, restarting nginx if anything has changed.

There is an example of something very similar in Pro Puppet on pages 145-146 using Apache. In the example the individual hostnames are gathered by apache inside the balancer block using an include statement, however it doesn't seem possible to do this in an upstream block in nginx.

So I think I need to somehow gather the hostnames into an array which I then use inside my conf file erb template to create the upstream server entries. My problem is how to do this. If I create a custom fact this will get evaluated before (I think) the resources are exported and gathered. If I use a custom function or some ruby inside the erb template that's evaluated on the master, not the agent. I do have a puppet agent running on the master server so I could collect the exported resources there and use a function, that's the best solution I can think of at the moment, just wondering if anyone else out there has had any success doing something like this?

Thanks,

Jon

 

[Greg Sutcliffe]

On 25 September 2012 12:03, Jon Ward <jgh...@gmail.com> wrote:
> Hi all,
>
> I'm attempting to use exported resources to dynamically generate an nginx
> config file. What I'd like to do if have all my web servers export their
> hostname inside a file resource, and have these collected by my loadbalancer
> and used in the config file, restarting nginx if anything has changed.

Nginx can do include directives, right? Can you do something like:

<upstream>
include conf.d/upstream/*.conf
</upstream>

and then have each host export a file in
conf.d/upstream/$hostname.conf containing $hostname:$port (or whatever
is needed).

Not sure if that will work, my nginx is a little rusty...

[Jon Ward]

Hi Greg,

Yeah that's what I'd hoped - my balancer.conf.erb template had this:

upstream <%= balancer_name %>_balancer {

  include /etc/nginx/conf.d.upstream_servers/*.conf;

}

However when I try to start nginx I get:

Starting nginx: [emerg]: "include" directive is not allowed here

A bit of googling (e.g. http://stackoverflow.com/questions/4165116/can-include-directive-be-used-within-upstream-block-of-nginx) seemed to suggest it's not possible.

[Bill Fraser]

Hi Jon,

I have seen something similar done with Puppet Labs' haproxy module:

https://github.com/puppetlabs/puppetlabs-haproxy

Essentially the balancer members export information about themselves
(hostname, port, listening service, etc) which is collected by a
balancer server. The configuration is then pieced together using the
concat module.

I believe that is what you are trying to accomplish here, and hopefully
you find it useful to see how it is done in another module.

Regards,
Bill

[Jon Ward]

Thanks Bill - looks like concat provides a potential solution, will give that a go.

Jon