adding/removing/updating configuration on-the-fly

[Mathijs]

Hi,

I'm thinking of moving our (rails)app-servers to nginx + switchpipe +
thin to replace nginx + mongrel_cluster + mongrel.

I have some questions on certain features that I would like to have
answered before making the move.

I noticed switchpipe picks up on configuration changes without
restarting/reloading.
Feels like an awesome feature. Does this impact performance? To me it
seems that polling the config-dir for changes isn't very fs-friendly.
Can the frequency be set somehow? Or won't it matter at all because of
some fs-caching mechanism in the OS? Or does it use inotify/dnotify/
fam/gamin?

Is there a way to 'kill' instances of just one application manually?
When capistrano upgrades an application, nginx will pick up on the
maintenance file and disable access to the app.
I would like to stop any running instances before upgrading, since the
'old' directory will be removed and replaced by the new one. I don't
want this to happen while the application-instances are still running
and accessing log files and other stuff.
Since I will set a somewhat high timeout-time(5 minutes or so, so
users that read a page for a few minutes won't have a small delay when
they start navigating around again), I can't count on the old
instances timing out during the upgrade, which means they will be
accessed again as soon as capistrano finishes and removes the
maintenance file. This will lead to errors since they will still run
the old version.

Will SSL connections work as expected?
Since nginx handles the ssl stuff and only passes an extra http-header
to the mongrels at the moment, I guess this will still work through
switchpipe.

How about uploads/downloads and stuff?
I saw there's a TODO item on big http bodies.
Does this mean that large files will be buffered in memory before
being sent?
Same for uploads?
I'm currently using the rails x_send_file plugin to make use of
nginx's sendfile support by only sending a special http header, which
is picked up by nginx which will then serve the real (big) file.
Will this still work?

I hope I'm not asking too many questions, but unfortunately there
isn't much documentation it seems.

Greetings,
Mathijs

[Peter Cooper]

On Feb 28, 11:02 pm, Mathijs <bluescreen...@gmail.com> wrote:
> I noticed switchpipe picks up on configuration changes without
> restarting/reloading.
> Feels like an awesome feature. Does this impact performance? To me it
> seems that polling the config-dir for changes isn't very fs-friendly.
> Can the frequency be set somehow? Or won't it matter at all because of
> some fs-caching mechanism in the OS? Or does it use inotify/dnotify/
> fam/gamin?

It just does it manually, but only checks once every five seconds.
This is a pretty non-intensive check, although naturally there will be
some effect (however minor). The delay could be changed, although
that's not currently a configuration file setting (but it can
certainly be made into one, and I will put that down to be done).

> Is there a way to 'kill' instances of just one application manually?

If you needed to you could just change the max_instances and
min_instances on that particular app's configuration file to suit.
It'll automatically pick up the changes and enforce them.

> When capistrano upgrades an application, nginx will pick up on the
> maintenance file and disable access to the app.
> I would like to stop any running instances before upgrading, since the
> 'old' directory will be removed and replaced by the new one. I don't
> want this to happen while the application-instances are still running
> and accessing log files and other stuff.

You could, I guess, have some way to remove and rebuild the app's
config file each time, which would have that effect. So:

* delete app's config file
* wait X seconds (usually 5)
* no instances now running
* update all files
* create new config file
* wait X seconds
* you're now up and running again

> Will SSL connections work as expected?
> Since nginx handles the ssl stuff and only passes an extra http-header
> to the mongrels at the moment, I guess this will still work through
> switchpipe.

No, not at all. From the FAQ:

<<
Does it support SSL?

No. The reason for this is that SwitchPipe just handles things cleanly
through a regular connection, with no handler for setting up a secure
connection / handshaking / etc. If you wanted to use SwitchPipe with
SSL, the best method would probably to put "Pound" in front of it, as
Pound can provide SSL services to regular HTTP daemons.
>>

SSL is a significant set of work over HTTP. SwitchPipe doesn't get
involved in protocols per-se, except for scanning HTTP headers as they
pass through, but SSL demands any link / proxy in the chain must fully
speak SSL and do all the handshaking, etc. Pound could be used for
this placed in front of SwitchPipe, however.

> How about uploads/downloads and stuff?
> I saw there's a TODO item on big http bodies.
> Does this mean that large files will be buffered in memory before
> being sent?

That's how it currently works, yes.

> I'm currently using the rails x_send_file plugin to make use of
> nginx's sendfile support by only sending a special http header, which
> is picked up by nginx which will then serve the real (big) file.
> Will this still work?

It should do. Headers are passed back and forth intact.

> I hope I'm not asking too many questions, but unfortunately there
> isn't much documentation it seems.

A lot more than most of the competitors though! :) I take it you've
seen: http://groups.google.com/group/switchpipe/web .. the FAQ is
quite long and might help in some areas.

Cheers,
Peter Cooper