We're researching using Go to build RESTful services.One requirement we've been given is that any HTTP server we write must be able to provide what it's known as "zero downtime", that is, code reloads must not cause ongoing requests to be lost, and must not cause an outage window where the HTTP server is not accepting requests.Another requirement is that the HTTP server must be able to be configured so it will listen on a UNIX socket, such that we don't need to worry about TCP port->service mappings.We know that it's possible to do this because servers like Gunicorn and NginX (via fork() and passing FDs), so this factors in our decision for a platform, and right now it weighs against writing Go-based services.So we've found some (albeit buggy) code that doesn't quite cut the mustard. http://blog.nella.org/?p=879What we're looking for is a contractor who will write a Go package for us, with a function which will do the following:- substitute for http.ListenAndServe() such that we can use it in our Go web services like that,- such that a UNIX signal sent to the program will cause it to fork a new program listening on the same socket, and exit the old one cleanly,- the function should be usable from within a goweb or gorillaweb app,- the function should be able to listen to UNIX sockets, and,- of course, the exiting program should exit cleanly rather than abort ongoing requests.Those are the requirements. We can pay you in U.S. dollars or Bitcoin (your choice).If this is not the appropriate forum for this type of request, please point me to the right forum.
I appreciate your advice and I already have this scaling model implemented for the frontend services.
Now I want a zero-downtime Go backend.
No, reconfiguring the frontend and reloading it twice every time a back end needs to be restarted (to swap old backends out and new backends in) is not an acceptable substitute. It also does not solve the problem that backends exiting will kill off any connections they are handling.
I appreciate your advice and I already have this scaling model implemented for the frontend services.
Now I want a zero-downtime Go backend.
No, reconfiguring the frontend and reloading it twice every time a back end needs to be restarted (to swap old backends out and new backends in) is not an acceptable substitute. It also does not solve the problem that backends exiting will kill off any connections they are handling.
So it should be a fairly trivial task to take your goagain code and make it look at sockets instead of ports, right? Let me know if you foresee any pitfalls...
On Friday, March 22, 2013 11:18:47 AM UTC-7, Kyle Lemons wrote:
It's relatively easy to write an HTTP "reverse proxy" like this in Go. This talk discusses a similar problem: http://talks.golang.org/2012/simple.slideAndrew
--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/41TPj4PWBI8/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
I get the feeling that you and I are talking about different problems.
I was referring to the problem of a zero downtime upgrade and restart of an HTTP backend, as explicitly defined above by me and others.
You seem to be talking about the general problem of high availability, which is larger in scope.
Now, I am aware that there are other problems that need solving for a comprehensive solution, like the problem of high availability, the problem of handling software and hardware failures, et cetera.
Making a Go program that can restart itself without losing connections is not a solution to those other problems. I never said it was nor would I suggest that, ever.
Let's get some common ground in terminology before criticizing each other's ideas, proposals and code.
Have a great day! :-)
Kyle,This is PHENOMENAL.I note that your code has a number of TCPListener interfaces. How feasible is it to rewrite it to use a more generic Listener interface, so it will work with UNIX sockets? That eliminates local access to the Go HTTP server, and eases the problem of mapping a constrained space of port numbers to backend servers in my frontend.
Hi Manuel,I've been playing with Go's net/http and ended up with a small patch that adds support for things I often need on web applications.These include: Unix sockets, ability to automatically use the contents of X-Forwarded-For as the Request.RemoteAddr, logging, and a few other things.freegeoip.net has been running on it for a couple of weeks and so far no problems at all. I'll probably blog about it when possible.
Cheers
In addition to Andrew's suggestion about making a frontend, I am working on open-sourcing a library that can provide zero-downtime restarts. Mine is similar in function to godoc.org/github.com/rcrowley/goagain though I think the API is much easier to use ;-).
On Thu, Mar 21, 2013 at 5:05 PM, Manuel Amador <manuel...@aditazz.com> wrote:
We're researching using Go to build RESTful services.One requirement we've been given is that any HTTP server we write must be able to provide what it's known as "zero downtime", that is, code reloads must not cause ongoing requests to be lost, and must not cause an outage window where the HTTP server is not accepting requests.Another requirement is that the HTTP server must be able to be configured so it will listen on a UNIX socket, such that we don't need to worry about TCP port->service mappings.We know that it's possible to do this because servers like Gunicorn and NginX (via fork() and passing FDs), so this factors in our decision for a platform, and right now it weighs against writing Go-based services.So we've found some (albeit buggy) code that doesn't quite cut the mustard. http://blog.nella.org/?p=879What we're looking for is a contractor who will write a Go package for us, with a function which will do the following:- substitute for http.ListenAndServe() such that we can use it in our Go web services like that,- such that a UNIX signal sent to the program will cause it to fork a new program listening on the same socket, and exit the old one cleanly,- the function should be usable from within a goweb or gorillaweb app,- the function should be able to listen to UNIX sockets, and,- of course, the exiting program should exit cleanly rather than abort ongoing requests.Those are the requirements. We can pay you in U.S. dollars or Bitcoin (your choice).If this is not the appropriate forum for this type of request, please point me to the right forum.
--
Any improvements on that? I wanted to use goagain but I'm using several listeners and I had to hack on goagain to support it. Howeve the way it handles file descriptors made my life hard. Did you open sourced your code or are you still waiting?
We will open source everything very shortly :-) we have come up with a mechanism that beats the previous FD passing and is compatible with the way systemd and upstart like to supervise services running "in the foreground".