Updating a running system

[André Thieme]

Is there an idiom for handling updates of a running system?
It should be easy to add new elements to a defroutes.
For example, we have:
(defroutes foo
(PUT "/resource1" res1))

(decorate foo
(with-something-1))

And then we make a change:
(defroutes foo
(PUT "/resource1" res1)
(GET "/resource2" res2))

When we evaluate this defroutes form the server begins to
handle also requests to /resource2.
Although one potential problem I see is:
We must evaluate the defroutes form AND the decorate form.
What happens if inbetween a request comes in? I think it
would get handled only by the handler for the route, without executing
first the with-something-1 handler.
But I would like to update all handlers in one atomic operation.

And when I have ten handlers through which a request may go
(or only through one of them, or the first three, etc.)?
(defn handler1 [request] ...)
(defn handler2 [request] ...)
(defn handler3 [request] ...)
...
(defn handler10 [request] ...)

(decorate system
(handler1)
(handler2)
...
(handler10))

In my local test system I update maybe the handlers 2 and 6
and add handler 11, but remove number 3.

In such a case I would like to
1. eval again:
(defn handler2 [request] ...)
(defn handler6 [request] ...)

2. ns-unmap the killed handler3:
(ns-unmap 'my.namespace 'handler3)

and
3. eval
(defn handler11 [request] ...)

After that:
4. re-eval
(decorate system
(handler1)
(handler2)
(handler4)
...
(hander10)
(handler11))

Operation 3 is no problem.
But Step 1 can be a problem. Namely if there is already a request
which entered the system. It will begin in the decorated handlers
and now goes through the chain.
But what I want to do is doing steps 1-4 atomically. That means:
all requests that already are in the system must be continued to
get handled as if I never made a change.
All new requests that come in will go through the handlers 1-11
(where 3 is missing, and 2 and 6 are updated and 11 is added).

Is there already an idiomatic way to do that?

[Shantanu Kumar]

At the risk of sounding like a shameless plug, I would suggest you to keep a tab on JWebMVC for some of the same things you mentioned.

http://code.google.com/p/bitumenframework/

The Clojure extension will let you modify the route-map in a transaction (STM), even in production deployment.

Regards,

Shantanu

[James Reeves]

On Oct 5, 7:46 am, André Thieme <splendidl...@googlemail.com> wrote:
> Is there an idiom for handling updates of a running system?

I think this topic has come up on the Clojure forums as well. Whilst
Clojure allows you to reload namespaces, I don't believe it does this
atomically; each def is evaluated sequentially. So if you have some
code like:

(defn foo [x]
(+ x 1))

(defn bar [x]
(prn (foo x)))

And then update your code to:

(defn foo [x y]
(+ x y))

(defn bar [x]
(prn (foo x 1))

Then you can get into a situation where the new "foo" is called by the
old "bar". In other words, reloading namespaces is not thread safe in
Clojure.

You could probably get around this by creating a custom class-loader,
but straight Clojure does not, to my knowledge, have the functionality
to safely reload namespaces in a threaded environment.

- James

[André Thieme]

I was more thinking about a single entry point into the routes.
Something like:

(defn sys1 [request] ...)
(defn sys2 [request] ...)

(defn handler1 [request] ...)
(defn handler2 [request] ...)
(defn handler3 [request] ...)
...
(defn handler10 [request] ...)

(defroutes system
(PUT "/foo" sys1)
(DELETE "/bar" sys2))

(decorate system
(handler1)
(handler2)
...
(handler10))

This is what I currently do in Compojure. The decorate will set up
handlers that run before the routes. And as I saw it begins with
the last decorated handler (handler10) and goes down to the first
one (handler1).

When now my defns above are instead anonymous functions then my
routes and their decoration could also be anon, or get a dummy
name. The route functions and the handler functions could all go
into a Map for example.
When I then want to make an update, I will put the updated functions
in a new Map and update:
(def routes2 {:sys1 (fn [request] ...), :sys2 (fn [request] ...)})
(def handlers2 {:handler7 (fn [request] ...) ...})

(defroutes temp3
(PUT "/foo" (:sys1 routes2))
(DELETE "/bar" (:sys2 routes2)))

(decorate temp3
(:handler1 handlers2)
(:handler2 handlers2)
...
(:handler10 handlers2))

(entry-point temp3)

A way to have multiple routes and handlers existing at the same
time would be good. And one single command would only do a reset!
on an atom, to decide at which function it starts. Then I can leave
all existing functions continue the handling of their current
requests and get the new ones online.
And the entry-point function or maybe def-entry-point macro could
also track how many requests are currently handled. A simple atom
that gets inc'ed when a new request enters the first handler, and
when the request got sent out the atom counter gets dec'ed.
Then entry-point can take a function which gets called as soon the
counter gets dec'ed to 0.
This function could be one that removes all references to the
vectors/maps of anon routes/handlers, so they can be garbage
collected.

(entry-point temp3
(fn []
(ns-unmap 'my.namespace 'routes2)
(ns-unmap 'my.namespace 'handlers2)))

I don't know the architecture of Compojure well enough.. but maybe
an addition of an entry-point function or a def-entry-point macro
can do the job?

[André Thieme]

James, is there a way to do something similar to what I suggested?

Let's say I implement
(defroutes temp1
...)

(decorate temp1
...)

But don't want to make this my servlet.
Instead I want:

(defroutes system
(ANY "/*" temp1))

and later
(jetty-server params "/*" (servlet system))

Then I want any requests that come in to be handled by
system, but this is doing nothing but forwarding the
requests to temp1.

I can make some middleware for temp1 which will bind a flag to
true as long at least one request is currently handled.
Then I can write a temp2 route+decoration, with all new functions
and updates and changes I want.
I eval
(defroutes system
(ANY "/*" temp2))
and have a freshly updated system which will now forward all
requests to the new handler.
In a loop, with a Thread/sleep 5 I wait until the flag is false
and then I can ns-unmap the temp1 stuff, and the GC will kill it.
Is that possible somehow?

[James Reeves]

On Oct 7, 12:21 pm, André Thieme <splendidl...@googlemail.com> wrote:
> James, is there a way to do something similar to what I suggested?

I'm not quite sure what you're asking. Yes, it's possible to arrange
your code so that you can safely reload namespaces, but every change
you make to that code from then on has the potential to break for any
running threads.

With regards to your 'decorate' problem, you could just change this:

(defroutes my-routes
(GET "/"
"Hello World")))

(decorate my-routes
(with-session :memory))

To:

(def my-routes
(-> (routes
(GET "/"
"Hello World"))
(with-session :memory)))

But that won't guarantee you're not going to have problems with your
other functions. I suspect that it will be very difficult to ensure
that an application can be safely reloaded in a production
environment.

- James

[André Thieme]

Hello.

On 7 Okt., 20:29, James Reeves <weavejes...@googlemail.com> wrote:
> On Oct 7, 12:21 pm, André Thieme <splendidl...@googlemail.com> wrote:
>
> > James, is there a way to do something similar to what I suggested?
>
> I'm not quite sure what you're asking. Yes, it's possible to arrange
> your code so that you can safely reload namespaces, but every change
> you make to that code from then on has the potential to break for any
> running threads.

I don't want to reload a namespace.
Instead I would add new (anon) functions at the repl to the namespace,
into a vector or map.
Those functions are my handlers, so, they are what is used in
defroutes
and decorate. It will not break running threads when I add those new
functions.

> With regards to your 'decorate' problem, you could just change this:
>
>   (defroutes my-routes
>     (GET "/"
>       "Hello World")))
>
>   (decorate my-routes
>     (with-session :memory))
>
> To:
>
>   (def my-routes
>     (-> (routes
>           (GET "/"
>             "Hello World"))
>         (with-session :memory)))
>
> But that won't guarantee you're not going to have problems with your
> other functions. I suspect that it will be very difficult to ensure
> that an application can be safely reloaded in a production
> environment.

This looks good. This def is this single entry point I was looking
for.
I can have:
(def route-handlers0 [(fn [request] ...), (fn [request] ...), ...])
(def decoration0 [(fn [request] ...), (fn [request] ...), ...])

(def system
(-> (routes
(GET "/route1" (get route-handlers0 0))
(PUT "/route1" (get route-handlers0 1))
(DELTE "/route2" (get route-handlers0 2))
...)
(get decoration0 0)
(get decoration0 1)
...))

When I now want to make a change I will have a
(def route-handlers1 [...])
(def decoration1 [...])
and eval again
(def system ...) with the new functions.

Do you think that this could break the server?
I would think that as all the old functions still
exist they can still be called. All ongoing connections
get handled by the first set of handlers. The function
call for that was created by the -> macro, and those
functions still exist.
But all freshly incoming requests will now get the new
route through the handlers.

I can have a middleware which will let me know when all
jobs for the old requests were handled. Then I can unmap
route-handlers0 and decoration0.
Does this make sense?

[James Reeves]

On Oct 8, 9:41 am, André Thieme <splendidl...@googlemail.com> wrote:
> I don't want to reload a namespace.
> Instead I would add new (anon) functions at the repl to the namespace,
> into a vector or map.

Then perhaps something like:

(def your-handlers (ref []))

(defn uber-handler
[request]
((apply routes @your-handlers) request))

- James