Re: Exception propagation design in Clojure web APIs.

[James Reeves]

I'd argue that using exceptions for control flow is something of an anti-pattern, even in Java.

In this case a better mechanism might be to use polymorphism. For instance:

(defprotocol Validatable

  (validation-errors [x] "Return the validation errors for x."))

(defn valid? [x]

  (empty? (validation-errors x)))

Then you can define a general function to validate and store that item in a database:

(defn store-valid [db x]

  (if (valid? x)

    (store db x)

    (validation-error-response x)))

- James

On 19 March 2013 16:43, Julien Dreux <julien...@gmail.com> wrote:

Hi all,

Coming from a Java background, I am having a hard time understanding how validation error propagation should work in clojure web APIs.

To be clear, this is similar to how my Java web service would be setup:

/** Method that validates the model, accesses the DB. If something went wrong, throw an exception */

public void validateAndCreateUser(User u) throws ValidationException, EmailAlreadyInUseException, ... {

  ...

  if(...) {

    throw new ValidationException(fieldName);

  } else if (...) {

    throw new EmailAlreadyInUseException(u.getEmail());

  }

}

/** Endpoint method, catches & formats the exceptions thrown by the db method. **/

@POST("/api/user/create")

public Response createUser (User u)  {

  ..

  try{

    validateAndCreateUser(u);

    return Response.ok();

  } catch (Exception e) {

    return generateExceptionResponse(e); //Method that maps exceptions to responses.

  }

}

For all of Java's clunkiness, this had the benefit of not having to write tons of if/else statements for validation handling. Exception were just thrown from anywhere, bubbling back up to inital call, and if not handled in the endpoint method, a specialized class mapped them into a proper response. The exceptions contained all the information needed to generate 'rich' error messages back to the client.

Being a Clojure newbie, I wonder what a good pattern is for a similar situation. So far, I have a method that validates models based on a schema, that returns

{:success true}

or 

{:success false :errors ["error 1" "error 2" ...]}

But I don't know how to avoid having to write if/else conditions of the sort in each function between my endpoint and db functions.

(if (validation :success)

  (follow-normal-path)

  (handle-validation-errors validation))

Any guidance appreciated.

Cheers,

Julien

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Laurent PETIT]

Hi I don't have much time to write a comprehensive answer, but if you want to follow the control via exceptions route, there's nothing preventing you from doing so.

You can (throw (ex-info msg map)), and then write a ring middle ware to handle exceptions globally, for instance.

HTH,

Laurent

[Dave Sann]

I am interested in this view that exceptions are an anti pattern. I have heard it voiced before.

I am not sure that I understand why.

As I see it you have a choices:

1. Handle in the result  - and test this result repeatedly all the way back to the caller

2. Handle "out of band" - Throw an exception, allow the stack to unwind and catch where it matters

[And maybe - but I am not very knowledgeable on this and it won't work today on the JVM anyway

 3. Use continuation passing style with TCO to shortcut the return to follow an exception path]

So, ignoring 3.

Why is 2 preferable over 1? There are certainly pros and cons.

Dave

[Dave Sann]

I am interested in this view that exceptions are an anti pattern. I have heard it voiced before.

I am not sure that I understand why.

As I see it you have a choices:

1. Handle in the result  - and test this result repeatedly all the way back to the caller

2. Handle "out of band" - Throw an exception, allow the stack to unwind and catch where it matters

[And maybe - but I am not very knowledgeable on this and it won't work today on the JVM anyway

 3. Use continuation passing style with TCO to shortcut the return to follow an exception path]

So, ignoring 3.

Why is 2 preferable over 1? There are certainly pros and cons.

Dave

On Wednesday, 20 March 2013 09:42:11 UTC+11, James Reeves wrote:

[Dave Sann]

That should have been "why is 1 preferable over 2"...

[John D. Hume]

One argument against using exceptions for commonplace occurrences (like invalid user input) is that the structure of the code may make it difficult to see where those things can pop up, which can lead to misunderstanding and introduction of bugs.

Even with Java's checked exceptions, where a certain method may be declared to throw a custom ValidationException or whatever, you can't see from the calling code which methods can throw one. You have to go look at the signatures of all called methods looking for them. (Or more likely you comment out the catch clause and see which calls your IDE underlines in red.) Using standard control-flow constructs make branch-points explicit.

[Christian Sperandio]

Use exceptions for control flow makes the developer's work easier but not the one of reader. In other terms the maintenance may become more difficult for the reason given by John.

And I think when exceptions are used for control flow, the original exception meaning is forgotten: Exceptions should inform about exceptional errors that shouldn't happen (code implementation error, network issues, disk failure,...). It's the opposite of user data checking :)

[Marko Topolnik]

Exceptions are a perfect tool for flow control, if used judiciously. The typical criticism revolves around their incompetent usage, and a more general criticism can be made against a mechanism that is subverted all too easily.

If you responsibly keep to the "good parts", exceptions could be the way to go. Validation is one example where I love them because it happens all around, but validation failures are all handled uniformly.

However, I would also urge you to explore other approaches, such as having a dynamically-bound variable that collects all the validation failures, which will potentially give you better diagnostics than the fail-fast behavior of validation excptions.

-marko

[Dave Sann]

Interesting.

Validation is the case I am thinking of - but it might also apply to any situation where you offer a different path of control - i.e. not stepping back up the stack. That's why I mentioned CPS. That's also (maybe) why true tail calls would be useful.

The mention of binding dynamic vars is interesting and I will look at this. It still leaves the question of how/when to pass control to actually manage or respond to the validation error. I have generally kept away from dynamic vars to date because I don't have a good feel for when and where they are and are not a good choice. This would be a great area for a blog post - or clojure docs doco -  if anyone who has used them extensively can offer a good view of pros and cons, whens and when nots :)

Marko, do you have a good example of doing what you say?

The reasons given for not using Exceptions are still not really convincing to me (yet). Although I do accept that some things may become more difficult to manage if you don't take care in what you do. This is generally true. I just see Exceptions as a way of managing control with pros and cons, not an anti pattern.

Maybe a more general question is what alternative control flows are available - which are possible but not available - and which are good for which situations.

Apologies to the original questioner for changing the focus of the recent posts.

Dave

[Marko Topolnik]

On Wednesday, March 20, 2013 1:06:54 PM UTC+1, Dave Sann wrote:

Marko, do you have a good example of doing what you say?

What I had in mind, related to OP's post, would be a middleware that does

(binding [*validation-failures* []] 

   (handler req) 

   (do-something-about *validation-failures*))

There would be a global function, such as add-failure, which would conj a new failure to the vector.

-marko

[Dave Sann]

Ok, but how do you take account of different processing (or non processing) needs in the handler (chain) given that something is awry half way down?

(note that in my case the handler is not pure - the validation precedes the need for a state change. The state change should occur if the validation passes and must not if it fails. I can't validate in advance because that would lead to a race condition).

[Ben Wolfson]

Exceptions are escape continuations; they should (modulo things like
"finally" if present) go directly to the nearest dynamically enclosing
handler. The fact that they have the name "exception" is unfortunate,
because it leads to purely nominal conceptions of when they should be
employed (i.e., in "exceptional situations"). That may have in fact
been the intention when they were introduced, but it's hard for me to
see why that's relevant. Oleg Kiselyov's implementation of generators
for ocaml is based on exceptions:
https://github.iu.edu/sabry/PPYield/blob/master/simple_gen.ml (as I
believe is the delimcc library, under the hood); it seems silly to
argue that since he makes *routine* use of exceptions, his
implementation is flawed.

--
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

[James Reeves]

On 20 March 2013 08:36, Marko Topolnik <marko.t...@gmail.com> wrote:

If you responsibly keep to the "good parts", exceptions could be the way to go. Validation is one example where I love them because it happens all around, but validation failures are all handled uniformly.

If validation happens "all around", that implies there is no one function that can test whether a value of data is valid for a given data store. This strikes me as a somewhat shaky foundation for a system.

There may be instances where it makes sense to use exceptions as a control flow mechanism, but I wonder whether it wouldn't be better to use something like CPS in those instances.

- James

[Marko Topolnik]

On Wednesday, March 20, 2013 4:34:32 PM UTC+1, James Reeves wrote:

On 20 March 2013 08:36, Marko Topolnik <marko.t...@gmail.com> wrote:

If you responsibly keep to the "good parts", exceptions could be the way to go. Validation is one example where I love them because it happens all around, but validation failures are all handled uniformly.

If validation happens "all around", that implies there is no one function that can test whether a value of data is valid for a given data store. This strikes me as a somewhat shaky foundation for a system.

The idea is that all validation functions share the same contract to call the appropriate add-failure function that registers the validation result.

 

There may be instances where it makes sense to use exceptions as a control flow mechanism, but I wonder whether it wouldn't be better to use something like CPS in those instances.

I can't picture how such a mechanism would work, and what benefit it would bring over the exceptions mechanism. CPS in Clojure means trampolining, which is quite an unwieldy, and I'd say "cheap" tack-on. A validating function would then be supposed to return a common, globally-defined "continuation", in fact just a simple function, that would redirect the flow towards the validation failure-handling case.

-marko

[Julien Dreux]

Thank you all for your answers, 

I like Marko's approach. 

What I had in mind, related to OP's post, would be a middleware that does
(binding [*validation-failures* []] 
   (handler req) 
   (do-something-about *validation-failures*))
There would be a global function, such as add-failure, which would conj a new failure to the vector.

Being a Clojure novice I had no idea that this was a possibility and it sounds like a better approach than exception throwing. Is this something similar to the way lib-noir does it? https://github.com/noir-clojure/lib-noir/blob/master/src/noir/validation.clj see the set-error fn. 

I would still code my own library because I am learning clojure at the moment. 

Question: how is it possible to guarantee that the *failure-list* or *error-list* is only available to the current request. Is that one of the properties of :dynamic?

[Marko Topolnik]

I would still code my own library because I am learning clojure at the moment. 

Question: how is it possible to guarantee that the *failure-list* or *error-list* is only available to the current request. Is that one of the properties of :dynamic?

Yes, that's built-in: with (binding ...) you establish a thread-local binding that is not visible from other threads = request handlers.

-marko 

[Marko Topolnik]

On Wednesday, March 20, 2013 5:51:59 PM UTC+1, Julien Dreux wrote:

Thank you all for your answers, 

I like Marko's approach. 

What I had in mind, related to OP's post, would be a middleware that does
(binding [*validation-failures* []] 
   (handler req) 
   (do-something-about *validation-failures*))
There would be a global function, such as add-failure, which would conj a new failure to the vector.

Being a Clojure novice I had no idea that this was a possibility and it sounds like a better approach than exception throwing. Is this something similar to the way lib-noir does it? https://github.com/noir-clojure/lib-noir/blob/master/src/noir/validation.clj see the set-error fn. 

Hm, that's really it, quite precisely; I wasn't aware of it. Seeing it now I realize that in my example I made the mistake of forgetting to wrap the failure vector inside an atom. Without that it won't work. 

-marko

 

[Julien Dreux]

Would there be a way of combining this approach with the use of preconditions in clojure?

E.g.

(defn create-user [user]

  {:pre validation/user? user}

  (db/insert! user))

[Cedric Greevey]

It will if you use set! to update the binding.

-marko

 

--

[James Reeves]

On 20 March 2013 16:41, Marko Topolnik <marko.t...@gmail.com> wrote:

On Wednesday, March 20, 2013 4:34:32 PM UTC+1, James Reeves wrote:

If validation happens "all around", that implies there is no one function that can test whether a value of data is valid for a given data store. This strikes me as a somewhat shaky foundation for a system.

The idea is that all validation functions share the same contract to call the appropriate add-failure function that registers the validation result.

I don't see why that would be necessary. Why not put all the validation logic in one place?

 

There may be instances where it makes sense to use exceptions as a control flow mechanism, but I wonder whether it wouldn't be better to use something like CPS in those instances.

I can't picture how such a mechanism would work, and what benefit it would bring over the exceptions mechanism. CPS in Clojure means trampolining, which is quite an unwieldy, and I'd say "cheap" tack-on. A validating function would then be supposed to return a common, globally-defined "continuation", in fact just a simple function, that would redirect the flow towards the validation failure-handling case.

I wasn't thinking of validation when I suggested CPS, because I don't see a need for any unusual control flow when validating data.

- James 

[Marko Topolnik]

On Wednesday, March 20, 2013 11:29:55 PM UTC+1, James Reeves wrote:

On 20 March 2013 16:41, Marko Topolnik <marko.t...@gmail.com> wrote:

On Wednesday, March 20, 2013 4:34:32 PM UTC+1, James Reeves wrote:

If validation happens "all around", that implies there is no one function that can test whether a value of data is valid for a given data store. This strikes me as a somewhat shaky foundation for a system.

The idea is that all validation functions share the same contract to call the appropriate add-failure function that registers the validation result.

I don't see why that would be necessary. Why not put all the validation logic in one place?

Indeed, why not. If the requirements allow it, it is definitely preferred.

 

 

There may be instances where it makes sense to use exceptions as a control flow mechanism, but I wonder whether it wouldn't be better to use something like CPS in those instances.

I can't picture how such a mechanism would work, and what benefit it would bring over the exceptions mechanism. CPS in Clojure means trampolining, which is quite an unwieldy, and I'd say "cheap" tack-on. A validating function would then be supposed to return a common, globally-defined "continuation", in fact just a simple function, that would redirect the flow towards the validation failure-handling case.

I wasn't thinking of validation when I suggested CPS, because I don't see a need for any unusual control flow when validating data.

That's enterely use case-dependent. In some project validation was specified as fail-fast, and it happened in the middle of business logic because the validation status couldn't be determined without computing some interim results. Throwing an exception from a function three-four levels down was the perfect choice in that case.

-marko 

[Julien Dreux]

For anyone looking to do something similar (i.e. using exception control flow for handling request errors & validation), I found the slingshot library invaluable for enhanced try and throw  leveraging Clojure. Going with 'native' try/throw was becoming nightmarish very quickly.

Thank you all for your input on this.

Julien