Making spec.test/intrument enable return value checking

[Adam P. Jenkins]

I wanted to float the idea of adding an option to clojure.spec.test/instrument to enable :ret and :fn spec validation against function outputs. So just calling

(stest/instrument symbols)

would behave as it does now, validating function input arguments, but

(stest/instrument symbols :check-outputs true)

would additionally validate the function outputs against the :ret and :fn specs for the instrumented functions.

The docs mention that instrument only enables input argument checking, because you should be using tests to validate function outputs. I agree that ideally an application's test suite will be thorough enough to catch all errors. However for prototypes where I don't have a full test suite, or any time an error turns up in a normal run that didn't get caught by a test, it would be useful to be able to enable output checking to help track down the error.

I made the change to add the :check-outputs option locally on my machine, and it was a very simple to change to just three functions in test.clj, and it seems to work fine. If I were to submit a patch, would this be a change that you'd be interested in? Or is there some good reason I'm missing for not supporting this option?

Thanks,
Adam

[Alex Miller]

instrument is concerned with validating inputs (that is, checking that external calls are properly invoking a function), not verifying that the function itself works (which is the domain of clojure.spec.test/check). One option for you now is to use s/assert to validate the :ret spec on return. 

(defn foo [...]

  ...

  (s/assert (:ret (s/get-spec `foo)) ret))

At the moment, we're not planning on changing this, however the future is a long time. :)  I don't think we need a ticket on it right now.