Is there a way to use spec/fdef function specs with clojure.test ?

[Khalid Jebbari]

Hello,

I'm struggling to find a way to to use the fdef specs I wrote in clojure.test tests. I can run them fine in the repl with spec/exercise-fn or spec.test/check, really nice when developping by the way. Now that I'm happy with the result I'd like to encode this knowledge in tests to prevent regressions. I don't need more tests that this, not specific property etc.

I found no way to plug the spec.test/check in clojure.test or easily reuse fdef specs. test.check/defspec and quickcheck expect properties as their argument. spec/describe return a LazySeq that I found hard to exploit without a lot of manual wiring, parsing and trial-and-errors.

If I had to write it by hand, it would look like :

(defspec myspec 100 (prop/for-all [one (spec/gen ::first-arg) 

                                                       two (spec/gen ::second-arg)]

                                        (is (true? (spec/valid? ::ret-spec (myfunc one two))))

The problem is that it's incomplete with regards to spec possibilities : spec/or, spec/nilable etc. and I use them. Also I the function changes (in any way) the test becomes irrelevant instantly.

A colleague resorted to manually calling spec.test/check in clojure.test and manually verifying the output of the function (the :result boolean, the :num-tests etc.). Feels way too manual, and doesn't report the shrunk value as nicely as test.check does.

Maybe I missed something completely. spec/describe seems the best bet to introspect the spec and use it in for-all calls. But still too manual.

Any help much appreciated.

[Khalid Jebbari]

I've managed to get something working. Very dirty and hardcoded most things though, so not reusable at all. The key here is to parse the return of spec/describe and retrieve the value for the :arg, convert it to real spec with eval and retrieve its generator.

(defspec a-test

         (let [args (nth (s/describe 'foo.core/bar) 2)

               spec-code (map #(if (symbol? %) (symbol "clojure.spec.alpha" (str %)) %) args)

               spec (eval spec-code)

               args-gen (s/gen spec)]

           (prop/for-all [argz args-gen]

             (s/valid? ::ret-spec (apply foo.core/bar argz)))))

In the end one shoot write a proper spec/describe parser ?

Or has someone a better idea ?

[Beau Fabry]

Can't you just assert on the return value of spec.test.check in a clojure.test test? ie, this is a test in one of our repos:

(deftest ^:spec generative
	(is (not-any? :failure (st/check [`export/invert-table->account-ids]))))

[Khalid Jebbari]

I could indeed, but if it fails, what does the report look like ? Does it display the output of `st/test` (which contains all details, notably args, return and smallest input ?

Khalid aka DjebbZ

@Dj3bbZ

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/0UjSFT926vg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[avichalp]

Hi Khalid,

Yes, you can get these details in the output.
I have found these helper functions useful for this purpose.

[Khalid Jebbari]

Thanks for the link. It almost works ! The problem I have is that stest/check returns a map with [:failure false] when it fails, which seems weird and prevent stest/abbrev-result to add meaningful data to the reporting. Not sure if it's a bug in spec.

Just to make things clear :

- Passing test :
(-> (stest/check 'my.ns/my-fn) first :failure)

=> nil

- Failing test :

(-> (stest/check 'my.ns/my-fn) first :failure)

=> false ;; I expected true, and it seems stest/abbrev-result too

Env :

[org.clojure/clojure "1.9.0-beta1"]

[org.clojure/spec.alpha "0.1.123"]

[org.clojure/test.check "0.10.0-alpha2" :scope "test"]

[Khalid Jebbari]

I created https://dev.clojure.org/jira/browse/CLJ-2246 since I'm pretty sure there's a bug.