instrumenting clojure.core

[Ryan Fowler]

Is there an effort to write specs for Clojure's core namespaces? Apologies if this has already been addressed.

I've been tinkering with trivial (and probably wrong) fdefs on clojure.core/map and clojure.core/identity. It seems that the spec exception messages are way better than what we have now.

Code is here: https://gist.github.com/ryfow/69a64e966d48258dfa9dcb5aa74005eb

=========================================================

Before instrumenting clojure.core/map

=========================================================

Form:             (into [] (map nil [1 2 3] [1 2 3]))

Exception Class:  java.lang.NullPointerException

Message:          nil

Form:             (into [] (map 1 [1 2 3] [1 2 3]))

Exception Class:  java.lang.ClassCastException

Message:          java.lang.Long cannot be cast to clojure.lang.IFn

Form:             (into [] (map identity 4))

Exception Class:  java.lang.IllegalArgumentException

Message:          Don't know how to create ISeq from: java.lang.Long

=========================================================

After instrumenting clojure.core/map

=========================================================

Form:             (into [] (map nil [1 2 3] [1 2 3]))

Exception Class:  clojure.lang.ExceptionInfo

Message:          Call to #'clojure.core/map did not conform to spec:

In: [0] val: nil fails at: [:args :fn] predicate: ifn?

:clojure.spec/args  (nil [1 2 3] [1 2 3])

Form:             (into [] (map 1 [1 2 3] [1 2 3]))

Exception Class:  clojure.lang.ExceptionInfo

Message:          Call to #'clojure.core/map did not conform to spec:

In: [0] val: 1 fails at: [:args :fn] predicate: ifn?

:clojure.spec/args  (1 [1 2 3] [1 2 3])

Form:             (into [] (map identity 4))

Exception Class:  clojure.lang.ExceptionInfo

Message:          Call to #'clojure.core/map did not conform to spec:

In: [1] val: 4 fails at: [:args :seq] predicate: sequential?

:clojure.spec/args  (#object[clojure.core$identity 0x15a04efb "clojure.core$identity@15a04efb"] 4)

=========================================================

Before instrumenting clojure.core/identity

=========================================================

Form:             (into [] (map identity [1 2 3] [1 2 3]))

Exception Class:  clojure.lang.ArityException

Message:          Wrong number of args (2) passed to: core/identity

=========================================================

After instrumenting clojure.core/identity

=========================================================

Form:             (into [] (map identity [1 2 3] [4 5 6]))

Exception Class:  clojure.lang.ExceptionInfo

Message:          Call to #'clojure.core/identity did not conform to spec:

In: [1] val: (4) fails at: [:args] predicate: (cat :one-argument identity),  Extra input

:clojure.spec/args  (1 4)

[Alex Miller]

Yeah, I've been working on parts of this and there will be more to say in the future.

[Leon Grapenthin]

That looks great already. I'm also interested in what the official workflow for adding specs to core is going to be and whether contributions are desired.

The last example seems like it could be better because the user has to infer how identity was called within map.

So what we'd want is a codependent spec where we can say if you give N sequences you have to give a fnN. Is this possible with the current ops?

[Alex Miller]

On Sunday, June 12, 2016 at 9:22:24 AM UTC-5, Leon Grapenthin wrote:

That looks great already. I'm also interested in what the official workflow for adding specs to core is going to be and whether contributions are desired.

Still much to be determined about this but I expect that specs for core will be provided and maintained by us, because while there are a lot of obvious specs in core, there are also a lot of far more subtle ones. Some of the predicates added in the latest alpha5 (seqable?, and the simple/qualified ident/keyword/symbol) will not coincidentally be useful in these. There are also a lot of cases where there is a judgement call on how precise to be (any likely tradeoffs with testing, performance, conciseness).

Also, there is a significant difference between specs for macros and functions in core. Macro args specs will always be checked during macroexpansion and this is a path toward better error messages for things like defn, ns, etc. I think it would make a lot of sense to deliver those with core. Specs for the functions in core are things you likely want instrumented at dev time, not at prod time, and fights a bit with direct linking. So, one option would be a dev build of Clojure, without direct linking and with automatic instrumentation of the functions in core. Another would be providing the core function specs externally or optionally alongside Clojure. We've only begun discussing how this will wind up.

I don't agree with the map spec provided because I expect the args and ret to be seqable?, not sequential?. For example, you can map over sets, which are seqable but not sequential. This kind of subtlety is why I expect we will provide the specs for core. Once they exist, we'll be interested in tickets on them of course.

There are also interesting :fn specs you can write on map - you could check that if the count of the coll args is 0 that the return is ifn? (a transducer) and that if there is at least one coll then the ret is a coll. If the colls are all finite, the return coll will have count equal to the minimum of the count of the arg colls. I think the transducer/coll difference is definitely worth spec'ing and would trigger errors in cases where some forgets the coll and accidentally gets a transducer. Is the cardinality check worth it? Probably not.

identity could have a :fn spec that checks that the input and output are identical. That's really the only thing you care about (other than the cardinality of the input). Is this spec really helping you though? Not every spec pays for it's weight and this may be one that isn't worth doing.

 

The last example seems like it could be better because the user has to infer how identity was called within map.

So what we'd want is a codependent spec where we can say if you give N sequences you have to give a fnN. Is this possible with the current ops?

Yes, with fspec - you could have a :fn spec on map that checked that the arity of the fn passed to map matched the number of colls you were passed. That would be better here than the identity spec.

[Leon Grapenthin]

Alex, thank you for the long and detailed response.

The direct linking issue and Ryans last example make me wonder whether the hard rule

"If A and B are both instrumented and if A ever calls B violating B's spec, A's spec is broken" 

holds? If so the direct-linking issue would probably be  a non-concern since instrumentation within the standard library would not matter because one could assume that the core lib has been tested already against it's own specs. If the latter is true (I'm not quite sure) it would appear worthwhile to look for a counter-example against the former in the standard library.

I'd highly appreciate if you could give an example for checking the arity of a passed in function since I don't see how to do that right now except for maybe using reflection?

[Alistair Roche]

Hi Leon,

I think you're looking for fspec, unless I'm misunderstanding something. I wrote up an example that might be helpful.

@Ryan thanks for starting this thread, and @Alex thanks for responding. It's been an interesting discussion!

Cheers,

[Ryan Fowler]

> On Jun 12, 2016, at 10:45 AM, Alex Miller <al...@puredanger.com> wrote:
>
> Still much to be determined about this but I expect that specs for core will be provided and maintained by us, because while there are a lot of obvious specs in core, there are also a lot of far more subtle ones.

I think this is the right choice and I'm looking forward to seeing the details. My gut says that the better error messages alone are going to help significantly in convincing colleagues to work through learning Clojure.

Awesome work so far.

Ryan

[Leon Grapenthin]

Thank Alistair, but that does not really address my question. Alex suggested using :fn of fspec to check arity of a higher-order argument.

But I could not find a tool to check function arity. Also I doubt :fn is going to work since I'd expect it to be invoked /after/ the call - i. e. the call would fail before the arity check.

Note that in your example you can only use spec/generic testing to check arity because you know the argument types. You can't test a generic higher order fn for just arity like this because the generator won't know the correct types to generate.

[Alistair Roche]

Oh, I see what you mean now, Leon. Apologies for not reading more closely! Yours is a much more interesting puzzle.

Here's an attempt I made, groping towards it using reflection, but I couldn't even get that to work. Would be curious to see what the solution is there, and even more so (like you) to see if it can be done without reflection.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/d_3V9MfLZmY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-- Alistair

[Alex Miller]

I was suggesting that you could do something like this (although I'm pretty sure this doesn't work right now):

(s/fdef map
  :args (s/cat :f (s/fspec :args (s/+ ::s/any))
               :colls (s/* seqable?))
  :ret (s/or :seq seqable? :transducer ifn?)
  :fn #(if (zero? (count (-> % :args :colls)))
         ;; transducer
         (ifn? (-> % :ret))
         ;; lazy seq
         (and (seqable? (-> % :ret))
              (= (count (-> % :args :f :args))
                 (count (-> % :args :colls))))))

In the map :args, spec the mapping function as well, then use :fn which can either relate the args and ret of the main function OR relationships between the args, as I'm doing at the very end. The input to :fn is the conformed output of the :args and :ret specs. 

But like I said, there are several problems with this right now and I need to discuss more with Rich whether something like this should be possible (mostly the args fspec is where I'm seeing issues.

[Francesco Bellomi]

I think map is a good example where the different arities have very different semantics, and maybe it would be practical to specify a separate spec for each arity.

In the unified spec, both :args and :ret have to resort to (more or less explicit) unions in order to express the sum of the separate cases, and :fn is complicated in order to disallow the undesired combinations.

Francesco 

[Leon Grapenthin]

Thanks Alex, I was experimenting along similar lines. It might work since the lazy seq is not realized at this point and :f might not have been called. Otherwise the last clause would always hold true or never be called (due to earlier ArityException). It covers more of a test for map itself (how does map invoke the lambda?) than that it spec's maps arguments.

I have tried various other ways like conformers etc. and have the impression that if checking and reporting on correct arity of lambdas before invocation is going to become a concern of spec it would appear that a new function like "arities f -> set of numbers" as a language feature would be truly helpful.

It is certainly going to be interesting where the line to a type system will be drawn. For instance generating and checking (in worst case only a subset of) required types for :f is only possible if one takes samples from the passed colls dynamically which I am not sure spec is capable of or designed for. 

Looking forward to further developments of spec :)