clojure.spec

[Rich Hickey]

Introducing clojure.spec

I'm happy to introduce today clojure.spec, a new core library and support for data and function specifications in Clojure.

Better communication

Clojure is a dynamic language, and thus far we have relied on documentation or external libraries to explain the use and behavior of functions and libraries. But documentation is difficult to produce, is frequently not maintained, cannot be automatically checked and varies greatly in quality. Specs are expressive and precise. Including spec in Clojure creates a lingua franca with which we can state how our programs work and how to use them.

More leverage and power

A key advantage of specifications over documentation is the leverage they provide. In particular, specs can be utilized by programs in ways that docs cannot. Defining specs takes effort, and spec aims to maximize the return you get from making that effort. spec gives you tools for leveraging specs in documentation, validation, error reporting, destructuring, instrumentation, test-data generation and generative testing.

Improved developer experience

Error messages from macros are a perennial challenge for new (and experienced) users of Clojure. specs can be used to conform data in macros instead of using a custom parser. And Clojure's macro expansion will automatically use specs, when present, to explain errors to users. This should result in a greatly improved experience for users when errors occur.

More robust software

Clojure has always been about simplifying the development of robust software. In all languages, dynamic or not, tests are essential to quality - too many critical properties are not captured by common type systems. spec has been designed from the ground up to directly support generative testing via test.check https://github.com/clojure/test.check. When you use spec you get generative tests for free.

Taken together, I think the features of spec demonstrate the ongoing advantages of a powerful dynamic language like Clojure for building robust software - superior expressivity, instrumentation-enhanced REPL-driven development, sophisticated testing and more flexible systems. I encourage you to read the spec rationale and overview http://clojure.org/about/spec. Look for spec's inclusion in the next alpha release of Clojure, within a day or so.

Note that spec is still alpha, and some details are likely to change. Feedback welcome.

I hope you find spec useful and powerful!

Rich

[adrian...@mail.yu.edu]

This looks incredible and it sounds like something which could immediately be put to good use in both hobby and production projects. Excited to test it out when the alpha is available. Thank you!

[Alan Thompson]

Looks great - eagerly awaiting a chance to use it.

Alan

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrey Grin]

 Is it planned to support recursive definitions? Example from.plumatic schema:

 

(def BinaryTree 
  (maybe ;; any empty binary tree is represented by nil
   {:value long 
    :left (recursive #'BinaryTree) 
    :right (recursive #'BinaryTree)}))

[Alex Miller]

Yes, you can create recursive definitions by registering a spec that refers to itself via registered name (a namespaced keyword).

[Sean Corfield]

Awesome! I know one company that’s going to jump straight on 1.9.0-alpha1! ☺

Heck, we may even start today with 1.9.0-master-SNAPSHOT since clojure.spec was committed a couple of hours ago ☺

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

[Nicola Mometto]

Some minor feedback while reading through `clojure.spec`:

`clojure.spec/->sym` has been implemented in a significant number of libraries already (see: https://crossclj.info/clojure/var->sym.html), I propose we include this in `clojure.core` under the name of `var->sym`

[scott stackelhouse]

Could someone describe what a spec of a seq in a seq would look like?  i.e. ['("a" "b" "c") '("d" "e" "f")].  I'm not quite "getting it."

--Scott

[Alex Miller]

Yeah, nested regexes are one of the places people are most likely to be tripped up. 

One of those inner lists could be speced as:

(def string-list (s/* string?)) 

;; or as (s/coll-of string?) - not a strong preference in this particular case but there are tradeoffs

And then the outer list is something like this:

(def outer (s/* (s/spec string-list))

where the s/spec is the key thing - that creates a new "sequential context". Otherwise, the string-list regex ops become part of the outer regex ops.

One difference here will be that the spec above will conform the string-list to a vector (as all sequential things conform to a vector). The coll-of approach would give you control over that though:

(s/conform (s/* (s/coll-of string? ())) ['("a" "b" "c") '("d" "e" "f")])

=> [("a" "b" "c") ("d" "e" "f")]

The benefit of using s/* in string-list is that if you wanted to include string-list inside another regex you could but with coll-of, it would always start a new collection:

(s/conform (s/cat :num integer? :strs string-list) [100 "a" "b"])

=> {:num 100, :strs ["a" "b"]}

So, tradeoffs.

[Alex Miller]

That schema looks weird to me as it seems to under-constrain leaf vs branch so this is not really equivalent but makes sense to me:

(spec/def ::tree (spec/or :leaf ::leaf :branch ::branch))

(spec/def ::leaf integer?)

(spec/def ::branch (spec/cat :left ::tree :right ::tree))

(spec/conform ::tree [[1 2] [3 4]])

=> [:branch {:left [:branch {:left [:leaf 1], :right [:leaf 2]}], :right [:branch {:left [:leaf 3], :right [:leaf 4]}]}]

Because you register specs in the registry by keyword that gives you the point of indirection to do recursive structures.

Generate some trees!

(gen/sample (spec/gen ::tree) 5)

=> (-1 -1 (0 0) (0 2) (-1 -1))

(gen/sample (spec/gen ::tree) 5)

=> (((-1 0) 0) 0 1 (0 1) (1 -3))

[scott stackelhouse]

Awesome, thanks!  I had misunderstood where the idea of a sequence was created (the sequential context as you put it).

[Andrey Grin]

Thanks, Alex. I've taken random example from plumatic.schema wiki to quickly illustrate the question. In my actual use case I dynamically generate schemas based on hundreds of model Java classes (beans) with mutual references so I need recursion (and also some way to model inheritance, that was is achieved with "conditional" in plumatic). Will try to implement the same with spec.

[Ivan Reese]

Is there anywhere we can read anything about the design process behind clojure.spec? I took a look at dev.clojure.org / JIRA, but I haven't yet found anything on the topic.

[Alex Miller]

clojure.spec has been developed internally for the last couple months. It combines a lot of different threads of thought. What are looking for?

There will be some more usage docs coming soon.

[Alex Miller]

I think you could apply the same idea - use a regular translation between Java class and registered name and then always refer to the registered name.

There are probably multiple approaches to modeling inheritance, so hard to recommend something without knowing more. In general, attributes coming from super classes can be modeled using the parent attribute name. And I would recommend looking into multi-spec for doing validation of a number of "types" that could be flowing into the same location by leveraging a multimethod that conditionally provides the spec to use.

[Rich Hickey]

I did most of the design of spec in a (paper) notebook.

The rationale tries to capture the salient driving forces.

If there is a specific question you have I’d be happy to answer.

Rich

> On May 23, 2016, at 4:11 PM, Ivan Reese <ivan...@gmail.com> wrote:
>
> Is there anywhere we can read anything about the design process behind clojure.spec? I took a look at dev.clojure.org / JIRA, but I haven't yet found anything on the topic.
>

[Andrey Grin]

Thank you. Yes, it seems that for recursion I can use the same approach. As for inheritance currently I just include all parent attributes in every child schema. As for spec definition multi-spec, as I understand it, requires "defmethod" for every class which is probably a problem for my case since all inheritance hierarchies are also determined at run-time. I will need to find some approach that is both dynamic and supports precise error messages in case validation fails.

[Ambrose Bonnaire-Sergeant]

Thanks Rich+team, this is awesome.

Instrumented vars via `fdef` do not seem to add :doc metadata yet

(which is advertised in the docstring for `fdef`).

Am I missing something?

Thanks,

Ambrose

[Alex Miller]

You can dynamically install defmethods at runtime too so you could catch this problem when it falls into the default, generate the schema, register it, then re-invoke. Theoretically.

[Rich Hickey]

fdef will not add doc metadata (see rationale re: not putting more stuff in the namespaces/vars), but specs will be present when you call ‘doc’. That doc enhancement was in a push later in the afternoon.

https://github.com/clojure/clojure/commit/4c8efbc42efa22ec1d08a1e9fa5dd25db99766a9

[Ambrose Bonnaire-Sergeant]

I see, thanks.

[Ambrose Bonnaire-Sergeant]

I'm having trouble calling `s/gen`, might be some sort of AOT compilation error.

(I'm guessing this line has something to do with it).

I'm using the latest master-SNAPSHOT.

(ns gen-load.core

  (:require [clojure.spec :as s]))

(s/gen integer?)

;CompilerException java.lang.NoClassDefFoundError: clojure/spec/gen$gen_for_pred, compiling:(gen_load/core.clj:4:1) 

Thanks,

Ambrose

[Nicola Mometto]

Possibly CLJ-1544 related?

[Nicola Mometto]

Looks like it is, in the meanwhile this patch should fix it: http://sprunge.us/XTiA

[Shaun Parker]

Thanks to everyone who worked on it! I'm excited there's finally an idiomatic way and can't wait to use it.

Is there any chance the three clojure.spec namespaces will be released as a library that could be use until 1.9.0 is released? It seems like we'd only be missing compile time macro checking/reporting and the updated clojure.repl/doc unless I'm missing something. It'd be nice to start gradually using everything else now.

[Ambrose Bonnaire-Sergeant]

I'm observing mutually recursive regex ops taking a long time to generate

test.check generators. Is this expected?

(s/def ::a (s/nilable (s/cat :a ::a

                             :b ::b

                             :c ::c)))

(s/def ::b (s/nilable (s/cat :a ::a

                             :b ::b

                             :c ::c)))

(s/def ::c (s/nilable (s/cat :a ::a

                             :b ::b

                             :c ::c)))

(time (s/gen ::a))

;"Elapsed time: 3993.431793 msecs"

nil

Thanks,

Ambrose

[Sean Corfield]

On 5/23/16, 2:29 PM, "Rich Hickey" <clo...@googlegroups.com on behalf of richh...@gmail.com> wrote:
>fdef will not add doc metadata (see rationale re: not putting more stuff in the namespaces/vars), but specs will be present when you call ‘doc’. That doc enhancement was in a push later in the afternoon.

Nice:

(require '[clojure.spec :as s])
(s/fdef clojure.core/symbol
:args (s/alt :separate (s/cat :ns string? :n string?)
:str string?
:sym symbol?)
:ret symbol?)

(doc symbol)
-------------------------
clojure.core/symbol
([name] [ns name])
Returns a Symbol with the given namespace and name.
Spec
args: (alt :separate (cat :ns string? :n string?) :str string? :sym symbol?)
ret: symbol?

Looks like a small bug in explain tho’ – is it worth filing a JIRA bug yet or wait until the Alpha?

(s/instrument-all)

(symbol 1) ;; this is fine…

clojure.lang.ExceptionInfo: Call to #'clojure.core/symbol did not conform to spec:
At: [:args :separate :ns] val: 1 fails predicate: string?
At: [:args :str] val: 1 fails predicate: string?
At: [:args :sym] val: 1 fails predicate: symbol?
:clojure.spec/args (1)

(symbol "a" :a) ;; this should explain that :a fails predicate: string?

java.lang.IllegalArgumentException: No matching clause: :clojure.spec/accept
clojure.spec/op-explain/invokeStatic spec.clj: 1196
clojure.spec/op-explain/fn spec.clj: 1192
clojure.core/map/fn core.clj: 2657
...
clojure.core/apply/invokeStatic core.clj: 646
clojure.spec/op-explain/invokeStatic spec.clj: 1196
clojure.spec/re-explain/invokeStatic spec.clj: 1259
clojure.spec/regex-spec-impl/reify/explain* spec.clj: 1282
clojure.spec/explain-data*/invokeStatic spec.clj: 143
clojure.spec/spec-checking-fn/conform! spec.clj: 520
clojure.spec/spec-checking-fn/fn spec.clj: 532

[George Singer]

Rich Hickey <richhickey <at> gmail.com> writes:

>
> I did most of the design of spec in a (paper) notebook.
>
> The rationale tries to capture the salient driving forces.
>
> If there is a specific question you have I’d be happy to answer.
>
> Rich
>

> > On May 23, 2016, at 4:11 PM, Ivan Reese <ivanreese <at> gmail.com>

wrote:
> >
> > Is there anywhere we can read anything about the design process behind
clojure.spec? I took a look at
> dev.clojure.org / JIRA, but I haven't yet found anything on the topic.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Clojure" group.

> > To post to this group, send email to clojure <at> googlegroups.com

> > Note that posts from new members are moderated - please be patient with
your first post.
> > To unsubscribe from this group, send email to

> > clojure+unsubscribe <at> googlegroups.com

> > For more options, visit this group at
> > http://groups.google.com/group/clojure?hl=en
> > ---
> > You received this message because you are subscribed to the Google
Groups "Clojure" group.
> > To unsubscribe from this group and stop receiving emails from it, send

an email to clojure+unsubscribe <at> googlegroups.com.

> > For more options, visit https://groups.google.com/d/optout.
>

I'd love to hear Rich give a talk about this (the design process, his
rationale, and so forth) -- especially how it relates to static type
systems. I know that there's write-up already on the clojure website, but
this would be a great talk also.

Can't wait to use this.

[Rich Hickey]

That one’s already fixed if you grab the latest.

[Rich Hickey]

Currently gens are not lazy, so entire tree is generated. This is because a) test.check isn’t lazy either and b) we want errors when asking for gen, not using it. But it is rough for recursive specs as you see.

For now you can limit the recursion depth to control the branching:

(binding [s/*recursion-limit* 3]
(time (s/gen ::a)))

[Sean Corfield]

On 5/23/16, 3:45 PM, "Rich Hickey" <clo...@googlegroups.com on behalf of richh...@gmail.com> wrote:
>That one’s already fixed if you grab the latest.

Confirmed! Thank you!

Sean

[Alan Moore]

Your timing couldn't have been better... we have been needing something like this. Thought of building something similar which would only have been an ad hoc, informally specified, bug ridden, slow implementation of half of clojure.spec.

Thank you for spending you hammock time on this important feature/library. Looking forward to using it.

Alan

[Leif]

Feedback:

The regex quantifiers seem to be greedy.  I suppose that is the standard choice, but it would be nice if the docs explicitly said that.

Maybe have a 'rep' macro for exact quantification?  My use cases for this are short sequences that would be handled just fine by 'cat', so I don't really need it.  But if someone else needs to specify long-ish but fixed-size sequences, or sequences with a specific length range, they should speak up now.

The docs should mention that you need to provide your own test.check dependency.  They do obliquely, but being explicit never hurts.

An example or two under the "Features > Overview > Maps" section would be nice.  The text makes many good points about the design decisions made for map specs, I'd like to see them in action.  (I am skeptical that a short example would convince me to namespace all my damn map keywords, though perhaps you'll lead me down the virtuous path.)

All that said, it looks good and I think I'll have fun trying it out.
--Leif

[Alex Miller]

Hi Leif, thanks for the feedback!

The use for regex ops is primarily expected to be syntax where greedy is typically a fine default (and where the sequence length is typically not long with exact quantification).

You can use the & operator though to add arbitrary predicates around another regex operator:

;; i1 is greedy here

(def nn (s/cat :i1 (s/+ integer?) :i2 (s/+ integer?)))

(s/conform nn [1 2 3 4 5])

=> {:i1 [1 2 3 4], :i2 [5]}

;; use s/& to bound i1 to length < 3

(def nn' (s/cat :i1 (s/& (s/+ integer?) #(< (count %) 3)) :i2 (s/+ integer?)))

(s/conform nn' [1 2 3 4 5])

=> {:i1 [1 2], :i2 [3 4 5]}

that predicate is arbitrary so could set any occurence operator needed there. In practice I have been using spec every day for weeks and I have only occasionally needed anything like this though.

Re the test.check dependency - this will be made more explicit in docs to come. But fair point.

I have written a lengthy guide with more examples which we'll release tomorrow. The rationale was not meant to serve as either tutorial docs or reference material but rather as ... the rationale. :)

On namespaced keywords, there are a couple language changes also on the way to provide namespaced literal maps (for reading and printing) and namespaced key destructuring. I expect these will also help in reducing the typing burden of namespaced keys.  See CLJ-1910 and CLJ-1919. It's likely these will be included in alpha2.

[se...@corfield.org]

I was a bit puzzled by the :req-un / :opt-un stuff in maps. Am I right that there is a requirement here for the keys to be namespaced but the actual namespace is completely irrelevant / ignored?

 

(defn thing [m]

  (+ (:a m) (:b m) (or (:c m) 1)))

(s/fdef thing

        :args (s/cat :map (s/and (s/keys :req-un [:x/a :y/b]

                                         :opt-un [:z/c])

                                 (s/map-of keyword? number?)))

        :ret  number?)

 

I tried several namespace prefixes here and it seemed I can use anything and they don’t even need to be consistent.

 

Wouldn’t it be easier to just allow un-namespaced keywords here? Or is there some aspect of the namespacing here that I’m missing?

 

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org

 

From: Rich Hickey
Sent: Monday, May 23, 2016 7:12 AM
To: clo...@googlegroups.com
Subject: clojure.spec

 

Introducing clojure.spec

 

I'm happy to introduce today clojure.spec, a new core library and support for data and function specifications in Clojure.

 

Better communication

 

Clojure is a dynamic language, and thus far we have relied on documentation or external libraries to explain the use and behavior of functions and libraries. But documentation is difficult to produce, is frequently not maintained, cannot be automatically checked and varies greatly in quality. Specs are expressive and precise. Including spec in Clojure creates a lingua franca with which we can state how our programs work and how to use them.

 

More leverage and power

 

A key advantage of specifications over documentation is the leverage they provide. In particular, specs can be utilized by programs in ways that docs cannot. Defining specs takes effort, and spec aims to maximize the return you get from making that effort. spec gives you tools for leveraging specs in documentation, validation, error reporting, destructuring, instrumentation, test-data generation and generative testing.

 

Improved developer experience

 

Error messages from macros are a perennial challenge for new (and experienced) users of Clojure. specs can be used to conform data in macros instead of using a custom parser. And Clojure's macro expansion will automatically use specs, when present, to explain errors to users. This should result in a greatly improved experience for users when errors occur.

 

More robust software

 

Clojure has always been about simplifying the development of robust software. In all languages, dynamic or not, tests are essential to quality - too many critical properties are not captured by common type systems. spec has been designed from the ground up to directly support generative testing via test.check https://github.com/clojure/test.check. When you use spec you get generative tests for free.

 

Taken together, I think the features of spec demonstrate the ongoing advantages of a powerful dynamic language like Clojure for building robust software - superior expressivity, instrumentation-enhanced REPL-driven development, sophisticated testing and more flexible systems. I encourage you to read the spec rationale and overview  http://clojure.org/about/spec. Look for spec's inclusion in the next alpha release of Clojure, within a day or so.

 

Note that spec is still alpha, and some details are likely to change. Feedback welcome.

 

I hope you find spec useful and powerful!

 

Rich

 

--

[Alex Miller]

On Tuesday, May 24, 2016 at 12:16:32 AM UTC-5, Sean Corfield wrote:

I was a bit puzzled by the :req-un / :opt-un stuff in maps. Am I right that there is a requirement here for the keys to be namespaced but the actual namespace is completely irrelevant / ignored?

No - the :req-un / :opt-un keys are used for two purposes:

1. The namespaced key is used to locate the spec in the registry

2. The unnamespaced key is used to find the value in the map

Once those are found, the spec is used to validate the attribute value.

 

 

(defn thing [m]

  (+ (:a m) (:b m) (or (:c m) 1)))

(s/fdef thing

        :args (s/cat :map (s/and (s/keys :req-un [:x/a :y/b]

                                         :opt-un [:z/c])

                                 (s/map-of keyword? number?)))

        :ret  number?)

 

I tried several namespace prefixes here and it seemed I can use anything and they don’t even need to be consistent.

Right - if no spec is found then nothing happens. But if you registered a spec as :x/a  or :y/b those specs would be used to validate the arg.

clojure+unsubscribe@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/clojure?hl=en

---

You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[se...@corfield.org]

OK, that explains a lot. Thank you. That needs to be clarified on the web page I think…

 

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org

 

clojure+u...@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/clojure?hl=en

---

You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Atamert Ölçgen]

Isn't s redefined in this example:

user=> (require '[clojure.spec :as s])

(s/def ::even? (s/and integer? even?))

(s/def ::odd? (s/and integer? odd?))

(s/def ::a integer?)

(s/def ::b integer?)

(s/def ::c integer?)

(def s (s/cat :forty-two #{42}

              :odds (s/+ ::odd?)

              :m (s/keys :req-un [::a ::b ::c])

              :oes (s/* (s/cat :o ::odd? :e ::even?))

              :ex (s/alt :odd ::odd? :even ::even?)))

user=> (s/conform s [42 11 13 15 {:a 1 :b 2 :c 3} 1 2 3 42 43 44 11])

{:forty-two 42,

 :odds [11 13 15],

 :m {:a 1, :b 2, :c 3},

 :oes [{:o 1, :e 2} {:o 3, :e 42} {:o 43, :e 44}],

 :ex {:odd 11}}

in http://clojure.org/about/spec

Kind Regards,
Atamert Ölçgen

◻◼◻

◻◻◼

◼◼◼

www.muhuk.com

[Alex Miller]

The first use is a namespace alias and the second is a var - they don't overlap in usage.

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Atamert Ölçgen]

Thanks Alex! I didn't know this.

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kind Regards,
Atamert Ölçgen

◻◼◻

◻◻◼

◼◼◼

www.muhuk.com

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Steve Miner]

On May 24, 2016, at 8:10 AM, Alex Miller <al...@puredanger.com> wrote:

The first use is a namespace alias and the second is a var - they don't overlap in usage.

Right, but it’s still a potential source of confusion.  The explanation distracts from the point of the example which is to demonstrate the new spec feature.

[JvJ]

So happy that there's now a standard way to do this!  It sounds like it would help with my latest project.

Unfortunately, my latest project is called "SPECS".... so I guess I'll have to find a different name.

[Rich Hickey]

And now, in the alpha release branching gens *are* lazy, so gen calls on recursive specs are fast.

[scott stackelhouse]

I'm having a problem writing a spec for a map with some required keywords and some optional keywords.  The caveat here is that the optional keywords are all or none... that is they are optional but if one is present they must all be present.

What I tried to write was:

(s/keys :req [::a ::b ::c] :opt [(and ::d ::e ::f)])  

and that fails an assertion.  It appears that the logicals (and, or) are not allowed in the optional section?

Am I thinking about this in the wrong way?  

--Scott

[Rich Hickey]

‘and' and ‘or’ are not currently supported in :opt

[Ambrose Bonnaire-Sergeant]

Thanks Rich. Now it seems test.check is eager to generate very large individual samples, which sounds

like a different problem. 

(s/def ::a (s/nilable (s/cat :a ::a

                             :b ::b

                             :c ::c)))

(s/def ::b (s/nilable (s/cat :a ::a

                             :b ::b

                             :c ::c)))

(s/def ::c (s/nilable (s/cat :a ::a

                             :b ::b

                             :c ::c)))

(time

  (count

    (binding [s/*recursion-limit* 2]

      (gen/sample (s/gen ::a) 3))))

;"Elapsed time: 50106.721779 msecs"

3

Thanks,

Ambrose

[scott stackelhouse]

Ok.  

Thanks all who have worked on this, btw.  It is incredibly timely for me and is already great help for a work project.

--Scott

[scott stackelhouse]

I restructured my data to make this section an optional sub-map, which I think is actually better anyway.

[Rich Hickey]

Fixed on master, thanks for the report.

[Stan Dyck]

I appreciate the wisdom of separating the existence of a key from the
conformance of the values of that key.

I wonder though if there is a way to specify co-constraints; a situation
where the conformance of the value of one key depends on the existence
of and perhaps the value of another key.

Say for example, I want to specify a ::content-type key and a ::body
key. If the ::content-type key exists and has a value of "image/png" I
want to make sure the ::body key also exists and satisfies my image?
predicate.

It looks like multi-specs get me to the neighborhood of a solution here
but I'm not sure they quite cover it.


Thanks,

StanD.

[Elliot]

Super super excited for this feature, thanks so much for creating this.

In the runtime-validation case, the guide mentions:

1. Calling `valid?` in a precondition

2. Calling `conform` in the fn implementation

However neither of these appear to use the `fdef`/`instrument` combo, which seems the closest to "type annotating" the function.  Would you ever expect to use fdef/instrument active in production for validation, or is that a misunderstanding of its use?

Thanks!

- E

[Kevin Corcoran]

Would it be a reasonable feature request to ask for Java classes to be implicitly converted to specs, the same as predicates?  FWIW, plumatic/schema allows classes to be used as schemas.

The spec guide contains this example:

(import java.util.Date) (s/valid? #(instance? Date %) (Date.)) ;; true

... and then, later, defines:

(s/def ::date #(instance? Date %))

If classes were implicitly converted to specs, ::date would be unnecessary, and the first example could be simplified to:

(import java.util.Date) (s/valid? Date (Date.)) ;; true

I do a lot of inter-op and this seems like it would be really convenient.

[Christophe Grand]

Speaking of recursive definitions: how to provide an override in a recursive spec?
Would it be possible to have path suffixes (in addition to paths not in replacement of) in the override map?

Thanks,

Christophe

On Mon, May 23, 2016 at 6:16 PM, Alex Miller <al...@puredanger.com> wrote:

Yes, you can create recursive definitions by registering a spec that refers to itself via registered name (a namespaced keyword).

On Monday, May 23, 2016 at 11:13:24 AM UTC-5, Andrey Grin wrote:

 Is it planned to support recursive definitions? Example from.plumatic schema:

 

(def BinaryTree 
  (maybe ;; any empty binary tree is represented by nil
   {:value long 
    :left (recursive #'BinaryTree) 
    :right (recursive #'BinaryTree)}))

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

On Clojure http://clj-me.cgrand.net/
Clojure Programming http://clojurebook.com
Training, Consulting & Contracting http://lambdanext.eu/

[Rich Hickey]

Once you are talking about some cross-key predicate you wish to satisfy you are talking about another property of the map itself. So, you can use composition to provide further constraints on the map:

(s/and (s/keys …) cross-cutting-pred)

keys is not going to be the place for that kind of cross-cutting logic, although it does have support for some keyset-membership logic (and/or in the :req set).

multi-specs are more for when you have an open set of possibilities. s/or is to multi-spec as c.c.cond is to multimethods, closed vs open dispatch.

[Rich Hickey]

> Would you ever expect to use fdef/instrument active in production for validation

No, definitely not. It’s that kind of runtime checking (and expense) that gives some dynamic lang checking systems a bad rep.

The philosophy is - generative testing has made sure your function complies with the specs. So, testing the :ret and :fn properties over and over is redundant and serves no point.

OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform *explicitly* to do so.

The intent is that running with wrappers (instrumentation) should be done only during testing.

[Gary Trakhman]

Is there a public way to get the registry's spec at a keyword?  I can envision other libs being built around the spec registry, and I'm trying to write a small one (coercions) to see if I can use spec for this use-case.  Is that non-sanctioned kind of thing going forward?

The likely candidate seems 'specize'.

[Gary Trakhman]

I answered my own question in the specific case, this seems to work:

(defn to-int

  [x]

  (try

    (Long/parseLong x)

    (catch Exception _

        nil)))

(s/def ::intable (s/conformer to-int))

(s/conform (s/cat :ints ::intable) ["5"])

> {:ints 5}

[Alex Miller]

You can get the registry with (s/registry), which is just a map of spec names (namespaced keywords) to the spec. Is that what you're looking for?

If you have registered (s/def ::foo string?)

then you can find the spec with (::foo (s/registry)) 

[Alex Miller]

As mentioned in the guide, while conformers can be used to implement coercions, that is something you should think about very carefully. When you register a spec with a conformer, you are making a choice for all possible future consumers of that spec as to how they are getting the conformed data back. That is a lot to "bake in" so tread very carefully, especially if you expect to share that spec with others.

[Elliot]

On Wednesday, May 25, 2016 at 5:30:03 AM UTC-7, Rich Hickey wrote:

The philosophy is - generative testing has made sure your function complies with the specs. So, testing the :ret and :fn properties over and over is redundant and serves no point.

OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform *explicitly* to do so.

Ah, that totally makes sense -- only the :arg validation and not the :ret/:fn properties are relevant to check on every invocation at runtime (rather than test time).  Thank you!

[kovas boguta]

On Wed, May 25, 2016 at 8:29 AM, Rich Hickey <richh...@gmail.com> wrote:

> Would you ever expect to use fdef/instrument active in production for validation

No, definitely not. It’s that kind of runtime checking (and expense) that gives some dynamic lang checking systems a bad rep.

The philosophy is - generative testing has made sure your function complies with the specs. So, testing the :ret and :fn properties over and over is redundant and serves no point.

OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform *explicitly* to do so.

The intent is that running with wrappers (instrumentation) should be done only during testing.

This seems like an important point, that didn't really come through (for me at least) in the docs so far. I was wondering about the runtime perf implications and what the expected usage patterns would be. 

Perhaps something worth emphasizing to the community. 

 

[Alex Miller]

Hopefully this will be made more clear in another guide about generators and testing. As soon as we have time to write it. :)

[JvJ]

Any plans for cljs support?

[Alex Miller]

Coming.

[Mars0i]

I'm very happy about clojure.spec.  I think. (!)

Two suggestions for the documentation page.  Just my two cents.

First, it would be helpful to begin with a description of what clojure.spec does and how it will be used, and one or two brief examples right at the beginning--ideally before the first or second page scroll.  The current "Rationale and Overview" page starts with a very long rationale section ("Problems", "Objectives", "Guidelines"), followed by a long and detailed "Features" section.  Both of these are needed, but throughout the rationale material, I was trying to infer what it was that clojure.spec really was, and then when I got to "Features", I still was trying to get a general sense of how clojure.spec supposed to be used before wading through an informal specification of its details.  No doubt I can study "Features" and synthesize my own general understanding of clojure.spec, but I don't think that should be necessary.  (When I say, "how it is used", I don't mean how it will interact with clojure.test.  That's important but secondary.  I need something more fundamental.)  The many postings in this thread show that many people already have the idea.  I assume that this is either because they've already worked with something similar, or did have the time to closely study the documentation and synthesize their own understanding, or maybe they're just smarter than I am. :-)

2. This is just confusing:

Communication

Species - appearance, form, sort, kind, equivalent to spec (ere) to look, regard                + -iēs abstract noun suffix Specify - species + -ficus -fic (make)

A specification is about how something 'looks', but is, most importantly, something that is looked at. Specs should be readable, composed of 'words' (predicate functions) programmers are already using, and integrated in documentation.

I gather that the indented part is excerpted from a dictionary entry for the word "Species", but at first I thought it was an example or a part of the specification of clojure.spec.  Just wasn't sure.  Coming as it does in a section called "Communication", and followed by text that includes "Specs should be readable"--when the "example" isn't readable, it's very confusing.  Cute, maybe, once you understand it, but unhelpful for someone who is trying to learn what clojure.spec is.


Thanks!  I still am not entirely clear about how clojure.spec should be used, but from what I can tell, it's very good thing and I'm very happy about it.

I was a participant in a few long discussions about improving Clojure docstrings, and I gather that clojure.spec might supplement or end up playing a role in modifying docstrings to provide succinct, clear descriptions of expected arguments, with potentially systematic display formatting, all of which would be great.  The other uses of clojure.spec, at least as I understand them, also seem extremely useful.

[JvJ]

Approximate ETA for this, if known?

[Rich Hickey]

user=> (s/def ::a (s/or :r (s/cat :a ::a)
:k keyword?))
:user/a

user=> (binding [s/*recursion-limit* 2]
(map first (s/exercise ::a 10)))

(((:B)) :? (:+/K) (:Xc_D.__.+HC/JaCD) ((:*3)) :gJ1z.o.+?.lC0/!-ZDN9 :D.-?I.q8.z/-5* (:F67jy+2M.bB_.h62Cp+?._X?b6gv4.x+7.Gz_6.v9Tt15/*) :!4J??+/-8?8_- ((:*JZAg**x!.qE3-.sh._?e/_!?T)))

user=> (binding [s/*recursion-limit* 2]
(map first (s/exercise ::a 10 {[:k] (gen/return :k)
[:r :a :k] (gen/return :a)
[:r :a :r :a :k] (gen/return :aa)})))

(:k (:a) ((:aa)) :k :k :k :k :k (:a) :k)

I’ll have to think about the suffixes

[Brent Millare]

What's the difference between clojure.spec/or and clojure.spec/alt? They seem to accept the same inputs, multiple keyword-predicate pairs, and output tagged values when used with clojure.spec/conform. Is clojure.spec/or usable within a regular expression?

[Gary Trakhman]

It seems like the :req impls should go further to disallow trash input, it's not immediately clear that they only allow namespaced keywords.

For example:

> (s/valid? (s/keys :req ["a"]) {:a 5})

true

> (s/valid? (s/keys :req [5]) {:a 5})

true

What ends up happening in practice, is (filter keyword? (flatten req)), which is suspect.  

Could/should spec be used to check itself? That's the lisp way, after all.  To do so here, we might need more map-checking builtins than s/keys.

On Wed, May 25, 2016 at 1:48 PM Brent Millare <brent....@gmail.com> wrote:

What's the difference between clojure.spec/or and clojure.spec/alt? They seem to accept the same inputs, multiple keyword-predicate pairs, and output tagged values when used with clojure.spec/conform. Is clojure.spec/or usable within a regular expression?

--

[Alex Miller]

I do actually have specs for spec and it would catch these. :)  You can try it yourself though:

(s/def ::s/req (s/* keyword?))

(s/def ::s/req-un (s/* keyword?))

(s/def ::s/opt (s/* keyword?))

(s/def ::s/opt-un (s/* keyword?))

(s/fdef s/keys

  :args (s/cat :form ::s/any :env ::s/any    ;; form/env unnecessary as of 1.9.0-alpha3

               :opts (s/keys* :opt-un [::s/req ::s/req-un ::s/opt ::s/opt-un ::s/gen]))

  :ret ::s/any)

(s/instrument #'clojure.spec/keys)

user=> (s/keys :req ["a"])

CompilerException java.lang.IllegalArgumentException: Call to clojure.spec/keys did not conform to spec:

In: [:req 0] val: "a" fails spec: :clojure.spec/req at: [:args :opts :req] predicate: keyword?

:clojure.spec/args  ((s/keys :req ["a"]) nil :req ["a"])

, compiling:(NO_SOURCE_PATH:26:1)

user=> (s/keys :req [5])

CompilerException java.lang.IllegalArgumentException: Call to clojure.spec/keys did not conform to spec:

In: [:req 0] val: 5 fails spec: :clojure.spec/req at: [:args :opts :req] predicate: keyword?

:clojure.spec/args  ((s/keys :req [5]) nil :req [5])

, compiling:(NO_SOURCE_PATH:27:1)

[Alex Miller]

s/or creates a spec and s/alt creates a regex op. Regex ops can be combined to describe a single sequence. Use s/alt if you are matching alternative regex expressions in a sequential context - in particular where the branches of the alt are themselves regex ops. Use s/or when matching a single item. There are indeed cases where you could use one or the other and even yield the same result, but they have different affordances and intent when used in combination with other things.

This is an example where s/alt decides between nested regex ops (and s/or won't work here):

user=> (s/def ::s (s/* (s/alt :key-pair (s/cat :a keyword? :b keyword?)

                              :nums (s/+ number?))))

:user/s

user=> (s/conform ::s [:a :b 1 2 3 :c :d :e :f 4 5])

[[:key-pair {:a :a, :b :b}] [:nums [1 2 3]] [:key-pair {:a :c, :b :d}] [:key-pair {:a :e, :b :f}] [:nums [4 5]]]

If we tried this with an s/or, it would try to match a new regular expression under the or and fail:

user=> (s/def ::s' (s/* (s/or :key-pair (s/cat :a keyword? :b keyword?)

                              :nums (s/+ number?))))

:user/s'

user=> (s/conform ::s' [:a :b 1 2 3 :c :d :e :f 4 5])

:clojure.spec/invalid

And you would instead match something like this:

user=> (s/conform ::s' [[:a :b] [1 2 3] [:c :d] [:e :f] [4 5]])

[[:key-pair {:a :a, :b :b}] [:nums [1 2 3]] [:key-pair {:a :c, :b :d}] [:key-pair {:a :e, :b :f}] [:nums [4 5]]]

[Alex Miller]

I forgot to mention that if you run this with alpha2, this example won't work due to a bug, but that is fixed in master for alpha3.

[Ambrose Bonnaire-Sergeant]

Rich,

Can you talk about the design process behind fspec?

What tradeoffs were in mind for fspec performing gen testing rather 

than a traditional function contract wrapper a la racket/contract?

Thanks,

Ambrose

On Mon, May 23, 2016 at 5:20 PM, Rich Hickey <richh...@gmail.com> wrote:

I did most of the design of spec in a (paper) notebook.

The rationale tries to capture the salient driving forces.

If there is a specific question you have I’d be happy to answer.

Rich

> On May 23, 2016, at 4:11 PM, Ivan Reese <ivan...@gmail.com> wrote:
>
> Is there anywhere we can read anything about the design process behind clojure.spec? I took a look at dev.clojure.org / JIRA, but I haven't yet found anything on the topic.

[Rich Hickey]

In my mind, fspec is no different than fdef.

With fdef you specify what a function does. How do we know it works? - generative testing. Wrappers only check individual calls.

Now you say your function returns a fn, and provides an fspec (same as fdef, a trio of specs) for that. How do we know the returned fn (which is often implemented anonymously, not a named thing we could fdef) works? It’s not a different question so it has the same answer - generative testing.

spec is about making sure your programs _work_ *ahead of time* by using generative testing, and leaving only application-domain required checks active at runtime (i.e. checking your program would otherwise do manually).

Contracts are about making sure your _broken programs fail_ as soon as possible at *(production?) runtime*, and do so at considerable runtime cost. If you only activate them during testing then quality depends the coverage and ranges of your test suite (usually poor).

spec is not a contract system.

Rich

[Rich Hickey]

To further clarify, you can and should turn on spec’s instrumentation *during testing* (and interactive development) to get contract-system style inter-function call checking. To the extent those tests are themselves generative (e.g. spec’s) the coverage and ranges will be good.

As to whether an fn-returning function, *when instrumented*, should return instrumentation-wrapped fns, I don’t know. I’d be concerned about tracking them lest you can never unstrument them.

[Okke Tijhuis]

Really great work. I am curious about a couple of things though.

One is the choice of names like s/*, s/+ and s/?. Is there a specific reason they were chosen? They aren't self-explanatory. It's not automatically clear that because they are in the clojure.spec namespace they resemble regex equivalents. And even then you have to know the regex meaning. Though I'm very familiar with regular expressions I find myself have to make a mental switch each time I come across one of those names. Especially because they don't appear inside a regular expression. Something like s/one-or-more would hardly need any explanation and be much easier to read.

Another thing I'm wondering about is if tools like Cursive/Cider will be easily able to determine if there's a spec. In quite a few cases it's hard to tell what arguments are expected, what a function returns or things like that. If I'm not mistaken you declare your specs elsewhere, so when browsing code you're unfamiliar with that information isn't directly available to you. It would be extremely nice to be able to have the spec optionally shown somewhere inline for example.

Something like this maybe:

(defn some-fn [:spec-ns/name n :spec-ns/age a]

    .....) => :spec-ns/result

[Brent Millare]

Thanks Alex, perfect explanatory example

[Leon Grapenthin]

Just had a chance to play around with spec. Looks like this is going to destroy a lot of problem space :) Thanks.

Probably a bug:

      (s/exercise

        (s/and

         set?

         (s/coll-of

          (s/with-gen keyword?

            #(gen/elements [:s1 :s2 :s3]))

          #{})))

;->

([#{} #{}] [#{} #{}] [#{} #{}] [#{:o-85:1ywl} #{:o-85:1ywl}] [#{} #{}] [#{} #{}] [#{} #{}] [#{:_Qi.Qj?dtMZh_s*3.x.sTxm9-E.NHr!?b5f0Ir2.u.+bof*-P.r.m_y**e0ntq.W+*.+?Urxe+Xp+/Q} #{:_Qi.Qj?dtMZh_s*3.x.sTxm9-E.NHr!?b5f0Ir2.u.+bof*-P.r.m_y**e0ntq.W+*.+?Urxe+Xp+/Q}] [#{} #{}] [#{} #{}])

Reverse argument order to s/and and it works.

[Rich Hickey]

The first predicate determines the generator.

[jeaye]

On Wed, May 25, 2016 at 05:38:53PM -0700, Daniel wrote:
> I'd love to see a blog which shows an example app leveraging core.typed,
> clojure.spec, and core.contracts to gradually provide greater confidence in
> a codebase. Sort of a step-by-step.

Agreed, that'd be great. clojure.spec is documented on its own, as are
the others, but tying them together is how we're going to get best
coverage.

[Daniel]

I'd love to see a blog which shows an example app leveraging core.typed, clojure.spec, and core.contracts to gradually provide greater confidence in a codebase.  Sort of a step-by-step with best practices.

[lvh]

Hi,


Is there a way to get a separate “repo” for specs to use with s/def? Use case: I have optional modules that are enabled/disabled at startup time. An input should only be valid if the plugin that defines that spec/behavior is included. I guess maybe I could move this to compile time by excluding/including certain namespaces, but that seems gauche… I know I can just use the spec literally, but having the s/def is convenient.


lvh

[Mars0i]

Upon further investigation, there are helpful examples on this page http://clojure.org/guides/spec, which was mentioned in the 1.9.0-alpha1 release notice, and there's a small example near the end of the Rationale and Overview page.  I still feel as if both pages kind of start in the middle rather than at the beginning by going over the guts before their purpose, but the guides page gets to the point fairly quickly.  I still think the "species" etymology and def is unhelpful, even though in some sense it's just the sort of thing I would like in a different context. 

Thanks again, in any event.

[Christophe Grand]

Thank you Rich, here is why I wasn't at ease with generating all paths:

*recursion-limit* is documented as being a soft limit, so I don't know even a lower bound or when to stop generating paths

Suffix match wouldn't be a panacea, it's just a (bad) quickfix.
Paths form a regular language so regexes as keys in the override map would be a better solution.

It would allow overriding a spec based on its context (any ancestor) while suffix match would only allow overriding a spec based on its closer ancestors (so one would still need to generate paths or suffixes to emulate that).

Christophe

[vandr0iy]

I'm reading clojure.spec documentation, and I want to know more about
how and where a similar module is to be used.
What is, say, a java equivalent to this functionality? Any examples in
some existing project?

Thank you in advance

[Rich Hickey]

Well, I think it depends on what you want to accomplish. For instance, there’s no saying that the best way to generate a custom version of a recursive spec is via overrides of its constituent parts. You could supply a new root level spec that generated compatible instances in a completely different (fixed or recursive) way.

As far as regex for overrides - they would have great expressivity but the tradeoffs would be a) having to exhaustively check them, and b) they are not guaranteed to uniquely match.

I have been thinking about allowing overrides of (registered) spec *names* in addition to (and alongside) paths. These overrides would cut across any specs, wherever the named spec was used.

[Wesley Hall]

spec is not a contract system.

Forgive me for I am about to sin :). 

I have a little RPC framework that I use to do simple remoting between clojurescript in the browser and ring based web services. I'm currently using schema to validate arguments received from clients and return appropriate exceptions upon non-conforming invocations. 

The idea of being able to perform generative testing against a specification for these functions is really appealing but if I am using generative testing to verify that my functions behave properly if invoked as intended it does feel like there would be some benefit to ensuring that the conditions under which the function has been tested are enforced at runtime for those functions on the edges of my API. 

I'd definitely prefer a manual conformity check over instrumentation in these cases, but it seems like an fspec cannot be used for this purpose (from within the function itself). I'd rather not define my specs twice. 

Seems like I might be destined to make cheeky instrument calls after each of these edge functions, in the same was the always-validate metadata is used in schema. 

Do I have a desperate need to be convinced otherwise? :)

[Rich Hickey]

If you name (register) your (sub)specs with s/def and you can reuse them as much as you like.

(s/def ::argi (s/cat :i integer?))
(s/def ::fnii (s/fspec :args ::argi :ret integer?))
(s/conform ::fnii +)
(s/valid? ::argi '(42))

However you are talking about calling ‘instrument’ so I don’t think you are in the HOF case. So you shouldn’t be using fspec but fdef:

(s/fdef fooi :args (s/cat :i integer?) :ret integer?)

(defn fooi [i]
(let [spec (-> `fooi s/fn-specs :args)]
(assert (s/valid? spec (list i)) (s/explain-str spec (list i))))
42)

(fooi "42")
user=> AssertionError Assert failed: In: [0] val: "42" fails at: [:i] predicate: integer?

Obviously some macrology could make this more succinct, as is being discussed elsewhere.

[Wesley Hall]

Thanks Rich, for this and your work in general. After 15 years of working with Java, it has been a real joy to find clojure (let's face it, that pun alone is pure gold!). 

I might try my hand at the macrology you describe as an exercise... everybody stand well back....

[Beau Fabry]

I'm in a similar position to you Wesley, I'm all for generative testing, but sometimes I prefer not to spec out the full depth of the tree (ie some input leaves with s/Any) and just turn on validation in staging and see what happens. The cpu-time wasted there doesn't matter much to me. Looking forward to seeing where things end up. Spec certainly seems really well thought through.

Aside: Great work on the docs Alex. Much much more comprehensive and readable than is usually available for an alpha feature in *any* language.

[Pedro Santos]

Hello,

How can I differentiate between the same keyword, but with difference semantics? Example:

{:attack 100

 :bonus {:attack {:category {:heavy 100}}}}

I have:

(s/def ::attack (s/and pos? #(<= 1 1000))

(s/def ::bonus (s/keys :req-un [::attack ; <--- 

Is there a way to do this?

--

Another question: is there a way to spec a fn that handles channels? Something like:

(s/fdef some-fn

 :args (s/cat :ch (s/chan ::unq/model))

 :ret (s/chan :unq/model))

--

And yet another: I have a game project with some test.check scenarios. The test run takes 8s without instrumenting and 22s with instrumentation, and this makes me think on starting to split tests in test suites. Is there something to handle this in clojure.test?

Thanks!

[Beau Fabry]

You just need to come up with qualified names for them:

user=> (require '[clojure.spec :as s])

nil

user=> (s/def :base/attack (s/and integer? pos? #(<= % 1000)))

:base/attack

user=> (s/def :base/bonus (s/keys :req-un [:bonus/category]))

:base/bonus

user=> (s/valid? :base/event {:attack 100 :bonus {:category {:heavy 100}}})

true

Not sure about channels.

[dom...@juxt.pro]

> OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform *explicitly* to do so.

With regards to this use-case, I was wondering if there were any plans for spec to allow for additional information to be passed to the predicates in some way?

A practical example for my question is user registration, I need to check that the new user's email isn't already taken by looking in the database.

The methods I've come up with for working with spec are:
- To bind a dynamic variable (*db* for example) containing my database reference, and have the predicate use that for looking up the email value.
    - This loses some locality in my code, and generally suggests a "bad code smell" in my experience.
- Update the user supplied data to include a reference to the db
    - I feel like this complects between the data being validated, with the information required to perform a validation.

Any suggestions for how I can utilize spec for this scenario?

Dominic

On Wednesday, 25 May 2016 13:30:03 UTC+1, Rich Hickey wrote:

> Would you ever expect to use fdef/instrument active in production for validation

No, definitely not. It’s that kind of runtime checking (and expense) that gives some dynamic lang checking systems a bad rep.

The philosophy is - generative testing has made sure your function complies with the specs. So, testing the :ret and :fn properties over and over is redundant and serves no point.



The intent is that running with wrappers (instrumentation) should be done only during testing.

> On May 24, 2016, at 7:43 PM, Elliot <ell...@deck36.net> wrote:
>
> Super super excited for this feature, thanks so much for creating this.
>
> In the runtime-validation case, the guide mentions:
>
> 1. Calling `valid?` in a precondition
> 2. Calling `conform` in the fn implementation
>
> However neither of these appear to use the `fdef`/`instrument` combo, which seems the closest to "type annotating" the function.  Would you ever expect to use fdef/instrument active in production for validation, or is that a misunderstanding of its use?
>
> Thanks!
>
> - E
>
>
> On Tuesday, May 24, 2016 at 11:12:59 AM UTC-7, scott stackelhouse wrote:
> I restructured my data to make this section an optional sub-map, which I think is actually better anyway.
>
> On Tuesday, May 24, 2016 at 11:08:27 AM UTC-7, scott stackelhouse wrote:
> Ok.  
>
> Thanks all who have worked on this, btw.  It is incredibly timely for me and is already great help for a work project.
>
> --Scott
>
> On Tuesday, May 24, 2016 at 10:57:26 AM UTC-7, Rich Hickey wrote:
> ‘and' and ‘or’ are not currently supported in :opt
>
>
> > On May 24, 2016, at 1:45 PM, scott stackelhouse <scott.sta...@gmail.com> wrote:
> >
> > I'm having a problem writing a spec for a map with some required keywords and some optional keywords.  The caveat here is that the optional keywords are all or none... that is they are optional but if one is present they must all be present.
> >
> > What I tried to write was:
> >
> > (s/keys :req [::a ::b ::c] :opt [(and ::d ::e ::f)])  
> >
> > and that fails an assertion.  It appears that the logicals (and, or) are not allowed in the optional section?
> >
> > Am I thinking about this in the wrong way?  
> >
> > --Scott

[Alex Miller]

On Friday, May 27, 2016 at 2:36:56 AM UTC-5, Pedro Pereira Santos wrote:

Hello,

How can I differentiate between the same keyword, but with difference semantics? Example:

If you have different semantics, you should use different keywords (either different name or namespace).

 

{:attack 100

 :bonus {:attack {:category {:heavy 100}}}}

I have:

(s/def ::attack (s/and pos? #(<= 1 1000))

(s/def ::bonus (s/keys :req-un [::attack ; <--- 

Is there a way to do this?

--

Another question: is there a way to spec a fn that handles channels? Something like:

Nothing right now but we've talked about this some. You have to keep in mind that channels also can have a transducer which means the read and write ends of a channel might have different specs. Tim Baldridge pointed out that you could use a map transducer that checked s/valid? on it's inputs - that could be used at either the beginning and/or end of a transducer chain. That's pretty brute force though and would add a lot of overhead.

 

(s/fdef some-fn

 :args (s/cat :ch (s/chan ::unq/model))

 :ret (s/chan :unq/model))

--

And yet another: I have a game project with some test.check scenarios. The test run takes 8s without instrumenting and 22s with instrumentation, and this makes me think on starting to split tests in test suites. Is there something to handle this in clojure.test?

That's the first timing report I've seen - pretty interesting! How many fns do you have instrumented? clojure.test does not support this directly but if you're using leiningen you can use their test selectors on how to do this.

 

Thanks!

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Pedro Santos]

Thanks Beau and Alex, I ended up using a different namespace and got things done!

About chans:

I was trying to use the spec mainly as a checker on test/dev. We have a system that communicates a lot via channels but we lose that extra check at those boundaries. Didn't think about the transducers, that complicates things.

About the test timing:

I've only instrumented one namespace and it has a some getter fns that are heavily used[1]. This may not be a typical scenario, because we test brute force AI and auto generate battles and play them to the end. But it's a heavy burden. Our travis build went from 9min to 47 min with that change. So we may need to re-organize the tests and don't instrument all the scenarios.

[1] https://github.com/orionsbelt-battlegrounds/obb-rules/blob/9fa5da24a3844982ba51e17e9adfe49f2aeea445/src/obb_rules/unit.cljc

[Rich Hickey]

You can’t give the same qualified spec name two meanings, that’s a main point of spec. However, your actual use case you use unqualified :attack in two different contexts. - just don’t map them to the same spec. Note that ::kw is just a shorthand in the examples for fully-qualified keys, they need not be in the same ns.

e.g. register specs under

:my.simple/attack
:my.complex/attack

and then in one context use (keys :req-un [:my.simple/attack …]) and in the other (keys :req-un [:my.complex/attack …])

[Georgi Danov]

I love the regexp approach, so decided to implement JSON parser in order to get acquainted with spec. The parser treats the input as char sequence.

The parser works, code is very readable, however performance is horrible — json file with 3.5k characters takes 350 seconds to process — 10 terms/sec.

Here is the spec https://gist.github.com/gdanov/3a0440255f7f4df262ce16ce87351a04