s/valid? does not tell me if the data is valid as supplied

[Jan Rychter]

I've been using spec for a while now, in a reasonably large code base (>30k lines of Clojure and ClojureScript) and there is an issue that bit me several times.

I use conformers for coercing data that is *almost* what I need, usually when reading from JSON (RethinkDB). Common conformers are keyword and set. And it works really well, except for one problem: there is no way to know if data has been conformed or not.

Calling s/valid? will tell me if the data is valid *if it has been conformed*. But what if it hasn't? Can I use the data? Is it "valid" according to the spec I wrote?

This is a very real problem: I've spent considerable time chasing bugs where there was a code path which did not call s/conform. The data passed all validations done with valid? and the bug manifested itself far down the road, where something expected a keyword instead of a string, or a set instead of a vector.

Here is a specific minimal example demonstrating what I'm talking about:

(ns spectest

  (:require [clojure.spec.alpha :as s]))

(s/def ::test-spec (s/and (s/conformer keyword) keyword?))

(s/conform ::test-spec "a") ;; :a

(s/valid? ::test-spec "a") ;; true

I expected the last valid? to return false, because my code does not expect a string, it expects a keyword, according to the spec.

I might be missing something, but I would much rather see valid? tell me if the data is valid for use (as supplied) and have a separate valid-when-conformed? which tells me if the data is, well, valid when conformed. It seems to me that the current valid? that does two things is confusing and not very useful for contracts.

At the very least I'd really like to see a function that tells me if the data is valid *as supplied*, as this is the function that I'd want to use when enforcing contracts everywhere in my code.

Alternatively, I could stop using conformers altogether, and write explicit data conversion functions. That might not be a bad idea, but it seems other people started using conformers, too, so eventually I'll hit the same problem again.

--J.

[Alex Miller]

This is exactly why we recommend that you not use conformers for coercion. Conformers were added primarily as a tool for building custom composite spec types (for example, we used it to build keys* from keys).

This is a common need though and I would be happier if spec did more to help you solve it in a way that minimized repetition and maximized the use of existing specs. I'm still thinking through what that would mean exactly. It's challenging right now to plug externally without rebuilding a significant part of spec, so that's obviously not ideal.

Ideally, you would be able to say the things that are important here:

- the spec of the incoming data (strings or whatever - JSON sourced is the major use case)

- the spec of the data I desire

- the coercion functions that can move from one to the other (there are probably a small number of these that are widely reused)

- some way to coerce+validate or coerce+conform

Building coercion into a single spec itself instead leads to the problem of not being able to know what the source data actually was, and that's really at odds with the spec philosophy and the notion of bidirectional conforming.

[Erik Assum]

FWIW, I’ve been using https://github.com/metosin/spec-tools

on a couple of projects lately, which helps nicely with

conformance and coercion. The main devs are very helpful on #ring-swagger on the Clojurians slack. 

Alex, how does spec-tools measure up to your thoughts on conformance/coercion?

Erik. 

-- 

i farta

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Miller]

On Tuesday, February 20, 2018 at 10:28:12 AM UTC-6, Erik Assum wrote:

FWIW, I’ve been using https://github.com/metosin/spec-tools

on a couple of projects lately, which helps nicely with

conformance and coercion. The main devs are very helpful on #ring-swagger on the Clojurians slack. 

Alex, how does spec-tools measure up to your thoughts on conformance/coercion?

spec-tools combines specs for your desired *output* with a coercion function, making the spec of the actual data implicit. I feel like this conflates too many things and obscures the actual input data value, which is the same problem the original poster had. Also, spec-tools introduces the notion of "type". spec intentionally avoids creating a new vocabulary of special words like this and in all cases relies on predicates or things mapped to predicates instead. I'm not a fan of this approach and I don't like the idea in CLJ-2116 much either - I think it's pretty unlikely this is where we will end up.

In general, I think a lot of the functionality in spec-tools either uses implementation internals that are almost certain to break as spec evolves or is at odds with the philosophies of spec as stated in https://clojure.org/about/spec (like the type vocabulary thing).

I do see the problems driving this, and I agree there is a gap to be filled here though. 

[Erik Assum]

Out of curiosity, are you at liberty to discuss how Cognitect solves the problem of validating/coercing values at the edges of the application when you (Cognitect) are doing consulting?

Erik.
--
i farta

[Alex Miller]

On Tuesday, February 20, 2018 at 11:24:19 AM UTC-6, Erik Assum wrote:

Out of curiosity, are you at liberty to discuss how Cognitect solves the problem of validating/coercing values at the edges of the application when you (Cognitect) are doing consulting? 

I'm not involved with those typically and I'm not familiar with their information sharing agreements, so I can't personally speak to that. Certainly you can transform, then validate with spec as one option.

[Sean Corfield]

Calling s/valid? will tell me if the data is valid *if it has been conformed*. But what if it hasn't? Can I use the data? Is it "valid" according to the spec I wrote?

 

If your spec includes coercions, you have inherently made the “is valid?” question include the coercion. Your ::test-spec accepts anything that can be used as the argument to keyword. This is why Cognitect keep recommending people do not do this. And I’m jumping into this thread because we _do_ include coercions in some of our specs at work… we’ve been heavy users of spec since the early alpha builds and we ran those alphas in production. But we are very conscious about our specs that coerce: we know and accept that they will work on “any string that can be coerced to <desired type>”. If we specifically want to check whether some data is <desired type> we use a different spec.

 

We do this for parameters in our REST API, for long, double, Boolean, date, etc – we have two specs for each: one that is a spec for the target type in the domain model (which in these cases is just defined as the appropriate built-in predicate), and one that is a spec for the API level (which accepts either the target type or a string that can be coerced to the target type). Then we use the appropriate spec at the appropriate “level” in our application.

 

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 

---

From: clo...@googlegroups.com <clo...@googlegroups.com> on behalf of Jan Rychter <jryc...@gmail.com>
Sent: Tuesday, February 20, 2018 2:41:38 AM
To: Clojure
Subject: s/valid? does not tell me if the data is valid as supplied

 

[Tommi Reiman]

tiistai 20. helmikuuta 2018 19.05.55 UTC+2 Alex Miller kirjoitti:

On Tuesday, February 20, 2018 at 10:28:12 AM UTC-6, Erik Assum wrote:

FWIW, I’ve been using https://github.com/metosin/spec-tools

on a couple of projects lately, which helps nicely with

conformance and coercion. The main devs are very helpful on #ring-swagger on the Clojurians slack. 

Alex, how does spec-tools measure up to your thoughts on conformance/coercion?

spec-tools combines specs for your desired *output* with a coercion function, making the spec of the actual data implicit. I feel like this conflates too many things and obscures the actual input data value, which is the same problem the original poster had. Also, spec-tools introduces the notion of "type". spec intentionally avoids creating a new vocabulary of special words like this and in all cases relies on predicates or things mapped to predicates instead. I'm not a fan of this approach and I don't like the idea in CLJ-2116 much either - I think it's pretty unlikely this is where we will end up.

I'm guilty of most parts of spec-tools and the CLJ-2116, so some comments.

Spec-tools was built because there was a real-life problem (coercion) and spec didn't cover it. This was summer 2016 and there is still no official solution for this.

New vocabulary, e.g. "type": everything besides coercion (json-schema mappings, form-resolution etc.) is already mapped directly to predicates and so could the coercion. Maybe we'll change that in next version. There will anyway be support to spec local configuration (for both coercion & spec transformation) via spec data: https://github.com/metosin/spec-tools/issues/96.

In general, I think a lot of the functionality in spec-tools either uses implementation internals that are almost certain to break as spec evolves or is at odds with the philosophies of spec as stated in https://clojure.org/about/spec (like the type vocabulary thing).

Hmmm... I know that data-specs use the functional (non-documented) version of the spec macros. They most likely will break at some point (but they are private in spec-tools too), but I guess the "functional specs" are anyway coming, so same things should be possible to do in the future, via just documented apis?

There are lot of regression, and some progression tests too in spec-tools.

I do see the problems driving this, and I agree there is a gap to be filled here though. 

This is good. 

There is also https://dev.clojure.org/jira/browse/CLJ-2251 by me, which would be my top1 wish for spec: one open `s/walk*` (replacing `s/conform*` and `s/unform*`) for specs and supporting different use cases: Would solve Jan's issues supporting just "validate", would bring first-class fast coercion (instead of current conform+unform is slow) and if it would be open, the support libs (like spec-tools) could extend by reusing the core, not by copying/rewriting.

Could you comment on that Alex?

Two-way coercion would be nice, but I think one-way coercion is ok too, much better than no coercion :) for JSON, Jackson can already write our Clojure back to JSON really fast, having a spec-transformation from clojure->JSON would be much slower anyway.

[Jan Rychter]

On Tuesday, February 20, 2018 at 4:53:33 PM UTC+1, Alex Miller wrote:

This is exactly why we recommend that you not use conformers for coercion. Conformers were added primarily as a tool for building custom composite spec types (for example, we used it to build keys* from keys).

I am afraid that ship has sailed. Looking around I see lots of cases where people do use conformers for coercion. At a first glance it seems very natural, and warnings not to do it are not easily found.

 

This is a common need though and I would be happier if spec did more to help you solve it in a way that minimized repetition and maximized the use of existing specs. I'm still thinking through what that would mean exactly. It's challenging right now to plug externally without rebuilding a significant part of spec, so that's obviously not ideal.

Ideally, you would be able to say the things that are important here:

- the spec of the incoming data (strings or whatever - JSON sourced is the major use case)

- the spec of the data I desire

- the coercion functions that can move from one to the other (there are probably a small number of these that are widely reused)

- some way to coerce+validate or coerce+conform

Building coercion into a single spec itself instead leads to the problem of not being able to know what the source data actually was, and that's really at odds with the spec philosophy and the notion of bidirectional conforming.

I'm glad you see the need, highlighting it was largely the point of my post. As for these requirements, I agree, although I'm not sure about the need to know about the source.

Regardless of larger future plans, I think my original suggestion still stands: it would be nice to have a function that would tell me if the data is valid as supplied.

And another minor point: when I call a validation function (as part of contract checking), I do not necessarily expect to deal with all kinds of exceptions that coercion functions might throw.

--J.

[Sean Corfield]

Looking around I see lots of cases where people do use conformers for coercion.

 

That doesn’t make them right 😊

 

At a first glance it seems very natural, and warnings not to do it are not easily found.

 

Every single time coercion comes up anywhere in the context of spec, someone says “don’t do that”, and they’ve been saying it since the earliest alpha versions of spec. You would be correct to point out that nothing in the spec overview or spec guide on clojure.org carries this caution, however (and I think it’s a reasonable “ask” for the guide to be updated to include such a caution).

 

My recommendation is to have a strictly non-coercive spec for the target data “type” / shape you want, and to have a second spec that combines the coercion you want with that spec. That way you have a way to tell if your uncoerced data conforms to the spec, as well as a way to do coercion in s/conform. They are – and should be – two separate specs and two separate operations. They represent different layers of abstraction inside your application (so “of course” they should be two separate specs, one built on top of the other).

 

Given that the overview and the guide don’t even mention s/conformer, I’m not sure where that recommendation should live. Alex, any thoughts on this, since you seem to be the one most often making the recommendation?

 

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 

From: clo...@googlegroups.com <clo...@googlegroups.com> on behalf of Jan Rychter <jryc...@gmail.com>

Sent: Wednesday, February 21, 2018 1:16:49 AM
To: Clojure
Subject: Re: s/valid? does not tell me if the data is valid as supplied

 

[Didier]

I would actually love it if Spec was extended to have the concept of types.

Something where every spec could be tied to a Type, and types could be constructed to have Hierarchies.

Not sure what the syntax would be like, but say:

(s/def :String ::name string?)

(s/def :String ::address (s/and string? (complement string/blank?)))

(s/def :Person ::person (s/keys :req [::name ::address]))

(s/def :Homeless ::homeless (s/keys :req [::name]))

(s/defisa :Homeless :Person)

Types would still be predicates, and all spec would be a Type too. Types would be optional though. A Type is the OR of all the specs predicates defined as that Type and its children types. So in the above, :Homeless is (s/or ::person ::homeless).

Now, this idea might need to be refined, but the goal would be so that Spec could be used as a modeling languages for other languages. So I could from a Spec auto-generate a Java class model, or a Ruby model, etc. Since now I can relate predicates to types myself. It could also allow for better static analysis.

Also, might make spec extra complicated and confused to mix both predicates and types, but that could depend how its done and managed.

[Alex Miller]

On Wednesday, February 21, 2018 at 7:34:00 PM UTC-6, Didier wrote:

I would actually love it if Spec was extended to have the concept of types.

Map specs are about attribute aggregation, not about types.

https://clojure.org/about/spec#_map_specs_should_be_of_keysets_only

 

Something where every spec could be tied to a Type, and types could be constructed to have Hierarchies.

You can effectively do this already. When you have s/keys maps which are sets of attributes, you can simply combine sets (base and extensions) using s/merge. Or when needing more polymorphism, s/multi-spec.

[Jan Rychter]

On Thursday, February 22, 2018 at 12:58:00 AM UTC+1, Sean Corfield wrote:

Looking around I see lots of cases where people do use conformers for coercion.

 

That doesn’t make them right 😊

Oh, I agree, I'm just being realistic. The cat is out of the bag. I just googled for 's/conformer' and found an entire library of conformers that do various data conversions.

  

At a first glance it seems very natural, and warnings not to do it are not easily found.

 

Every single time coercion comes up anywhere in the context of spec, someone says “don’t do that”, and they’ve been saying it since the earliest alpha versions of spec. You would be correct to point out that nothing in the spec overview or spec guide on clojure.org carries this caution, however (and I think it’s a reasonable “ask” for the guide to be updated to include such a caution).

 

My recommendation is to have a strictly non-coercive spec for the target data “type” / shape you want, and to have a second spec that combines the coercion you want with that spec. That way you have a way to tell if your uncoerced data conforms to the spec, as well as a way to do coercion in s/conform. They are – and should be – two separate specs and two separate operations. They represent different layers of abstraction inside your application (so “of course” they should be two separate specs, one built on top of the other).

Also agree. I think coercion and conformance verification are two separate concepts and should not be conflated. I like the idea with two specs, as this lets me get some reuse.

 

Given that the overview and the guide don’t even mention s/conformer, I’m not sure where that recommendation should live. Alex, any thoughts on this, since you seem to be the one most often making the recommendation?

As a spec user, I would expect to learn about this from the spec guide.

Also, I still believe that in the short term a function that validates a spec without calling conformers would be useful.

best regards,

--Jan

[Didier]

Map specs are about attribute aggregation, not about types.

I understand the design philosophy. I think that's great. What I meant I guess is that it would be nice to also have a declarative way to relate Types to Specs, for when Specs fall short of types. Then you'd have the best of both world. Specs where Types fall shorts, and Types where Specs fall shorts.

You can effectively do this already. When you have s/keys maps which are sets of attributes, you can simply combine sets (base and extensions) using s/merge. Or when needing more polymorphism, s/multi-spec.

I guess I'm talking about Types as in primitive types included. So if I create a String type, I can describe what is the set of values for the type using a spec. Similarly, I now know that the Type and the Spec used to describe its possible set of values are equivalent. So I now know the relationship between them.

The real reason I want this is so that I can auto-generate client code in different languages for my Specs.

I'm thinking of it as a more generic Swagger OpenAPI specification.

Maybe what I'm thinking off doesn't make sense, its all kind of at the blurry idea phase in my head.

Currently, I manually have an equivalent set of Java Classes which correspond to the specs of my data which gets shared between Java and Clojure systems. And I manually have functions which converts a piece of data from a given Spec, into its Java equivalent representation and back. I've been thinking about how I could have the equivalent Java classes be auto-generated, as well as the conversion be automatic. The biggest challenge is knowing what Java types would a given Spec map too. So I thought if I my spec could have a mapping on them too, then it wouldn't be too hard. When you write your Spec, you could associate it with an enclosing Type, and then from the Specs I might be able to auto-generate the Java model classes.

Anyways, the whole thing is incubating in my head. And I can probably do it on top of Spec if I really needed too.

[Wilker]

I've written a library that tries to solve this, instead of making conforms that do coercion, it uses the spec and a separated registry to conform the value (similar to what spec does to find the generators for a given spec). You can find the library at: https://github.com/wilkerlucio/spec-coerce. This way you don't complect the conformance with coercion.

[Alex Miller]

Nice!

[Dave Dixon]

I've been using the macro below to make "types" in the context of clara-rules, where "type" has a specific semantic. If you're going to do something like this, you definitely should have a very well-defined notion of what "type" means. 

(defmacro def-derive

     "Macros to wrap useful pattern of defining a spec and calling

      derive on the spec and a \"parent\" spec to create a hierarchy."

     ([child-name parent-name]

      `(def-derive ~child-name ~parent-name ~parent-name))

     ([child-name parent-name spec]

      `(do

         (#?(:clj clojure.spec.alpha/def :cljs cljs.spec.alpha/def)

           ~child-name (#?(:clj clojure.spec.alpha/merge :cljs cljs.spec.alpha/merge) ~parent-name ~spec))

         (derive ~child-name ~parent-name))))

[Simon Belak]

I’m a pretty heavy user of spec including a lot of coercion. The way I’ve always done coercion is to enumerate all accepted shapes first and do coercion last. For your example this would be:

(s/def ::foo (s/and (s/or :string string?
:kw keyword?)
(s/conformer (fn [[tag x]]
(case tag
:string (keyword x)
x)))))

Ps. also don’t forget s/conformer can return :s/invalid to signal failure.