Java like static typing for Clojure?

[Didier]

I know a lot of people like to say how unhelpful Java like static typing is, and only more powerful type systems of the ML family add value, but I've been wondering recently if for Clojure it wouldn't make more sense to simply extend the type hints to enable an optional Java like static typing scheme.

It is my understanding that ML style static typing is incredibly difficult to add properly and without compromise to a dynamic language. That doing so limits the scope of type inference, rendering the task of adding type info more tedious then in ML languages themselves.

ML style static typing provide enhanced safety grantees, but seem to add too much complexity to Clojure to be practical. What about a Java like static typing scheme though?

I haven't found in practice that the safety of Clojure was an issue, as the REPL workflow tend to promote quite a lot of testing. So I'm not too worried about needing the state of the art of provable correctness for my programs. What has been a biggest cause of issue to me was refactoring and shared code base across a team. Those last two use cases are actually pretty well handled by Java like static type checking. Is it a powerful type checker, not really, but it enables most trivial type errors to be caught early, and it allows easier integration points for other devs to follow, as well as documentation for functions, better tools support and easier refactoring, while also enabling performance optimizations.

I have limited knowledge in typing systems, and have no idea how easy it is to implement them, but as a user of Clojure, I feel like I would find an optional Java like static typing a great addition, one that I am more willing to use and benefit from then Typed Clojure's more complex ML style type checking.

What do other think?
Can anyone with better knowledge tell me if this would be feasible or if adding such gradual typing system is effectively as hard as adding ML style type checking?

[A. Levy]

Have you heard of Typed Clojure? (http://typedclojure.org)

It is an optional, gradual type system for Clojure. Does it offer enough to be useful or are you looking for something different?

[Alan Thompson]

Be sure to check out Plumatic Schema (previously Prismatic Schema) if you haven't already.  There is also a good Clojure Conj video from 2013.

Alan

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Timothy Baldridge]

I highly recommend this talk on Spec by Stu Halloway. Spec is a new feature coming in Clojure 1.9, and this talk goes into the pros/cons of static typing and tests and shows how there can be a better way. 

https://www.youtube.com/watch?v=VNTQ-M_uSo8

--

“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Stephen Lester]

Hi there!

There is an 'optional type system' for Clojure in 'core.typed':

https://github.com/clojure/core.typed

But it also seems like others who were using such a type system have stepped away from it:

https://circleci.com/blog/why-were-no-longer-using-core-typed/

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Colin Yates]

I have finally found time to watch Stu's video. It was very useful
introducing clojure.spec (even if I thought he over-egged the pudding
somewhat ;-)).

I know there is a back port, but are there any other strategies of
using this on 1.9 and building against 1.8? I was thinking of building
a tower of yaks such that a lein 1.9 profile included a src directory
which had all of the spec declarations in which the 1.8 profile used
in production excluded. I am hesitant to rely on 1.9 at the moment -
how stable have others found it?

>>> clojure+u...@googlegroups.com

>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an

>>> email to clojure+u...@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to

>> clojure+u...@googlegroups.com

>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to clojure+u...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to

> clojure+u...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to clojure+u...@googlegroups.com.

[Sean Corfield]

We have 1.9 Alpha 13 in production.

We’ve had nearly all the 1.9 Alpha builds in production. After all, right now 1.9 = 1.8 plus clojure.spec with some minor additions to clojure.core (new predicates).

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

[Colin Yates]

Thanks Sean.

[Colin Fleming]

I recently spent a bunch of time researching exactly this. My motivation was that my main project, Cursive, suffers from a ton of NPEs which I find very difficult to manage. I wanted to see how difficult it would be to have a typed Clojure-like thing, using something similar to Kotlin's type system. Kotlin uses a type system which is similar to Java and has Java interop as a primary goal, which I would also need since Java interop is essential to me. It fixes a bunch of flaws in the Java type system and adds new features like nullable types, which I now find it difficult to live without.

Before anyone asks, spec is not useful for me because it relies heavily on generative testing to increase your confidence in your functions. I can't use generative testing because my application is tied to a large Java codebase which I cannot model to any useful degree. Essentially, spec recommends runtime tests at the boundary of your system, and nearly my entire system is interop boundary. I'm not interested in runtime checks except where absolutely necessary - Kotlin does this for me transparently, spec doesn't. 

Here's a short list of my findings. I'm happy to expand on any of these points if anyone is curious. It bears repeating - Java interop is non-negotiable for me, and that makes a lot of this harder than it would be otherwise.

Disclaimer: I'm no programming language expert. This was hard for me, and a surprising amount of it was new to me. I'd appreciate any corrections or clarifications.

1. Type systems are hard. I for one didn't appreciate the complexity that goes into making them easy to use. Don't be fooled by the 20-line implementations of Hindley-Milner.
2. In particular, generics are very hard, and variance for generic objects (i.e. the intersection of generic objects and OO) is the source of much difficulty.
3. Type systems are split into two main camps - nominal typing (like Java, where the types are identified by names) and structural typing, where the type of an object is defined by it's "shape", like Go's interfaces.
4. One of the major benefits of Clojure is its heterogeneous collections, a.k.a. "just use a map". This is very difficult to maintain in a type-safe manner without losing most of the benefit.
5. There are two main things I was interested in from a type system - type checking (i.e. making sure that your program's types are correct) and type inference (working out what the types of things are from the code, so you don't have to annotate everything). Type checking is relatively straightforward, but type inference can be very hard indeed.
6. Much of the complication comes from the need to interop with Java. ML-style interface essentially doesn't work if you want to maintain compatibility with Java since it cannot be interfaced in a useful way with nominal OO systems. In particular, method overriding is basically impossible to represent.
7. #6 implies that you cannot have a language with good Java interop and global type inference, i.e. you will definitely be annotating your function parameter types, and your function return types. I'm ok with this since IMO it's a good practice anyway.
8. Once you start thinking about the fact that you no longer have heterogeneous collections, and start thinking about what the alternatives would look like, you start to realise that you'd end up with basically the ML family style of data types - homogeneous collections, typed tuples and ADTs. I'm actually ok with that too, since they're a very nice set of primitives and I think they probably represent 95% of how people use Clojure's collections anyway.
9. Structural typing seems like it might be a good fit for something like Clojure's maps. However, mixing structural and nominal typing and OO seems to be the path to madness.

I think that's it. This is still a project I'd like to tinker with at some point, but I think it's fair to say that I dramatically underestimated the amount of work a useful Java-compatible type system would be. I still think it seems like a nice point in the language design space though, which is curiously unfilled on the JVM (some of you may have noticed that I basically want F# with nullable types).

Cheers,

Colin

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Daniel]

Just curious... What do you think the primary contributing factor is for Cursive's NPEs? Java interop?

[Colin Fleming]

This is a discussion that I've had a couple of times. I don't think that interop is the main factor here, I think it's more that I'm programming against a large codebase I don't understand well (I can't since it's around 4 million LOC). I suspect that if I were programming against a large undocumented Clojure codebase I'd have similar problems, or worse. 

Note that Java interop often gets the blame here, but it's worth pointing out that in Clojure everything is Java interop in the end. (+ 1 x) or (str/trim x) will give you an NPE as easily as in Java if you pass nil in x. And even if Java interop is the root cause of the problem, supposedly interop is idiomatic in Clojure, and interfacing with existing Java systems is a key use case for continuing Clojure adoption. I don't think that the type of application that Cursive is is as much of an outlier as people think - anyone integrating with an existing system will probably face similar problems. Indeed, after my previous conversations, several people contacted me to say that their experience matched mine in similar systems.

People have also asked me about whether nil punning helps. In my experience, it allows for some elegant code but it has a really terrible tendency to mask bugs and push them much further into the system than would otherwise be the case. It would be nice to be able to say that I'm expecting a collection to be non-nil, and to get an NPE at the point of the collection call if it's nil due to a bug, rather than just silently returning nil and continuing. 

Also, when tracking down an NPE from a bug report, if there are collection methods in the stack trace I now have to consider (at least) 3 cases: the collection is unexpectedly nil, the collection is unexpectedly empty, or the collection unexpectedly contains a nil value. This greatly increases the space of problems that I have to consider as possibilities when trying to work out what's going on.

On 21 October 2016 at 05:39, Daniel <double...@gmail.com> wrote:

Just curious... What do you think the primary contributing factor is for Cursive's NPEs? Java interop?

[Alex Miller]

Just as a counter-anecdote, I have worked on large Clojure codebases (both ones I developed and ones I was unfamiliar with), including ones that interfaced with existing Java codebases, and have not experienced this problem to the degree you describe Colin. So while I believe these problems exist, I don't think it's a foregone conclusion that they must.

FYI, I've found that when working with a partially instrumented spec'ed clojure.core, that unexpected nil colls (and colls containing nil) are reported earlier and better (because they occur at the point they are introduced rather than later and several layers down in some RT method).

[Colin Fleming]

Sure, I'm not arguing that all large projects suffer from this, and I'm not entirely sure why Cursive does so badly. But the argument I often see online is the opposite - that Clojure codebases never suffer from this, and if they do then it must be because of interop, or the application must be strange in some way. I think this sort of argument is equally disingenuous, and like I say, I've heard numerous anecdotes to the contrary. 

One theory about why Cursive is particularly prone to this that I forgot to mention - Cursive is unusual in that it is run on users' desktops. When writing, for example, a web application, assuming you're validating your input you have a reasonable handle on the expected space of inputs, and you can test various combinations of your application state. For Cursive, that is much harder - I currently support 6 versions of the underlying IntelliJ platform, and they run on 3 operating systems. There are two main IntelliJ configurations (Ultimate and Community). These variations are actually surprisingly stable and don't tend to create a lot of problems, but users can also have any combination of plugins installed, and they can sometimes interact in surprising ways - I've had bugs logged against Cursive that have turned out to be bugs in the Scala plugin, and vice versa. Plugins can hook into all sorts of different IDE functionality at different levels and this can interact in strange ways. I've had a lot of bugs of this sort, and it's plausible that many of the bugs that are currently unresolved are a result of this in some way. 

In this sort of situation, a static type system which provides universal guarantees (this value can never be null) is more useful than a contract system (no null values have been seen yet for the test inputs you've tried). There's simply no way I can test all combinations, or reproduce all combinations that users might have running. While the spec'ed core definitely sounds like something I should try, it will still only work for the combinations that I actually test unless I leave it running in production builds.

[Josh Tilles]

Out of curiosity, did you try Typed Clojure? It certainly has its rough edges, but you sound willing to bear the burden of annotating code with types (at the top-level, at least) and I think its treatment of Java interop does what you want: unless instructed otherwise, the typechecker assumes that arguments to Java methods must not be nil and that any Java method may return nil.

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Colin Fleming]

I tried it a couple of years ago, and my impressions were more or less the same as CircleCI's here. I found the type annotation burden much higher than using a typed language such as Kotlin, the type checking was very slow and the boundary between typed and untyped code was really onerous. Ambrose has done some great work recently though, so I should check it out again. However my general feeling is that retrofitting something like Typed Clojure onto an existing language is always going to be more difficult and fraught with problems than having a language which was designed with the types in mind in the first place.

Another possibility which I haven't had time to explore properly is Allen Rohner's spectrum.

Honestly, the easiest solution to my problem is probably just to use Kotlin, which was designed by JetBrains for almost exactly my use case, has great IDE support, and has a lot of smart people working full-time on it. However that has a couple of problems: 1) it would make me sad and 2) I would no longer be dogfooding Cursive all the time, which is a valuable source of finding bugs during development. But at least I'd be spending all my time on developing Cursive features, and not chasing NPEs or investigating all this.

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Colin Yates]

"making me sad" is unsustainable - problem solving with 1s and 0s is
hard enough as it is without using demotivating tools :-).

[Colin Fleming]

Absolutely, both that and the dogfooding are compelling arguments :-)

>> clojure+unsubscribe@googlegroups.com

>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to clojure+unsubscribe@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to

> clojure+unsubscribe@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to clojure+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Sean Corfield]

On 10/21/16, 10:40 AM, "Colin Fleming" <clo...@googlegroups.com on behalf of colin.ma...@gmail.com> wrote:

> Honestly, the easiest solution to my problem is probably just to use Kotlin, which was

> designed by JetBrains for almost exactly my use case, has great IDE support, and has

> a lot of smart people working full-time on it.

 

And, to be fair, Kotlin has as a design goal to help address Java’s NPE issue. Whereas Clojure has, as part of its design, idiomatic nil-punning. If you’re doing a lot of Java interop, Kotlin is going to be a better fit.

 

As for Typed Clojure, we’ve tried it a few times and the problems cited by CircleCI and by yourself are why we’ve given up on it each time. I will say, in Typed Clojure’s defense, that it gets better and better each time I try it so it’s definitely going in the right direction – but it is a very hard problem to solve!

 

Pretty much the only time I ever see NPEs is when my Clojure code touches Java interop. And, yes, that can mean numeric ops (since those are implemented directly in Java) and string manipulation (again, implemented on top of Java).

 

As an experiment, I tried a version of clojure.string where nil was always treated as “” and it does indeed avoid the NPEs but it comes at a performance cost (calling str or adding nil conditions). In the domain in which I work, nil -> “” is pretty much universally the right thing so it’s a cost we’re considering swallowing, for the extra simplicity it would bring to our code (i.e., creating a drop-in replacement of clojure.string that implements all of the functions with added str calls as needed – many can be handled mechanically).

 

We don’t do much numeric work so we don’t hit NPEs in that Java interop boundary very often. There tho’ there is almost no argument that nil -> 0 would be the “right thing” so suffering NPEs instead of some NonNumericArgumentException thing isn’t such a horrible trade off.

 

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 

[Colin Yates]

Ironically I ran into an issue where I was receiving "" instead of nil
which caused some interesting behaviour.

For those who find these things interesting, this was for capturing
criteria in the UI which was sent to the server to filter. The
behaviour was:

- form is nil, server ignores the query
- enter some text, server sends the top N matching records
- delete the text, you get the first top N records

I am sure you have guessed, but the cause was HTML input fields start
off as nil, enter something and then delete it and you get an empty
"". The server was checking (not (nil? criteria)) rather than (seq
criteria)....there is a reason (seq ....) is idiomatic :-).

[Antonin Hildebrand]

You could travel to the future and use ClojureScript with Kotlin as compilation target and a version of clojure.spec which resolves a subset of known constructs to Kotlin type annotations at compile-time :-p

Actually that idea of having a library of compile-time-recognizable spec constructs (like nil? string? or instance? predicates) is something I would like to see in ClojureScript today. It would generate type annotations for Closure Compiler to catch some class of simple bugs at compile time. But this is off-topic for this thread...

[Daniel]

> In this sort of situation, a static type system which provides universal guarantees (this value can never be null) is more useful than a contract system (no null values have been seen yet for the test inputs you've tried). There's simply no way I can test all combinations, or reproduce all combinations that users might have running.

Isn't a major selling point of generative testing to create loads of unique cases you can't invent on your own?

You don't trust it to do that? Is that from personal experience? Genuinely curious because I am a little excited about using it in a project at work but this is disheartening.

[Colin Yates]

Generative testing is great but defining the contract gets more
complex the further away from a 'unit' you get. It is easy to define
extensive generators for (defn length [s]). It is a much bigger
problem to generate extensive inputs for every call-site of (length)
and then every call-site of the call-site of (length) and so on.

This is not a problem limited to clojure.spec, it is a limitation to
any example based testing, and in that sense, a helpful simplifying
lie is to think of generative testing as example based testing on
steroids.

Type systems work across the code base as a whole and are therefore
(in another helpful simplifying lie) more extensive.

The other consideration is that type systems limit the
'specifications' you can give it, clojure.spec specifications can be
as rich as your code and are therefore unbounded.

Don't get me wrong, I am just starting to get into clojure.spec and I
too am very excited, but given the specs are effectively unbounded, it
isn't going to catch all the errors that types are. However, BECAUSE
it is unbounded it can be far more powerful.

Just my 2p (and I haven't had any coffee yet so the above might be
complete drivel).

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to

> clojure+u...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Colin Fleming]

Yes, that is a major selling point. Generative testing is really cool, and you should definitely not be disheartened - it's a tool like any other, with its strong points and weak points (like static typing, too). It's definitely not universally applicable, and even more than standard testing, it requires you to write your code keeping in mind how you will test it. It happens that in my case I believe that static typing is a better fit, but that doesn't mean that it won't work well for your use case. You should definitely be excited to try it and see - if it works well in your case, it's pretty magical.

The limitation in my case is that generative testing works best when testing pure functions. Because so much of my system involves calls out to a huge mutable blob of Java, it doesn't work well for me. 

i actually don't have a lot of experience with generative testing, but here are two examples from the little experience I have, one which worked well and one which didn't. 

The one that worked well was testing a binary object serialiser. This was in Java several years ago, so a lot of the complexity in my code was writing generators - spec (via test.check) really helps with this and it would be relatively trivial to achieve nowadays what took me a long time back then. But once I had a generator which generated random objects, I then just ran a bunch of tests which generated an object, serialised it, deserialised the resulting bytes, and checked that the deserialised object was equal to the original one. It was really useful, and caught a whole lot of edge case bugs I would never have caught otherwise. Things to note are that the serialisation and deserialisation are pure functions, and the success condition is very easy to define.

One that I have not yet found a good way to test generatively is the paredit code in Cursive, which occasionally suffers from edge case bugs. I'd also like to test code refactorings. However here, the generation is much much harder (I'd have to generate valid code in some way) and then I'd have to randomly refactor it and somehow check that the refactored version is equivalent. Here both the generation and the success condition are extremely hard to define, and the risk is that in your success condition you really just end up reimplementing your original algorithm and then what you're testing is actually that those two algorithms are the same, not necessarily that they do what you expect. I was lucky enough to talk at length to Thomas Arts at Curry On last year and I asked him about this. He had done something similar for Erlang code, and had a very complicated code generator. They generated code that produced some output, then randomly refactored it, compiled both versions, ran them both and checked that the output was the same. His conclusion was that in that case, the tests weren't worth it - they took forever to run and were very brittle.

On 22 October 2016 at 02:54, Daniel <double...@gmail.com> wrote:

> In this sort of situation, a static type system which provides universal guarantees (this value can never be null) is more useful than a contract system (no null values have been seen yet for the test inputs you've tried). There's simply no way I can test all combinations, or reproduce all combinations that users might have running.

Isn't a major selling point of generative testing was that it creates loads of unique cases you can't invent on your own?

You don't trust it to do that? Is that from personal experience?  Genuinely curious because I am a little excited about using it in a project at work but this is disheartening.

[Erik Assum]

Colin,

I think the points you bring up here are very interesting, and reflect ideas that I have not yet been able to formulate. 

I certainly would think there is enough "stuff" in here for a talk, and also some ideas into how to grnerative test "bread and butter" code, as can I, and if so, how, generative-test my REST-api. Can I do it in some more meaningful way than just checking for a non 5xx status code?  How do I property base test my functions which take my domain "objects". Do I have to write generators for them, and am I then ending up in the same pain as I do writing mocks in Java? I know that I can compose generators, but I might end up needing users with e.g. genuine Norwegian addresses. 

These are my questions wrt generative testing in my world. 

Erik. 

-- 

i farta

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Timothy Baldridge]

>> but I might end up needing users with e.g. genuine Norwegian addresses. 

I think that's an interesting point, and it's a problem I've encountered several times myself. However I think it can be solved several ways. 

1) If your validation function "norwegian-addr?" is a simple predicate, then the answer is actually pretty trivial, you can create a generator for that predicate. There are some test.check generators for arbitrary regexes, and those can be composed. 

2) If your validation function hits a database of know addresses, then you have another possibility: generate your test data from the accepted data set. I've taken this approach on several projects. I have a database of known Product IDs, and I query that when creating my test.check generators.

3) If your validation function hits a database you can also generate that database from test.check. A "is-valid-addr?" function that takes a database should be generic enough to query for any address that exists in a DB. So generate your DB with data from test.check, then generate the inputs to your functions, then let it run. 

#3 here is especially powerful. When we talk about creating generators for "domain objects", it should be recognized that the domain itself could be generated by test.check. 

[Didier]

Spectrum (https://github.com/arohner/spectrum) seems quite interesting. I haven't tried spec yet, but form quick glance, it looks just as annoying as typed clojure to use.

I think I'm imagining something simpler though. Say a system that just used already existing Clojure type hints, and performed compile time checks about to whatever level is possible given the provided annotation.

So say I did:

(def ^int something "test")

This would fail at compile time.

Similarly:

(def ^String something "test")
(inc something)

It would fail at compile time.

Or:

(defn inc2 [^Long a]
  (inc (inc a)))
(inc2 1.80)

Would also fail.

And hopefully this:

(defn ^Long inc2 [^Long a]
  (inc (inc a)))
(def temp ["test", 2.48])
(map inc2 temp)

Would also fail. Because the type checker would infer that the vector has String or Double as type, and inc2 expects none of these.

Or:

(conj ["a", "b"] "c")

would still work.

(conj ^Vec<String> ["a", "b"] 10)

would fail.

Now I don't know, maybe this is too simple, not really useful, or maybe its non trivial to implement. But I feel just small type checks like that could be useful, and they wouldn't be too burdening.

On Saturday, 15 October 2016 15:14:08 UTC-7, Didier wrote:

I know a lot of people like to say how unhelpful Java like static typing is, and only more powerful type systems of the ML family add value, but I've been wondering recently if for Clojure it wouldn't make more sense to simply extend the type hints to enable an optional Java like static typing schemI the.

[Mikera]

In my moments of insanity / hammock time I've toyed with making a typed variant of Clojure. Somewhat inspired by core.typed, but I feel that to be effective a type system needs to be deeply integrated with the compiler and standard library, not just a standalone tool.

Types would themselves be an abstraction, supporting the following operations:

- Intersection with another type (N or And)

- Union with another type (U or Or)

- Checking is a type is a subclass of another type (extends?)

- Checking if a value conforms to the type (instance?)

Abstractions themselves would be possible to define as types, so you could have the Seqable type:

(defabstraction Seqable ....)

(defn process-sequence [^Seqable s]

  (let [things (seq s)]   ;; guaranteed to work by type system

     ...)

Java classes would conform to the Type abstraction, so you can still use Clojure-style Java type hints for interop.

Anything passed to a typed parameter would be verified to be either:

- A valid subtype, in which case everything is OK

- Something that cannot be a valid subtype, in which case you get a compiler error

- A potentially valid value (either untyped or an intersecting type), in which case a runtime check is inserted (and maybe an optional warning...)

Types could be parameterised, so you can have types like:

(ListOf Integer)

(Nullable String)

(ArrayOfShape [10 10])

(ExactValue "Foo")

(Or String Integer)

Occurrence typing would mean that stuff like the following is valid:

(defn foo [^(Nullable (Or String Integer)) x]

  (cond 

     (nil? x) ....

     (Integer? x) (+ x 5)           ;; valid addition, can prove it is an Integer

     (String? x) (.charAt x 0)    ;; no type hint required, can prove it must be a non-null string....

     ))

Since branching on type is a common idiom, it would also support type-based overloading of functions:

(defn foo

  ([^Null x] ....)

  ([^Integer x] ....)

  ([^String x] ....))

Untyped code would look still look like regular Clojure. Types would always be optional.