Errors

[pa...@pwjw.com]

Hi!

Boy I really think you've all done a nice job with Clojure. I've used quite a few environments over the years and clojure + CIDER + etc is a great experience. The immutability and threading are smart. I've been able to finish a hobby project in clojure which I've been mulling for a long time and never found the right environment. Super stuff.

And the error messages are not good.

So I was wondering: Is this a philosophical thing? Or is it an effort thing? And if it is an effort thing, is there some sort of plan for what effort to put in? And if so, can I help by closing tickets?

Here's one, for instance, which bugs me (using a build of branch master in clojure from github just now)

user=> (+ 1 1)

2

user=> (1 + 1)

ClassCastException java.lang.Long cannot be cast to clojure.lang.IFn  user/eval17 (NO_SOURCE_FILE:10)

OK but what sort of noob would actually type in 1 + 1? (That's a joke. Lots of people would. And I do sometimes when switching between java and clojure) But that same error pops up if you misalign in parenthesis or confuse an argument order or do a variety of other things.

And that error message is terrible. I mean I know why it is there. Clojure needs the first element of a list be the function position and 1 is not a function, it is a long. But you have to squint rather generously at that message to get to the problem. Or just learn what that message "really" means, which seems a bit unfriendly.

And so when I go look inside the source 

user=> (print-stack-trace *e 20)

java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn

 at user$eval9.invokeStatic (NO_SOURCE_FILE:6)

    user$eval9.invoke (NO_SOURCE_FILE:6)

    clojure.lang.Compiler.eval (Compiler.java:6978)

    clojure.lang.Compiler.eval (Compiler.java:6941)

Compiler.java has

                                IFn fn = (IFn) fexpr.eval();

                                return fn.invoke();

Which of course throws a class cast exception.

But there's so much information right there. Imagine this (in pseudo-code) as

Object f = fexpr.eval();

if (f instanceof IFn) return (IFn)f.invoke()

else throw ClojureNotAFunctionEvaluationError "First position in form at line " ... " of environment " ... " is not a function object. You handed me " + fexpr.first() " in form " ... " which has type " fexpr.first().class() " and value " fexpr.first().toString()

or whatever (like maybe don't toString values - that's why you'd want a plan. And maybe use a string table so you can internationalize. Again, plan :) )

so I almost want to whack this sort of stuff into a local copy as I hit error messages which bug me, but that seems foolish. And anyway I'm new here.

Sorry if I have accidentally stepped into some sort of landmine or if this has been rehashed a million times.

But I figured, in the spirit of being willing to help if there's help to be applied, I would ask if there's some sort of thinking on this topic for 1.9.1 or 1.10 or something?

Thanks

  Paul

[Gary Trakhman]

I think it has been rehashed often and core is very conservative about changes, but the current latest and greatest for improving many kinds of errors is going to be clojure.spec, which runs parallel to the actual execution path, so as not to effect things like performance or old code that depends on the specific weird behaviors in core.  http://clojure.org/guides/spec

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James Reeves]

On 6 December 2016 at 01:28, <pa...@pwjw.com> wrote:

And the error messages are not good.

So I was wondering: Is this a philosophical thing? Or is it an effort thing? And if it is an effort thing, is there some sort of plan for what effort to put in? And if so, can I help by closing tickets?

This is an issue that's been discussed often.

The fundamental problem is that in a dynamically typed language, good error messages are often at odds with runtime performance. The more checks we add to catch specific scenarios, or to provide more descriptive scenarios, the more performance tends to be impacted.

However, Clojure 1.9.0 may have a solution to that in the form of specs. We can turn on specs selectively at development time, so we get the benefit of detailed error messages, while in production we can turn them off for performance.

- James

[Paul Walker]

Yeah I understand that tradeoff. I cherrypicked a case where the runtime difference would be tiny but others are harder I’m sure.

I guess I will go figure out how spec applies to my project.

Thanks as always. This group is so responsive. Appreciated.

- Paul

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/ANKq6XD1nW8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Sean Corfield]

> I cherrypicked a case where the runtime difference would be tiny

 

I wouldn’t be so sure of that. Adding any additional code to the function call logic is going to impact almost every single expression in Clojure and that “tiny” difference is going to add up pretty fast.

 

As others have noted, runtime performance is a Big Deal™ for Clojure and that means that some things are traded off for that speed.

 

Also, as others have noted, this topic comes up repeatedly, probably a handful of times a year in some form or other. After using Clojure in production for just over five and a half years now, all I can say is: you get used to the error messages and you develop an intuition for what most of them mean. Several of the Clojure/core folks have said at various times over the years that Clojure is deliberately not optimized for novices, on the grounds that whilst everyone starts out as a novice, most of your time spent with Clojure is beyond that stage (and an optimization for novices can be a de-optimization for everyone else).

 

My pet peeve is that clojure.string is not nil-safe. (str nil) produces “” but if you pass nil to pretty much any clojure.string function, you get a NPE instead. You might think “Hey, it would only be a tiny difference if clojure.string functions accepted nil and treated it as an empty string!” … until you create a fork of Clojure with that modification and run some benchmarks … J

 

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Mikera]

On Tuesday, 6 December 2016 10:47:42 UTC+8, James Reeves wrote:

On 6 December 2016 at 01:28, <pa...@pwjw.com> wrote:

And the error messages are not good.

So I was wondering: Is this a philosophical thing? Or is it an effort thing? And if it is an effort thing, is there some sort of plan for what effort to put in? And if so, can I help by closing tickets?

This is an issue that's been discussed often.

The fundamental problem is that in a dynamically typed language, good error messages are often at odds with runtime performance. The more checks we add to catch specific scenarios, or to provide more descriptive scenarios, the more performance tends to be impacted.

Runtime checks are clearly bad for performance, but this kind of thing can typically be done at compile time, with no runtime cost.

[Alex Miller]

On Monday, December 5, 2016 at 7:28:21 PM UTC-6, pa...@pwjw.com wrote:

Hi!

Hi Paul,

 

Boy I really think you've all done a nice job with Clojure. I've used quite a few environments over the years and clojure + CIDER + etc is a great experience. The immutability and threading are smart. I've been able to finish a hobby project in clojure which I've been mulling for a long time and never found the right environment. Super stuff.

And the error messages are not good.

If I can make a request, it would be that instead of saying "errors are bad" (which is honestly, not helpful by itself), but to step back and either take a broader approach towards systemic issues or to file and work tickets for specific cases/messages, so they can actually be fixed.

 

So I was wondering: Is this a philosophical thing?

There is certainly no philosophy that errors should be bad. I would say that the state of things right now is more a side effect of prioritization and choices over many years. Rich's focus has typically been on building powerful expressive tools vs the beginner experience (I'd recommend watching his Design, Composition & Performance talk for more background on that). That said, there is no reason not to also improve the experience for newer users.

 

Or is it an effort thing? And if it is an effort thing, is there some sort of plan for what effort to put in? And if so, can I help by closing tickets?

My first wish would be that you log tickets for stuff like the one below. Providing patches would be a helpful next step. Everything is gated by the rate at which Rich looks at tickets though.

 

Here's one, for instance, which bugs me (using a build of branch master in clojure from github just now)

user=> (+ 1 1)

2

user=> (1 + 1)

ClassCastException java.lang.Long cannot be cast to clojure.lang.IFn  user/eval17 (NO_SOURCE_FILE:10)

OK but what sort of noob would actually type in 1 + 1? (That's a joke. Lots of people would. And I do sometimes when switching between java and clojure) But that same error pops up if you misalign in parenthesis or confuse an argument order or do a variety of other things.

And that error message is terrible. I mean I know why it is there. Clojure needs the first element of a list be the function position and 1 is not a function, it is a long. But you have to squint rather generously at that message to get to the problem. Or just learn what that message "really" means, which seems a bit unfriendly.

And so when I go look inside the source 

user=> (print-stack-trace *e 20)

java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn

 at user$eval9.invokeStatic (NO_SOURCE_FILE:6)

    user$eval9.invoke (NO_SOURCE_FILE:6)

    clojure.lang.Compiler.eval (Compiler.java:6978)

    clojure.lang.Compiler.eval (Compiler.java:6941)

Compiler.java has

                                IFn fn = (IFn) fexpr.eval();

                                return fn.invoke();

Which of course throws a class cast exception.

But there's so much information right there. Imagine this (in pseudo-code) as

Object f = fexpr.eval();

if (f instanceof IFn) return (IFn)f.invoke()

else throw ClojureNotAFunctionEvaluationError "First position in form at line " ... " of environment " ... " is not a function object. You handed me " + fexpr.first() " in form " ... " which has type " fexpr.first().class() " and value " fexpr.first().toString()

or whatever (like maybe don't toString values - that's why you'd want a plan. And maybe use a string table so you can internationalize. Again, plan :) )

Another thing to keep in mind is that you may be underestimating the full set of paths that lead to this particular error and whether the error message you are crafting makes sense in all of them. In runtime code, constructing better errors changes the size of the generated code, which can affect the ability of the JVM to inline code and lead to unexpected performance degradations (we've had to pull error message improvements for this in the past). I say this not to dissuade us, but just to highlight that things are not always as easy as they seem at first.

 

so I almost want to whack this sort of stuff into a local copy as I hit error messages which bug me, but that seems foolish. And anyway I'm new here.

Logging bugs is helpful. A search for dupes is helpful, but not essential. I am mostly aware of what's in the system already and will dedupe as needed.

 

Sorry if I have accidentally stepped into some sort of landmine or if this has been rehashed a million times.

But I figured, in the spirit of being willing to help if there's help to be applied, I would ask if there's some sort of thinking on this topic for 1.9.1 or 1.10 or something?

I have spent some time thinking about systemic classes of problematic errors and error reporting. One class of problems come from macros with complex syntax - these tend to have hand-written parsers that make a lot of assumptions and also typically report their errors as the location of the macro code rather than the user code. Another class are functions in core that tend to either fail on unexpected types or bottom out several layers down in RT java code. All of these are substantially improved by the presence of specs which can produce more accurate errors at the earliest point in the stack where an invalid value is passed. Another area that Stu and I have talked about a bit is in error reporting, particularly around compiler errors (as compared to runtime errors) - right now we are tied to only an internal compiler exception class for reporting errors with location, but this could definitely be better. And then I think some of the existing tooling does a pretty poor job of showing the best info when an error is received.

Alex

[Mars0i]

On Monday, December 5, 2016 at 10:36:59 PM UTC-6, Sean Corfield wrote:

Several of the Clojure/core folks have said at various times over the years that Clojure is deliberately not optimized for novices, on the grounds that whilst everyone starts out as a novice, most of your time spent with Clojure is beyond that stage (and an optimization for novices can be a de-optimization for everyone else).

This makes sense for anyone with significant programming experience.  It's a shame, though, that Clojure would be a great language for novice programmers except for the error messages, which are probably a show-stopper for some p teachers thinking of using it for teaching.  I know that there are people involved in Clojure Bridge and other teaching programs, but as a teacher (in another discipline), I personally would have a hard time choosing Clojure over Scheme at this point, if I was teaching an intro programming course.  Clojure's probably a better language for later growth though, given its easy integration with Java and Javascriptl.

I'm not arguing for a change.  I like run-time performance, too.  Spec messages are not any good for novice programmers, either, but I'd guess that down the road it will be possible to build a friendly error system on top of spec.  Can't do that as easily with JVM stacktraces. :-)

[Paul Walker]

Wow thank you for the fulsome responses

A couple of thoughts. I brashly said

> I cherrypicked a case where the runtime difference would be tiny

and I appreciate the clarifications - especially that my naive add-one-bytecode calculation in my head (the code I pasted has a pretty meaty “analyze” call called before it every time which is why I thought I cherry-picked) didn’t think about things like byte code inlining or, really, the pile-of-sand one-more-bytecode syndrome. So thanks for that!

And my favorite response from Alex:

> If I can make a request, it would be that instead of saying "errors are bad" (which is honestly, not helpful by itself), but to step back and either take a broader approach towards systemic issues

This is so reasonable I’m embarrassed I was a bit glib in my first note. I mean it’s not like they are C++ template errors from 1998  (my gold standard for bad errors). So let me expand on “bad” a little. 

I think the error messages are “bad” in 3 particular but related ways. I’d be happy to expand on these, but the themes are:

1: They are “large” and “far from my code”. This is partly an issue with java stack traces, but the intermixing of java and clojure and the presentation of clojure names in java name-like things in the stacks is obscuring. The line numbers are obscured instead of displayed prominently. And so on.

2: Similarly, they expose internals of the runtime implementation to the user. Stacks intertwine java and clojure (CIDER error screen has a toggle to supress the stack partially for this). They show java exceptions as clojure errors when they aren’t part of a clojure program. etc…

3: These, plus a few other things, mean the error messages are semantically far away from the program you write. You stay happily in LISP-y land until you get a stack, then you realize you are in java. There are lots of examples of this, but take the dumb one I picked - (1 + 1) in a REPL. That shows two concepts (java.lang.Long and clojure.lang.IFn) which you don’t deal with in lisp. I’ve written quite a lot of clojure in the last 5 months and never typed clojure.lang.IFn once. It’s a java error message but I’m a clojure programmer. Instant semantic break. And since many error messages are generated at the runtime level, which is java / JVM, I experience this semantic break often. Add the excellent comments in this thread on macro expand non-locality of error and the problem expands.

So if you imagine a guide like “the error should be as close to the error causing code as possible and presented in the semantics of the problem” then I think I conclude clojure messages are “bad”. (But I love clojure) 

It seems spec, inasmuch as it acts as a sort of optional type system which is in the space of the clojure language and produces information in that space, would address many of these. 

And as to why I mentioned this: I mean I haven’t had a problem. I just built a dual mental model of clojure and the clojure runtime while I learned the language and swapped between when an error popped. And I’ve written a lot of java (and interpreters and compilers too) so that was easy enough for me to do. And I have been hugely productive in your kit. I love clojure! 

But indeed for teaching it’s an obstacle, but also for teaching’s adjacent problem, using a language or toolset in a group of programmers of mixed abilities, I was thinking errors would be something I’d improve.

But really, it seemed like a place I could lend a shoulder if work was afoot, so the comments on the ticketing approach are appreciated and I will dig into them.

Thank you for your rapid and thoughtful responses

- Paul

[Alex Miller]

On Tuesday, December 6, 2016 at 5:55:52 AM UTC-6, Paul Walker wrote:

Wow thank you for the fulsome responses

A couple of thoughts. I brashly said

> I cherrypicked a case where the runtime difference would be tiny

and I appreciate the clarifications - especially that my naive add-one-bytecode calculation in my head (the code I pasted has a pretty meaty “analyze” call called before it every time which is why I thought I cherry-picked) didn’t think about things like byte code inlining or, really, the pile-of-sand one-more-bytecode syndrome. So thanks for that!

I think in the case you picked that you're right that the performance impact is minimal. This happens at compile-time and generally we are less sensitive there than in calls during the runtime. Colin's talk from last year's conj gives a good idea of some of the challenges in working on this stuff. https://www.youtube.com/watch?v=kt4haSH2xcs

 

And my favorite response from Alex:

> If I can make a request, it would be that instead of saying "errors are bad" (which is honestly, not helpful by itself), but to step back and either take a broader approach towards systemic issues

This is so reasonable I’m embarrassed I was a bit glib in my first note. I mean it’s not like they are C++ template errors from 1998  (my gold standard for bad errors). So let me expand on “bad” a little. 

I think the error messages are “bad” in 3 particular but related ways. I’d be happy to expand on these, but the themes are:

1: They are “large” and “far from my code”. This is partly an issue with java stack traces, but the intermixing of java and clojure and the presentation of clojure names in java name-like things in the stacks is obscuring. The line numbers are obscured instead of displayed prominently. And so on.

Personally, I don't have a problem with stack traces - there's a ton of useful information in a stack trace. The issues of "distance from code" are interesting in Clojure though. One area I mentioned is due to errors in macros where it reports either the error in the macro rather than in the source OR where you get a runtime code that is based on code that was written by the macro (not by you). The first problem is significantly improved by spec. Another source of "distance" is laziness - sometimes a nested operation will not error until a lazy seq is forced, possibly far from the core of the problem. Eager transformations (we have more options for this now with transducers) can help with that, as can spec in reporting more accurately the error when it occurs.

 

2: Similarly, they expose internals of the runtime implementation to the user. Stacks intertwine java and clojure (CIDER error screen has a toggle to supress the stack partially for this). They show java exceptions as clojure errors when they aren’t part of a clojure program. etc…

Again, Clojure is an unapologetically hosted language. There is no desire to hide that fact or to shield you from it. That means that learning Clojure inevitably means learning something about the JVM or Java as well. 

 

3: These, plus a few other things, mean the error messages are semantically far away from the program you write. You stay happily in LISP-y land until you get a stack, then you realize you are in java. There are lots of examples of this, but take the dumb one I picked - (1 + 1) in a REPL. That shows two concepts (java.lang.Long and clojure.lang.IFn) which you don’t deal with in lisp. I’ve written quite a lot of clojure in the last 5 months and never typed clojure.lang.IFn once. It’s a java error message but I’m a clojure programmer. Instant semantic break. And since many error messages are generated at the runtime level, which is java / JVM, I experience this semantic break often. Add the excellent comments in this thread on macro expand non-locality of error and the problem expands.

I get this and it's unfortunate when it exposes to you more than you need to know initially. But at the same time, I think developing that knowledge is essential to make the best use of Clojure in the long term.

 

So if you imagine a guide like “the error should be as close to the error causing code as possible and presented in the semantics of the problem” then I think I conclude clojure messages are “bad”. (But I love clojure) 

It seems spec, inasmuch as it acts as a sort of optional type system which is in the space of the clojure language and produces information in that space, would address many of these. 

And as to why I mentioned this: I mean I haven’t had a problem. I just built a dual mental model of clojure and the clojure runtime while I learned the language and swapped between when an error popped.

That seems like a good thing to me. :) 

 

And I’ve written a lot of java (and interpreters and compilers too) so that was easy enough for me to do. And I have been hugely productive in your kit. I love clojure! 

But indeed for teaching it’s an obstacle, but also for teaching’s adjacent problem, using a language or toolset in a group of programmers of mixed abilities, I was thinking errors would be something I’d improve.

There are some interesting gradations even in this depending on whether you're talking very new programmers, or experienced programmers new to Clojure, or experienced *Java* programmers new to Clojure, etc. Elena Machkasova has been working through how to present Clojure error messages for new developers at U of Minnesota. An older talk on this is at https://www.youtube.com/watch?v=k5erDyDPzgc. Elena has actually done some user studies on this as well (which Cognitect funded). I don't think she would say that this work is appropriate for all Clojure developers though.

[Paul Walker]

Very useful. So it is a *bit* philosophical :) I guess the unabashedly hosted philosophy is one I wasn't so sure was intentional but it's good to understand; and embracing that changes the way you'd teach or introduce the language. Gotcha. Where I thought I "had to form" a dual mental model and I should have thought I "got to learn" one quickly. Cool!

And yeah lazyness and locality of error is always always hard. 

I loved elenas talk. At about the 16 minute mark she takes to task exactly the message I picked. Her solution looks tricky for non-instructional uses though, like when she rebinds map. And a big pile of stack regexps is tricky to maintain beyond teaching use cases I could imagine.

Anyway, thanks for the links and thanks for the thoughtful answers. I really appreciate it. It has me thinking.

Be well,

  - Paul